You Need a Change Control Process for your Software Projects
Protect your systems from human error with a few simple steps
Have you ever screwed up some production code or website when you deployed your fresh new changes?
I know I have!
If you’re a developer long enough, you’re going to end up doing something that seemed perfectly fine…until it breaks everything and you’re running around in a mad panic to fix things before someone notices.
It doesn’t have to be that way.
For just a little bit of extra work and a little bit of poking around and writing things down, you can avoid messing things up when you deploy your software.
Change Control is Good
Change control “reduces the possibility that unnecessary changes will be introduced to a system without forethought, introducing faults into the system or undoing changes made by other users of software.” Taking a moment to think through how your change can affect existing systems is a vitally important part of software development.
The more complex a system is, the more care one must take when introducing new or modified code into it.
So how do you exercise that care? Do you walk around and talk to every other developer, DBA, project manager, and product owner in your company? Do you read through every line of code currently in production and then spend a week looking at process flow charts and making test adjustments?
Imagine how much time that would take…
Fortunately, a functional change control process is fairly simple, and the most important part of it is simply another set of eyeballs on the plan.
The One Thing You Have to Do
Impact analysis is the one thing that needs to be a part of every change control action. No matter how monumental or insignificant you think your change is, you need to take a step back and look at what it could potentially do to your production systems.
The bigger the change you are introducing, the more impact analysis you have to do.
Is your change going to lock some users out of the system?
If you’re changing database schemas, will it destroy some existing data or break the current database layer code on existing websites?
Are you changing a class definition, are there any child classes that you will then break if you don’t also update them?
If you don’t know much about a system or process that your new change is going to touch, go talk to someone. Ask questions and bring in other developers to help you investigate.
Even if you built the entire system and you know it front to back, you still need to do this step. Maybe you’ve forgotten something you did a couple of years ago. Maybe you’re carelessly rushing through this change. Maybe you’ve made a mistake in your code or your overall logic.
Do the impact analysis, because you’ll be kicking yourself later if you break something that takes hours to fix but could have been avoided.
What’s Your Backout Plan?
When preparing to push a code or server change out into your production environment, it’s not enough to do an impact analysis. You also need a backout plan.
What are you going to do when you make the change in production, go to verify your change went well, and then everything is broken?
Hey, it happens. It happens to the best of us.
It should be simple enough to fix that little oops, but like everything else, the bigger the change the more thought you have to put into rescuing you from yourself.
Have you made backups of everything that’s in production now, in the area you’re updating? Did you make a list of old settings that you’re updating before you update it? Do you have a list of all the things you need to check to make sure your rollout went smoothly so you can find potential problems?
This leads us to…
Write it All Down
Ah, documentation. The bane of all software engineers.
Or is it?
Writing all of this down is important, so don’t skip this step. What good is a plan if you think it up on a Friday and then it’s all forgotten by Monday? There is another reason why you need to write everything down, which I’ll get to in a moment, but the first and simplest reason is just so you keep everything straight and don’t forget parts of your production rollout plan.
Whether you have an official change control document you have to fill out, a ticket you need to create in Service Desk, VersionOne, Jira, etc., or you just need to send out an email to your group, you need to write it all down.
Documenting your impact analysis, implementation plan, checklists, backout plan, and any other piece of information you may need for this change is important. You need to be able to track what you do over time. You need to be able to prove to people that you thought through your change in case something goes wrong and people are mad. There may be procedural requirements within your company or even legal requirements to document changes, so writing everything down can keep you covered.
Another Set of Eyes
One of the most important reasons you need to write it all down is so you can give your plan and analysis to another engineer to look at.
It doesn’t matter whether you have a formal process for review and approvals or you just walk over to someone else’s desk and say, “Hey, can you take a look at this?” Another set of eyes may see problems you overlook.
For example, this individual may look at your implementation plan and say, “Oh, this is a bad time to roll this out because of all those people who use this at the time of day you want to make the change. Better do it off-hours.”
Or they might say, “What about System X? Nobody ever thinks about System X, and this might affect it.”
If you have some downtime baked into your code rollout, will it have a financial impact? If so, somebody on the business side of things might also need to look at your plan.
No More Blowing Things Up
Having a change control process (with associated documentation) can keep you from blowing things up. Businesses don’t like their smooth, profit-making machines out of operation, and developers don’t like doing extra work cleaning up messes.
So your best option for smooth sailing through your deployments is to take a bit of time to think through what could blow up and write down a plan so it doesn’t happen.
Feel free to subscribe to my YouTube channel for more content like this and also not like this.
