When a website or application deploys unsuccessfully, someone’s weekend is usually ruined by having to clean up the mess. That’s the least of the resulting problems, though. When major bugs make their way out to production, your agency looks bad to your clients.
A set of good practices and processes is the key to avoiding website downtime and buggy application deployments. A dependable web development partner can ensure adherence to an automated and repeatable deployment strategy.
If your clients’ deployments go south, their sites and applications can suffer revenue-draining downtime as ecommerce, ads, or subscription-based services go dark. Here’s what has to happen to avoid these deployment problems.
How to Solve Website & App Deployment Problems Before They Start
Any time new code deploys to a live site or web application, there is a chance for error. Just because there is a chance for error doesn’t mean we have to accept that error as inevitable, though. Consider a burning building.
It’s not enough to know there’s a fire. That’s why we don’t just have smoke detectors. We also have fire escapes, fire extinguishers, sprinkler systems, and signage posted in commercial buildings to guide us to safety.
Putting actionable processes in place to guide deployments back to safety is necessary, too. Solutions can’t be thought up in the moment. Planning for common deployment issues is the first step toward mitigating deployment risk.
In our experience deploying code, launching new websites and apps, and migrating client sites, we’ve identified a few typical problem areas.
Oversights That Endanger Web Development and Software Deployments
Whether it’s a client’s internal development team or your web development partner, a qualified party needs to be on the lookout for potential deployment problems. Most can be traced back to:
- A lack of code standardization. We follow basic grammar rules because standardization in language allows for effective communication. The same applies to writing code. Teams that aren't following strict code style rules may not catch certain bugs before they make it out to production.
- Too much manual task completion. Certain tasks are complex enough to warrant manual solutions. Whenever possible, though, it’s best to automate. Automation eliminates human error, provided it’s set up properly.
- Insufficient testing. Code deployments require thorough testing at all stages. This can happen in the production environment (live site) itself, or in a staging site that mirrors it as closely as possible.
Now that you know some of the common deployment pitfalls, let’s dig a little deeper into how to avoid them. We’ve seen these best practices pay off with consistently smooth deployments.
Avoid Deployment Pitfalls Through Peer Review, Code Freezes, and Automation
Depending on your application, website architecture, etc., deployments can vary from the simple to the complex. In either case, the following steps help prevent the majority of issues that might arise during a deployment.
Ahead of and during your clients’ deployments, special attention must be given to:
Thorough Peer Reviews, Allowing for Team Consensus and Buy-In
Before any new code is integrated within a codebase it needs to be put through a human review process. A peer review is best completed by an individual with the same or similar core competencies as the party introducing the code. That way, they’ll know exactly what kinds of issues they’re looking for and be able to offer solutions.
If your clients have an internal development team, they need to understand the importance of peer reviews. This is a non-negotiable step for our own team. Why? It ensures a certain level of code quality while also creating accountability. The more people there are who understand how the software is made and how new features are architected, the more there are to catch problems and help when something does go wrong.
What this code review looks like often depends on the makeup of the development teams involved. On our team, everyone writes code with a high level of expertise. With this even playing field, our process is just a simple step of peer-reviewing any changes to a codebase and calling out anything that might be improved. If your clients/teams have more junior developers on staff, it’s a good idea to pair that junior talent with more senior developers as reviewers. It’s all about having confidence in what you’re deploying before you push the button. The more eyes on the code beforehand, the better.
Code Freezes, Ensuring Code Is Deployed Exactly As Intended
We’ve seen it happen with our own clients. They test everything one week, and it looks fine — but then a week passes. When they say, “...okay, go ahead and deploy,” they don’t realize someone has made additional, untested changes during that past week. Those untested changes can and often do wind up causing problems in deployment.
Depending on the size of a project, multiple teams may contribute to different parts of an application. Those multiple parts coalesce into a single release and must be tested together as a whole. Testing features in isolation (also an important step) can mean certain problems are missed that only surface once they are combined. The overlap itself may be where bugs are introduced. Code freezes to the rescue!
A code freeze doesn’t need to be a complicated process. For example, we just don’t deploy on Fridays. Doing so might introduce bugs that leak into the weekend. That can lead to substantial downtime and revenue loss as online stores, ads, subscriptions, etc. are inoperable. We essentially code freeze every Thursday, meaning no new code is merged (and thus deployed) into production until Monday. It’s our way of ensuring we know exactly what we’re launching, and that what we’re launching has been thoroughly tested without any gotchas sneaking in along the way. Oh, and that we aren't ruining anyone's weekend.
Automate Wherever Possible, Creating Standardization and Eliminating Human Errors
Clients often come to us to help them "fix their deployments." They complain of deployments going poorly with their existing development partner and they want to figure out how to avoid having their weekends ruined. In almost every case, we find that their biggest problem is that the deployments are being handled manually, oftentimes by a single individual. Entire applications being deployed by a single person, attempting to coordinate multiple steps, across servers, and crossing their fingers all goes well.
There is a better way: automation. Computers happen to be really good at automation and today's tools make deployment automation much easier than, say, a decade ago. With the right infrastructure in place, every step of a website or web application deployment process can be automated.
With automation in place, developers can focus on improving the app, instead of worrying about how the code gets out to production. Stakeholders alike do not have to worry about "pushing the button," because a computer handles all the necessary and monotonous steps, prone to human error. Confidence grows, there are fewer surprises, and everyone is happier.
Common areas of opportunity for automation include:
- Code styling, which can be accomplished through an automated linter program.
- Fetching and integrating code into the live environment.
- Hosting or DNS changes required before deployment.
- Importing configuration files.
- Clearing caches (CDN, application, etc.).
Clearing caches and importing config files aren’t particularly complicated steps. But they need to get done, and automating these kinds of tasks ensures they’re not overlooked by a human.
The other major benefit of deployment automation is that the automation itself can help expose deployment issues. When using a standard deployment script/tool to deploy to Staging or QA environments, you can help expose issues that might only crop up during a deploy. Better that this happens on QA than production!
Tactical Automation Items: Backups, Testing, and Rollback Strategies
Two key areas of automation where we want to shine the spotlight are backups and testing. Code and databases always require backups. We strongly advocate not only for automating backups but also for automating their testing.
We’ve run into a common backup issue with multiple agencies. They automated their database backups, but they never test those backups. Then, when something went wrong and they needed to rely on those backup files, they learned the hard way their backups were actually silently failing. Automating the testing of your backups addresses this concern. One way we do this is by using our database backups daily to build our QA environments. We know they are working because as team members test new work on our QA environments they are also testing last nights' database backup! If these were broken, we'd know really quickly.
When all else fails, it’s important to have a strategy for getting back to where the site or app was before the problem hit. That way, you can roll right back to the last known stable state. Think of it as a last resort, but one you'll someday wish you had.
Modern Platform as a Service (PaaS) Providers Address Website and App Deployment Needs
Today, there are plenty of PaaS providers to choose from. That means items like automated deployments, nightly backups, simple rollbacks, or automated preview/QA environments are just part of the package. But even a great PaaS provider, like Platform.sh, can only take things so far.
Not every application is suitable for a PaaS solution. Some might be incompatible, while others may have a governing body dictating the use of on-site vs. cloud-based servers. In such cases, a web development partner who can check every box to ensure a smooth deployment is a must.
Even if a website or application is the perfect fit for PaaS hosting, working with a reputable web development partner is still a worthwhile investment. With so many of the issues above addressed by the hosting service, there is more budget for your web development partner to work on providing your clients with more value elsewhere.
One final note — while these tools may all be provided by a PaaS provider, utilizing those tools is a different story. Someone needs to actually integrate with said service. The included tools facilitate all of this, but they can’t do everything. A web development partner helps get the most out of these platforms.
We hope you find this guide for successful deployments helpful. If you have a deployment on the horizon and you're looking for help we'd love to talk.
