How to implement Continuous Deployment with Drupal

The Guardian, one of the most trusted news media, took a different approach for their membership and subscriptions apps. Rather than emphasising on lengthy validation in staging environments, The Guardian’s Continuous Deployment pipeline places greater focus on ensuring that the new builds are really working in production. Their objective was to let the developers know that their code has run successfully in the real world instead of just observing green test cases in a sanitised and potentially unrepresentative environment.

Thus, The Guardian reduced the amount of testing run pre-deployment and extended the deployment pipeline constituting feedback on tests run against the production site. Such is the significance of utilising a lightweight Continuous Deployment pipeline which has helped a large organisation like The Guardian to focus on production validation instead of a large suite of acceptance tests. Such benefits can be witnessed in the Drupal-based projects as well where Continuous Deployment can allow us to iterate on Drupal web applications at speed.

Read more on the implementation of Continuous Integration and Continuous Delivery with Drupal

A Brief Timeline of Continuous Deployment

Agile Aliiance has stated that the origins of Continuous Deployment can be traced in the early 2000s. In 2002, Kent Beck, creator of Extreme Programming, has mentioned Continuous Deployment in the early discussions (unpublished) of applying Lean ideas to software where undeployed features are seen as inventory. However, it took multiple years for it to be refined and codified.

Later, in the proceedings of Agile 2006 Conference, the first article describing the core of Continuous Deployment - The Deployment Production Line - came into the limelight. Published by Jez Humble, Chris Read and Dan North, it was a codification of the practices of numerous ThoughtWorks UK teams.

By 2009, the practice of Continuous Deployment became well established as can be seen through the article called Continuous Deployment at IMVU by Timothy Fitz. Not only it is beneficial in Agile processes, but its great features can be extracted for methodologies such as a Lean startup or DevOps.

Continuous Deployment in focus

While Continuous Integration refers to the process of automatically building and testing your software on a regular basis, Continuous Delivery is the logical next step which ensures that your code is always in a release-ready state. The ultimate culmination of this process is the Continuous Deployment.

In Continuous Deployment, every alteration that passes all stages of your production pipeline is released to the customers

In Continuous Deployment, every alteration that passes all stages of your production pipeline is released to the customers with no human intervention and only a failed test will deter a new alteration to be deployed to production. It is a spectacular way to aggrandise the feedback loop with your customers and take pressure off the team as is takes away the so-called ‘release day’ from the equation. It allows the developers to emphasise on creating software and they can see their work going live minutes after they have put in all their efforts on it.

Why Should you Consider Continuous Deployment?

Continuous Deployment benefits both the internal team who are implementing it and the stakeholders in your company.

For internal team

• Instead of performing a weekly or a monthly release, moving to feature-driven releases enables faster and finer-grained upgrades and helps in debugging and regression detection by only altering one thing at a time.
• By automating every step of the process, you make it self-documenting and repeatable.
• By making the deployment to the server fully automated, a repeatable deployment process can be created.
• By automating the release and deployment process, you can constantly release the ongoing work to the staging and QA servers thereby giving visibility fo the state of development.

Moving to feature-driven releases enables faster and finer-grained upgrades

For stakeholders in the company

• Instead of waiting for a fixed upgrade window, you can release features when they are ready thereby getting them to the customer faster. As you are constantly releasing to a staging server while developing them, internal customers can see the alterations and take part in the development process.
• Managers will see the result of work faster and progress will be visible when you release more often
• If a developer needs a few more hours to make sure that the feature is in perfect working condition, then the feature will go out a few hours later and not when the next release window opens.
• Sysadmins will not have to perform the releases themselves. Small, discrete feature releases will enable easier detection of the alterations that have affected the system adversely. 

Continuous Deployment Tools

Unit tests and functional tests put the code into as many execution scenarios as possible for predicting its behaviour in production. Unit testing frameworks consist of NUnit, TestNG and RSpec among others.
 
IT automation and configuration management tools like Poppet and Ansible manage code deployment and hosting resource configuration. Tools like Cucumber and Calabash can help in setting up integration and acceptance tools.
 
Monitoring tools like AppDynamics and Splunk can help in tracking and reporting any alterations in application or infrastructure. Performance due to the new code. Management tools like PagerDuty can trigger IT incident response. Monitoring and incident response for Continuous Deployment setups should be to real-time for shortening time to recovery when there are hassles with the code.
 
Rollback capabilities are essential in the deployment toolset to detect any unexpected or undesired effects of new code in production and mitigate them faster. Moreover, canary deployment and sharding, blue/green deployment, feature flags or toggles and other deployment controls can be useful for organisations looking to safeguard against user disruption from Continuous Deployment.
 
Some applications can deploy in containers such as Docker and Kubernetes for isolating updates from the underlying infrastructure.

Continuous Deployment with Drupal

A digital agency worked with Drupal 8, Composer, Github, Pantheon and CircleCI around Continuous Integration and Deployment. The project involved moving from internal hosting to the cloud (in this case, Pantheon), moving the main sites from Drupal 7 to Drupal 8 and implementing a new design.

To the cloud

Pantheon was chosen as the cloud host for new Drupal sites. Initially, it was chosen for features like ‘Cutom Upstreams’, one-click core updates, simple deployments between development, Test, and Live environments, Multidevs, and the fact that each is a Git repo a heart. Terminus (Pantheon CLI tool) was heavily used and appreciated.

Migration to Drupal 8

It focussed on two main umbrella sites and one news site to serve both umbrella sites. It did a content refresh which showed that only content that needs to be migrated are the news articles. The configuration management of Drupal 8 was found to be nicer than the Drupal 7.

Custom Design

As the Drupal is not the only web platform they were using, instead of building a Drupal theme, they built a platform-agnostic project with a new look and feel. It was based on the Zurb foundation and was just HTML, CSS, and JavaScript.
 
Grunt was used as the build tool. So when they have a new release, they would just commit and push to Github. That triggers a CircleCI workflow which tags a new release and publishes the release artefact as an npm package to Artifactory. From there, npm package can be pulled into any project including Drupal.
 
It should be noted that the published package includes only the CSS, JS, libraries and other assets. After the publishing, a static site is created with the package and corresponding HTML templates on a cloud host as a reference implementation.

Deployment Process

They had an ‘upstream’ repo on Github named umbrella-upstream which is a composer-based Drupal 8 project with a custom install profile comprising of custom modules, package.json, and deploy scripts. Each of the sites (umbrella-site X, umbrella-site Y, etc.) was also in a Github repo as composer-based Drupal 8 project and had umbrella-upstream configured as a remote.
 
When they push an alteration to the upstream repo, a set of CircleCI workflows gets started that runs some Codeception acceptance tests and the alterations get merged from umbrella-upstream down to each umbrella-site X/Y repo.
 
Then, another CircleCI workflow builds, tests and pushes a full Drupal umbrella-site X/Y install to the corresponding Pantheon site X/Y all the way up right to the test environment. Quicksilver hooks were used to send any alterations Pantheon back to the site repos.

Entire Workflow involved:

• Code alterations and Git commit in custom design repo
• Npm update custom-design -save-dev, grunt and Git commit in umbrella-upstream repo

Finally, the alterations show up in the Test environment of each site on Pantheon.

Conclusion

It is of paramount importance that you keep iterating and deploy software at speed and with efficacy. Continuous Deployment is a great strategy for software releases wherein code commit that passes automated testing phase is automatically released into the production environment.
 
Drupal deployment can benefit to a great extent through the incorporation of Continuous Deployment in the project development process. The biggest advantage of doing so is that it makes the alterations visible to the application’s users.
 
Opensense Labs is committed towards the provision of wonderful digital experience to the organisations with its suite of services.
 
To make your next Drupal-based project supremely efficacious through the implementation of Continuous Deployment, ping us at [email protected]