Managing Drupal Configuration with Automated Checks

Managing Drupal configuration files and keeping them in sync with a database requires intense attention to detail and process. The importance of following the correct sequence of events can be documented, but mistakes still happen. Additionally, trying to get our robot friends such as Dependabot to play by the same rules presents a challenge.

The ideal sequence of configuration commands can even vary by where the configuration operation is performed. This message from Drupal Slack illustrates the requirements nicely.

Yes it should be like this:

Locally:

1. drush updb

2. drush config:export + commit

Deploy:

1. drush updb

2. drush config:import

Missing Config Updates

When Drupal core or a Drupal module releases an update, it can alter the structure or values of configuration. This is fine as long as those changes make it back into the tracked configuration files. If the “Local” process described above is followed, this isn’t an issue because the changes are exported and tracked.

However, if a tool such as Dependabot performs core or contributed module updates, it will not have a database to run updates on, and it will most certainly not be able to export those config changes and commit them. After the Dependabot update is deployed, the active database configuration and the config tracked in code are no longer in sync. Subsequently, the output of drush config:export will now show a diff between the active configuration in the database and the tracked configuration files. This can lead to unintended consequences down the line.

Config Process & Development

Suppose a Dependabot PR that resulted in config changes as described above was just merged. Then a developer grabs a fresh database and the latest code, and they go about their work while making some changes to the site’s configuration locally. They export their changes, but the diff now includes unrelated changes. This is a result of the config changes made in the database from the previously deployed Dependabot update that were never exported and committed.

This pattern will repeat and become harder to untangle until someone decides to include the changes in an unrelated PR or create a separate PR to bring things back into sync. This isn’t fun for anyone. The best solution is to avoid this entirely.

At Chromatic, we have made attempts to monitor release notes for signs that a config change may be included, but naturally some slip through the cracks. The only way to truly avoid this and keep ourselves accountable is to automate the process of checking for config changes on every pull request.

Automate a Configuration Checking Solution

Any code branch should be able to import its configuration into a production database and subsequently export the active configuration without a resulting configuration diff. Thus the solution becomes to run a standard deployment which includes a configuration import, run configuration export (drush config:export), and verify that the output confirms everything is synced.

 [notice] The active configuration is identical to the configuration in the export directory (../drupal-config/sync).

Note: This will likely only be done in pre-production environments, but that is where we want to catch our problems anyways!

Our Solution

We have automated our deployments to pre-production environments, such as Tugboat, with a standardized Ansible based deployment tool. In a recent release we added functionality to conditionally run drush config:export and check the output for active configuration is identical, and we were in business.

# Identify Drupal configuration changes in core or contrib updates that need
# to be exported and may have been missed.
- name: config structure check
  shell: "{{ deploydrupal_drush_path }} config:export -y"
  args:
    chdir: "{{ deploydrupal_core_path }}/sites/{{ deploydrupal_site_name }}"
  register: deploydrupal_config_check_result
  when:
    - deploydrupal_config_check_structure
  failed_when: "deploydrupal_config_check_structure_string not in deploydrupal_config_check_result.stderr"

Every build now checks for configuration synchronization between our code and database, keeping us accountable as we go.

Checking Configuration Content Changes

The process outlined above will catch changes to configuration schemas, as the changes are made via importing configuration and not blown away by it. However, the potential for including a config change via an update hook (ran via drush updatedb), and the config change to be lost still remains.

A great example of this is an update to the Google Authenticator login module that changed the name of certain configuration keys. The resulting Dependabot PR failed to run in our QA environment alerting us to the problem. It was resolved by following the correct steps locally and committing the updated configuration changes. Due to Google Analytics loading on every page and failing loudly it was caught, but the issue might not always be so easily found. Detecting this problem programatically is quite simple in theory, but more difficult in practice. The basic process is:

• Ensure all configuration is imported and up to date (drush config:import).
• Run the update hooks (drush updatedb).
• Export configuration and check for a diff (drush config:export).

This is fine until you realize that the drush deploy command and the suggested deployment command order noted above make it clear that we need to run our database updates (drush updatedb) before our configuration import (drush config:import) when deploying code.

Any pre-production deployment process should mirror the sequence of commands run during a production deployment to properly test the changes, which creates a problem. There are various ways around this with enough additional infrastructure and creativity, but based upon our research, none of them could be implemented with just changes to a deployment script alone.

Many updates that result in a change to configuration keys/values will often cause a failed build during the pre-production deployment process, which accomplishes the same end goal of alerting us to the configuration synchronization problem. However, that won’t always be the case, so we will continue to search for a way to fully validate the configuration.

The Value of Configuration Maintenance

The configuration system empowers us to do incredible things. However, it must be used carefully and maintained meticulously if we want it to be the canonical source of truth for our sites. The subtle yet important differences in workflows under different scenarios must be respected by everyone, humans and robots alike. With proper care, well-managed configuration will continue to foster reliable deployments and simplify the development and code review process for everyone.