Managing Complex Configuration with Drupal 8

Configuration management (CM) in Drupal 8 is great. Especially when every environment is configured with the same values and there is no need for values that vary by environment or values that editors can override. However, when more complex configuration scenarios arise, we must find solutions to problems that core CM functionality doesn’t address (yet). Let’s walk through some of the scenarios you might encounter.

"I need a different set of modules enabled/disabled per environment along with their configuration." or "I need different pieces of configuration enabled/disabled depending on the current environment."

For example, you may want to keep Field UI, Views UI, and other interface modules disabled on production, while ensuring they are enabled on other environments. Conversely, you may need development modules to be disabled on production along with keeping their configuration out of the database, but maintain their configuration across development sites. Or perhaps you need to disable production focused modules, like caching modules, on development environments.

Configuration Split allows you to do all of this by creating separate chunks or “splits” of configuration that you can then enable/disable per environment. Custom configuration import and export Drush commands are provided with this module. These will replace the standard config-import and config-export in your workflow for now, but integration with the standard Drush commands seems to be in the works.

Check out the following for additional information and in-depth tutorials:

Update: An issue has been filed for getting this type of functionality integrated with Drupal core.

Update: This issue explains why custom drush commands are no longer needed thanks to the filters provided by Config Filter.

"I have a set of configuration files in a module that I want to be made active when the module is installed, but allow editors to make subsequent changes to the configuration values."

For example, you may need to deploy some updated entity type descriptions or field descriptions, but after they are in place, an editor should be able to update the values and not have the values reverted after the next config import is performed. To pull this off there are a couple options:

Configuration Rewrite allows configuration YML files to be placed in a module and those values will override the active configuration upon the module's installation. The module's project page says it best:

Place any configuration .yml in your module's config/rewrite folder and declare config_rewrite a dependency to your module. Full and partial configuration changes are saved on installation.

Drush CMI tools could also help in this scenario. It provides a Drush command (drush cimy) with a flag that allows a specified configuration folder to be imported for installation only. This same configuration is then added to a special config-ignore.yml file that instructs the subsequent imports to ignore the given configuration.

drush cimy --source=/path/to/config-export --install=/path/to/config-install

"I have some configuration that I never want exported or imported."

Suppose there is a subset of configuration that is frequently altered by editors, so attempting to keep the configuration files up to date with the latest values would be nearly impossible. (There is a module that makes this possible, which we will address shortly).

Drush CMI tools, as mentioned in the previous scenario, solves this by providing a set of Drush commands that allow a defined subset of configuration to be ignored during the import and export process. The drush cimy process is very similar to the drush config-import --partial import process, but solves for the configuration deletion problem that --partial introduces. The Partial Imports section of the Drush CMI tools GitHub page explains this issue in much greater detail.

This process very is similar to Configuration Split in that it allows selected configuration to be ignored, the key difference is this module only allows one subset of ignored configuration and requires it to be ignored on all environments. Thus the ideal use case seems to be for configuration that would be changed by editors and should not be reverted upon each deploy.

"I have a set of configuration files outside of the primary configuration folder that I want to use to continuously override the active configuration."

Suppose you have a multi-site code base that shares a configuration directory, but you need to override subsets of that on a per site basis. Or suppose you have a bunch of configuration that should be altered if a given module is enabled, but otherwise it should use the configuration from the main config directory.

Config Override allows you to pull this off. To use it, you place configuration YML within a module or in a designated folder and the values will override the active configuration. This continuously overrides the active configuration, unlike previous approaches.

Unless there is a large amount of configuration to override, using the standard configuration override system seems like a better approach. Although, there is something to be said for keeping the configuration in YML files instead of having it in PHP files.

"I want my current active configuration to continue to be stored in the database, and I also want it to be continuously exported to code and committed to the version control system."

The previous scenario allowed active config to not be overridden by ignoring designated subsets of configuration during import and export. In this scenario, we take the opposite approach and instead seek to always keep the config files and the active config in the same state by updating the source config files.

Configuration Tools does this by exporting the current active configuration to YML files as changes are made and can optionally commit the changes to a repository as well. This not only keeps everything in code at all times which in beneficial by itself, but it also allows developers to keep their local environments using the latest configuration from production without a database dump.

"I don't want anyone editing configuration that is stored in code since it will be overridden by the import process during the next deployment."

Another option for handling the file state and active state synchronization issue is to simply not allow the active state to deviate from the values set in the configuration files.

Configuration Read-only mode does this by completely disallowing editing of values stored in configuration.

This approach represents the opposite end of the spectrum from the approach taken by the Config Tools module.

"I have a folder of configuration specific to each environment that I want put into the active configuration using YML files."

This scenario involves the need to denote a folder that will contain all of the configuration that is specific to a given environment. There are a couple options here that solve the problem in alternate ways depending on your preferences.

Environment Config looks for a folder based upon the path set in the DRUPAL_CONFIG_DIR environment variable, where YML files with environment specific values can be placed. This limits you to one environment per PHP instance as explained on the module's Drupal page, "Currently only one file per environment is possible - won't work if eg dev and stage are on the same machine as they'll be sharing the DRUPAL_CONFIG environment variable."

The config values this module creates/changes will be included during the next config export, which will reflect the current environment's values. Thus putting the values for the current environment into the shared configuration files, which may not be desired. The module page calls out this potential issue and addresses it with the following:

IF YOU EXPORT YOUR ACTIVE CONFIGURATION (eg by running drush cex) THEN YOUR ENVIRONMENT-SPECIFIC CONFIGURATION WILL BE EXPORTED. This will not usually be a problem provided you have the same set of environment-specific configurations on each environment you deploy to (because the exported setting will be overridden by their environment-specific counterparts).

If that doesn’t work for you, we came up with a custom solution which imports environment specific configuration during configuration import that would be performed during each deployment. It has the same issues with exporting the overridden configuration as the Environment Config module, however no additional contrib modules are needed for this approach. So it is potentially a simpler approach depending on your appetite for custom code.

This does requires that the settings.php file be aware of the current environment, so ensure that you are setting some variable that you can use to switch the active configuration.

In settings.php create a new entry in the $config_directories array that we will call environment, then set the path to the folder containing the config files based upon the current active environment.

// Set the environment configuration directory based upon our custom environment variable.
switch ($settings['environment']) {
case 'LOCAL':
$config_directories['environment'] = '../config/local';

$config_directories['environment'] = '../config/prod';

Finally, ensure this additional directory gets included with the deploy commands.

drush config-import environment --partial

Note that one way around the exporting of overridden environment specific config values issue that is noted above, is excluding the config files included in the site specific config folders from the primary configuration folder. Then importing both the environment specific configuration folder and the full configuration using the --partial flag is as follows:

drush cim sync --partial
drush cim environment --partial

Unfortunately, two partial imports do not equal a full import. Two partial imports will never delete configuration, thus encountering the partial deletion issue that Drush CMI Tools solves with the --delete-list flag for their custom Drush commands.

"I have a subset of configuration that I need to export to a tarball."

Config partial export simply exports selected configuration to a tarball. If you are using git to manage your configuration the use cases for this seem limited, but it might be just what is needed for some special circumstance.

Reducing Complexity, Increasing Repeatability

No matter what your scenario, formalizing the configuration import and export process is crucial to avoiding configuration syncing issues and making your workflows repeatable and stable. As you explore the options above, you will find that there are alternate Drush commands, additional arguments and flags to existing Drush commands, and quite a few modules that can all come together to suite your specific needs.

How you solve your scenario is up to you, but the important thing is being able to run the exact same series of commands. Whether you are just changing your local branch, or a performing a full production deployment, consistency is fundamental to the success of your configuration management workflow. To aid in this effort, we recommend not only putting your deployment commands into a version controlled file, but also creating custom Drush commands that automate the configuration export process and the configuration import/deployment commands when working locally to ensure uniformity.

Simple yet still Complex

Finally, none of this is rocket science, but keeping all of the edge cases, variants, and environment differences is not easy. CM in Drupal 8 is miles ahead of our previous Features-based workflows and it is only going to continue improving. In the interim, these tools allow us to address some of the challenges of successfully managing large scale web platforms with complex configuration requirements.

As you probably noticed, we didn’t advocate for one approach over another, as it really depends on each site’s specific needs. With all the options out there, we would love to hear how you have implemented this on your site, so hit us up on Twitter @chromatichq and let’s continue the conversation!