Mar 15 2019
Mar 15

Migrations are fraught with unexpected discoveries and issues. Fighting memory issues with particularly long or processing heavy migrations should not be another obstacle to overcome, however, that is often not the case.

For instance, I recently ran into this no matter how high I raised the PHP memory_limit or lowered the --limit flag value.

$ drush migrate:import example_migration --limit=1000

Fatal error: Allowed memory size of 4244635648 bytes exhausted (tried to allocate 45056 bytes) in web/core/lib/Drupal/Core/Database/Statement.php on line 59

Fatal error: Allowed memory size of 4244635648 bytes exhausted (tried to allocate 65536 bytes) in vendor/symfony/http-kernel/Controller/ArgumentResolver/DefaultValueResolver.php on line 23

It should be noted that the --limit flag, while extremely useful, does not reduce the number of rows loaded into memory. It simply limits the number of destination records created. The source data query has no LIMIT statement, and the processRow(Row $row) method in the source plugin class is still called for every row.

Batch Size

This is where query batch size functionality comes in. This functionality is located in \Drupal\migrate\Plugin\migrate\source\SqlBase and allows for the source data query to be performed in batches, effectively using SQL LIMIT statements.

This can be controlled in the source plugin class with via the batchSize property.

 * {@inheritdoc}
protected $batchSize = 1000;

Alternatively, it can be set in the migration yml file with the batch_size property under the source definition.

  batch_size: 10
  plugin: example_migration_source
  key: example_source_db

There are very few references that I could find in existing online documentation. I eventually discovered it via a passing reference in a Drupal.org issue queue discussion.

Once I knew what I was looking for, I went searching for how this worked and discovered several other valuable options in the migration SqlBase class.


 * Sources whose data may be fetched via a database connection.
 * Available configuration keys:
 * - database_state_key: (optional) Name of the state key which contains an
 *   array with database connection information.
 * - key: (optional) The database key name. Defaults to 'migrate'.
 * - target: (optional) The database target name. Defaults to 'default'.
 * - batch_size: (optional) Number of records to fetch from the database during
 *   each batch. If omitted, all records are fetched in a single query.
 * - ignore_map: (optional) Source data is joined to the map table by default to
 *   improve migration performance. If set to TRUE, the map table will not be
 *   joined. Using expressions in the query may result in column aliases in the
 *   JOIN clause which would be invalid SQL. If you run into this, set
 *   ignore_map to TRUE.
 * For other optional configuration keys inherited from the parent class, refer
 * to \Drupal\migrate\Plugin\migrate\source\SourcePluginBase.
 * …
abstract class SqlBase extends SourcePluginBase implements ContainerFactoryPluginInterface, RequirementsInterface {

Migration Limit

Despite the “flaws” of the --limit flag, it still offers us a valuable tool in our effort to mitigate migration memory issues and increase migration speed. My anecdotal evidence from timing responses from the --feedback flag shows a much high migration throughput for the initial items, and a gradually tapering speed as a migration progresses.

I also encountered an issue where the migration memory reclamation process eventually failed and the migration ground to a halt. I was not alone in this issue, MediaCurrent found and documented this issue in their post Memory Management with Migrations in Drupal 8.

Memory usage is 2.57 GB (85% of limit 3.02 GB), reclaiming memory. [warning] Memory usage is now 2.57 GB (85% of limit 3.02 GB), not enough reclaimed, starting new batch [warning] Processed 1007 items (1007 created, 0 updated, 0 failed, 0 ignored) - done with 'nodes_articles' The migration would then cease to continue importing items as if it had finished, while there were still several hundred thousand nodes left to import. Running the import again would produce the same result.

I adapted the approach MediaCurrent showed in their post to work with Drush 9. It solved the memory issue, improved migration throughput, and provided a standardized way to trigger migrations upon deployment or during testing.

The crux of the solution is to repeatedly call drush migrate:import in a loop with a low --limit value to keep the average item processing time lower.

Our updated version of the script is available in a gist.

So the next time you are tasked with an overwhelming migration challenge, you no longer need to worry about memory issues. Now you can stick to focusing on tracking down the source data, processing and mapping it, and all of the other challenges migrations tend to surface.

Mar 14 2019
Mar 14

When loading or interacting with entities in Drupal 8, we often use the EntityTypeManagerInterface interface, which is the brains behind the entity_type.manager service that is provided in many of the Drupal core base classes.

This often appears in one of the following ways:



Either approach returns an instance of EntityStorageInterface. Each entity type can define a class that extends EntityStorageBase and adds additional custom methods that are applicable to a given entity type.

The node entity type uses this pattern in \Drupal\node\NodeStorage to provide many of its commonly used methods such as revisionIds() and userRevisionIds().

The benefits of adding custom storage methods becomes more apparent when you begin to work with custom entities. For example, if you have a recipe entity type, you could have a loadAllChocolateRecipes() method that abstracts the query and conditions needed to load a subset of Recipe entities.

The resulting call would look like this:

/* @var $recipes \Drupal\recipe_module\Entity\Recipe[] */
$recipes = $this->entityTypeManager

A custom storage handler class is integrated with an entity via the annotated comments in the entity class.


 * Define the Recipe entity.
 * @ContentEntityType(
 *   id = "recipe",
 *   label = @Translation("Recipe"),
 *   handlers = {
 *     "storage" = "Drupal\recipe_module\RecipeStorage",

Then in the storage handler class, custom methods can be added and existing methods can be overridden as needed.

 * Defines the storage handler class for Recipe entities.
class RecipeStorage extends SqlContentEntityStorage {

   * Load all recipes that include chocolate.
   * @return \Drupal\example\Entity\Recipe[]
   * .  An array of recipe entities.
  public function loadAllChocolateRecipes() {
    return $this->loadByProperties([
      'field_main_ingredient' => 'chocolate',

Manual SQL queries can also be performed using the already provided database connection in $this->database. Explore the Drupal\Core\Entity\Sql\SqlContentEntityStorage class to see the many properties and methods that you can override or leverage in your own methods.

Again, the NodeStorage and TermStorage offer many great examples and will demystify how many of the “magic” methods on these entities work behind the scenes.

For example, if you ever wondered how the Term::nodeCount() method works, this is where the magic happens.


 * {@inheritdoc}
public function nodeCount($vid) {
  $query = $this->database->select('taxonomy_index', 'ti');
  $query->addExpression('COUNT(DISTINCT ti.nid)');
  $query->leftJoin($this->getBaseTable(), 'td', 'ti.tid = td.tid');
  $query->condition('td.vid', $vid);
  return $query->execute()->fetchField();

The next time you need to write a method that returns data specific to an entity type, explore the use of a storage handler. It beats stuffing query logic into a custom Symfony service where you are likely violating single responsibility principles with an overly broad class.

This potentially removes your dependency on a custom service, removing the need for extra dependency injection and circular service dependencies. It also adheres to a Drupal core design pattern, so it is a win, win, win, or something like that.

Jan 31 2019
Jan 31

It is easy to see Twig code samples and just accept that label is some magic property of an entity now and use it in your template.

<h3>{{ node.label }}</h3>

Then you come across an example that calls referencedEntities, and it quickly becomes apparent some magic is going on.

{% for entity in node.example_reference_field.referencedEntities %}
  <li>{{ entity.label }}</li>
{% endfor %}

The secret is that in Drupal 8, methods are now exposed in Twig templates.

However, not all methods are exposed (including referencedEntities in the example above). The TwigSandboxPolicy class defines the whitelisted method names and prefixes that are allowed. It also allows for these settings to be overridden via settings.php if needed.

$settings['twig_sandbox_whitelisted_methods'] = [
$settings[twig_sandbox_whitelisted_prefixes] = [

Note that additional items cannot be appended to the array due to the logic in TwigSandboxPolicy. The existing defaults must be included in the custom override.

See Discovering and Inspecting Variables in Twig Templates for additional information and background.

Example Use Case

Let’s use an example to further explore how this could be used. Suppose we have a user entity available in our template and want to retrieve some additional data.

The first step is overriding the core user class so we can add custom methods.

 * Implements hook_entity_type_build().
function example_entity_type_build(array &$entity_types) {
  // Override the core user entity class.
  if (isset($entity_types['user'])) {

Next we can add custom methods that allow retrieval of data following some business rules.

namespace Drupal\example\Entity;

use Drupal\user\Entity\User;

 * Extends the core user class.
Class ExampleUser extends User {

   * Load custom data associated with the user.
  public function getExampleData() {
    // Return the entity returned by some business logic.
    return $entity;

With the user class override in place, we can pass a user to our template.

return [
  ‘#theme’ => ‘example_template’,
  ‘#user’ => \Drupal::currentUser(),

Then in our template, that has a user entity passed in with the variable name user, we can access our custom method.

{% set entity = user.exampleData %}
<h4>{{ entity.label }}</h4>
<p>{{ entity.example_field.value }}</p>

Note that we defined our method as getExampleData, but we called exampleData’. This is due to thegetprefix being whitelisted byTwigSandboxPolicy. Had we named our methodloadExampleDatait would not have worked. However, addingloadExampleData` to the whitelisted method names would fix it.

Sticking to methods that strictly retrieve data is best, and it honors the intentions set forth in TwigSandboxPolicy.

$whitelisted_methods = Settings::get('twig_sandbox_whitelisted_methods', [
  // Only allow idempotent methods.

The ability to call and expose methods on entities, fields, etc allows for simple theme hook definitions, while still allowing the template easy access to nearly any data it needs without bloated preprocess hooks. The real power begins when you chain these methods and entity references fields together to drill into your data model all from your template.

It should be noted that this can begin to blur the lines in the MVC pattern. The view layer can quickly begin taking over logic previously owned by the controller. It’s up to you how far to take it, but either way, it is another powerful tool that Drupal 8 and Twig offer to make development easier.

Oct 12 2018
Oct 12

When I began using Drupal, I was a site builder. I could find a combination of contrib modules to do just about anything without writing a line of code. I made Views that did the seemingly impossible with crazy combinations of filters and field formatters. I built Contexts that placed my blocks exactly where I wanted them, no matter how complex the requirements.

This all changed when I began writing modules and became a developer.

You have a problem? I’ll have a custom hook, a class override, and an event listener created shortly. Custom code to the rescue.

However, owning custom code, which is what having custom code in your repo really is, is much like owning anything else that is custom. It offers the incredible freedom of building exactly what you need, but it comes at a cost.

Take a car for example. A custom car can be built exactly to your specifications with no compromises. However when it comes time for maintenance, there will be no readily available parts. No one else is trained in repairs of your car, so the local auto shop can’t help when it breaks down. Diagnosing and fixing the car is all on you. All of a sudden that generic car that doesn’t meet all your needs, but will suffice with a few modifications isn’t looking so bad.

Enough with the car analogy. Let’s examine the costs specifically related to owning custom code.

  • Upfront cost to write, revise, and review the code.
  • Regression tests must be written to cover the custom functionality.
  • New developers are not familiar with it and take longer to ramp up.
  • Documentation needs to be written, updated, and maintained.
  • Security issues must be monitored and fixed quickly.
  • Code must be updated when the site is migrated to a new platform.
  • Compatibility must be maintained with other libraries as they are updated.

Despite these costs, custom code may still be the right answer depending on your needs. When evaluating the best approach, asking the following questions will often help.

  • Are the downsides of an existing solution that doesn’t exactly meet your needs less substantial than the costs of a custom solution?
  • Can functionality from an existing library or framework be extended or overridden to provide a small targeted change that limits the scope of custom code needed?
  • Would a contribution to an existing module/library solve the issue and allow future development to be done with assistance by a open source community?

If after a review of all of the options, a custom solution is truly needed, adhering to the following will help minimize the custom costs.

  • Strictly adhere to community code standards to make collaboration easier.
  • When extending functionality, only override the methods needed to make your change. Leverage the parent class as much as possible.
  • Architect code to allow for loose couplings, and leverage loose couplings when integrating with others’ code.

None of this is to say that custom code is wrong. It is often needed based upon a project’s requirements. This is simply an appeal to stop and analyze the needs at hand, instead of defaulting to a custom solution that may provide a quick fix, but come with many hidden long-term costs.

So perhaps I was on to something during my site building days by avoiding code. As a developer’s goal is not to write more code, it is to solve problems, build out functionality, and if necessary—write code. Ultimately it’s about delivering value, whether that is using existing tools, writing custom code, or a happy medium somewhere in between.

Aug 22 2018
Aug 22

A pull request (PR) is like a product looking for a buyer. The buyer, or pull request reviewer in this analogy, is trying to make an informed decision before they purchase (approve) the product. Thus as a PR creator, you are a salesperson and understanding the steps of the sales process will help you sell your product and allow buyers to make an educated decision on a product with features they may not be familiar with.

The Sales Process


Identifying customers and their need is often a prerequisite for a code change. The ticket or user story often already define the need, so the customer comes to you. So far, this sales process is easy!


Before writing code, it is important to research the codebase, evaluate existing solutions on the market, i.e. code that already does what you need. Understand the context of your code changes and the need behind them, similar to understanding a product or market before you go on a sales call.


As the solution is implemented, be constantly thinking about the needs of the buyer, or the pull request reviewer in this case. Code comments should be left with intentionality, always with the perspective of how can I best help the reviewer of this PR understand that my solution is correct and make them buy that my changes are needed. Remember that you may also be selling this same code to yourself months later when you are trying to figure out why it is there and how it works.


Get your slide deck ready! The code changes are complete and it’s time to sell everyone on the changes you made. Others are now relying upon you to give them the information and context they need to make an informed purchase. Use the preparation, work, and knowledge from the previous steps to document the benefits of your code and why someone should buy it.

Some concrete examples of this are:

  • Determine whether this should really be multiple smaller PRs.
  • Perform an initial self-evaluation of your PR and try to find edge cases and flaws.
  • Find the obvious mistakes/major oversights or development code left behind.
  • Explain how this PR changes this because of that.
  • Leave comments on your own PR explaining why one approach was chosen over another.
  • Evaluate if those PR comments really should be code comments. More often than not, the answer is yes.
  • Remember that changes without context are very difficult to review.
  • Include screenshots of your code in action.
  • Provide links where reviewers can demo your code and see it in action.
This PR comment should be moved to the code.

This PR comment should be moved to the code.

Handling objections

Often there will still be questions on the features your product provides or how it responds for a given use case. Sometimes you might be trying to sell the wrong product all together. Either way, treat questions and feedback as opportunities to improve your product or the presentation process to make future sales go more smoothly.

The buyer has an important role to play in the process too. The buyer’s feedback will help future sales processes go more quickly and improve the quality of future products. For those who often find themselves reviewing pull requests, we have a lot of thoughts on improving this practice as well.


With feedback in hand, it’s now time to close the sale. Don’t rush through this process though, be sure to ask questions and understand why changes were requested, always with an eye on making future sales easier. Then address any concerns, refactor or fix your product, explain the changes, and make a final push to get the PR approved and merged, in other words: sold!


Hit the like button on some of the best feedback and send some thank you emojis on Slack. Note the areas of expertise of each reviewer so you can ask yourself what they would like to see in a future product and prepare to do it all over again, as the next ticket is surely not far behind!


This “sales” approach is not about process for the sake of process, nor is it intended to be a rigid workflow. The value of selling your code is in the benefits it brings to the pull request workflow within your development process. Some of these benefits are:

  • Better initial code through self reflection and review.
  • Decreased time spent by others reviewing pull requests.
  • Fewer bugs and regressions that slip through approval process as reviewers are better informed.
  • Increased code quality as reviewers are able to focus on higher level code architecture due to increased understanding.

What other benefits do you see to this process? Do you have a favorite PR selling trick we missed? Most importantly, did we sell you on the process?

May 08 2018
May 08

With the advent of the media initiative, Drupal now has a clear best practice way of handling images, audio, and other forms of media. Many sites (including ours) were previously using simple file fields to store images. To update our content model a migration was in order. However a traditional migration using the Migrate API seemed like overkill for the scope of our content. This is likely the case for many other small sites, so let’s walk through our solution step by step.

Create Media Entities

The first step is to create the new media entities. For this example we will be using a media bundle named image. A file field named field_media_image should be added to this new entity.

Create Media Reference Fields

Next we need to create entity reference fields to replace our existing file fields on all of our content types. Leave the file fields in place for now, as we will use them as the source of our migration.

Migrate Image Files

With our limited content, we were able to do this process in an update hook without the aid of the batch process. This same process could be easily adapted for batch integration if needed though.

 * Create media entities from existing file fields.
function example_update_8401() {
  // Load all of the article and page nodes.
  $nodes = \Drupal::entityTypeManager()
    ->loadByProperties(['type' => ['article', 'page']]);
  foreach ($nodes as $node) {
    // Verify that the file field has a value.
    if (!empty($node->field_file_image->entity)) {
      // Create the new media entity and assign it to the new field.
      $node->field_media_image->entity = example_create_media_image_entity(
        sprintf('Updated image for node "%s".', $node->getTitle())

To aid in the creation of our media image entity, we created a helper function to encapsulate its creation.

use Drupal\media_entity\Entity\Media;

 * Creates a media image entity from a file entity.
 * @param \Drupal\file\FileInterface $file
 *   The existing file object.
 * @param string $alt
 *   The image alt text.
 * @return \Drupal\media_entity\Entity\Media
 *   The media entity.
function example_create_media_image_entity(FileInterface $file, $alt = NULL) {
  $media_entity = Media::create([
    'bundle' => 'image',
    'uid' => '1',
    'name' => $file->alt,
    'status' => Media::PUBLISHED,
    'field_media_image' => [
      'target_id' => $file->id(),
      'alt' => $alt,
  return $media_entity;


With the new field in place and the data successfully migrated, the file field can now be deleted from our nodes. This can be done via the admin UI or with another update hook depending on your site’s workflow.

Shiny New Toy

With the new media entity content model, additional data can be easily associated with each image by simply adding fields to the media entity. Now functionality such as caption text, copyright information, or alternate crop data can be handled with ease!

Mar 07 2018
Mar 07

“Decoupled Drupal” sounds cool and just about everyone else seems to be either doing it or talking about it, so it must be the best solution for you, right? Well, maybe. As with most things, the answer is more nuanced than one might think. In this post, we’ll explore some of the questions to ask along the way.

What is “decoupled Drupal”?

In a conventional web project, Drupal is used to manage content and to present it. In this sense, it is monolithic because your entire project sort of “lives” inside Drupal.

In contrast, decoupled Drupal — also known as “headless Drupal” — is a strategy by which Drupal is used strictly as a content management system without a presentation layer. That is, there are no public-facing themes and templates, there is only content and the administrative UI. In this strategy, Drupal exposes an API (application programing interface) for other applications to consume. This opens the door for a broad gamut of languages, frameworks, and platforms to ingest and present your content.

What Are the Benefits Of Decoupling?

This is, perhaps, the most important question. Decoupled architectures alleviate many of the problems that often accompany monolithic content management systems. Some of these are:

  • Simplified upgrade paths. A CMS upgrade doesn’t have to affect the presentation layer, only the CMS.
  • Easier troubleshooting. Bugs are easier to locate when your platform is composed of smaller, discrete parts instead of one giant application.
  • Easier to harden against regressions. Back-end developers can introduce changes that break front-end functionality more easily in a monolithic application. In a headless strategy, the API can be tested easily, helping you catch potentially breaking changes earlier in the process.
  • Focused tooling; granular deployments. Developers need only concern themselves with the setup and tooling specific to their part of the stack. Back-end devs don’t need to compile CSS; front-end devs don’t have to worry about composer installs. Likewise, deployments in decoupled projects can be much smaller and more nimble.
  • Clearer separation of scope and concerns. Since the CMS is just an API, it is easy to establish clear roles based on your content strategy. In contrast, a monolithic CMS can make it difficult to draw a line between syndication of content and serving a website.
  • Improved performance. While a monolithic Drupal site can perform very well, the need to frequently bootstrap Drupal for user requests can slow things down. Depending on your project’s needs and your caching strategy, these requests might be inevitable. A headless architecture allows you to build a front-end application that handles these requests more efficiently, resulting in better performance.
  • Divide and conquer development. Teams are able to develop in a decoupled manner. Development of a new feature can happen simultaneously for both the front-end and back-end.

By now you’ve probably noticed that headless Drupal has many advantages. If you’re ready to get on the decoupled bandwagon, hold your horses! As great as it sounds, this strategy introduces paradigm shifts on several levels. We’ll walk you through some high-level considerations first, and then go into some implementation challenges that we’ve faced on our own decoupled projects.

What To Consider Before Decoupling

Before you get into the nitty gritty details, you must look at all the factors that can influence this decision. Here are the high-level concerns you should consider before making your decision:

Your Team

The first thing to consider when examining whether Decoupled Drupal is right for your project is your team. Having your project split into multiple codebases means that your developers will naturally coalesce around their part of the tech stack. These subteams will need to communicate effectively to agree on roles and negotiate request/response interaction contracts.

Also consider developer skill levels: Are your most experienced problem-solvers also very strong Drupal developers? If so, you might need to hire a subject matter expert to help strengthen your front-end application team.

Project managers and operations staff will also find themselves supporting two or more teams, each with their own resourcing and infrastructure needs.

Decoupled is great if…

  • Team members’ experience levels and expertise is evenly distributed, or…
  • You are able to hire subject matter experts to offer guidance where your team needs it.
  • Project managers can handle multiple projects with separate teams.
  • Your DevOps team can handle the added complexity of satisfying the infrastructure needs of different applications.

Interacting With Your Data

When decoupling Drupal from the presentation layer, you’ll need to decide whether Drupal should be the owner and sole arbiter of all your content. This is a no-brainer if all of your content is in Drupal, but when other services complement your data, should your front-end application talk directly to those other services, or should it go through the Drupal API? There is no one correct answer to this question.

For example, you might have comment functionality for your project provided by a third-party service. From a performance perspective, it makes sense for your front-end application to make a separate and parallel request to the comments API while it requests data from your Drupal API. However, if multiple client applications need access to the comments, it might make sense to fetch this content from Drupal so as to reduce the number of implementations of this functionality.

Decoupled is great if…

  • You have a strong grasp on your data needs.
  • You establish clear roles for which applications/services are responsible for each piece of data.

Hosting Architecture

Separating the content from the display has serious implications. Evaluating if your current hosting provider can even support this architecture is the first step. If a change is needed, the feasibility of this needs to be evaluated. Are there existing long term contracts, security restrictions, data migration restrictions, etc. that may impede this process? With the separated hosting, a hidden benefit is the ability to put the backend application behind a VPN or firewall to further prevent unauthorized access.

Decoupled is great if…

  • You have a hosting provider capable of hosting a presumably non-Drupal front-end.
  • The hosting provider is capable of adding caching and other tooling between the various parts of your new architecture.
  • You want the added security of restricting editorial access to users within an internal network.

Serving Content to Multiple Clients

Decoupling is often a means to enable creating content once and displaying it on multiple sites. This is an inherent benefit, but this benefit adds complexity. For example, is there a need to allow editorial control for a custom color scheme per client site? Will content display the same title, published date, promo text, etc. on all sites and will content use the same URL alias on all sites? If it does, how will duplicate content affect SEO, and how will canonical meta tag values be set across sites?

Decoupled is great if…

  • Content needs to be created and managed in a single application and distributed to multiple platforms.
  • The content can remain unencumbered by per site design and metadata requirements.
  • You have an editorial process that does not expect a high degree of design/layout control.
  • You are prepared to handle the complexity of managing canonical alias values and other metadata to meet SEO needs.

URL Aliases

The URL alias is the backbone of a website. In a decoupled world, the alias could be considered a front-end only concern, but that is a simplistic approach that ignores several complicating factors. For instance, is there a unique identifier in the alias or can we find the content associated with the request without a unique identifier? Additionally, where are alias-to-entity mappings stored and managed? If editors have traditionally held control over aliases, or entity field values were used to generate an alias, pushing this logic out of the back-end might not be feasible.

Decoupled is great if…

  • The URL alias values have a unique identifier that makes API requests easy.
  • Editors are are not expecting control over the alias from the authoring platform.
  • Content attributes are not used to build the alias


A web page is more than the content on the page. In a modern web application, there is a plethora of metadata used to construct each page. This can include ad targeting values, meta tag values, and other microdata such as JSON-LD. Again, these may be considered web issues, i.e a front-end concern, but if these values are to be consistent or if content is used on multiple platforms, they suddenly become a shared issue, i.e. a back-end concern.

Decoupled is great if…

  • Content will need to share ad targeting data logic and other metadata across multiple consuming platforms.
  • The metadata logic that powers meta tags, JSON-LD, analytics, etc. can be generated with standardized rules.

Menu Items

The ease of menu management in a monolithic site is often taken for granted. The complexity of managing menus in a decoupled architecture is evident when you answer questions such as Where are menus created, ordered, and managed? How are menus shared across multiple sites? Will the menu paths work across sites?

Decoupled is great if…

  • The site has a static menu that can be created and managed in the front-end application.
  • A single menu can be shared across multiple clients without the need for per site menus.


Redirect logic is another element of the routing architecture that is complicated by going decoupled. This is especially true on sites with multiple or complex redirect needs that must be handled in a single redirect, or where editors expect a high level of control over the redirects.

Decoupled is great if…

  • There is no need for redirect control by the editorial staff, with all redirects being managed by broad server-level rules.
  • You have an architecture that supports combining multiple redirect rules into a single redirect.

Is Decoupled Right for Me?

A decoupled architecture offers a wide array of benefits, but some requirements handled easily by a monolithic infrastructure are suddenly born again by decoupling. Your business leadership must understand the implications of a decoupled architecture and the site’s architecture should mirror the business priorities. Finally, your team should possess the necessary technical leadership, management and communication skills, and web architecture expertise to tackle decoupling.

None of these challenges are insurmountable, they simply require a thorough analysis of your business needs and an understanding of the tradeoffs incurred with the available solutions. Depending on your specific needs, many of the benefits can outweigh the cost of increased complexity.

So should you decouple?

As you can tell, it’s complicated, but we hope this post helps you determine if it’s right for you. We love helping organizations navigate these questions, so please do reach out to us if you need a closer look at your particular situation.

Sep 18 2017
Sep 18

Multiple people frequently collaborate to author content in Drupal. By default, Drupal only allows a single author. This makes crediting multiple users in a byline impossible with core functionality.

The author data is stored in the uid column of the node_field_data table, which stores an INT value. Considering that this functionality is deeply integrated in Drupal core we’ll have to use a different approach.

To solve this, we simply create a new field that allows for multiple User entity references. We now have the ability to easily associate one or many users with a piece of content.

With our new “Authors” field in place, new content will use our new user reference field, however we will need to update existing content to use this field to provide consistent data. Alternatively, we could create logic to check both fields values when constructing the author data to display.

We opted for updating existing content to maintain data parity. A simple update hook in my_module.install does the trick:

 * Set value for new author field.
function my_module_update_8001() {
  $nodes = \Drupal::entityTypeManager()
    ->loadByProperties(['type' => 'article']);
  foreach ($nodes as $node) {
    \Drupal::logger('my_module')->notice(sprintf('Updated node %s', $node->getTitle()));

The final step is updating content to use our new field. The steps required here will vary widely depending on your templating and site building approach. However the basics come down to:

  • Verify that the existing Author field on the needed content types is hidden.
  • Verify that the new field displays the correct data from the referenced User entities.

With these simple steps, your content is now ready for storing and displaying the names of every user who helped with the authoring process. Now the only thing left to do is sort out fights over who gets to be listed first!

Apr 20 2017
Apr 20

Many Drupal 7 sites relied upon hook_boot and hook_init for critical functionality that needed to be executed on every page, even pages cached by Drupal. Oftentimes these hooks were incorrectly used and an alternate approach would have been much more performant by triggering logic exactly where it was needed instead of on every page. For example, drupal_add_js could be used to add JavaScript for a particular block or theme hook globally, but using a targeted preprocess function is often the correct approach. However, in some cases, these hooks were indeed the correct solution. Both hook_boot and hook_init were deprecated in Drupal 8, so an alternate approach will be needed.

Cacheable Logic

In Drupal 7, hook_init offered a hook that was fired on every page that Drupal did not cache. Drupal 8 offers this same functionality, using the Event Subscriber pattern as detailed on the hook_init change notice. These pages provide detailed examples, which walk you through the process of setting one up. How to Register an Event Subscriber in Drupal 8 also provides examples of event dispatching code.

Uncacheable Logic

Uncacheable logic should seldom be needed, as the solution is often to use proper cache settings on your render arrays. In the rare case that it is indeed needed, there are two viable options depending on the situation.

Using settings.php

The change notice page states:

A module that needs to run on cached pages should prompt its users to add code in settings.php

Note that Services are not instantiated yet when this code runs, thus severely limiting the available functionality. So while this approach is by no means ideal, it is available. The StackMiddleware approach below is a better approach that offers deeper integration with Drupal functionality.

Using StackMiddleware

This comment on the hook_boot change notice page provides an example of using StackMiddleware. It provides 95% of the functionality needed to run logic on cached pages by utilizing a tagged service with the http_middleware tag. Since the new class is a service, it will have full access to other core and contrib services, allowing for much greater functionality. The example shows the following for a module’s *.services.yml file:

    class: Drupal\mymodule\StackMiddleware\MyModule
      - { name: http_middleware, priority: 180, responder: true }

This is a pretty standard service definition, but note the items added to the tags property that register our service with the http_middleware tag and also set a priority. In order to bypass the page cache, understanding the page_cache.services.yml file is helpful. There, a similar definition can be found, but with a higher priority value.

    class: Drupal\page_cache\StackMiddleware\PageCache
    arguments: ['@cache.render', '@page_cache_request_policy', '@page_cache_response_policy']
      - { name: http_middleware, priority: 200, responder: true }

Higher priority services are run first. So to trigger logic before the page cache module takes over the request, a priority greater than 200 is needed.

    class: Drupal\mymodule\StackMiddleware\MyModule
      - { name: http_middleware, priority: 210, responder: true }

With this change in the services files, and proper setup of the service as described by the comment, the http_middleware.mymodule service should now be called on every page load, even on fully cached pages.

namespace Drupal\example\StackMiddleware;

use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpKernel\HttpKernelInterface;

 * Performs a custom task.
class ExampleStackMiddleware implements HttpKernelInterface {

   * The wrapped HTTP kernel.
   * @var \Symfony\Component\HttpKernel\HttpKernelInterface
  protected $httpKernel;

   * Creates a HTTP middleware handler.
   * @param \Symfony\Component\HttpKernel\HttpKernelInterface $kernel
   *   The HTTP kernel.
  public function __construct(HttpKernelInterface $kernel) {
    $this->httpKernel = $kernel;

   * {@inheritdoc}
  public function handle(Request $request, $type = self::MASTER_REQUEST, $catch = TRUE) {
    // Custom logic goes here.

    return $this->httpKernel->handle($request, $type, $catch);

Verifying the Results

A quick and easy way to test all of this is to simply add \Drupal::logger('test')->notice(‘not cached’); into the functions triggered by each of the approaches above. Ensure that the Drupal cache is enabled, and simply refresh a page while watching your log (drush ws --tail). Then verify the logic is being called as expected.

Mar 01 2017
Mar 01

What is a Code Review?

A good code review is like a good home inspection. It ensures that the original architectural drawings were followed, verifies that the individual subsystems are built and integrated correctly, and certifies that the final product works and adheres to standards. In the world of software development, a “code inspection” often takes the form of a pull request. A pull request, by its nature, simply shows line by line information for all of the updated or added files. This allows developers, who didn't write the code, to perform a code review and approve the changes before they are merged into the codebase.

Why Code Reviews are Important

Code reviews are not just a quality assurance (QA) process with a different name. They offer a wide variety of benefits that go well beyond the inherent advantages of a QA process.

Training / Mentorship

Deadlines often mean that time is not set aside for mentoring developers, or when it is, finding scenarios to illustrate various ideas are hard to find. Code reviews offer a great way to organically integrate training and mentorship into everyday workflows. It allows developers to learn from each other and leverage the years of experience from everyone involved. Senior developers are able to impart background knowledge of past decisions and provide guidance on a variety of best practices.


Knowing that another developer will be looking over your changes line by line forces you to think more deeply about your code as you write it. Instead of thinking “what is the fastest way to accomplish this task,” you are forced to think about the right way to build functionality. Your end goal is code that not only fulfills the scope of the ticket, but that is easy for others to understand and review.

Sharing the Load / Burning Down the Silos

Software development often requires specialization. When everyone’s work is visible in a pull request, it offers others the chance to see how unfamiliar problems are solved. This allows everyone to ask questions that help others, encourage additional learning, and diversify developers’ knowledge. Also, when things do break, more members of the team will have know-how to quickly track down the bug.

Code Quality

Peer reviews help ensure that any code that is committed to the codebase is not only functional, but meets quality standards. Typically, this means that it:

  1. Works.
  2. Has good documentation.
  3. Meets code style guidelines and best practices.
  4. Is performant.
  5. Doesn't add unnecessary lines (uses existing APIs).
  6. Properly abstracts code where appropriate (adheres to D.R.Y. principles).
  7. Is future-proofed as well as possible.


Senior developers may know about needs for future planned functionality or the needs of another internal team and can help steer current architecture and code changes to support those efforts. Some additional work now may pay huge dividends in the future by simplifying the addition of new features and maintaining existing functionality.

Reducing Risk

A peer review requires a pull request that is able to be comprehended by another developer. By introducing a peer review workflow, pull requests often decrease in size to aid the review process. This reduces risk, as often this relies upon smaller more discreet change sets instead of monolithic changes in a single pull request. Additionally, by having developers with varying areas of expertise, potential security risks are reduced by catching issues during the peer review process instead of after they’ve made it out to production.

How to Do a Code Review

Creating a Pull Request

For a peer review process to reach its full potential, it requires a skillful reviewer, but also a well-prepared pull request. A pull request might receive many reviews, but the first review it should receive is from its author. When preparing your own pull request for review by others, there are many things that can be done. These include:

  • Provide context for the changes by including a summary.
  • Document outstanding questions/blockers if further progress is pending feedback.
  • If necessary, clear up any potential areas of confusion ahead of time with preemptive comments.
  • Realize that if confusion could exist, perhaps the code or the code comments need improvement.
  • Attach screenshots of the functionality in action to provide additional context.
  • Ensure reviewers are aware of the state of your pull request and are notified when their review is needed.

An ideal PR

Reviewing a Pull Request

Providing feedback on a pull request is a lot like coaching, it requires building trust, adjusting to the situation and the player, and treating everything as a teachable moment. Let’s walk through some of Chromatic’s tried and true techniques.

  • Cite requests with links to posts, code standards guidelines, etc. when pointing out lesser known things or to build rapport with a developer not aware of your expertise.
  • Ask leading questions that help other developers see things from your perspective instead of just saying "change this."
  • Keep it fun and where possible, introduce some self-deprecating humor. Bring up your own past mistakes as evidence for why an alternate approach should be considered.
  • Acknowledge when you are nitpicking to make feedback more palatable.
  • Avoid pointing out problems without providing solutions.
  • Denote a few instances of a repeated mistake or enough to establish a pattern, then request that they all be fixed.
  • Ask questions. Doing so allows you to learn something and the author is forced to think critically about their decisions.
  • Explain the why of your requested change, not just the what.
  • Avoid directing negativity at the developer, instead focus on how the code could be improved.
  • Enforce the use of a code sniffer to prevent wasting time nitpicking code style.
  • Make note of the good parts of the code code and celebrate when you or others learn new techniques through the process.
  • Avoid bikeshedding on trivial issues.
  • Realize when optimization is no longer useful and avoid over engineering solutions.
  • Remember that the goal isn’t to find fault, it is to improve the codebase and teach others along the way.
  • Written communication may not come off in the same tone as it was intended, so error on the side of friendliness.
  • Treat others how you want to be treated. This is a simple rule, but such an important one. Consider how you would feel receiving this review, and be sure to write your review accordingly.

Establishing a Workflow

A pull request won’t merge itself, request feedback, or implement the feedback it was given. Thus, having clear expectations for who creates, updates, reviews, and merges a pull request is crucial. For example, after a pull request has been created, is it the responsibility of the developer to solicit feedback until it is approved and merge it themselves, or does a development manager take ownership of every pull request and merge it when they deem it ready? A good workflow should have just enough rules to work, without overly burdening developers and will likely vary per project or per team. The correct workflow is the one that works for you, the important part is that you have one, and that it includes a code review.

In “Review”

Adding a code review step to your development workflow might slow things down at first. However, in short order it will invariably show itself to be a worthwhile investment. Code reviews are an invaluable tool for mentoring, improving accountability, sharing knowledge and enhancing overall code quality. Now that you have reviewed this post, if you approve it, then merge it into your team’s workflow.

The bad puns are done, but if you want to continue the discussion on code reviews, reach out to us on Twitter @chromatichq.

Feb 10 2017
Feb 10

Drupal has an excellent field system with unique field types for storing just about every kind of data. When entering data that requires selecting an option from a predefined set of values with an optional default value the List field is often used.

The List field type provides a UI for entering allowed values and choosing a default value. This is great when the option set is small and the default value is always the same. However, often the editorial workflow calls for options that vary per content type and a default that is not consistent.

Despite the UI giving no indication of this functionality existing, the List field supports the setting of dynamic allowed values and dynamic default values.

static allowed values in UI

For our scenario, let’s suppose that there is an Alignment field (field_alignment), that had “Left” and “Right” as options. However, the Article content type was added, and it needed a “Center” alignment option, while the other content types needed to only have the original options. Additionally existing content defaults to “Right” alignment, but the Article content type should use “Center” as the default value.

With that in mind, let’s walk through how to do this in Drupal 8 using the allowed_values_function and default_value_callback field configuration values.

Allowed Values

First we will alter our allowed values to add “Center” as an alignment option for Article content.

In the field instance definition configuration file, set the allowed_values_function key. Note that the allowed_values entries can be removed once an allowed_values_function is in place.


type: list_string
  allowed_values: {  }
  allowed_values_function: 'example_allowed_values_function'
module: options
locked: false
cardinality: 1

Now setup the allowed values function and place it anywhere in the global namespace (e.g. a .module file):

use Drupal\Core\Entity\ContentEntityInterface;
use Drupal\field\Entity\FieldStorageConfig;

 * Set dynamic allowed values for the alignment field.
 * @param \Drupal\field\Entity\FieldStorageConfig $definition
 *   The field definition.
 * @param \Drupal\Core\Entity\ContentEntityInterface|null $entity
 *   The entity being created if applicable.
 * @param bool $cacheable
 *   Boolean indicating if the results are cacheable.
 * @return array
 *   An array of possible key and value options.
 * @see options_allowed_values()
function example_allowed_values_function(FieldStorageConfig $definition, ContentEntityInterface $entity = NULL, $cacheable) {
  $options = [
    'left' => 'Left',
    'right' => 'Right',
  // Add a custom alignment option for Article nodes.
  if ($entity->bundle() == 'article') {
    $options['center'] = 'Center';

  return $options;

Once the function is in place, the UI will reflect the change and disable manual editing of the allowed options.

dynamic allowed values in UI

Default Values

Now we will leverage our newly added “Center” alignment option and set it as the default value for this field when authoring Article content.

In the field base definition configuration file, set the default_value_callback.


translatable: false
default_value: {  }
default_value_callback: 'example_default_value_function'
settings: {  }
field_type: list_string

Next, setup the default value function anywhere in the global namespace as follows:

use Drupal\Core\Entity\ContentEntityInterface;
use Drupal\Core\Field\FieldDefinitionInterface;

 * Sets the default value for the alignment field.
 * @param \Drupal\Core\Entity\ContentEntityInterface $entity
 *   The entity being created.
 * @param \Drupal\Core\Field\FieldDefinitionInterface $definition
 *   The field definition.
 * @return array
 *   An array of default value keys with each entry keyed with the “value” key.
 * @see \Drupal\Core\Field\FieldConfigBase::getDefaultValue()
function example_default_value_function(ContentEntityInterface $entity, FieldDefinitionInterface $definition) {
  $default = 'right';
  // Article nodes should default to center alignment.
  if ($entity->bundle() == 'article') {
    $default = 'center';

  return [
    ['value' => $default],

With all of this in place, re-import the configuration and test it out. The authoring interface should now reflect the alternate allowed values per content type as well as the custom default value. So the next time the List field doesn’t quite meet your needs, give it a second chance, it might just be the right field for the job!

Jan 17 2017
Jan 17

Configuration management (CM) in Drupal 8 is great. Especially when every environment is configured with the same values and there is no need for values that vary by environment or values that editors can override. However, when more complex configuration scenarios arise, we must find solutions to problems that core CM functionality doesn’t address (yet). Let’s walk through some of the scenarios you might encounter.

"I need a different set of modules enabled/disabled per environment along with their configuration." or "I need different pieces of configuration enabled/disabled depending on the current environment."

For example, you may want to keep Field UI, Views UI, and other interface modules disabled on production, while ensuring they are enabled on other environments. Conversely, you may need development modules to be disabled on production along with keeping their configuration out of the database, but maintain their configuration across development sites. Or perhaps you need to disable production focused modules, like caching modules, on development environments.

Configuration Split allows you to do all of this by creating separate chunks or “splits” of configuration that you can then enable/disable per environment. Custom configuration import and export Drush commands are provided with this module. These will replace the standard config-import and config-export in your workflow for now, but integration with the standard Drush commands seems to be in the works.

Check out the following for additional information and in-depth tutorials:

Update: An issue has been filed for getting this type of functionality integrated with Drupal core.

Update: This issue explains why custom drush commands are no longer needed thanks to the filters provided by Config Filter.

"I have a set of configuration files in a module that I want to be made active when the module is installed, but allow editors to make subsequent changes to the configuration values."

For example, you may need to deploy some updated entity type descriptions or field descriptions, but after they are in place, an editor should be able to update the values and not have the values reverted after the next config import is performed. To pull this off there are a couple options:

Configuration Rewrite allows configuration YML files to be placed in a module and those values will override the active configuration upon the module's installation. The module's project page says it best:

Place any configuration .yml in your module's config/rewrite folder and declare config_rewrite a dependency to your module. Full and partial configuration changes are saved on installation.

Drush CMI tools could also help in this scenario. It provides a Drush command (drush cimy) with a flag that allows a specified configuration folder to be imported for installation only. This same configuration is then added to a special config-ignore.yml file that instructs the subsequent imports to ignore the given configuration.

$ drush cimy --source=/path/to/config-export --install=/path/to/config-install

"I have some configuration that I never want exported or imported."

Suppose there is a subset of configuration that is frequently altered by editors, so attempting to keep the configuration files up to date with the latest values would be nearly impossible. (There is a module that makes this possible, which we will address shortly).

Drush CMI tools, as mentioned in the previous scenario, solves this by providing a set of Drush commands that allow a defined subset of configuration to be ignored during the import and export process. The drush cimy process is very similar to the drush config-import --partial import process, but solves for the configuration deletion problem that --partial introduces. The Partial Imports section of the Drush CMI tools GitHub page explains this issue in much greater detail.

This process very is similar to Configuration Split in that it allows selected configuration to be ignored, the key difference is this module only allows one subset of ignored configuration and requires it to be ignored on all environments. Thus the ideal use case seems to be for configuration that would be changed by editors and should not be reverted upon each deploy.

"I have a set of configuration files outside of the primary configuration folder that I want to use to continuously override the active configuration."

Suppose you have a multi-site code base that shares a configuration directory, but you need to override subsets of that on a per site basis. Or suppose you have a bunch of configuration that should be altered if a given module is enabled, but otherwise it should use the configuration from the main config directory.

Config Override allows you to pull this off. To use it, you place configuration YML within a module or in a designated folder and the values will override the active configuration. This continuously overrides the active configuration, unlike previous approaches.

Unless there is a large amount of configuration to override, using the standard configuration override system seems like a better approach. Although, there is something to be said for keeping the configuration in YML files instead of having it in PHP files.

"I want my current active configuration to continue to be stored in the database, and I also want it to be continuously exported to code and committed to the version control system."

The previous scenario allowed active config to not be overridden by ignoring designated subsets of configuration during import and export. In this scenario, we take the opposite approach and instead seek to always keep the config files and the active config in the same state by updating the source config files.

Configuration Tools does this by exporting the current active configuration to YML files as changes are made and can optionally commit the changes to a repository as well. This not only keeps everything in code at all times which in beneficial by itself, but it also allows developers to keep their local environments using the latest configuration from production without a database dump.

"I don't want anyone editing configuration that is stored in code since it will be overridden by the import process during the next deployment."

Another option for handling the file state and active state synchronization issue is to simply not allow the active state to deviate from the values set in the configuration files.

Configuration Read-only mode does this by completely disallowing editing of values stored in configuration.

This approach represents the opposite end of the spectrum from the approach taken by the Config Tools module.

"I have a folder of configuration specific to each environment that I want put into the active configuration using YML files."

This scenario involves the need to denote a folder that will contain all of the configuration that is specific to a given environment. There are a couple options here that solve the problem in alternate ways depending on your preferences.

Environment Config looks for a folder based upon the path set in the DRUPAL_CONFIG_DIR environment variable, where YML files with environment specific values can be placed. This limits you to one environment per PHP instance as explained on the module's Drupal page, "Currently only one file per environment is possible - won't work if eg dev and stage are on the same machine as they'll be sharing the DRUPAL_CONFIG environment variable."

The config values this module creates/changes will be included during the next config export, which will reflect the current environment's values. Thus putting the values for the current environment into the shared configuration files, which may not be desired. The module page calls out this potential issue and addresses it with the following:

IF YOU EXPORT YOUR ACTIVE CONFIGURATION (eg by running drush cex) THEN YOUR ENVIRONMENT-SPECIFIC CONFIGURATION WILL BE EXPORTED. This will not usually be a problem provided you have the same set of environment-specific configurations on each environment you deploy to (because the exported setting will be overridden by their environment-specific counterparts).

If that doesn’t work for you, we came up with a custom solution which imports environment specific configuration during configuration import that would be performed during each deployment. It has the same issues with exporting the overridden configuration as the Environment Config module, however no additional contrib modules are needed for this approach. So it is potentially a simpler approach depending on your appetite for custom code.

This does requires that the settings.php file be aware of the current environment, so ensure that you are setting some variable that you can use to switch the active configuration.

In settings.php create a new entry in the $config_directories array that we will call environment, then set the path to the folder containing the config files based upon the current active environment.

// Set the environment configuration directory based upon our custom environment variable.
switch ($settings['environment']) {
  case 'LOCAL':
  $config_directories['environment'] = '../config/local';

  case 'PRODUCTION':
  $config_directories['environment'] = '../config/prod';

Finally, ensure this additional directory gets included with the deploy commands.

$ drush config-import environment --partial

Note that one way around the exporting of overridden environment specific config values issue that is noted above, is excluding the config files included in the site specific config folders from the primary configuration folder. Then importing both the environment specific configuration folder and the full configuration using the --partial flag is as follows:

$ drush cim sync --partial
$ drush cim environment --partial

Unfortunately, two partial imports do not equal a full import. Two partial imports will never delete configuration, thus encountering the partial deletion issue that Drush CMI Tools solves with the --delete-list flag for their custom Drush commands.

"I have a subset of configuration that I need to export to a tarball."

Config partial export simply exports selected configuration to a tarball. If you are using git to manage your configuration the use cases for this seem limited, but it might be just what is needed for some special circumstance.

Reducing Complexity, Increasing Repeatability

No matter what your scenario, formalizing the configuration import and export process is crucial to avoiding configuration syncing issues and making your workflows repeatable and stable. As you explore the options above, you will find that there are alternate Drush commands, additional arguments and flags to existing Drush commands, and quite a few modules that can all come together to suite your specific needs.

How you solve your scenario is up to you, but the important thing is being able to run the exact same series of commands. Whether you are just changing your local branch, or a performing a full production deployment, consistency is fundamental to the success of your configuration management workflow. To aid in this effort, we recommend not only putting your deployment commands into a version controlled file, but also creating custom Drush commands that automate the configuration export process and the configuration import/deployment commands when working locally to ensure uniformity.

Simple yet still Complex

Finally, none of this is rocket science, but keeping all of the edge cases, variants, and environment differences is not easy. CM in Drupal 8 is miles ahead of our previous Features-based workflows and it is only going to continue improving. In the interim, these tools allow us to address some of the challenges of successfully managing large scale web platforms with complex configuration requirements.

As you probably noticed, we didn’t advocate for one approach over another, as it really depends on each site’s specific needs. With all the options out there, we would love to hear how you have implemented this on your site, so hit us up on Twitter @chromatichq and let’s continue the conversation!

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web