Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Jul 27 2021
Jul 27

This article assumes a basic knowledge of the building of custom modules, the Drupal 8 / 9 Migration system, and the processes behind creating customised migrations from a previous version of Drupal.

One of the more common components of any migration from a previous version of Drupal is the need to migrate files. In Drupal 7 there was a core ‘File’ entity type and on pretty much all of our clients' sites we would also have the contributed module File Entity enabled. This extended the core file functionality and gave the ability to add fields to the file entity, have separate file types, integrate with views and more.

As of Drupal 8.4.x and above, Drupal Core has the concept of a ‘Media’ entity type that allows you to upload, manage and reuse files and multimedia assets. This acts as a replacement for the standard core File entity upload field and once you have configured a Media entity reference field on your entity type, it allows you to reference previously uploaded files from a Media library, instead of having to upload a new file every time.

Internally, a Media entity references a file entity and this process happens automatically for uploads made through the site frontend. When you upload a file to a Media entity reference field that has been set to use the 'Media Library' field widget, Drupal will handle importing the file, creating the entry in the file_managed table, creating the Media entity and then referencing the newly created file in the file reference field on the media entity. You can then reference this newly created media entity in any other media entity reference fields that allow referencing the same media entity bundles as this field.

The Core Drupal 8 / 9 migration of files doesn’t automatically allow you to migrate from Files to Media entities. The core file migration plugin for Drupal 7 to 8 / 9 just provides a migration route for Files to Files, and any auto-generated entity to entity migrations (provided by the migrate_drupal core module) will only support a File reference field to a File reference field. However, we want the ability to migrate into Media entities, so read on to see how to accomplish this with a bit of custom code.

The Process

The process of getting Files migrated as Media entities is a two-step process. First, we need a migration to migrate the Drupal 7 Files into our Drupal 8 / 9 site as Files, and then we need one or more secondary migrations (one per file/media type) to create Media entities from those now-migrated Files.

Creating the custom module

First of all, we’ll need to create a new custom module (if you haven’t already got a custom migration module in place already!). You can name this anything you want - usually, we’d use a project name prefix followed by the name of the module but for this tutorial, we’ll just call the module ‘example_custom_migration’ for simplicity.

example_custom_migration.info.yml

name: Example Custom Migration
description: Includes a custom migration for files.
package: example

type: module
core_version_requirement: ^8.8.0 || ^9.0
dependencies:
  - migrate:migrate

This is a minimal module info.yml file in which we include a dependency on the core migrate module.

Writing the migration source plugin

Next up we’ll need to create our custom migration source plugin. This is the plugin that we will reference in our migration(s) later on. We are going to be extending the core File migration source plugin to give the ability to filter the source files we want to migrate by file type, which we’ll need later on in order to determine which files will be referenced by which Media entity type.

We’ll name the plugin class file FileByType.php and it will live inside of our custom module in the directory src/Plugin/migrate/source. Be sure that the directory structure matches that exactly as Drupal 8 uses the PSR-4 standard for PHP autoloading. For more information about the PSR-4 standard with regards to Drupal 8 / 9 development, read this article.

We’ll start writing the class by extending the core File migration process plugin.

src/Plugin/migrate/source/FileByType.php

We only need to override two methods provided by the File class, query() and fields().

Our query() method override will handle filtering the files by a configured file type that we can pass into the plugin. The fields() method override will include the names of any extra fields that we will be returning in addition to the fields from the base File class.

The query() function

First of all, let's write the query function. The start of the function will call the parent query() method which will build up the query to query the Drupal 7 file_managed table for all files that are not stored under the temporary:// scheme, also providing the ability to only return files for a specified scheme based on the scheme configuration parameter. We don’t need to modify anything about the innards of the parent method so we can just call it here to save having the same code duplicated.

/**
 * {@inheritdoc}
 */
public function query() {
  $query = parent::query();
}

Next, we’ll add our customisations to support filtering the files by a specific file type. Later on, this will be passed in as a configuration option from our migration.

// Filter by file type, if configured.
if (isset($this->configuration['type'])) {
  $query->condition('f.type', $this->configuration['type']);
}

As we are migrating into Media entities, we may want additional data such as image alt and title text values from the Drupal 7 database. So we’ll support (optionally) returning this data as well.

// Get the alt text, if configured.
if (isset($this->configuration['get_alt'])) {
  $alt_alias = $query->addJoin('left', 'field_data_field_file_image_alt_text', 'alt', 'f.fid = %alias.entity_id');
  $query->addField($alt_alias, 'field_file_image_alt_text_value', 'alt');
}

// Get the title text, if configured.
if (isset($this->configuration['get_title'])) {
  $title_alias = $query->addJoin('left', 'field_data_field_file_image_title_text', 'title', 'f.fid = %alias.entity_id');
  $query->addField($title_alias, 'field_file_image_title_text_value', 'title');
}

Later on, when we are writing the migration for ‘Image’ media items we will be including two additional configuration lines in the call to our source plugin, get_alt and get_title. When we write the migration for ‘Document’ media items, we don’t want alt text and title text as they aren’t applicable, so we will omit them from the configuration there.

The finished query() function will look like this:

/**
 * {@inheritdoc}
 */
public function query() {
  $query = parent::query();

  // Filter by file type, if configured.
  if (isset($this->configuration['type'])) {
    $query->condition('f.type', $this->configuration['type']);
  }

  // Get the alt text, if configured.
  if (isset($this->configuration['get_alt'])) {
    $alt_alias = $query->addJoin('left', 'field_data_field_file_image_alt_text', 'alt', 'f.fid = %alias.entity_id');
    $query->addField($alt_alias, 'field_file_image_alt_text_value', 'alt');
  }

  // Get the title text, if configured.
  if (isset($this->configuration['get_title'])) {
    $title_alias = $query->addJoin('left', 'field_data_field_file_image_title_text', 'title', 'f.fid = %alias.entity_id');
    $query->addField($title_alias, 'field_file_image_title_text_value', 'title');
  }
}

If you wanted to add any further logic in this function to return any other field data (your fieldable files may have extra field data that you need!) then you could do so here.

The fields() function

In the fields function, we want to return the additional fields that this source plugin now provides (file type, alt text, title text), in addition to the standard fields the File plugin already provides.

/**
 * {@inheritdoc}
 */
public function fields() {
  $fields = parent::fields();
  $fields['type'] = $this->t('The type of file.');
  $fields['alt'] = $this->t('Alt text of the file (if present)');
  $fields['title'] = $this->t('Title text of the file (if present)');
  return $fields;
}

We first call parent::fields() to get the list of fields from the parent method and then add our additional fields onto the $fields array afterwards. If you were to add further logic to the query() method which returned further additional fields, you would also want to add these here as well.

The finished source plugin

The finished source plugin should now look like this:

configuration['type'])) {
      $query->condition('f.type', $this->configuration['type']);
    }

    // Get the alt text, if configured.
    if (isset($this->configuration['get_alt'])) {
      $alt_alias = $query->addJoin('left', 'field_data_field_file_image_alt_text', 'alt', 'f.fid = %alias.entity_id');
      $query->addField($alt_alias, 'field_file_image_alt_text_value', 'alt');
    }

    // Get the title text, if configured.
    if (isset($this->configuration['get_title'])) {
      $title_alias = $query->addJoin('left', 'field_data_field_file_image_title_text', 'title', 'f.fid = %alias.entity_id');
      $query->addField($title_alias, 'field_file_image_title_text_value', 'title');
    }
  }

  /**
   * {@inheritdoc}
   */
  public function fields() {
    $fields = parent::fields();
    $fields['type'] = $this->t('The type of file.');
    $fields['alt'] = $this->t('Alt text of the file (if present)');
    $fields['title'] = $this->t('Title text of the file (if present)');
    return $fields;
  }

}

Writing the Migrations

As you will hopefully know already, since Drupal 8.1.x, migrations are now Plugins instead of Configuration, which means we can just add them into a migrations folder inside of our custom module. You no longer have to include the migrations as config in a config/install folder inside of your module or worry about deleting and re-importing said config to pick up any changes that were made. You simply need to do a cache rebuild and the changes will be picked up automatically.

The First migration - Drupal 7 Files to Drupal 8 / 9 Files

Create the migrations directory inside of your custom module which will contain this migration and the other ones later on. Then create the file that will contain our migration plugin and call it example_custom_migration.upgrade_d7_file.yml

At this point, the innards of our migration plugin definition will be pretty much the same as the core Drupal 7 migration file plugin. At this point, we are not using our custom migration source plugin that we wrote earlier. The standard d7_file plugin provided by core will suffice to initially migrate all of the files, regardless of their type.

id: upgrade_d7_file
class: Drupal\migrate\Plugin\Migration
migration_tags:
  - 'Drupal 7'
  - Content
migration_group: migrate_drupal_7
label: 'Public files'
source:
  plugin: d7_file
  scheme: public
  constants:
    source_base_path: /path/to/your/drupal7/webroot
process:
  fid: fid
  filename: filename
  source_full_path:
    -
      plugin: concat
      delimiter: /
      source:
        - constants/source_base_path
        - filepath
    -
      plugin: urlencode
  uri:
    plugin: file_copy
    source:
      - '@source_full_path'
      - uri
  filemime: filemime
  status: status
  created: timestamp
  changed: timestamp
  uid: uid
destination:
  plugin: 'entity:file'

As you can see, this is pretty standard stuff for a migration plugin and we aren’t doing anything overly complex yet. This plugin definition will ensure we grab all the files from the Drupal 7 site and migrate them as files onto our Drupal 8 / 9 site.

You may notice the ‘migration_group’ key which isn’t something you would find with core migration plugins. This is an optional key that the migrate_plus module allows you to define in your migration plugin that allows you to group your migrations together and share configuration between migrations. This enables you to then run the migration(s) as a group through both the Migrate UI and through the command-line (Drush). This is a feature we use quite a lot when building Drupal to Drupal migrations! You don’t have to have migrate_plus in order to run these migrations but the grouping stuff is very useful.

Next up, we need to write the migration(s) to turn these Files into Media entities.

The Secondary migration(s) - Drupal 8 / 9 Files to Media

For this bit I’m assuming that you already have created the various media entity types in the site UI for ‘image’ and ‘document’ media types. If you haven’t, be sure to go ahead and create them and if you decide to name them something different you’ll have to update the relevant places in the scripts.

Images

Now we have the file migration in place, we’ll begin by writing the migration for the Image media entities. Create a new file named example_custom_migration.upgrade_d7_file_to_media_image.yml inside of the migrations folder with the following contents.

id: upgrade_d7_file_to_media_image
class: Drupal\migrate\Plugin\Migration
migration_tags:
  - 'Drupal 7'
  - Content
migration_group: migrate_drupal_7
label: 'Migrate Media image entities'
source:
  plugin: d7_file_by_type
  scheme: public
  type: image
  get_alt: true
  get_title: true
  constants:
    source_base_path: /path/to/your/drupal7/webroot
process:
  field_media_image/target_id:
    -
      plugin: migration_lookup
      migration: upgrade_d7_file
      source: fid
    -
      plugin: skip_on_empty
      method: row
  thumbnail/target_id:
    plugin: migration_lookup
    migration: upgrade_d7_file
    source: fid
  field_media_image/alt: alt
  field_media_image/title: title
  status: status
  created: timestamp
  changed: timestamp
  uid: uid
destination:
  plugin: 'entity:media'
  default_bundle: image
migration_dependencies:
  required:
    - upgrade_d7_file

Let’s break this down section by section.

In the source section, you can see that this time we are now using our custom source plugin d7_file_by_type that we wrote earlier. We are also passing in a couple of additional options to the plugin; get_alt and get_title that will enable the plugin to return the additional alt text and title text data that we want for our image media entities.

In the process section we write to the appropriate fields on the media entity; field_media_image/target_id and thumbnail/target_id to save the references to our file, field_media_image/alt and field_media_image/title to save the alt and title values. We also save the standard entity data like our file migration before; status, created, changed, and uid.

The destination plugin we use this time is ‘entity:media’ because we are now creating a media entity, and the default_bundle is set to ‘image’ as that’s the media bundle type we are using for this migration.

Finally, we add a required migration_dependency on our upgrade_d7_file migration. This will ensure that the migration will only run once the file one has fully ran and imported all of the available items.

Documents

The document migration will be pretty similar to the image migration that we wrote above.

id: upgrade_d7_file_to_media_document
class: Drupal\migrate\Plugin\Migration
migration_tags:
  - 'Drupal 7'
  - Content
migration_group: migrate_drupal_7
label: 'Migrate Media file entities'
source:
  plugin: d7_file_by_type
  scheme: public
  type: document
  constants:
    source_base_path: /path/to/your/drupal7/webroot
process:
  field_media_file/target_id:
    -
      plugin: migration_lookup
      migration: upgrade_d7_file
      source: fid
    -
      plugin: skip_on_empty
      method: row
  field_media_file/display:
    plugin: default_value
    default_value: 1
  status: status
  created: timestamp
  changed: timestamp
  uid: uid
destination:
  plugin: 'entity:media'
  default_bundle: document
migration_dependencies:
  required:
    - upgrade_d7_file

The source section is the same except this time we pass in a type of document to our source plugin and as we don’t need the alt and title text data that we needed for the image media, we’ll omit the get_alt and get_title configuration lines.

The process section again is similar, except this time we are setting the referenced file inside of field_media_file/target_id and we also set the field_media_file/display to 1 which will enable the file to be displayed when viewing content. Again, we omit the get_alt and get_title bits and the rest of the configuration here are the same entity properties as before.

We keep the destination plugin the same (entity:media) but the default_bundle is changed to ‘document’.

Before running the migrations.

Before we are able to run our migrations, we need to make sure that your Drupal 8 / 9 site knows how to connect to your Drupal 7 site. If you are already mid-way through writing migrations from Drupal 7 to Drupal 8 / 9 then you will most likely already have this in place, but if not, you need to ensure your settings.php (or settings.local.php - wherever your current site database connection details are present!) has an entry with the connection details to the Drupal 7 site.

e.g. inside of settings.local.php

$databases['drupal_7']['default'] = [
  'database' => ‘your_database_name’,
  'username' => 'your_database_user_name',
  'password' => 'your_database_user_pass',
  'prefix' => '',
  'host' => 'localhost',
  'port' => '3306',
  'namespace' => 'Drupal\\Core\\Database\\Driver\\mysql',
  'driver' => 'mysql',
];

Note the connection key drupal_7 there, in the $databases definition. We also need to ensure that our migration group knows that we should be using this key if it doesn’t already.

If you have already got a migration_group configuration setup as part of a Drupal -> Drupal site migration (you most likely will if you have migrate_plus enabled and have begun work on a migration) then ensure in the shared_configuration section at the bottom that you have the following:

shared_configuration:
  source:
    key: drupal_7

If you don’t have a migration group config already, then create a new one named migrate_plus.migration_group.migrate_drupal_7.yml

langcode: en
status: true
dependencies: {  }
id: migrate_drupal_7
label: 'Import from Drupal 7'
description: ''
source_type: 'Drupal 7'
module: null
shared_configuration:
  source:
    key: drupal_7

Put it into your site’s configuration directory (e.g. config/sync) and import the config.

If for some reason you aren’t using migrate_plus with migration groups then you’ll need to ensure in the source section of each migration plugin we have written earlier that you have

key: drupal_7

in the plugin definition. e.g. (for the document plugin this would be as follows)

source:
  plugin: d7_file_by_type
  scheme: public
  type: document
  key: drupal_7
  constants:
    source_base_path: /path/to/your/drupal7/webroot

Running the migrations

Nearly there (I promise!). Now you have your migration plugins all correct and in place, it’s time to try them out! First, ensure your module is enabled if it isn’t already (either via the extend menu in the UI or via Drush - drush en example_custom_migration).

Next, you can either try running the migrations through the UI (admin/structure/migrate) or via Drush (if you are using the migrate_tools module version 4.x / 5.x and Drush 9.x / 10.3.x OR Drush 10.4.x+ which includes most of the Drush commands the migrate_tools module included..) I’m a Drush man myself, so I’d either run the migrations individually with these commands

  • drush migrate:import upgrade_d7_file
  • drush migrate:import upgrade_d7_file_to_media_image
  • drush migrate:import upgrade_d7_file_to_media_document

or as a group (if you are using migrate_plus with the group support) with

  • drush migrate:import --group=migrate_drupal_7

although if you are writing these migrations as part of a larger migration with other migrations in the group, you’ll probably just want to run them individually to be sure they are working correctly.

All being well, after running the migrations you should find all the relevant files have been copied into your new site’s public files directory and you should be able to see all your lovely files as media entities by visiting /admin/content/media. (Great Job!)

Referencing the media in migrations that previously referenced files

Any migrations that you have previously written that referenced files and fid’s instead of media entities should now be updated. This is a relatively trivial task as the following example details. Please note that the fields you are migrating into will need to be recreated as Entity Reference fields, referencing the Media entity type. If you have let Drupal handle migrating your fields from a previous version then it will have created these fields as File entity reference fields.

This is an example of what part of your process plugin may look like before and after, on an entity migration for an image field named ‘field_images’.

Before (File)

field_images:
  plugin: sub_process
  source: field_images
  process:
    target_id: fid
    alt: alt
    title: title
    width: width
    height: height

After (Media)

field_images:
  plugin: sub_process
  source: field_images
  process:
    target_id:
      plugin: migration_lookup
      migration: upgrade_d7_file_to_media_image
      source: fid
      no_stub: true

And here is an example of what part of your process plugin would look like before and after for a document field named ‘field_documents’.

Before (File)

field_my_documents:
  plugin: sub_process
  source: field_my_documents
  process:
    target_id: fid
    display: display

After (Media)

field_my_documents:
  plugin: sub_process
  source: field_my_documents
  process:
    target_id:
      plugin: migration_lookup
      migration: upgrade_d7_file_to_media_document
      source: fid
      no_stub: true

If your file field in Drupal 7 allows both images and documents, then your field in Drupal 8 / 9 should allow the same. You can reference both media migrations in the process as follows:

field_my_documents:
  plugin: sub_process
  source: field_my_documents
  process:
    target_id:
      plugin: migration_lookup
      migration:
        - upgrade_d7_file_to_media_document
        - upgrade_d7_file_to_media_image
      source: fid
      no_stub: true

Extending further

We have covered the basics on how to get images and document files migrated, but you can (and we have on lots of client’s sites) get this running for Video files as well in a similar manner with minimal changes.

You just need to create another file to media migration plugin for video files, in addition to the image and document ones already made. Ensure that the type of file passed into the source plugin section of the migration is ‘video’ and the default_bundle of your entity:media destination in the destination section is ‘video’. Then you should be hot to trot!

Bonus - How to have a dynamic file source_base_path

You may be wondering, “What if I don’t want to hard code my source_base_path in the source plugin definition of my migrations?”

This is another common requirement we have when writing our own migrations, and luckily the solution isn’t too tricky with a bit of custom code. You may find this useful if you are going to run this migration across different environments (e.g. your dev machine, testing server, production environment when the time comes) and you don’t want to keep having to change your migration plugin files each time.

In your migration source plugin for both the file and file -> media migrations, change

source_base_path: /path/to/your/drupal7/webroot

to

source_base_path: /

You don’t strictly have to do this as we are going to override it anyway but I think having a blank path gives a clearer indication that we haven’t explicitly set one, as we are going to override the value. Of course, you could also have a comment above the line explaining that this will be overridden if you so wish.

Migration Event Subscriber

In order to override the souce_base_path, we’ll create an Event Subscriber in our custom module. This will enable us to subscribe to the appropriate Migrate events and will allow us to modify the data at the correct point in time. For more detailed information about event subscribers, see this article we wrote way back in 2016 ‘Drupal 8 Event Subscribers - the successor to alter hooks’.

You’ll first want to create a services.yml file directly inside of the custom module as well named example_custom_migration.services.yml. This will let us define our migration subscriber so that Drupal knows about it.

services:
  example_custom_migration.d7_file_migration_subscriber:
    class: Drupal\example_custom_migration\D7FileMigrationSubscriber
    arguments: ['@config.factory']
    tags:
      - { name: event_subscriber }

Next, create this file inside of the src folder in our custom module. You can name it what you want really but for this example, we’ll call it D7FileMigrationSubscriber.php.

configFactory = $configFactory;
  }

  /**
   * {@inheritdoc}
   */
  public static function getSubscribedEvents() {
    $events[MigrateEvents::PRE_IMPORT] = 'onMigrationPreImport';
    return $events;
  }

  /**
   * Act on a migration before it imports.
   */
  public function onMigrationPreImport(MigrateImportEvent $event) {
    $migrate_id = $event->getMigration()->id();

    // Inject the correct source_base_path to the file migrations.
    $file_migrate_ids = [
      'upgrade_d7_file',
      'upgrade_d7_file_to_media_image',
      'upgrade_d7_file_to_media_document'
    ];

    if (in_array($migrate_id, $file_migrate_ids)) {
      $source_config = $event->getMigration()->getSourceConfiguration();
      $source_base_path = $this->configFactory->get('example_custom_migration.settings')->get('files_source_base_path');
      $source_config['constants']['source_base_path'] = $source_base_path;
      $event->getMigration()->set('source', $source_config);
    }
  }
}

There is a lot going on here which I'm not going to get into in this article. (The article I linked to above should help explain it all if you aren’t familiar with concepts such as event subscribers and dependency injection!) But the end result of this code will be to swap out the source_base_path in our source plugin configuration to one defined by a config entry named ‘example_custom_migration.settings’ with a key of ‘files_source_base_path’

Finally, all you need to do next is just pop this line into the bottom of your settings.local.php (which should be different per environment) replacing ‘/path/to/your/drupal7/webroot’ with the path to the Drupal 7 site webroot for the current environment.

$config['example_custom_migration.settings']['files_source_base_path'] = '/path/to/your/drupal7/webroot';

This will then allow you to have a different path set on a local dev build, another team member’s local dev build, testing and production servers as well! Neat huh?

Photo by Kaboompics.com on Pexels

Jul 20 2021
Jul 20

Sometimes it can be handy to have extra pages for a node (or any entity). For example:

  • To show different sets of information on separate pages for a single product, page, or thing.
  • So you can set different access requirements on each page for a node.
  • You want to block access to the ordinary route (e.g. node/123 and its aliased equivalent) for some reason, but you still want some other page to represent that node.

I've found a few people with that need on Drupal slack before, so I thought I'd write a guide because it's surprisingly easy!

My imaginary requirement

I'm going to imagine my client has asked me to set up an additional page for products, to show information relevant to someone that has already bought the product. We'll call it the 'Product support' page. This page could be public, but shown as a separate tab for products, or even just hidden as a link only sent out to customers that have ordered the specific product. Access could even be restricted to those people entirely. All the content to show on the page would come from fields on the product. We don't really want to go creating another separate entity to hold data that's only relevant to the product in question anyway. This makes sense from a data point of view as all the fields are for one thing, whether for existing or potential customers. (Of course, Field Group could be used to split those things into tabs on the product's edit form, or a similar technique to this article's could be used to separate them out into an additional edit page!)

First, set up a view mode for your node/entity

Just like teasers can be configured separately to the full content version of a node, you can create your own view mode. Add it from /admin/structure/display-modes/view on your site, and then enable it from the 'Custom display settings' section at the bottom of the 'Manage display' tab for your entity type/bundle. Once that's done, navigate to the sub-tab for your view mode and choose the fields you'll want to show.

So I'll make one called 'Product support', and configure only fields like 'Support helpline', 'Manuals' and 'Returns information' to show in this view mode. I will probably go hide those fields in the usual (default/full) mode too.

Next, set up the page to use that view mode

HTML page routes for viewing entities are usually set up by the 'route_provider' specified in an entity type's annotation block. For nodes, that's Drupal\node\Entity\NodeRouteProvider, which dynamically sets up the view, edit & delete routes (take a look!). But you can define your route with some simple YAML. In a custom module, add (or use an existing) mymodule.routing.yml file, with the following code. Replace 'mymodule' with your module's machine name, and 'product_support' with the machine name of your view mode:

mymodule.product_support:
  path: '/node/{node}/product-support'
  defaults:
    _entity_view: 'node.product_support'
    _title: 'Product support'
  requirements:
    node: '\d+'
    _entity_access: 'node.view'
    _custom_access: 'mymodule_product_support_access'

The last line contains the name of a function you might want to use to add any custom access logic. The function could just be in your .module file. In my case, I might look up the current user's orders to check whether they have bought the product before allowing access. You might want to at least restrict which bundles the route works for, like so:

/**
 * Extra access callback for product pages.
 */
function mymodule_product_support_access(\Drupal\Core\Entity\ContentEntityInterface $node) {
  return \Drupal\Core\Access\AccessResult::allowedIf($node->bundle() === 'product');
}

But you can just omit that _custom_access: line from the YAML entirely to just use the same access that the ordinary node page has, no problem.

Rebuild the site caches; you're done!

Now you can access /node/123/product-support to view a different page of content for your product! If you want to set it up as a tab where the usual 'View'/'Edit' tabs of a node would be, then use the following YAML in a mymodule.links.task.yml file in your module. Again, replace 'mymodule' and 'product_support' as appropriate, and then rebuild the site caches to see it work:

mymodule.product_support:
  route_name: mymodule.product_support
  title: 'Product support'
  base_route: entity.node.canonical
  weight: 5

All of this can work for any entity type - try just replacing 'node' in each of these code snippets with the machine name of the entity type that you want to use.

There's a module for that

There are modules like View Mode Page which let you set these extra pages up in the UI. But I've found my additional pages usually need some additional bespoke logic though, whether for access or something else on the page. So given how little actual code is needed, I tend to just make them this way. But it does have some handy features, like supporting URL aliases for nicer paths.

Entity Form Mode also claims to help you make separate edit pages for form modes, which is very handy. But again, I find these tend to have even more interesting bespoke requirements with custom access logic. But see how you get on. Making these in custom code can be just as easy anyway - just replace the _entity_view: 'node.product_support' part of the routing YAML with _entity_form: 'node.my_form_mode', and the _entity_access part should use 'node.update' instead of 'node.view'.

I'm sure there are some interesting use cases for this - let me know in the comments what you've needed custom entity routes for!

Photo by Thant Zin Oo on Unsplash

Jun 14 2021
Jun 14

I'm a fan of configuring things for display through Drupal's admin UI. It gives site builders confidence and power. What if you want to place blocks or views listings in amongst fields on pages of content? For example, to display:

  • A listing (view) of related content, such as accessories for a product
  • A standard contact block, advert, or some other calls to action in the middle of the content, exactly where the user is best 'caught' in their journey, rather than having to stick those in sidebars or after all the content fields.
  • Some specific value(s) pulled from fields on some indirectly related entity, through a token, such as details from a taxonomy term representing the 'section' that a page is in.
  • Consistent relevant links on user profiles to take people to common destinations

Drupal's usual blocks system allows you to put these in sidebars or above/below the usual node fields, but not between them. You could use a 'heavyweight' system like Layout Builder, Panels, or Display Suite, but those tend to entirely change the way you configure or edit your content. You could get a developer to override twig templates or write custom PHP. But is there a middle ground?

Well, of course! You might have noticed some modules already allow their page additions to be moved around amongst the usual fields on content. See these rows without widget or format settings in this following screenshot, which aren't for ordinary fields at all? Wouldn't it be great to be able to add your own?

Profile display settings with 'extra fields' highlighted

This is where the Entity Extra Field module (entity_extra_field) comes in. It supports embedding blocks, views or values to be replaced via tokens. So a site builder can set these up to be managed just like ordinary fields on the page (whether it's a node, term, paragraph, or any other type of content). Each one would act as a sort of 'pseudo-field', rendered as part of a display mode amongst the ordinary fields. It also works for form modes - so you can display useful content beside existing field widgets, perhaps displaying relevant related data to editors in the places that they would be entering content about that data.

Entity Extra Field supports visibility conditions (just like blocks, but for views & tokens too) and passing & selecting contexts for blocks. These give it quite a lot of power - for example, to conditionally hide a field rather than just using Drupal's ordinary display settings for it. So I believe this module does a better job than the older EVA module (for views), and my own similar EBA module (for blocks) did. In fact, I recommend that anyone using my EBA module in D7 should use Entity Extra Field in its place when moving to Drupal 9. Here are some screenshots of its interface - first for selecting a view to add to content:

Entity extra field administrative interface for selecting a view to add to an entity display.

And for choosing a block to display - in this case, a custom one that requires a context:

Entity extra field administrative interface for selecting a context-aware block to add to an entity display.

Each 'Extra field' gets shown on all entities of the type/bundle they are configured on. So there's no need to constantly remember to add a common block or view every time you create/edit a page. If you do want to have different ones on different pages, then you should use Views Reference or Block Field. These modules provide true fields for editors to choose which view/block to display on each individual page.

The code inside Entity Extra Field uses hook_entity_extra_field_info(), which acts just like its Drupal 7 predecessor, hook_field_extra_fields(), which I've written about before. So you could write code using that to add your own page additions too - but given that blocks, views, and content accessible via tokens are possibly the most common things to embed, that suddenly feels unnecessary. Even as a developer, I'm glad to avoid writing code that would need maintaining anyway.

I've been privileged to be able to contribute fixes & functionality to the Entity Extra Field project, resulting in a recent new release. My time for that was essentially sponsored by ComputerMinds and one of our clients who would use this site-building capability, especially around block contexts. So thanks to them! And of course a big thank you goes to Travis Tomka (droath) for making the module, and accepting my many issues & patches!

Photo by Y. Peyankov on Unsplash

Jun 09 2021
Jun 09

Defining your own Drupal block plugins in custom code is really powerful, but sometimes you can feel limited by what those blocks have access to. Your block class is like a blank canvas that you know you'll be pulling data into, but how should you get that data from the surrounding page? Often you have to resort to fetching the entity for the current page out of its route parameters (e.g. on a node page), in order to get the values out of its fields that you want to display. Plugins can actually have a context passed to them directly - which can be common things like the logged-in user, or the node for the page being viewed. Let's have a look at how to tell Drupal what your plugin needs, so you don't have to do the donkey work.

If you've created a block plugin, you'll already be aware of the annotation comment just above the class name at the top of your PHP file. It might look something like this:

Spot that last property in the annotation: the context_definitions part. That's where the block is defined as requiring a node context. The 'entity:node' part tells Drupal that the context should be a node; Drupal supports it just being 'entity' or various other things, such as 'language' or even 'string'. You can very easily get hold of the context for your block, e.g. in the build() method of your class, allowing your block to adapt to its surroundings:

/**
 * {@inheritdoc}
 */
public function build() {
  $entity = $this->getContextValue('node');

  return $entity->field_my_thing->view();
}

I've used a very simple tip from our article on rendering fields for the output of this method. But the key here is the use of $this->getContextValue('node');. Our block class is extending the BlockBase base class, which gives us that getContextValue() method to use (via ContextAwarePluginTrait, which you could use directly in your plugin class if you're not extending BlockBase). The 'node' parameter that we've passed to it should match the key of the context definition array up in the class annotation - it's just a key that you could rename to anything helpful. Plugins can specify multiple contexts, so distinguish each with appropriate names.

In this basic case of using a node, the chances are that you're just wanting to use the node that the current page is for. Drupal core has 'context provider' services - one of which provides exactly that. Most basic Drupal installations probably won't have other context providers that provide nodes, so the node just gets automatically passed through, without you having to do anything else to wire it up. Brilliantly, the block will only show up when on a node page, regardless of any other visibility settings in the block placement configuration. You can bypass that by flagging that the context is optional in its definition - spot the 'required' key:

context_definitions = {
  "node" = @ContextDefinition("entity:node", required = FALSE, label = @Translation("Node")),
}

A slightly more interesting example is for users, as Drupal core can potentially provide two possible contexts for them:

  1. The currently logged-in user, or at least the entity object representing the anonymous user if no-one is logged in.
  2. The user being viewed - which will only be available when visiting an actual user profile page.

When there are more than one possible contexts available, block placement configuration forms offer the choice of which to use. So you might want a block in a sidebar on profile pages to show things to do with the user who owns that profile - in which case, select the 'User being viewed' option in the dropdown. Otherwise the data your block shows will just be about you, the logged-in user, even when looking at someone else's profile. Internally, your selection in the dropdown gets stored as the context mapping, which you can see in the exported configuration files for any context-aware block (including those automatically selected due to only one context being available).

If all this talk of Contexts is reminding you of something, it's because the concept was brought into core after being used with Panels in older versions of Drupal. Core's Layout Builder now uses it heavily, usually to pass the entity being viewed into the blocks that represent fields etc that you place in each section. For anyone that really wants to know, those blocks are defined using a plugin deriver - i.e. a separate class that defines multiple possible block plugins, dynamically. In the case of Layout Builder, that means a block is dynamically created for every field an entity can have. If you use plugin derivers, you might need dynamically set up context definitions too. So in the deriver's getDerivativeDefinitions() method, you could have something like this, the PHP equivalent of the regular block's static annotation:

/**
 * {@inheritdoc}
 */
public function getDerivativeDefinitions($base_plugin_definition) {
  $derivative['context_definitions'] = [
    'node' => new ContextDefinition('entity:node', $this->t('Node')),
  ];

  $this->derivatives['your_id'] = $derivative;
  return $this->derivatives;
}

I've only lightly touched on context provider services, but you can of course create your own too. I recently used one to provide a related 'section' taxonomy term to blocks, which pulls from an entity reference field that nearly all entity types & bundles on a site had. The blocks display fields from the current page's parent section. It made for a common interface separating that 'fetching' code from the actual block 'display' code. I recommend understanding, copying & adapting the NodeRouteContext class (and accompanying service definition in node.services.yml) for your case if you have a similar need.

I hope this awareness of context allows your blocks to seamlessly adapt to their surroundings like good chameleons, and maybe even write better code along the way. I know I've had many blocks in the past that each had their own ways of pulling relevant data for a page. Contexts seem like the answer to me as they separate fetching and display data, so each part can be done well. Getting creative with making my own context types and context providers was fun too, though probably added unnecessary complication in the end. Let me know in the comments what context-aware plugins enable you to do!

Photo by Andrew Liu on Unsplash

Apr 01 2021
Apr 01

The last year has highlighted to us all how important it is for the global community to come together and solve problems. We rate ourselves highly at ComputerMinds, and figure it's time to share and stretch our abilities to the full. So I'm here on this special day to announce that we are branching out beyond just resolving bugs on websites, to fixing any kind of bugs in any problem space. There's so much market potential, we're really quite excited at the possibilities for bringing innovative solutions to the world!

Medical bugs

The ugliest bug of them all, COVID-19, has been such a terrible challenge for us all. We've been inspired by the countless heroes across the world who have stood up in the face of it, so we want to help too by putting our services to more significant tests than just Drupal websites. We have continued to serve our clients during the pandemic as well as we can so far, but now it's time for us to help with other kinds of bugs. We'll start with the common cold and flu, and work our way up to the bigger beasts. We believe in our approach and that our experience will propel us to find solutions. To help us with this, we'll team up with the best in the business with offices in Bristol and Coventry.

Pest control

Inspired by the wonderful pest control hawk that flies around our Coventry office, we will help fix your bug problem. Our Drupal experience has taught us to search for the root problems and to stop at nothing to go down debugging rabbit holes - so we are perfectly suited to this industry too. Unwanted animals and insects, beware! But we've also got a strong ethical heartbeat too - we always want to do things the right way, after all. We'll continue to work with existing partners to campaign for sustainable pest control, and against unnecessary culling.

Lifestyle bugs

Life coaching is a blossoming market. We believe too many rush ahead to give advice about making lifestyle changes, before pausing to eliminate 'bugs' in people's lives that hold them back. Too many web projects need rescuing because of the issues that hold them back - and these are often much deeper than mere software issues, but go down to 'people' problems. We're ready to bring our experience from these situations to help people become the best versions of themselves. We recognise that in some scenarios, people need to give themselves more slack in their expectations, whilst others need to the right pressures applied to improve performance. Sometimes proper recovery from major trauma must be prioritised; for others we are well-placed to encourage physical exercise for all the benefits and widened perspective it brings; for others laziness is the 'bug' we will identify and help clients overcome!

Engineering

We've already been solving problems in electrical engineering and patching up domestic engineering horror stories. We've worked with some genuine motor engineering history beneath us. So now we're opening up to offer our services to the great people of Bristol and Coventry. Our cities are well known for engineering feats, so we are keen to partner with the local firms that are facing bugs in their work. Our contribution towards the Coventry Motofest event demonstrates our passion for great engineering - the traditional kind, not just software engineering.

Software problems - not just Drupal

Drupal will always be our specialism, but the knowledge gained over the years from those projects gives us plenty of wisdom for any software project. We can already offer consulting for commerce websites built on other platforms. You probably already knew that we have great experience in building websites with GatsbyJS, and native mobile apps in other technologies. We are ready for you with all sorts of advice that can be applied to nearly any web project, whether that be advice on analytics, A/B testing solutions or development methodologies. But we're not limiting ourselves to the internet either any more - bugs will always be found in all kinds of software!

So what are you waiting for? Find out how we can help fix your bugs!

Jan 12 2021
Jan 12

The vast majority of community-contributed Drupal 8 modules now have releases that are compatible with Drupal 9, but what can you do if you need to use a module that doesn’t? Well, you’re likely to find a compatibility patch in the project's issue queue. But the tool most of us use with composer to apply patches, cweagans/composer-patches, is a plugin that can only make its changes after composer reads a package's metadata about its compatibility (and dependencies). So for contrib modules that haven't yet committed their patches, attempting to apply the patch in the usual way doesn't help. For example, before commerce_migrate was compatible with Drupal 9 (update: it is now!), when I tried applying the necessary patch this way, composer just gave me an enormous and confusing error:

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Conclusion: don't install drupal/commerce_migrate 2.0.0-rc3
    - Conclusion: don't install drupal/commerce_migrate 2.0.0-rc2
    - Conclusion: remove drupal/core 9.0.9
    - Installation request for drupal/commerce_migrate ^[email protected] -> satisfiable by drupal/commerce_migrate[2.0.0-rc1, 2.0.0-rc2, 2.0.0-rc3].
    - Conclusion: don't install drupal/core 9.0.9
    - drupal/commerce_migrate 2.0.0-rc1 requires drupal/core ^8.7 -> satisfiable by drupal/core[8.7.0, 8.7.0-alpha1, 8.7.0-alpha2, 8.7.0-beta1, 8.7.0-beta2, 8.7.0-rc1, 8.7.1, 8.7.10, 8.7.11, 8.7.12, 8.7.13, 8.7.14, 8.7.2, 8.7.3, 8.7.4, 8.7.5, 8.7.6, 8.7.7, 8.7.8, 8.7.9, 8.8.0, 8.8.0-alpha1, 8.8.0-beta1, 8.8.0-rc1, 8.8.1, 8.8.10, 8.8.11, 8.8.12, 8.8.2, 8.8.3, 8.8.4, 8.8.5, 8.8.6, 8.8.7, 8.8.8, 8.8.9, 8.9.0, 8.9.0-beta1, 8.9.0-beta2, 8.9.0-beta3, 8.9.0-rc1, 8.9.1, 8.9.10, 8.9.11, 8.9.2, 8.9.3, 8.9.4, 8.9.5, 8.9.6, 8.9.7, 8.9.8, 8.9.9].
    - Can only install one of: drupal/core[8.7.0, 9.0.9].
    - Can only install one of: drupal/core[8.7.0-alpha1, 9.0.9].
...
    - Installation request for drupal/core (locked at 9.0.9) -> satisfiable by drupal/core[9.0.9].


Installation failed, reverting ./composer.json to its original content.

The error in situations like this goes on and on, including scary messages like 'Conclusion: remove drupal/core', and sometimes about all sorts of seemingly unrelated packages! Ultimately, composer is trying to bend over backwards to use versions of Drupal and its dependencies that would work with the module, but it won't find any. Or if it does, you might find yourself downgraded to an old version of Drupal, which you certainly don't want!

So what's the solution?

We need to override the compatibility metadata for the contrib module.

Drupal sites use https://packages.drupal.org/8 as the metadata provider, which you'll find listed in your project's root composer.json file under a repositories section. This section instructs composer where to get the metadata from ... so if you add your own 'package' repository to be used first, composer will happily use that, ignoring what drupal.org tells it:

"repositories": [
    {
      "type": "package",
      "package": {
        "name": "drupal/commerce_migrate",
        "type": "drupal-module",
        "version": "dev-8.x-2.x",
        "source": {
          "type": "git",
          "url": "https://git.drupalcode.org/project/commerce_migrate.git",
          "reference": "9ac26262b3443d20e69cea69652abc2dee39fee5"
        }
      }
    },
    {
      "type": "composer",
      "url": "https://packages.drupal.org/8"
    }
  ],

In this example, for the commerce_migrate module, we specify the latest commit (at the time of writing) from its development branch (8.x-2.x), as that's what the patch is intended to get ultimately applied to. That branch also matches our requirement in our composer.json file: "drupal/commerce_migrate": "dev-8.x-2.x". (Note that composer needs the dev- prefix when using git branches.)

So this section redefines the module's dev-8.x-2.x version for composer. As we require that version, we get the snapshot of the codebase at that specific commit - skipping the metadata that drupal.org would have added which would restrict its compatibility.

Now, the patch can be added in the usual patches section, and then running composer update drupal/commerce_migrate will apply it successfully!

This technique can also be used to override other package metadata for composer, such as a change in dependencies. But beware - it means you're no longer using the real releases for the package, and in this example, locks to a specific commit. So you will need to keep an eye out for when the patch gets committed to the module, and for any other work in its codebase that should be incorporated over time. It's a little like taking a snapshot of the project, and applying just the specific changes you need - at the cost of losing out on any goodness that the maintainers may add over time. This approach does avoid forking the codebase away at least. As this replaces the module’s metadata entirely, it’s worth checking for any dependencies or other compatibility metadata that its own (potentially now patched) composer.json file might have, and copying them to your project’s root composer.json file.

Given how quickly the community & its leaders have rallied to make modules D9-ready, I wouldn't be surprised if this trick can be removed in a few months' time, as maintainers continue to commit these (often trivial) patches. Any projects that don't, may even get covered automatically some day - plus, if their maintainers aren't doing this, they're also unlikely to be adding much of note in the meantime anyway, so it's pretty safe. I would just keep an eye out for any new releases - especially security ones - to those projects.

Get notified

I suggest finding the issue that currently tracks making the module compatible with Drupal 9. This may be linked to from the project page, or it's usually one of the more recent/active ones in its issue queue. If you're logged into drupal.org (and you should be - sign up if you haven't!), then you can press a 'Follow' link that has a nice green star next to it, to get email notifications of updates. The 'View all releases' link from a project page takes you to a page that has an RSS feed, which can be used to subscribe to notifications of new releases too. That's well worth doing for a number of reasons! If you didn't know it, you can also subscribe to notifications of security releases - head to https://www.drupal.org/security where you'll find instructions at the top of the sidebar (which is below the list of posts if you're on a mobile).

Thanks to heddn, dpi and larowlan for this solution. Photo by Josh Carter on Unsplash.

Nov 11 2020
Nov 11

TLDR : Check your cookie popup!

It seems everyone is talking about core web vitals at the moment, spurred on by Googles’ recent announcement that page experience (which includes core web vitals) will start influencing search ranking in May 2021.

We won’t go into detail on web vitals in this post - there is plenty of information already on the web, including from Google themselves on the excellent web.dev site.

Instead, this post will look specifically at cumulative layout shift - or CLS as it’s known. I like to think of this as the jerkiness measure of your site - the higher the number the worse the jerkiness.

Getting your CLS score is pretty easy - you can use pagespeed or lighthouse (built right into chrome). I will assume (because you are reading this post) that you have done this already AND you are not happy with your score. So - what next - how do you go about diagnosing and fixing the issue?

The reading online makes lots of great suggestions, mostly around helping the browser reserve space for elements that are late loaded (images, video etc). This is all good stuff - however - it didn’t help me and the sites I was working on.

So first off I switched to incognito mode (I wanted to simulate a fresh first-time visit to the site - for various reasons) in chrome, loaded the site and then fired up the performance tab in the chrome dev tools. This is a scary looking tab - there is a lot going on here - and you can safely ignore 98% of it (at least for now). You just need to make sure the ‘Screenshots’ tick box is ticked - then hit the little refresh icon - this will refresh the page and start profiling the page load. It’s worth pointing out that this is running locally so you can use it on your local dev sites.

This takes a few seconds and when complete you should have a lovely looking film strip of screenshots along the top - mouseover and you get a larger version. Start at the left and move rightwards - and keep your eyes open for any obvious content shifts.

I did this for 3 individual and unrelated client sites - and found the cookie popup to be the culprit in 2 of the sites - it was appearing mid-way through the load and pushing ALL the content down - fixing this took around .4 off the CLS score (which is a lot). The other site had the ENTIRE expanded mobile menu briefly rendering at the top of the page - which then disappeared (as it should) and shifted all the page content up by about 800px! Fixing this took about .5 off the CLS score.

So - the takeaway from this is probably before you spend ages fiddling with width and height attributes on your img tags, have a quick look to make sure there isn’t some low hanging fruit massively impacting your CLS.

Oct 27 2020
Oct 27

Over the last year or so, I've got quite engaged with Drupal slack. I've loitered in channels like #d9readiness and #config, discussed issues with members of the security team, and asked questions to module maintainers (and received answers!). But most of all, I've helped people out in the #support channel. This has been an interesting experience in many ways, so I thought I'd share my reflections. The Drupal slack workspace is intended for the community, so if you're reading this - it's probably for you too. Hopefully my thoughts might help prepare you to use it as an effective tool.

Ultimately, I've got stuck in as a way to contribute to the Drupal community in a new way as part of our CM contribution challenge. That has given me motivation to give more than I get, but I've been pleasantly surprised how much I - and ComputerMinds - have benefitted from being part of Drupal slack. When we've been unsure about the status of a module or the way forward with an issue, it's been great to be able to reach out to exactly the right people. Especially in the current climate that limits our ability to meet people in-person (e.g. at Drupalcon). So, thank you to those people that have helped me :-) We've even received sponsorship to work on some module issues after discussion on slack - we ended up becoming responsible for releasing security fixes for the Commerce Ingenico project after it had been shut down due to vulnerabilities. That's a win for us, and it's a win for the community, who can use that module once again.

Slack has benefits, but also drawbacks, as it is yet another system clamouring for attention in a crowded digital world. So it's important to use it appropriately for the good of the community, but also wisely for your own sanity. Drupal's challenging learning curve means that there are a lot of people out there wanting help, and the #support channel is full of them. (I've spotted some of the most experienced drupallers asking for help there too; so it's not just for newbies.)

Learning Curve for Popular CMSSource: Learning Curve for Popular CMS

Sustaining a good level of support is a challenge, especially doing so with a friendly manner. My approach has been to answer questions I know I can help with and avoid those that I know other people would be better placed to answer. Some questions can suggest a shaky foundation of understanding, so unless I'm sure I can help fix that, I tend to stay away from those too. Often a question is a symptom of a deeper issue - either with someone's website, or their understanding of Drupal components. On the whole, I believe I have helped a lot of people - and not just with their immediate questions. Some people come back to me with direct messages weeks or months later, knowing that I might be able to help.

Unfortunately my mental health has suffered a bit since the coronavirus pandemic began, so I've had to keep an eye out for things that may be contributing to that. Slack offers connection to people 24/7 - which can be good, but it's not a truly deep connection, and can be relentless. While my motivation for interacting with Drupal slack was to try and contribute to the community, that's not entirely for it's own sake. ComputerMinds want to help Drupal and its community flourish - but that's partly because we want that for our own benefit! So I decided to focus on supporting people that might be within closer reach, by skipping over most posts from outside UK business hours. (I'll happily reply to DMs/threads asynchronously though if necessary.)

In conclusion, I think Drupal slack is a really handy tool for connecting with the Drupal community. I'm very glad to help people through it, even if that means only making a difference slowly, to one person at a time. There are some incredibly helpful people on there who I see answering questions again and again. I tried to be one of those for a while, though I learnt to take care to make my offering sustainable. If you need help, try out Drupal slack. If you don't .... then you can probably be a help to someone there! Most of us are a mix of those extremes anyway. Try to be kind on slack - to yourself and others. I figure if we all help each other, we might make climbing that Drupal learning cliff a bit more personal, and a bit more pleasant. Win-win!

Photo by Paul Gilmore on Unsplash

Jul 28 2020
Jul 28

Many of us at ComputerMinds have always taken pride on doing Drupally things the right way whenever possible, and then helping the community do so too. One of these things is displaying values from fields on content entities. We wrote before about how to do this in Drupal 7 and Drupal 8. It's now the turn of Drupal 9! Thankfully, this updated version is basically the same as the last one, as D9 is very similar to D8 on the surface, but with old cruft ripped out to allow it to continue improving. So the short answer to "How can I show a field programmatically?" is still:

$entity->field_whatever->view();

Isn't that great? Your existing code for Drupal 8 already works with Drupal 9! That was the aim of the update; to make the upgrade incredibly easy. Whereas upgrading from Drupal 7 can be a mammoth task (you might want to get in touch with us to help!), the jump to D9 is much simpler.

So is anything different? Most changes are buried well within Drupal's innards. The most relevant difference for displaying fields in Drupal 8 as opposed to Drupal 9 is that if you were originally loading the entity object ($entity) using the entity.manager service (e.g. from \Drupal::entityManager()), you now need to use the entity_type.manager service (e.g. \Drupal::entityTypeManager()).

Our previous article on rendering fields in D8 contains much more detail, which is still totally valid for Drupal 9. That will help you tweak the formatter settings to view a field with, or how to get raw values out of the field. For example:

// Render an image field with a specific image style.
$entity->field_my_image->view([
  'type' => 'image',
  'label' => 'hidden',
  'settings' => array(
    'image_link' => 'content',
    'image_style' => 'square_icon',
  ),
]);

// Get the raw value out of a single-value link field.
$link = $entity->field_web_address->uri;

A comment on that article did point out that you can get fatal errors if you use this code too naïvely. That's because magic methods are used here, with the assumption that you are sure the field exists on the entity. If you don't, then just break the chaining down:

// The $field variable will just be null if the entity doesn't have this field.
if ($field = $entity->field_whatever) {
  $to_show = $field->view();
}

Alternatively, you can use get() methods instead of the magic methods. But if you do, you'll probably want to surround your code with Exception handling to catch InvalidArgumentException exceptions, as the magic method getters are more lenient in more scenarios.

Photo by Belle Hunt on Unsplash

Jul 14 2020
Jul 14

I recently released a new contributed module to aid translation on Drupal 7 sites: Entity Translation: Separated Shared Elements Form (ETSSEF). Yes, it has a convoluted name! It finally resolves a suggestion from years ago in an Entity Translation project issue, to allow editing untranslatable fields separately to translatable ones. One of our clients has a multilingual product database site with a few hundred fields on their content, so anything like this that could reduce the size of their editing forms is useful. I figure the best way to demonstrate this is with a recipe that blends it together with some other super (but generally obscure) modules. I hope you can spot parts that may be helpful for your projects!

Screenshot of the top of an edit translation form, including Shared tab and Add fields to form widget

The Recipe
 

Ingredients

Take a look at each of these project pages linked above for a flavour of what each module will bring to the mix.


Recipe Difficulty Rating: Intended for experienced Drupal cooks only; others may prefer to try our takeaway service.
 

Method
  1. Enable each of the modules listed above, and set the admin theme to be used.
     
  2. Configure the various Field storage modules. Try to understand what each of these is doing, and adjust appropriately for you if necessary:
    // Turn off storage of revision info for your content type that has many fields.
    $bundle = 'farmer';
    variable_set('field_sql_norevisions_entities', array(
      'node' => array(
        $bundle => 1,
      ),
    ));
    // Default to using Blob storage (1 table instead of 1000s).
    variable_set('field_storage_default', 'field_sql_blob_storage');
    // Relabel the options in the UI to make the distinction clear.
    variable_set('field_storage_ui_relabel_options', array(
      'field_sql_blob_storage' => 'Retrievable only',
      'field_sql_storage' => 'Sortable & Filterable',
    ));
    // Load default-sql-storage fields in batches of 20 (instead of 1 at a time).
    variable_set('field_sql_storage_group_load_max_fields', 20);
    
     
  3. Set up your content type with many many fields. Choose 'Retrievable only' for the storage type for any fields that don't really need to be used for querying against, or sorting/filtering in lists. This will improve performance, as all the field data for those is stored together in a 'blob' column in the database so can be loaded (& unserialized) in a single go, rather than requiring select queries from so many different individual database tables.
     
  4. Configure nodes of this type to use entity translation (field translation) and head to the entity translation settings at /admin/config/regional/entity-translation. Set their 'Shared elements on translation forms' setting to 'Only display on their own separate forms':
    Entity translation settings showing Shared elements configured to show on their own forms
    This ensures that untranslatable fields are just edited on the initial Edit tab (in a 'Shared' secondary tab); with just translatable fields in the translation forms. When there are so many fields, it's worth slimming down forms as much as possible! This also has the advantage that untranslatable data can be edited without needing to touch any specific translation.
     
  5. Override the edit form for your node type in a custom module, to use the Field Attach Form Selective module, so that fields are only shown on the form as they are filled in. This vastly reduces the amount of stuff on the form. I've written a gist that demonstrates this, and includes wiring it up to work nicely with ETSSEF. You must copy the entire contents of node_form() from node.pages.inc in Drupal core into the farmer_node_form() function, but replace the call to field_attach_form() at the bottom, with a call to field_attach_form_selective(). Use the same arguments.
     
  6. I then added some classes and CSS to the secondary tabs on the page to show the flag icons next to each language, as well as repeating the current tab name in the page title. I then added CSS to fix the page header in place so that editors easily retain that contextual information as they scroll down the giant forms. Otherwise it's too easy for them to forget which language they are editing!
Season to taste

Now when you edit your content type, your forms will be much slimmer and your site will run far smoother with hundreds of fields. As with any recipe, take the bits of this that are to your taste, ignore others, or blend it into your own creations! This was only for D7, so bringing the ideas over to Drupal 8/9 in some form would be an obvious thing to do. I’d love to hear of other ingredients you use to help when editing content forms with enormous amounts of fields, translatable or not.

Photo by Maarten van den Heuvel on Unsplash

Jul 07 2020
Jul 07

Drupal 7 introduced the brilliant feature of letting users cancel their own account and with it various options for what to do with content they've created when they are cancelled. One of these options is to:

Delete the account and its content.

Which can prove somewhat problematic if used incorrectly.

You see, Drupal is very good at the latter part: deleting all the content created by the user. It's not very good at warning someone that they are about to delete potentially a lot of important content.

The scenario

Let me set the scene for you. Someone had an account on a Drupal site and did a lot of work, making pages etc. Then they left the organisation. Someone else comes along and after a while thinks: I should clean up all these old user accounts and delete them, we don't need them any more.
Unfortunately they use the aforementioned Delete the account and its content option.

A few days pass and then they notice that the cookie policy page has gone missing. And they are sure that the FAQ section had more than 3 questions in it.
Oh dear.

They now face a serious problem. They have two 'easy' options to resolve it:

  1. Restore a database backup from before they deleted the user to recover all the lost content.
  2. Attempt to manually re-create all the content that was deleted.

However, they've been using the site in the interim and have changed lots of content. So have other users of the site. They can't simply restore a database backup from before all the content was deleted because they'd lose all the changes since then. But they also size up the volumes of missing content, and they simply aren't sure what content has gone missing, but know that it's hundreds of pages. Also the references between content have been broken, content that still exists on the site is trying to reference content that isn't there. So now not only do they need to re-create content but they have to go around fixing all the other site content that references that content. Oh my.

The third option

There is another way:

  1. Automatically re-create all the content that was deleted.

But how?

If you've got a decent backup from before the deletion happened then you contact your friendly ComputerMinds and we'll help you out by following something along the lines of the below. If you don't have a decent backup, then you're toast: Learn your lesson and start making backups of your data that you can restore from!

But you've got that backup, right? Ideally from as close as possible to, but not after, the account and content being deleted. So let's see what you/we do with it:

We're going to repeat the deletion and work out how to put it all back.

Begin by restoring the code, files and database from your backup to a development machine.
Load up the site in your browser and get ready to perform the exact same operation that caused the problem in the first place, but don't perform it yet!

Now, identify tables that contain changes that you don't really care about, the more the merrier. I'm thinking the watchdog table, any cache_* tables etc. You might need expert knowledge of the site to make this list as long as possible, it'll help later because you can really cut down the amount of noise and work you'll have to do later.

Once you've done that you want to make a 'pre-delete' database dump. Something like this:

drush sql-dump --structure-tables-list='sessions,cache,watchdog' > pre-delete.sql

Now, go back to your browser and cancel the account in the same way that was done before, so: Delete the account and its content.

Once the deletion has happened we want to run the same drush command as before, but save the results to another file.

drush sql-dump --structure-tables-list='sessions,cache,watchdog' > post-delete.sql

Now we essentially have two database snapshots, the difference between the two is all the content that was deleted. So we'll aim to produce a set of SQL queries to restore all that to the production database.

I had very mixed results with trying to get two MySQL dump files that would diff easily in a way that would leave the correct INSERT/UPDATE statements to put all the content back. Comparing the two dump files pre-delete.sql and post-delete.sql directly just didn't seem to work.

Percona to the rescue!

There's a tool in the Percona suite called pt-table-sync that will diff two databases and produce a set of SQL statements that would make the data consistent between the two, i.e. the SQL 'diff'.

There's a final wrinkle that means that you actually need another database server at this point, because pt-table-sync can only sync from one server to another, not between two databases on the same server. However, in the age of Vagrant or Docker getting multiple MySQL servers running on your machine is no big issue. I'm going to suppose you have two database servers running on ports 3306 and 3307 on your local machine.

Restore each of the SQL dump files from before to an identically named database on the servers respectively. Then you can get pt-table-sync to produce the magic:

pt-table-sync --print --databases=db_name h=127.0.0.1,P=3306 h=127.0.0.1,P=3307 > content-restore.sql

To make the diff go the right 'way' make sure the server with the post-delete.sql file is listed first in the command line. And you may need to adjust the command to get it to connect to your servers correctly.

Once you've done that content-restore.sql should contain a set of SQL commands that you could run on the production server to restore all the deleted content. However, I'd recommend doing one final manual look through the file and making sure that nothing is going to run against tables that don't really matter or that can't be recovered in other ways.
It's a text file so review it line by line and understand what each line is going to do and make sure they are the expected changes!

Once you've done all that you can execute the content-restore.sql file on your production server and that should restore everything that was deleted from the database!

Wrap up

So we've done this twice now, for different clients. We were happy that we were able to recover their content and not force them to either lose all other changes made or have to re-create a lot of pages.
We learnt so much the first time we did this, that the second time it was actually a fairly smooth process that didn't take very long at all despite having to restore thousands of pieces of content. We've also taken steps to stop people from using this particularly dangerous option when cancelling a users account.

Obviously all of the above relies on having backups of your database, and being able to retrieve a point-in-time, not just the 'latest' one. If you don't have this in place already, go now and get that sorted!
If you have backups, maybe bookmark this page so that if you ever need to recover a large amount of accidentally deleted content you'll know a (fairly) easy way that works well.

Jun 23 2020
Jun 23

So we challenged ourselves to contribute back to the Drupal community this year. How are we doing? Here's a simple update on what each of us has done so far. Hopefully we'll see other ComputerMinds team members join this list by the end of the year.
 

Christian Sanders

You may have noticed Christian's recent article on updating jQuery. That work included producing patches to Drupal core itself and the jquery_update project. Christian has also pushed for user interface improvements in paragraphs (which I hope to implement later this year).

James Silver

James updated a patch for using tokens in webform components. He continues to own a whole 17 (yes, 17!) sandbox projects on drupal.org.

James Williams (me!)

I've already written in more detail on my recent contribution to the XML sitemap project and to recommend sponsoring contributions. The Drupal 8 release of the Language Hierarchy project was my first sponsored contribution, and since then I've made it available for Drupal 9 as well. I've started answering support requests in Drupal slack on a semi-regular basis, but otherwise most of my contributions are still around code. Here's a summary:

Mike Dixon

Even our head 'mind still gets in on the act. He has reported and solved a bug in the schema.org metadata module (used for SEO) and is helpful when he can to respond to support requests for modules that we use. And as the boss, we should be grateful that he encourages us all to use some of our company time to give back to the Drupal world!

Nathan Page

We kicked off this challenge with Nathan's dive down the rabbit hole, and before long Nathan had even dabbled in reviewing and producing patches for Drupal core. I'd say that was the aim of the challenge: to grow as developers and encourage others to help the Drupal project - so I think we can safely say, 'mission accomplished'. Nathan has since moved onto pastures new (Hi Nathan if you're reading this!), but we're delighted to see his contributions have continued.

Steven Jones

Perhaps our biggest all-time contributor to Drupal, Steven has continued to provide brilliant contributions. I consider him my mentor in this field, and I'm not the only one. Some of his contributions are to the infrastructure that some of us use with Drupal. For example, making Valet+ support Drupal sites better. Here's a summary of how he's helped out on drupal.org so far this year:

Stephen Tweeddale

Steve's chief contribution this year is actually outside of the Drupal ecosystem - he has continued to maintain a plugin for GatsbyJS sites: gatsby-source-git. This uses a git repository as a source for pages to go into a static Gatsby site, just like the regular one for Markdown files within a site. (Did you know ComputerMinds do GatsbyJS, not just Drupal?!) The two systems are being used together on more and more projects nowadays, so I think it's only fair on Steve to include this here ;-)

~

We're proud of all this! We've been involved with sorting a couple of security issues in contrib modules too, which are rightly kept secret. Of course there's plenty of scope to do more, especially by giving more of our time to help out beyond code.

Why not join us in taking on this challenge, to make a contribution to Drupal every month this year? Good luck! Let us know how you get on in the comments below.

Jun 16 2020
Jun 16

There are some key files like robots.txt and .htaccess which are often tweaked for Drupal websites. These can be considered part of the 'scaffolding' of a site - they control the way the site works, rather than its content or design. Any new release of Drupal core that includes changes to them specifically mentions that they need updating, as those changes may have to be merged with any customisations made on your site. For example, there was a security release that added rules to .htaccess, which were essential for any site to incorporate and the template settings file, default.settings.php, also gets regular updates which are easy to miss out on. The new Drupal Scaffold composer plugin can now ensure that these files are always up-to-date by default. But that can mean it's now too easy to lose customisations, as those files are taken out of our direct control. (They now behave like files from external dependencies, which are usually excluded from version control.)

It's not a good idea to 'hack' (i.e. make changes to) core files. Drupal developers even dissuade each other from doing this by joking about bad things happening to kittens! But while these scaffolding files may come from core, they all live outside of Drupal 8's /core directory. (A full list of these files is near the bottom of this article.) This leaves them vulnerable to the forgetful developer coming along and tweaking them without thinking. To be fair, it's quite right to expect to be able to tailor them for SEO, specific business requirements, performance gains, debugging needs or whatever.

So the Scaffold composer plugin provides some ways to customise these files in a 'nice' way, all of which require some little edits to your project's root composer.json file.
 

  1. Simply append or prepend some lines

    Create a file containing the lines that you want to add, and reference it within the 'extra' section:

      "extra": {
        "drupal-scaffold": {
          "file-mapping": {
            "[web-root]/robots.txt": {
              "append": "assets/my-robots-additions.txt"
            }
          },
          ...
        }
      }

    Replace 'append' with 'prepend' as the key if needed. This is great for robots.txt, which usually just wants some additions beyond what Drupal normally provides. I've used it for default.settings.php to suggest some useful project-specific config overrides for developers.

  2. Override a file entirely

    Create the file you want to use instead of core's version, and reference it within the 'extra' section:

      "extra": {
        "drupal-scaffold": {
          "file-mapping": {
            "[web-root]/robots.txt": "assets/robots-override.txt"
          },
          ...
        }
      }

    This loses out on any improvements that Drupal may add over time, but is handy if you want to take back control of the file entirely. For example, some SEO agencies like to determine the contents of robots.txt entirely (although the RobotsTxt module may be more useful for that). To entirely exclude a file, map it to false (the RobotsTxt module requires that). 

  3. Patch a file

    Create a patch of changes that you want to make, and use the post-drupal-scaffold-cmd script event hook:

      "scripts": {
        "post-drupal-scaffold-cmd": [
          "cd docroot && git apply -v ../patches/my-htaccess-tweaks.patch"
        ]
      }

    This is really useful if you have specific changes to merge into a specific place of a scaffolded file, like in .htaccess. This ensures you get the benefit of updates made by core to the file.

    Pro tip: run composer install; git diff -R .htaccess > patches/my-htaccess-tweaks.patch to produce the patch if .htaccess is still under version control!

Once these are in place, you can then ensure to remove and exclude all the scaffolded files from version control, if you haven't already. Here's example commands you could run to remove them. Make sure to replace 'docroot' with your webroot subdirectory.

# Commands to remove scaffolded files
git rm .editorconfig .gitattributes --ignore-unmatch;
cd docroot;
git rm .csslintrc .eslintignore .eslintrc.json .ht.router.php .htaccess index.php robots.txt update.php web.config modules/README.txt profiles/README.txt themes/README.txt example.gitignore INSTALL.txt README.txt sites/README.txt sites/development.services.yml sites/example.settings.local.php sites/example.sites.php sites/default/default.services.yml sites/default/default.settings.php --ignore-unmatch

...and a snippet you could paste into your project's .gitignore file. (Again, replace 'docroot' if necessary.) This should then be committed for this to all work out.

# Lines to add to your project's .gitignore file.
# Files from the Drupal scaffold for composer.
/.editorconfig
/.gitattributes
docroot/.csslintrc
docroot/.eslintignore
docroot/.eslintrc.json
docroot/.ht.router.php
docroot/.htaccess
docroot/example.gitignore
docroot/index.php
docroot/INSTALL.txt
docroot/README.txt
docroot/robots.txt
docroot/update.php
docroot/web.config
docroot/sites/README.txt
docroot/sites/development.services.yml
docroot/sites/example.settings.local.php
docroot/sites/example.sites.php
docroot/sites/default/default.services.yml
docroot/sites/default/default.settings.php
docroot/modules/README.txt
docroot/profiles/README.txt
docroot/themes/README.txt

A current list of the files can be found in core's composer.json file.

Good luck - you can now rest assured that the Drupal kittens will rest in peace ?

Photo by Dan Diza on Unsplash

Jun 04 2020
Jun 04
Drupal 9 logo

Update: Since writing this article the EOL of Drupal 7 has been extended from November 2021 until November 2022.

It’s here! June 3rd 2020 marks the official release date of the first production ready version of Drupal 9. It feels like Drupal 8 was only released a short while ago but it turns out it’s been 4.5 years already! The release of any new major version of Drupal is an exciting milestone in the project’s history and with a shiny new brand logo in place, Drupal 9 is ready to hit the ground running.

We are already well underway doing upgrade builds for our clients that are still on Drupal 7, and for those that are on Drupal 8 already, we will be shortly getting those running on Drupal 9 too.

With the release of Drupal 9 now out in the wild, you only have until November 2021 November 2022 until Drupal 7 reaches end of life (EOL) with Drupal 8 still reaching EOL November 2021. By that time those versions will no longer receive any further security updates. So it’s imperative that any Drupal 7 or 8 sites out there have plans to complete the upgrade to Drupal 9 in time before then because the work required for Drupal 7 sites to upgrade may be pretty significant. One of our developers Christian wrote an excellent article on upgrading your existing Drupal 7 site in May last year, so be sure to check that out if you haven’t already.

So what new features can you expect when upgrading a Drupal 8 site to Drupal 9? Well, nothing immediately obvious. The first release of Drupal 9 (9.0.0) contains the exact same features as the final minor release of Drupal (8.9.0), just with updated dependencies that Drupal 9 relies upon and there’s been a clear out of deprecated code from the 8.x codebase. This means Drupal can continue to improve in future with less baggage holding it back.

olivero d9 themeWork in progress 'Olivero' theme for Drupal 9

This is an exciting new front end theme currently in the works called ‘Olivero’ - which aims to give Drupal 9 a fresh modern default theme - and a new default admin theme called ‘Claro’. New and exciting features of Drupal 9 will start arriving with the release of Drupal 9.1 onwards...

At ComputerMinds we still believe strongly in Drupal as the number one platform of choice to build our sites upon and it’s evident that others share that view as there are now over 1 million websites powered by Drupal. It remains a very powerful platform which allows us to build exactly the kind of experience that a client wants, tailored to their specific needs, whilst being able to scale up to potentially handle millions of users and pieces of content. (If your site gets that big!)

Being open source itself and using other open source technologies like Symfony, Composer, Twig and PHPUnit to power it means it’s now more widely accessible than ever to other PHP developers that may have experience with other frameworks using some of the same underlying technology.

Whether it’s powering nonprofit websites such as the Rainforest Alliance, Government websites such as the Colorado General Assembly, providing a superior CMS editing experience for a global magazine like The Economist, helping boost Tourism numbers to Fiji, promoting your favourite music artist like Bruno Mars or helping you to find your favourite recipe on BBC Good Food, Drupal really can do it all.

Drupal 9 is available to download right now from drupal.org so what are you waiting for? Go check it out! (Of course those of you with a properly managed workflow will be upgrading from 8 to 9 using composer, so get that composer file updated!)

If you’ve liked what you’ve read and feel like you’re ready - or in a position - to start thinking about a site upgrade, why not start a conversation with us today by using our contact form. We’d love to hear from you and look forward to seeing what benefits we can bring to your site.

May 05 2020
May 05

We've been busy recently, but that doesn't stop us at ComputerMinds contributing back to the Drupal community! For our latest multilingual website, we needed an XML sitemap with alternate links and hreflang attributes. This site uses separate domains for each language - for example, www.example.se (??) and www.example.no (??). Search engines need these alternate links to help them understand how to match up each translation of a page, which are distributed across these different domains. But this site is built on our existing Drupal 7 e-commerce platform that uses the XML sitemap project, which has no support for alternate links (nor entity translation).

Sometimes contributing is glamourous (think of getting new features into Drupal core, or creating new modules for the community), but other times it just involves stepping back to gain some perspective, and do some menial tasks. This was the latter! There have been two long-running issues on drupal.org about getting the functionality I needed, both created way back in 2012:

So my challenge was to wade into these two, and the 281 comments on them, to figure out how to make progress. It turned out that I'd actually dipped into the first one nearly 2 years ago, and my colleagues had used work from them before too. But a lot changes in that time! The patches needed updating ('re-rolling') to work with the most recent code of the XML sitemap project itself and to work with the latest versions of entity_translation. I particularly enjoyed spotting a comment from our very own Mike Dixon, who threw a spanner in the works with a patch that confused everyone!

Eventually I created updated patches, resolved some bugs, and incorporated some additional valuable work that hadn't yet been reviewed by anyone. These patches ensured our client would be satisfied, and hopefully someone else will come along to review & approve them some day too.

Perhaps the most interesting thing to have come out of the work was a snippet of PHP I relied on to process links that needed adding to the sitemap. The XML sitemap project provides a UI to rebuild things in a batch, but recent changes to respect access mean that even that does not quite build the sitemap entirely. Instead, the work is done via cron, including a queue. Queues are pretty brilliant for ensuring background processes happen at some point, without hitting timeouts, or having to code up boilerplate code in hook_cron() implementations. But there's no way to limit what runs on cron (without additional modules) - I didn't want other modules to go doing other things when I was working on this, I just wanted my sitemaps to be built! So I came up with this, which is otherwise almost entirely pinched from drupal_cron_run():

function limited_cron($cron_modules, $queue_keys = array()) {
  $queues = module_invoke_all('cron_queue_info');
  drupal_alter('cron_queue_info', $queues);

  // Limit to the queue(s) that we specifically want.
  $queues = array_intersect_key($queues, array_flip($queue_keys));

  foreach ($queues as $queue_name => $info) {
    DrupalQueue::get($queue_name)->createQueue();
  }

  $implementations = module_implements('cron');
  $implementations = array_intersect($implementations, $cron_modules);
  foreach ($implementations as $module) {
    // Do not let an exception thrown by one module disturb another.
    try {
      module_invoke($module, 'cron');
    }
    catch (Exception $e) {
      watchdog_exception('cron', $e);
    }
  }

  foreach ($queues as $queue_name => $info) {
    if (!empty($info['skip on cron'])) {
      // Do not run if queue wants to skip.
      continue;
    }
    $callback = $info['worker callback'];
    $end = time() + (isset($info['time']) ? $info['time'] : 15);
    $queue = DrupalQueue::get($queue_name);
    while (time() < $end && ($item = $queue->claimItem())) {
      try {
        call_user_func($callback, $item->data);
        $queue->deleteItem($item);
      }
      catch (Exception $e) {
        // In case of exception log it and leave the item in the queue
        // to be processed again later.
        watchdog_exception('cron', $e);
      }
    }
  }
}

// Only run the cron implementations and queues that the
// XML sitemap project provides.
limited_cron(array('xmlsitemap', 'xmlsitemap_node'), array('xmlsitemap_link_process'));

This allowed me to very easily get my sitemap built up quickly, by repeatedly calling limited_cron() with just the things that I needed to run. Note that it doesn't act exactly the same as the normal Drupal cron, which runs as the anonymous user and avoids updating the session. But I have found myself returning to use this on other projects for other modules' cron queues. Hopefully you might find it useful too :-)

Photo by Ian on Unsplash

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web