Author

Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Sep 30 2021
Sep 30

For the past two years I’ve been working on something less visible but no less important.

Since DrupalCon Amsterdam 2019 (an actual in-person conference — sounds surreal in 2021, doesn’t it?!) I’ve been working on Acquia Migrate Accelerate, or “AMA” for short. In a few days, another DrupalCon Europe is starting … so perfect timing for a recap! :D

Why?

Drupal 8 comes with an awesome migration system built in, originating in the Migrate Drupal 7 module. It standardized many migration best practices. But it still required a huge time investment to learn it.

Of course, there’s the “Migrate Drupal UI” (migrate_drupal_ui) module in Drupal core. But that does not allow for granular migrations. It allows for a one-shot migration: you see which things will be migrated and which won’t. You can click a button and hope for the best. It only works for the very simplest of sites. It is impressively minimal in terms of the code it needs, but unfortunately it also means one pretty much needs to be a expert in migrations to use it successfully.

It will be of little help as soon as you run into important data you want to migrate for which no migration path exists.

See Mauricio Dinarte’s excellent “31 days of Drupal migrations”. In those 31 blog posts, you’ll learn to know and appreciate the migration system (I sure did!). Unfortunately, that still won’t fully prepare you: you’ll need to decipher/reverse engineer the intricacies of how the data gets stored in Drupal 7 with its entities, revisions and fields — and with each field type having its own intricacies — and map that to Drupal 9 equivalents.

And how does one migrate straight from Drupal 7 with its more fragmented ecosystem?

For example: media handling. There are easily a dozen approaches possible in Drupal 7. Each in use on tens of thousands of sites. In Drupal 8 & 9, everything has standardized on Media and Media Library. But how do you get your Drupal 7 site’s content in there?

Another example: location data. location was very popular, but is now dead. geofield was equally popular but still alive. geolocation was less popular but now more. addressfield was popular, address is the successor. None of the Drupal 9 modules offer Location’s feature set. How do you migrate this data?

Goal

The goal for AMA (the vision of https://www.drupal.org/u/grasmash and especially https://www.drupal.org/u/webchick!) is to empower the non-technical user to be able to perform migrations. A UI that is aimed at the site builder POV: one should be able to select which content types (also vocabularies, menus, et cetera) make sense to migrate, and then not have to bother with technical details such as “migration plugins” or YAML files.

Acquia Migrate Accelerate:

For example, AMA shows just “Page” in the UI. Under the hood (and you can see this in the UI too, but it’s just not prominent), that corresponds to the following migration plugin definitions:

  • d7_node_type:page
  • d7_field_instance:node:page
  • d7_field_formatter_settings:node:page
  • d7_field_instance_widget_settings:node:page
  • d7_node_complete:page
  • d7_url_alias:node:page
  • node_translation_menu_links:node:page

In other words: the support configuration for nodes of the page bundle (the first 4), then all actual entity/field data (d7_node_complete:page), followed by URL aliases and menu links referencing pages.

However, to be able to do this, we need many more migrations in Drupal core to be derived: view modes, fields, formatters and widget should all have an entity type+bundle-specific derivative. That’d allow each bundle to be migrated individually. Which enables the site builder to check that their pages and everything related to it has been correctly migrated before moving on to the next data concept to migrate. So far we’ve not yet been able to convince the migration system maintainers of the value of this.

(AMA does many more things, but that’s not in scope of this blog post.)

Closed & Open

Acquia understandably wants its customers to be able to use AMA, and not its competitors’ users. Like all Drupal modules, the AMA module is GPL 2+ licensed. Only the React UI is closed source. The automated recommendations engine is closed source. Obviously the code to spin up AMA environments in Acquia Cloud is closed source .

But … all of the work that goes into making migrations reliable is open source. At the time of writing, we have ~100 unique patches that are being applied, 39 of which to Drupal core! While I was writing this, https://www.drupal.org/project/drupal/issues/3190818 got committed, plus a few were committed recently but did not yet ship in a 9.2.x point release, so soon that number will be lower :)

An overview

In the past 20 months we’ve hardened many migrations, and created new ones from scratch. Thousands of Drupal sites have already benefited — many more than there are Acquia customers.

The highlights:

Overall, 29 Drupal core patches and 18 Drupal contrib patches have been committed! Plus another 36 core patches and 32 34 contrib patches are successfully being used, and will hopefully land in the near future. (Not counting commits to the migration modules we now (co-)maintain.) Many dozens of migration paths from Drupal 7 have been stabilized, especially by https://www.drupal.org/project/media_migration.

A comprehensive overview (all patches are uncommitted unless stated otherwise):

D7D9 for all

We aim to continue to do to the work to get patches committed: address feedback, add test coverage, and so on. We want to help everyone migrate from Drupal 7 to 9!

Teamwork

These many hardened migrations are thanks to the titanic work of:

If you found this interesting, check out Gabe’s write-up of the application architecture that powers the awesome React-based UI that Peter built.

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web