Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough

Updating a D6 -> Drupal 8.0.x migration to Drupal 8.1.x

Parent Feed: 

We have several Drupal 6 to Drupal 8 upgrade projects going on, which is particularly challenging given how quickly the Drupal Migration system is changing. Given that a couple of them are nearing launch, and were missing some node references, I set out to get the content updated from the production sites before launch.

When we started migrating these sites, you could run the Migrate Upgrade scripts passing the database credentials, and get a full list of what could be upgraded and what could not. And it would migrate all the stuff. The whole point of having a migration system like this is so that you can keep on incrementally migrating content until the new site is ready to go, do a final migration, and go live!

The first surprise: with migration in 8.1.x, that is no longer supported with the built-in automatic Drupal to Drupal upgrade -- it's a one-shot, one way deal.

Oh crap.

Fortunately, it's pretty easy to use migrate_upgrade to generate a custom migration, and with some database mangling, you can get your new migration to pick up the mappings from a previous auto-migration done in 8.0.x.

The main gotcha here is that node content still isn't showing up as migrated -- but for us, that was good enough -- that's what we wanted to update from production anyway. The big win here is that all of the rest of the dependencies for the migration -- users, input filters, taxonomy terms, blocks, menus, settings -- are all now mapped and so we can keep the changes we've made on the new site without clobbering with an entirely new migration.

Caution! Extremely technical backflips ahead...

Ok, not really... but this is tricky stuff, involving multiple wipes/reloading of your database. Make sure you do this all on a test copy!

Before starting: Gather your tools, backups

Before doing anything, I would recommend backing up your database. And make sure your tools are up to date. Here are the tools I use for this process:

  • drush 8.x
  • drupal console 1.0.0 (currently at beta3)
  • bash, sed
  • a working Drupal 8.0 site (it's ok if you've already upgraded it to 8.1)

Step 1: Back up database, upgrade all code

drush sql-dump > site-80.sql
(do whatever you usually do to upgrade)
drush dl --select migrate_tools migrate_upgrade migrate_plus # select the latest dev versions of these
drush updb
drush cr # cache rebuild
drush sql-dump > site-81.sql

Note: for some reason, drush dl migrate_upgrade wants to install in ~/.drush, and not in the site -- this caused me a bit of confusion, and lots of errors when trying to update! Be sure it installs in a location that doesn't get overridden by the 1.x version, which is not compatible with Drupal 8.1.

For changes as major as these, I like to have places I can roll back to. git tags can be useful, along with database snapshots at each step.

Step 2: Save your config

The config by default is stored under sites/default/files -- we exclude that path from git, and so we recommend moving it somewhere inside your git tree. You can set this in the settings.php file, to whatever path you would like:

$config_directories['sync'] = 'config/sync';

... the above configuration puts it under the site root, in config/sync.

Once you have that set:

drupal config:export
git add config
git commit

... and you have your current configuration exported and saved to code.

Step 3: Apply any core patches, prep the destination

In particular, we wanted node references migrated to D8 entity references: Add migrate support for Drupal 6 node & user reference

At this point you should enable all the modules that have migrations you want picked up, and generally make sure your target environment is ready.

If you make any further changes, export and commit your config again, and create a new database backup BEFORE creating your new upgrade.

Step 4: Create your custom migration module, and create the new migration

Here I borrowed heavily from other posts:

The gist of it:

drupal generate:module # Create a new module to contain the migration configuration
gedit sites/default/settings # Add another database credential array for the D6 database, using a key "upgrade"
drush migrate-upgrade --configure-only # Creates all the migration config objects it can find
drupal config:export

At this point you have a brand new migration in the config, and a module ready to store this config. Assuming your module is in module/custom/site_migrate:

mkdir -p module/custom/site_migrate/config/install
cp config/sync/migrate_plus.migration* module/custom/site_migrate/config/install/
rm module/custom/site_migrate/config/install/migrate_plus.migration_group.default.yml

... this exports your migration config to code, skipping the default migration_group that would conflict with migrate_plus.

Step 5: Mangle all the data

Ok. Now we're to the tricky part. We're essentially going to revert to where we were at the end of step 3, and then enable our migration module, with the mapping tables from the original migration.

So...

git add modules/custom/site_module
git commit
git clean -df config/sync
git checkout HEAD config/sync

... to get your code base back to the right spot.

Now, the guts of the change. The new migrate_upgrade uses mostly the same migration names as the old, except with upgrade_ inserted at the beginning of the name. This means you need to rename your migrate_map and migrate_message tables to pick up those original mappings.

There's one other gotcha: at the moment, drush migrate-upgrade looks for a database key named "upgrade" but in the exported configuration, it changes to expect a database key named "drupal_6". So you'll need to edit your setting.php file to change that, after step 4.

Here's how I renamed the tables. Exactly what you need to change may vary based on the version of your mysql libraries and drush, but this worked for me:

cp site-81.sql site-mm.sql
sed -i "s/ALTER TABLE `migrate_message_/ALTER TABLE `migrate_message_upgrade_/g" site-mm.sql 
sed -i "s/ALTER TABLE `migrate_map_/ALTER TABLE `migrate_map_upgrade_/g" site-mm.sql 
sed -i "s/^LOCK TABLES `migrate_message_/LOCK TABLES `migrate_message_upgrade_/g" site-mm.sql 
sed -i "s/^LOCK TABLES `migrate_map_/LOCK TABLES `migrate_map_upgrade_/g" site-mm.sql 
sed -i "s/^INSERT INTO `migrate_message_/INSERT INTO `migrate_message_upgrade_/g" site-mm.sql 
sed -i "s/^INSERT INTO `migrate_map_/INSERT INTO `migrate_map_upgrade_/g" site-mm.sql 
sed -i "s/^CREATE TABLE `migrate_message_/CREATE TABLE `migrate_message_upgrade_/g" site-mm.sql 
sed -i "s/^CREATE TABLE `migrate_map_/CREATE TABLE `migrate_map_upgrade_/g" site-mm.sql 

Step 6: Run your migration!

Almost there...

drupal database:drop # this deletes any extraneous tables that might cause issues
drush sqlc < site-mm.sql  # reload the database with the changed tables, and without the second migration
drush en -y site_migrate
drush migrate-status

Presto! You should see most of your previous migrations, with most of the things already imported!

Now, if you migrate, you'll most likely get duplicate nodes of everything you previously imported. I would recommend bulk-deleting them, and then you should be able to proceed with:

drush migrate-import --all

... and get all node content migrated, along with any new users, menu items, files, etc.

But most importantly, you can now use the new migration improvements and same techniques as if you had started the migration in 8.1.x!

Thanks to Mike Ryan for assistance.

Author: 
Original Post: 

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web