Author

Aug 15 2019
Aug 15

Once, the Drupal community had Mollom, and everything was good. It was a web service that would let you use an API to scan comments and other user-submitted content and it would let your site know whether it thought it was spam, or not, so it could safely publish the content. Or not. It was created by our very own Dries Buytaert and obviously had a Drupal module. It was the service of choice for Drupal sites struggling with comment spam. Unfortunately, Mollom no longer exists. But there is an alternative, from the WordPress world: Akismet.

Akismet is very similar to Mollom. It too is a web service that lets you use an API to judge if some submitted content is spam. The name is derived from the Turkish word kismet, which means fate or destiny. The A simply stands for automatic (or Automattic, the company behind both WordPress and Akismet). It was created for WordPress, and like Mollom was once for Drupal, it is the service of choice for WordPress sites. However, nothing is keeping any other software from making use of it, so when you download the Akismet module, you can use it with your Drupal site as well. Incidentally, the module is actually based on the code of the Mollom module.

There is no stable release of the module, currently. In fact, there is no release at all, not even an alpha release. Except for development releases, that is. This means that for now it might not be an option for you to deploy the module. Hopefully, this changes soon, although the last commit is over a year ago at the time of writing. A mitigating circumstance, though, is that Drupal.org itself seems to be using this module as well, albeit in the Drupal 7 version (this article will be discussing the D8 version).

Adding Akismet to your Drupal site

How to add the module will depend on your workflow. Either download a development release, or - when you use a composer-based workflow - add the module like so:

$ composer require drupal/akismet:^[email protected]

Then, enable the module through the Extend admin screen (/admin/modules).

Basic configuration

In order to configure the module, you will first need an Akismet API key. To get this, register at https://akismet.com. If you have a wordpress.com account (which you might have from either wordpress.com itself, or e.g. because you also have Gravatar) you can sign in with it.

Once you've obtained an API key, you can go to /admin/config/content/akismet/settings to configure Akismet.

The choice what to do when Akismet is not available is probably dependent on how busy your site is. If it is very busy, and you do not get tons of spam, you probably want to accept all submissions. If you get a lot of spam and not very many actual contributions, you might want to block everything. If you both have a high traffic site and get a lot of spam, good luck. Of course, you can always look at a second line of defense, like the Honeypot module

The second two settings - Show a link to the privacy policy and Enable testing mode - seem to be left-overs from the Mollom module, because neither of them seem to do anything. I created issues for both the privacy policy and the testing mode.

While Mollom had a requirement to either show a link to its terms of use on protected forms, or have your own terms of use that made it clear you make use of the Mollom service, Akismet doesn't seem to have such a requirement. (Of course, it is a good idea to add something about your use of Akismet in your terms of use or your privacy policy).

Testing with Akismet is possible to either pass "viagra-test-123" as the body of the message, or "[email protected]" as the email address; these will always result in a spam classification. This seems to trigger the "unsure" scenario, eventhough that doesn't actually fully work, currently (see further). Forcing a ham (the opposite of spam - I didn't make this up) response is a bit trickier, because it would involve setting some parameters to the web service you do not have control over from the outside. Especially the testing mode might be a nice feature request for the Akismet module. Ideally, the module would work similar to the Mollom module, where you could simply send in a comment with "ham", "unsure" and "spam" to test. As said, I created an issue to flesh out this functionality.

The advanced configuration hides settings to control whether to only log errors and warnings, or all Akismet messages, and a timeout for contacting the Akismet server. Especially in the beginning you might want to log all messages to monitor whether things are working as they should. 

Configuring which forms to protect

When having finished the basic configuration for the module, it is time to configure the forms you want to protect. This happens on the path /admin/config/content/akismet. Here, click the "Add form" button to start configuring a form.

When clicking the button, the module will ask you which form you wish to configure. Out of the box, the module will offer to protect node, comment user and contact forms. A hook is offered to add additional forms, although either a module will need to implement the hook itself, or it will have to be done for it. Here, I'm choosing to just protect the comment form, as I am suffering from quite a lot of comment spam. Once you've chosen a form, it will show the form fields you might want to pass to the Akismet web server for analysis.

You'll basically want to select anything that is directly controlled by the user. The obvious candidate is the body, but also the subject, user name, email address, website and hostname will contain clues whether something is spam or not.

Next, you get to select what happens when Akismet decides content is or might be spam. Akismet may report back that it is sure something is spam. If it says something is spam, but does not pass back this certainty flag, the Drupal module says Akismet is "unsure", which is actually a term that can be traced back to the Mollom roots of this module. You may tell the module it should then retain the submission for manual moderation, although this doesn't seem to work correctly, at the moment. I created an issue in the issue queue for that. What I'm seeing happening is that the post is discarded, just like when Akismet is sure.

Click Create Protected Akismet Form to save the protection configuration. You're now ready to catch some spam. You can look at the watchdog log (/admin/reports/dblog) to see the module reporting on what it is doing. Akismet itself also has a nice dashboard with some graphs showing you how much ham and spam it detected on your site.

Reporting spam to Akismet

Sometimes, Akismet might wrongly accept some piece of content that is actually spam. Or, when the moderation queue mechanism actually works properly, you probably want to let Akismet know that yes, something is in fact spam (you might also want to let Akismet know it didn't correctly identify spam, i.e. report false positives. This is a feature of the web service, but is currently not in the module; another feature request, it seems. I've submitted one in the issue queue).

The module comes with action plugins for comments and nodes that let you unpublish the content and report it as spam to Akismet at the same time. You can add it to your comment views by changing their configuration at /admin/structure/views/view/comment (you will need the Views UI module enabled to be able to configure views). Unfortunately, it seems that also with this functionality there is an issue, the action doesn't actually unpublish. A patch is available in the linked issue and of course the workaround is to first use the Akismet action, and then use the standard unpublish action.

Find the configure link for the Comment operations form. Click the link and, in the modal that opens, find the list of checkboxes for the available actions. Enable Report to Akismet and unpublish and save the configuration. Repeat for the Unapproved comments display. This will mean you will now have the action available in the actions dropdown on the comments overviews at /admin/content/comment.

Adding this to a node view will be similar, although chances are that when you have end users submitting nodes, you likely also have some dedicated views for your specific use case, such as Forum posts.

Issues created as a result of this blog post

Please note that I did not intend to "set anyone to work" with creating these. I simply wanted to record some findings and ideas from writing up this blog post.

Aug 05 2019
Aug 05

When deploying changes to a Drupal environment, you should be running database updates (e.g. drush updb, or through update.php) first, and only then import new configuration (which itself is supposedly the result of running the same update hooks on a development environment). The reason for this is that update hooks may want to update configuration, which will fail if that configuration is already structured in the new format (you can't do without updates either; update hooks don't just deal with configuration, but may also need to change the database schema and do associated data migrations). So, there really is no discussion; updates first, config import second. But sometimes, you need some configuration to be available before executing an update hook.

For example, you may want to configure a pathauto pattern, and then generate all aliases for the affected content. Or, you need to do some content restructuring, for which you need to add a new field, and then migrate data from an old field into the new field (bonus tip: you should be using post update hooks for such changes). So, that's a catch-22, right?

Well, no. The answer is actually pretty simple, at least in principle: make sure you import that particular configuration you need within your update hook. For importing some configuration from your configuration sync directory, you can add this function to your module's .install file:

/**
 * Synchronize a configuration entity.
 * 
 * Don't use this to create a new field, use 
 * my_custom_module_create_field_from_sync().
 *
 * @param string $id
 *   The config ID.
 *
 * @see https://blt.readthedocs.io/en/9.x/readme/configuration-management/#using-update-hooks-to-importing-individual-config-files
 */
function my_custom_module_read_config_from_sync($id) {
  // Statically cache storage objects.
  static $fileStorage, $activeStorage;

  if (empty($fileStorage)) {
    global $config_directories;
    $fileStorage = new FileStorage($config_directories[CONFIG_SYNC_DIRECTORY]);
  }
  if (empty($activeStorage)) {
    $activeStorage = \Drupal::service('config.storage');
  }

  $config_data = $fileStorage->read($id);
  $activeStorage->write($id, $config_data);
}

Use it like this:

my_custom_module_read_config_from_sync('pathauto.pattern.landing_page_url_alias');

As you might have seen in the docblock above that function, it is not actually suitable for creating fields. This is because just importing the configuration will not create the field storage in the database. When you need to create a field, use the following code:

/**
 * Creates a field from configuration in the sync directory.
 *
 * For fields the method used in kankernl_custom_read_config_from_sync() does
 * not work properly.
 *
 * @param string $entityTypeId
 *   The ID of the entity type the field should be created for.
 * @param string[] $bundles
 *   An array of IDs of the bundles the field should be added to.
 * @param string $field
 *   The name of the field to add.
 *
 * @throws \Drupal\Core\Entity\EntityStorageException
 */
function my_custom_module_create_field_from_sync($entityTypeId, array $bundles, $field) {
  // Statically cache storage objects.
  static $fileStorage;

  // Create the file storage to read from.
  if (empty($fileStorage)) {
    global $config_directories;
    $fileStorage = new FileStorage($config_directories[CONFIG_SYNC_DIRECTORY]);
  }

  /** @var \Drupal\Core\Entity\EntityStorageInterface $fieldConfigStorage */
  $fieldStorage = \Drupal::service('entity_type.manager')
    ->getStorage('field_storage_config');

  // If the storage does not yet exit, create it first.
  if (empty($fieldStorage->load("$entityTypeId.$field"))) {
    $fieldStorage
      ->create($fileStorage->read("field.storage.$entityTypeId.$field"))
      ->save();
  }

  /** @var \Drupal\Core\Entity\EntityStorageInterface $fieldConfigStorage */
  $fieldConfigStorage = \Drupal::service('entity_type.manager')
    ->getStorage('field_config');

  // Create the field instances.
  foreach ($bundles as $bundleId) {
    $config = $fieldConfigStorage->load("$entityTypeId.$bundleId.$field");
    if (empty($config)) {
      $fieldConfigStorage->create($fileStorage->read("field.field.$entityTypeId.$bundleId.$field"))
        ->save();
    }
  }
}

And, once again, a usage example:

my_custom_module_create_field_from_sync('node', ['basic_page', 'article'], 'field_category');

The function will check whether the field already exists, so it is safe to run again, or to run it for a field that already exists on another bundle of the same entity type.

Note that when using post update hooks, it will be important to create a single hook implementation that applies all required actions for what should be considered a single change, because there are no guarantees about the order of post update hooks. So that would for example constitute:

  1. Create a new field.
  2. Migrate data from the old field to the new field.
  3. Remove the old field.

Hopefully, this helps someone get out of that catch 22. Whatever you do, don't run your config import before your database updates.

Jul 10 2019
Jul 10

At last month's DrupalJam XL in Utrecht, the Netherlands, I gave Gabor Hojtsy's presentation on the state of Drupal 9. It was recorded - thanks DrupalJam organization! - so here is the video. You might also want to view Gabor's own presentation from DrupalCamp Belarus.

You'll need to turn up the audio, because it seems that it was recorded using the camera, not the fancy microphone I'm wearing.

[embedded content]

Aug 24 2018
Aug 24

Drupal 8 will actively complain when your site does not have a hash_salt configured, which usually gets generated when installing the site. (The complaint, mind you, might be fairly obscure; your site might just say "The website encountered an unexpected error. Please try again later." Depending on your error reporting settings, the message might be a bit more helpful). If, for example, you "install" a site by copying over a database and files, you will not have this.

When you need a fresh site hash, you can use the following Drush command.

drush php-eval 'echo \Drupal\Component\Utility\Crypt::randomBytesBase64(55) . "\n";'

Alternatively, if you would like to put it in a file, so that you can read the file from somewhere outside your webroot, you could do:

drush php-eval 'echo \Drupal\Component\Utility\Crypt::randomBytesBase64(55)' > salt.txt

To quote the comment from default.settings.php:

 * Example:
 * @code
 *   $settings['hash_salt'] = file_get_contents('/home/example/salt.txt');
 * @endcode

Aug 24 2018
Aug 24

Drupal 8 will actively complain when your site does not have a hash_salt configured, which usually gets generated when installing the site. (The complaint, mind you, might be fairly obscure; your site might just say "The website encountered an unexpected error. Please try again later." Depending on your error reporting settings, the message might be a bit more helpful). If, for example, you "install" a site by copying over a database and files, you will not have this.

When you need a fresh site hash, you can use the following Drush command.

drush php-eval 'echo \Drupal\Component\Utility\Crypt::randomBytesBase64(55) . "\n";'

Alternatively, if you would like to put it in a file, so that you can read the file from somewhere outside your webroot, you could do:

drush php-eval 'echo \Drupal\Component\Utility\Crypt::randomBytesBase64(55)' > salt.txt

To quote the comment from default.settings.php:

 * Example:
 * @code
 *   $settings['hash_salt'] = file_get_contents('/home/example/salt.txt');
 * @endcode

Mar 22 2018
Mar 22

Last weekend, the DrupalCamp Ruhr was held in Essen, Germany. I was fortunate enough to have been selected as a speaker. I've now made the slides available online.

The slides are at Drupal 8 Configuration Management, Workflows for site development. It is an evolution of the talk I did last year at the Dutch DrupalJam.

The presentation uses Reveal.js, which means it has some neat tricks up its sleeve. You can have a look at the speaker notes bij pressing 's' (they're no secret) and zoom out to an overview of slides by pressing 'o' or esc. Should you want a print version, you can pass ?print-pdf as a GET-argument in the URL, provided you use Chrome or Chromium to do this (the presentation itself should work fine in any browser, although I've only really tested it in Safari on the Mac). I recommend tweaking the styling a little in your browser to set text color to black and not printing backgrounds, or it will take a lot of ink.

Jan 20 2018
Jan 20

If you have the Media module for Drupal 8 installed (not a requirement to use media with Drupal 8, so this post may not apply to you), you need to remove it before you can upgrade to the latest core version (8.4). Unfortunately, there are a few gotchas involved with the process. This blog post is about getting rid of the old contrib Media module, so the site can be updated to Drupal 8.4 in a subsequent step. This is based on my personal experience. YMMV, as they say.

Originally I published this post on december 25th of 2017. Merry Christmas! Since this blog was added to Planet Drupal yesterday, I thought it would benefit others to re-date this to today, so it would appear in the Planet feed.

Having to maintain this site in my spare time, it has been far too long for me to upgrade it to Drupal 8.4 (it actually went live on Drupal 8.3 when 8.4 was already final, because the platform had been running for a while for another site and frankly, I just wanted it live—something about a plumber and leaky pipes). The site had been built with the contrib Media module, which turns out to be a bit of a mistake (just because of the problems I outline below, not because there was anything wrong with the module), but that's water under the bridge.

Where the Media module for Drupal 7 is the core for a (the?) leading Media solution, for Drupal 8 that responsibility moved to the new media_entity module, as well as a bunch of extension modules that add specific media types. In short, Media in Drupal 8 is much more modular. The Media module itself still existed, but it was basically a quick start suite of configuration and a few tweaks to how things worked out of the box.

The reason removing the contrib Media module is needed is that from 8.4 on, media_entity was integrated into core, except that is is now called... Media module. This means the contrib Media module would have a name collision with the new Media module in core. So, it needs to be disabled. Which is mostly OK (you can retain most of the configuration it adds), except for a new nasty gotchas.

Note that this is not about moving your site to the core media entity type. That is something that will have to happen eventually. You will have to wait for all extension modules (image, video, maybe gallery, Twitter or Instagram, etc.) to have versions that work with the core entity type, as well as for the upgrade path to be complete. This needs to happen, though, if you currently have Media module and you need to move to Drupal 8.4 (and you do need to; 8.3 stopped being supported when 8.4 came out).

Gotcha #1: Route "view.media.media_page_list" does not exist

When I first tried disabling the contrib Media module (which I will simply call Media module for the remainder of this blog post), I was greeted by this lovely error message. In full, it read:

Symfony\Component\Routing\Exception\RouteNotFoundException: Route "view.media.media_page_list" does not exist. in Drupal\Core\Routing\RouteProvider->getRouteByName() (line 190 of core/lib/Drupal/Core/Routing/RouteProvider.php).

It went along with the dreaded "An unexpected error occurred" as well as a giant stack trace, but that's just due to my development environment settings.

The View is what the Media module added in place of the View that's also added by media_entity module. I haven't quite figured out how this situation came to be, but there is still a View in the exported configuration, presumably the one added by media_entity. However, as far as I can tell, there were no actual changes to the exported configuration, so it is still a bit puzzling. Luckily, the solution turned out to be fairly straightforward; the View had its status key set to false in the exported configuration yml-file (views.view.media.yml):

langcode: nl
status: false
dependencies:
  config:
    - core.entity_view_mode.media.media_library
    - media_entity.bundle.gallery
  module:
    - entity_browser
    - media_entity
    - user

Changing it to true and re-importing configuration got rid of this problem (don't forget to remove media in core.extension.yml if you do this; chances are you didn't get a chance to actually export the configuration after disabling the Media module, so re-importing your exported configuration would happily re-enable it again—in fact, you could skip the step of disabling the module through the interface and simply remove its line from core.extension.yml).

Gotcha #2: Media module contains some tweaks

The Media module contains some useful tidbits (mostly tweaks to standard core or contrib functionality). If you are getting ready to update a site to the next minor core version, you are not in the state of mind to deal with functional regressions and/or changes (especially to find out how to turn regressions into changes). At least, I wasn't. That's why I created the Media Glue sandbox module, that basically just contains the css, javascript and hook implementations that Media did, except under a different name space so it does not collide with the new core Media module. It does contain one little change, which involves the styling for the entity browser. The Media module contained a small bug that caused the items to jump around in the browser when hovering over them (at least in my browser of choice, Safari). The css for the new module has been tweaked so the elements are the same size for both regular and hover states.

I will probably want to figure out how I want to get rid of this module at some point, as I usually prefer to stay closer to base functionality as opposed to install all sorts of little modules that may add small niceties; extra modules can break and also add a slight performance penalty. There also apparently are various modules that do something similar, which would appear to be better starting points. But that's stuff for another blog post.

Gotcha #3: When you decide you don't need Media entity slideshow

As said, Media module came with a bunch of configuration. It added all that configuration as install configuration (not optional), so it had dependencies on extension modules that were required for all that configuration. As I previously mentioned, I'm no great fan of stuff I don't really need; all it can do is break. So I figured I could remove some of the extras that came with Media. Mostly, this is no problem. You can disable the Document, Instagram and Twitter modules if you don't need them. Any configuration dependent on them, like media entity bundles and display settings, will be removed. (The same likely goes for the video embed module, but I haven't tried).

However, when you elect to remove the Media Entity Slideshow module, Drupal will make an odd report about configuration that will be removed; in addition to the usual suspects (the media bundle and all its derived configuration like fields and displays) it will want to remove the Media Embed and Media Library entity browsers as well as the Media library View. The last one is the key here. The other two depend on it, because they use it as their browser view. The View itself contains two displays that are used for the Entity Browser that is meant for creating galleries (which, oddly, does not get removed, in my experience). These displays contain a filter option to exclude galleries (in order to avoid gallery inception). Thus, the View depends on its existence and if we wish to remove the bundle, we need to remove the View. 

At this point, the safest option is to abort the uninstalling of the module and go and edit the troublesome View. You need to remove the two displays:

  1. Gallery media select modal
  2. Gallery User Media select modal

Save the View and export the configuration.

Unfortunately, this did not remove the actual dependency from the View, only the displays that are the reason for having the dependency. Open the views.view.media_libary.yml file in your editor and look for all values containing gallery. This will turn up everything that had to do with the media bundle. Remove the lines you find. Save the file and re-import configuration. 

Go and uninstall the module again. If the View and entity browsers are still being listed, try and clear the cache. Finally, remove the gallery_media_library entity browser (admin/config/content/entity_browser).

After the module is uninstalled, do another config export.

Next steps

That's basically it. At this point, you will have to release your changes through your DTAP street (if that doesn't mean anything to you, don't worry; it simply means you have to get these changes into production) to prepare your site for the actual move to Drupal 8.4. You can not remove the Media module before getting it disabled (if you have a clever release process, you might be able to do removals at the very end, which would allow you to do things in one go).

If you use a composer-based build process, you will need to remove drupal/media from your composer.json. At the same time, you will need to add back any modules that were previously dependencies of Media that you wish to keep. The full list of dependencies for (from its .info.yml file) Media looks like this:

dependencies:
  - media_entity:media_entity
  - media_entity_image:media_entity_image
  - video_embed_field:video_embed_field
  - video_embed_field:video_embed_media
  - media_entity_slideshow:media_entity_slideshow
  - media_entity_instagram:media_entity_instagram
  - media_entity_twitter:media_entity_twitter
  - media_entity_document:media_entity_document
  - slick_media:slick_media
  - entity_browser:entity_browser
  - entity_browser:entity_browser_entity_form
  - entity_embed:entity_embed
  - dropzonejs:dropzonejs_eb_widget
  - image_widget_crop:image_widget_crop
  - drupal:link
  - drupal:editor
  - inline_entity_form:inline_entity_form
test_dependencies:
  - media_entity:media_entity
  - media_entity_image:media_entity_image
  - video_embed_field:video_embed_field
  - video_embed_field:video_embed_media
  - media_entity_slideshow:media_entity_slideshow
  - media_entity_instagram:media_entity_instagram
  - media_entity_twitter:media_entity_twitter
  - media_entity_document:media_entity_document
  - slick_media:slick_media
  - entity_browser:entity_browser
  - entity_browser:entity_browser_entity_form
  - entity_embed:entity_embed
  - dropzonejs:dropzonejs_eb_widget
  - image_widget_crop:image_widget_crop
  - drupal:link
  - drupal:editor
  - inline_entity_form:inline_entity_form

Find out the versions you have installed currently and require them explicitly in your composer.json. If you added everything you need to keep, you can go ahead and remove media itself. Rebuild the site and make sure everything still works. You are now ready to actually take the step and update to Drupal 8.4.

Dec 25 2017
Dec 25

If you have the Media module for Drupal 8 installed, you need to remove it before you can upgrade to the latest core version (8.4). Unfortunately, there are a few gotchas involved with the process. This blog post is about getting rid of the old contrib Media module, so the site can be updated to Drupal 8.4 in a subsequent step. This is based on my personal experience. YMMV, as they say.

Having to maintain this site in my spare time, it has been far too long for me to upgrade it to Drupal 8.4 (it actually went live on Drupal 8.3 when 8.4 was already final, because the platform had been running for a while for another site and frankly, I just wanted it live—something about a plumber and leaky pipes). The site had been built with the contrib Media module, which turns out to be a bit of a mistake (just because of the problems I outline below, not because there was anything wrong with the module), but that's water under the bridge.

Where the Media module for Drupal 7 is the core for a (the?) leading Media solution, for Drupal 8 that responsibility moved to the new media_entity module, as well as a bunch of extension modules that add specific media types. In short, Media in Drupal 8 is much more modular. The Media module itself still existed, but it was basically a quick start suite of configuration and a few tweaks to how things worked out of the box.

The reason removing the contrib Media module is needed is that from 8.4 on, media_entity was integrated into core, except that is is now called... Media module. This means the contrib Media module would have a name collision with the new Media module in core. So, it needs to be disabled. Which is mostly OK (you can retain most of the configuration it adds), except for a new nasty gotchas.

Note that this is not about moving your site to the core media entity type. That is something that will have to happen eventually. You will have to wait for all extension modules (image, video, maybe gallery, Twitter or Instagram, etc.) to have versions that work with the core entity type, as well as for the upgrade path to be complete. This needs to happen, though, if you currently have Media module and you need to move to Drupal 8.4 (and you do need to; 8.3 stopped being supported when 8.4 came out).

Gotcha #1: Route "view.media.media_page_list" does not exist

When I first tried disabling the contrib Media module (which I will simply call Media module for the remainder of this blog post), I was greeted by this lovely error message. In full, it read:

Symfony\Component\Routing\Exception\RouteNotFoundException: Route "view.media.media_page_list" does not exist. in Drupal\Core\Routing\RouteProvider->getRouteByName() (line 190 of core/lib/Drupal/Core/Routing/RouteProvider.php).

It went along with the dreaded "An unexpected error occurred" as well as a giant stack trace, but that's just due to my development environment settings.

The View is what the Media module added in place of the View that's also added by media_entity module. I haven't quite figured out how this situation came to be, but there is still a View in the exported configuration, presumably the one added by media_entity. However, as far as I can tell, there were no actual changes to the exported configuration, so it is still a bit puzzling. Luckily, the solution turned out to be fairly straightforward; the View had its status key set to false in the exported configuration yml-file (views.view.media.yml):

langcode: nl
status: false
dependencies:
  config:
    - core.entity_view_mode.media.media_library
    - media_entity.bundle.gallery
  module:
    - entity_browser
    - media_entity
    - user

Changing it to true and re-importing configuration got rid of this problem (don't forget to remove media in core.extension.yml if you do this; chances are you didn't get a chance to actually export the configuration after disabling the Media module, so re-importing your exported configuration would happily re-enable it again—in fact, you could skip the step of disabling the module through the interface and simply remove its line from core.extension.yml).

Gotcha #2: Media module contains some tweaks

The Media module contains some useful tidbits (mostly tweaks to standard core or contrib functionality). If you are getting ready to update a site to the next minor core version, you are not in the state of mind to deal with functional regressions and/or changes (especially to find out how to turn regressions into changes). At least, I wasn't. That's why I created the Media Glue sandbox module, that basically just contains the css, javascript and hook implementations that Media did, except under a different name space so it does not collide with the new core Media module. It does contain one little change, which involves the styling for the entity browser. The Media module contained a small bug that caused the items to jump around in the browser when hovering over them (at least in my browser of choice, Safari). The css for the new module has been tweaked so the elements are the same size for both regular and hover states.

I will probably want to figure out how I want to get rid of this module at some point, as I usually prefer to stay closer to base functionality as opposed to install all sorts of little modules that may add small niceties; extra modules can break and also add a slight performance penalty. There also apparently are various modules that do something similar, which would appear to be better starting points. But that's stuff for another blog post.

Gotcha #3: When you decide you don't need Media entity slideshow

As said, Media module came with a bunch of configuration. It added all that configuration as install configuration (not optional), so it had dependencies on extension modules that were required for all that configuration. As I previously mentioned, I'm no great fan of stuff I don't really need; all it can do is break. So I figured I could remove some of the extras that came with Media. Mostly, this is no problem. You can disable the Document, Instagram and Twitter modules if you don't need them. Any configuration dependent on them, like media entity bundles and display settings, will be removed. (The same likely goes for the video embed module, but I haven't tried).

However, when you elect to remove the Media Entity Slideshow module, Drupal will make an odd report about configuration that will be removed; in addition to the usual suspects (the media bundle and all its derived configuration like fields and displays) it will want to remove the Media Embed and Media Library entity browsers as well as the Media library View. The last one is the key here. The other two depend on it, because they use it as their browser view. The View itself contains two displays that are used for the Entity Browser that is meant for creating galleries (which, oddly, does not get removed, in my experience). These displays contain a filter option to exclude galleries (in order to avoid gallery inception). Thus, the View depends on its existence and if we wish to remove the bundle, we need to remove the View. 

At this point, the safest option is to abort the uninstalling of the module and go and edit the troublesome View. You need to remove the two displays:

  1. Gallery media select modal
  2. Gallery User Media select modal

Save the View and export the configuration.

Unfortunately, this did not remove the actual dependency from the View, only the displays that are the reason for having the dependency. Open the views.view.media_libary.yml file in your editor and look for all values containing gallery. This will turn up everything that had to do with the media bundle. Remove the lines you find. Save the file and re-import configuration. 

Go and uninstall the module again. If the View and entity browsers are still being listed, try and clear the cache. Finally, remove the gallery_media_library entity browser (admin/config/content/entity_browser).

After the module is uninstalled, do another config export.

Next steps

That's basically it. At this point, you will have to release your changes through your DTAP street (if that doesn't mean anything to you, don't worry; it simply means you have to get these changes into production) to prepare your site for the actual move to Drupal 8.4. You can not remove the Media module before getting it disabled (if you have a clever release process, you might be able to do removals at the very end, which would allow you to do things in one go).

If you use a composer-based build process, you will need to remove drupal/media from your composer.json. At the same time, you will need to add back any modules that were previously dependencies of Media that you wish to keep. The full list of dependencies for (from its .info.yml file) Media looks like this:

dependencies:
  - media_entity:media_entity
  - media_entity_image:media_entity_image
  - video_embed_field:video_embed_field
  - video_embed_field:video_embed_media
  - media_entity_slideshow:media_entity_slideshow
  - media_entity_instagram:media_entity_instagram
  - media_entity_twitter:media_entity_twitter
  - media_entity_document:media_entity_document
  - slick_media:slick_media
  - entity_browser:entity_browser
  - entity_browser:entity_browser_entity_form
  - entity_embed:entity_embed
  - dropzonejs:dropzonejs_eb_widget
  - image_widget_crop:image_widget_crop
  - drupal:link
  - drupal:editor
  - inline_entity_form:inline_entity_form
test_dependencies:
  - media_entity:media_entity
  - media_entity_image:media_entity_image
  - video_embed_field:video_embed_field
  - video_embed_field:video_embed_media
  - media_entity_slideshow:media_entity_slideshow
  - media_entity_instagram:media_entity_instagram
  - media_entity_twitter:media_entity_twitter
  - media_entity_document:media_entity_document
  - slick_media:slick_media
  - entity_browser:entity_browser
  - entity_browser:entity_browser_entity_form
  - entity_embed:entity_embed
  - dropzonejs:dropzonejs_eb_widget
  - image_widget_crop:image_widget_crop
  - drupal:link
  - drupal:editor
  - inline_entity_form:inline_entity_form

Find out the versions you have installed currently and require them explicitly in your composer.json. If you added everything you need to keep, you can go ahead and remove media itself. Rebuild the site and make sure everything still works. You are now ready to actually take the step and update to Drupal 8.4.

May 31 2017
May 31

Last week, on wednesday May 24th, the annual Dutch Drupal conference DrupalJam had its 2017 edition in "De Fabrique" in Maarssen. After having been an attendee for quite some years, this time I submitted a proposal for a talk, which was accepted. I'm now happy to report the slides are available online: Drupal 8 Configuration Management, Workflows for Site Development.

The presentation uses Reveal.js, which means it has some neat tricks up its sleeve. You can have a look at the speaker notes bij pressing 's' (they're no secret) and zoom out to an overview of slides by pressing 'o' or esc. Should you want a print version, you can pass ?print-pdf as a GET-argument in the URL. It will look horrific on screen, but will come out OK on paper or in PDF form, provided you use Chrome or Chromium to do this (the presentation itself should work fine in any browser, although I've only really tested it in Safari on the Mac).

Dec 03 2016
Dec 03

The last three days, I was at the code sprint for the Drupal Camp Munich 2016, hosted at the Hubert Burda Media offices. It was a great experience. It was my first community code sprint, I've met some great people and I've learned a ton in three days. I have to thank the organizers of the sprint as well as the camp organizers, and of course my employer One Shoe to give me this opportunity. We have been working on the Media Initiative, to get a solid Media solution into Drupal core in an upcoming minor release (e.g. 8.3 or 8.4). Christian Fritsch, one of the main developers for the Media Initiative, asked me to have a look at one particular issue that is quite important to get sorted out, because it aims to solve a weird usability problem that certainly wouldn't fly for a core component. I've been able to push it quite a bit forward in those three days. Until I got to getting Javascript tests to run on my setup, that is.

There are some solid tutorials available to get you up and running with Javascript testing for Drupal. The one I was pointed to is this one from Alex Pott. Everything seemed to go OK, until I actually went to run the tests. The test would consistently get a 403 when trying to go to the login page of the test environment. When running the test from my Mac directly, the Simpletest install didn't seem to spin up correctly. In the end, it seems my setup with Vagrant is at the heart of the problem.

Giving this some thought*, I realised that the test environment having to run on the Vagrant box, while the test itself was running on the Mac was the likely culprit. The tutorial stresses that it is important that the tests are ran with a user with the proper permissions. Presumably, because it will need to initialize the environment which will then need to be accessed by the webserver to run the actual test. So, what do you do? Install phantomJS on Vagrant? Sure, that would have worked, most likely. But what worked too, was just setup a little port forward from port 8510 on the Vagrant box to the same port on the host:

ssh -L 8510:192.168.33.1:8510 localhost

This is issued on the vagrant box. You may need to change the IP address of your host, if it is different, and you will need to provide the password for the vagrant account on the vagrant box (which usually is just "vagrant").

I both love it and hate it when the solution to a problem is a one liner.

*: This sort of insight usually doesn't come to me at the end of a long day of hard work, but instead in bed, in the shower, or, in this case, on the Lufthansa flight to Amsterdam while gazing out of the window at all the pretty lights below.

Feb 21 2016
Feb 21

This is a subject that seems to come up again and again. I figured this out once before, but back then I was using Eclipse, while I am now a PhpStorm devotee. Also, I've long since started using Vagrant, which means that every debugging scenario is now "remote". Usually, this is not a huge issue, but with command line debugging, this presents a bit of a challenge.

Randy Fay has done a nice write-up back in 2013, which covers most of the bases. It does leave one missing link, which seems to have come up only recently; a recent change in drush has broken debugging through xdebug, because it is now using pcntl_exec() to execute the actual script (don't ask me for the technical details, I haven't delved into that too much). As is so often the case, Stack Exchange provided the missing piece of the puzzle; it's possible to circumvent the "decoupling", that seems to occur due to the pcntl_exec(), by using drush.launcher as the entry point.

As much for my own reference as anyone else's, these are the key points from both Randy's post and the SE topic:

  • Set up a "PHP Web Application" for debugging the command line. The sole purpose of this is to be able to provide a path mapping when running the command in Vagrant.
  • Enable xdebug debugging for the command line in your Vagrant box. In my case, this simply meant symlinking the same xdebug.ini from my /etc/php5/cli/conf.d directory as I was using in the /etc/php5/apache/conf.d for web debugging.
  • All executed code needs to be available in the project, including drush. You can accomplish this by e.g. installing drush as a composer dependency (also, remember to execute drush from your project).

The following you will need to do every debugging session:

  • Use PhpStorm's "Listen for PHP Debug connections" button
  • Set the remote debug client on the command line using (or whatever is the IP-address for your host machine when coming from the Vagrant box; the xdebug.remote_connect_back that is likely in your config will not work for the command line):
    export XDEBUG_CONFIG="idekey=phpstorm remote_host=192.168.33.1"
    
  • Set the server configuration. Make sure the name you use matches the server name you configured in PhpStorm:
    export PHP_IDE_CONFIG="serverName=cli"
    
  • Execute drush by substituting "drush.launcher" for the regular drush command. Make sure you use the drush copy from your project. For example:
    ../vendor/drush/drush/drush.launcher migrate-reset-status posts
    

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web