Updating Drupal 8 core with Composer has proven to be a problematic process for many developers. For some, this is nearly as upsetting as the fact that the Composer logo is actually a conductor, and some have abandoned the platform entirely, opting to stick with Drupal 7.

The process isn’t always as simple as running composer update drupal/core and going about your day — the update from 8.3 to 8.4 was notoriously difficult and I recently experienced an issue while updating from 8.4.5 to 8.5.0. In this article, I’ve provided instructions for updating D8 core with Composer, plus some tips for dealing with common issues.

This is especially important now as we await a highly critical security update to all versions of Drupal, to be released on Wednesday, March 28, 2018. This level of security update is quite rare, but the update needs to be implemented on all sites as soon as possible after its release.

As the PSA linked to above notes, the Drupal Security Team will be providing security releases for unsupported minor versions 8.3.x and 8.4.x due to the issues many have encountered when updating from these versions. If you’re still on one of those versions, the update may be more straightforward if you stick with the release for that minor version.

General instructions for updating core

First, let’s cover the steps needed to update Drupal 8 core with Composer.

1. To update the core package, run:

    composer update drupal/core --with-dependencies -v
   
   

   It’s recommended to run the update command with the --with-dependencies flag to update any of Drupal core’s dependencies as well.

2. To capture any included database updates, run drush updb -y.
3. To capture any included configuration changes, run drush config-export -y and commit the changes.

All three of these steps are necessary whenever the core package is updated.

Dealing with errors

Core version doesn’t update

If you run the composer update command but core isn’t updating, edit your composer.json file to include the specific version of core you want, e.g. ^8.5. Then, run the composer update command again.

Composer command outputs errors

Composer may not be able to resolve all of the dependencies of core and will output an error like this:

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Conclusion: don't install drupal/core 8.5.0
    - Conclusion: don't install drupal/core 8.5.0-rc1
    - Conclusion: don't install drupal/core 8.5.0-beta1
    - Conclusion: don't install drupal/core 8.5.0-alpha1
    - Conclusion: don't install drupal/core 8.6.x-dev
    - Conclusion: remove symfony/config v3.2.9
    - Installation request for drupal/core ^8.5 -> satisfiable by drupal/core[8.5.0, 8.5.0-alpha1, 8.5.0-beta1, 8.5.0-rc1, 8.5.x-dev, 8.6.x-dev].
    - Conclusion: don't install symfony/config v3.2.9
    - drupal/core 8.5.x-dev requires symfony/dependency-injection ~3.4.0 -> satisfiable by symfony/dependency-injection[3.4.x-dev, v3.4.0, v3.4.0-BETA1, v3.4.0-BETA2, v3.4.0-BETA3, v3.4.0-BETA4, v3.4.0-RC1, v3.4.0-RC2, v3.4.1, v3.4.2, v3.4.3, v3.4.4, v3.4.5, v3.4.6].
    - symfony/dependency-injection 3.4.x-dev conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-BETA1 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-BETA2 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-BETA3 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-BETA4 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-RC1 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-RC2 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.1 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.2 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.3 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.4 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.5 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.6 conflicts with symfony/config[v3.2.9].
    - Installation request for symfony/config (locked at v3.2.9) -> satisfiable by symfony/config[v3.2.9].

This happens when one of Drupal’s dependencies is updated and the new version requires an updated version of another package. To resolve this, include the dependency package causing the issue in the composer update command. The --with-dependencies flag this will ensure that the dependency’s dependencies are also updated. To fix the error above, I ran:

composer update drupal/core symfony/config --with-dependencies -v

You’re not alone

If you continue to run into problems, the best advice I can give you is to search for the specific update you’re trying to make. Every time I’ve had an issue I’ve been able to find discussions online regarding that specific update and potential resolutions.

In fact, when I got the error above while trying to update to 8.5.0, I found this helpful article by drupal.org user eiriksm and was able to resolve the issue. Check out the article and its comments for more discussion on how to deal with Composer issues when updating Drupal 8 core.

EXPECTED_SIGNATURE=$(wget -q -O - https://composer.github.io/installer.sig) && \
php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" && \
php -r "if (hash_file('SHA384', 'composer-setup.php') === '${EXPECTED_SIGNATURE}') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" && \
php composer-setup.php && \
php -r "unlink('composer-setup.php');" && \
mv composer.phar /usr/local/bin/composer

Our clients are often looking to reach their audiences via email campaigns, and MailChimp is one of the solutions we frequently recommend for this. MailChimp makes it easy to create and manage email campaigns while also providing beneficial analytics on user behavior.

Earlier this year I wrote a blog post showing how to use Composer Manager along with the Mailchimp API v2.0 PHP package to subscribe users to mailing lists in a Drupal 6 or 7 custom module without the need for the Mailchimp contributed module.

However, since then, MailChimp API v3.0 was released and Mailchimp announced that v2.0 (and all prior versions) will no longer be supported after 2016.

So in this blog post, I’ll demonstrate how to accomplish the same objective using the new MailChimp API v3.0, and I’ll expand the tutorial to also include some Drupal 8 specifics.

Background

To quickly summarize the key takeaways from my previous blog posts on Composer Manager and subscribing users to MailChimp lists using the old API:

• Composer is a tool for managing PHP libraries that your project depends on.
• Challenges arise managing project-wide dependencies when custom and contributed modules specify their own unique dependencies.
• Composer Manager is a contributed module for Drupal 7 (and formerly Drupal 6) that addresses these challenges and allows contributed and custom modules to depend on PHP libraries managed via Composer.
• Using a Composer managed PHP package for the MailChimp API, we can easily subscribe users to MailChimp lists in a Drupal custom module without relying on the Mailchimp module.
• While the Mailchimp contributed module is great, sometimes all you need is a simple, lightweight method for subscribing users to mailing lists.

One important development since my previous posts is that Composer Manager has been deprecated for Drupal 8. Improvements introduced in Drupal 8.1.0 allow modules to rely on Composer managed dependencies without the need for the Composer Manager module.

Implementation

There are a few steps we must take so that we can subscribe users to mailing lists in our custom module. We’ll review each of these steps in detail:

• Add the MailChimp API v3.0 PHP library as a dependency of our custom module.
• Ensure that the library is installed for our project.
• Properly use the library in our custom module to subscribe users to mailing lists.

Specify the dependency

ThinkShout maintains the Mailchimp contributed module and we were very excited to see that as part of the effort to “get Drupal off the island” they also released a PHP library for MailChimp API v3.0.

To use this new library, we must specify it as a dependency of our custom module. We do that in a composer.json file that sits in our custom module’s root directory and requires that library via the following code:

{
  "require": {
    "thinkshout/mailchimp-api-php": ">=1.0.3"
  }
}

Install the library

Composer is intended for projects and therefore requires a Drupal site to have a single composer.json, so things get complicated when individual modules specify their own dependencies.

For Drupal 7 sites (or still active Drupal 6 sites), the Composer Manager contributed module handles this by merging the requirements specified by each custom and contributed module’s composer.json files into a single, consolidated, site-wide composer.json file.

So for Drupal 6/7 projects we’ll need Composer Manager installed and enabled.

Once enabled, we can generate the consolidated composer.json and then install all of the site’s dependencies that file specifies (including the MailChimp API v3.0 PHP library specified by our custom module) in one of two ways:

From the command line, we can run the following drush commands:

$ drush composer-json-rebuild
$ drush composer-manager install

Alternatively, we could include the following lines in an update hook:

// Re-build Composer Manager composer.json and run composer update.
drush_composer_manager_composer_json_rebuild();
drush_composer_manager('update');

For Drupal 8 sites, the process is slightly different. As mentioned previously, as of release 8.1.0, Drupal core directly uses Composer to manage dependencies and the Composer Manager module is no longer necessary. For Drupal 8 sites, we should follow the Drupal.org instructions for managing dependencies for a custom project. Following those instructions ensures that all of the site’s dependencies, including the MailChimp library specified by our custom module, are installed.

Use the library

Once we have the MailChimp API v3.0 PHP library installed, we can use it in our custom module to subscribe users to mailing lists.

We suggest creating a dedicated function for subscribing users to email lists which can then be called throughout the custom module. For our purposes, we modeled that function off of the Mailchimp module (version 7.x-4.6) mailchimp_subscribe_process() function.

We implemented the following function, which can be reviewed and modified for your specific purposes:

<?php
/**
 * Add an email to a MailChimp list.
 *
 * This code is based on the 7.x-4.6 version of the Mailchimp module,
 * specifically the mailchimp_subscribe_process() function. That version of
 * the Mailchimp contrib module makes use of the ThinkShout PHP library for
 * version 3.0 of the MailChimp API. See the following for more detail:
 * https://www.drupal.org/project/mailchimp
 * https://github.com/thinkshout/mailchimp-api-php.
 *
 * @see Mailchimp_Lists::subscribe()
 *
 * @param string $api_key
 *   The MailChimp API key.
 * @param string $list_id
 *   The MailChimp list id that the user should be subscribed to.
 * @param string $email
 *   The email address for the user being subscribed to the mailing list.
 */
function mymodule_subscribe_user($api_key, $list_id, $email) {

  try {
    // Set the timeout to something that won't take down the Drupal site:
    $timeout = 60;
    // Get an instance of the MailchimpLists class.
    $mailchimp = new \Mailchimp\MailchimpLists($api_key, 'apikey', $timeout);

    // Use MEMBER_STATUS_PENDING to require double opt-in for the subscriber. Otherwise, use MEMBER_STATUS_SUBSCRIBED.
    $parameters = array(
      'status' => \Mailchimp\MailchimpLists::MEMBER_STATUS_PENDING,
      'email_type' => 'html',
    );

    // Subscribe user to the list.
    $result = $mailchimp->addOrUpdateMember($list_id, $email, $parameters);

    if (isset($result->id)) {
      watchdog('mymodule', '@email was subscribed to list @list.',
        array('@email' => $email, '@list' => $list_id), WATCHDOG_NOTICE
      );
    }
    else {
      watchdog('mymodule', 'A problem occurred subscribing @email to list @list.', array(
        '@email' => $email,
        '@list' => $list_id,
      ), WATCHDOG_WARNING);
    }
  }
  catch (Exception $e) {
    // The user was not subscribed so log to watchdog.
    watchdog('mymodule', 'An error occurred subscribing @email to list @list. Status code @code. "%message"', array(
      '@email' => $email,
      '@list' => $list_id,
      '%message' => $e->getMessage(),
      '@code' => $e->getCode(),
    ), WATCHDOG_ERROR);
  }
}

With that function defined, we can then subscribe an email address to a specific Mailchimp mailing list through the following function call in our custom module:

mymodule_subscribe_user($api_key, $list_id, $email);

Conclusion

By taking advantage of the modern PHP ecosystem built on reusable Composer managed packages, we can easily build or adapt a custom module to subscribe users to mailing lists without the MailChimp contributed module.

Lastly, a special thanks to ThinkShout for their hard work maintaining the MailChimp module and creating the library, on which this approach depends!

Drupal 8 is out, Drupal 8.1 will be out before we know it, but it seems contrib is still catching up. One question that seems to keep coming up is around installing a Drupal 8 module. In this post I will look at the different ways to install a module and resolving dependencies.

Deploy, Search API Solr Search, and Commerce are just a few of the modules with Drupal 8 releases that require dependencies loaded via composer. This means you can’t just download the module’s ZIP file, unzip it in your modules directory, and enable it. You need to install the dependencies.

One simple way to do this is the Composer Manager module. This module has extensive documentation on how to use it. Essentially what it does it merge the composer.json that ships with core and the composer.json files from all the contrib module you have, then downloads the dependencies. You may notice that this will also update core dependencies, but this is a good thing! The core composer.json has been written in such a way that it won’t introduce API breaking dependencies and only uses stable releases. So you will benefit from any bug fixes or security fixes rolled out in these dependencies before they’re rolled out with Drupal.

The two other ways we’re going to look at involve directly using Composer.

Just the dependencies
It’s possible to carry on installing Drupal modules exactly as you always have, download the zip or tarball, then unzip it into your modules directory. As mentioned earlier this will not install the dependencies, therefore you will need to look inside the module’s composer.json file, see what the dependencies are, and install them manually. Let’s take Deploy module as an example, this depends on the dev-master version of relaxedws/replicator. So, go to your Drupal docroot and run the command composer require relaxedws/replicator:dev-master. This will add relaxedws/replicator to Drupal’s composer.json, download it, and put it in the vendor directory ready for the module to make use of. This will not change any other other dependencies you have. Then to update the dependencies you can either run composer update to update relaxedws/replicator and all core dependencies or composer update relaxedws/replicator to just update the relaxedws/replicator package.

The module too
If you install all you Drupal modules via composer, all of the dependencies will automatically be installed too. First you will need to add a new repository, so run composer config repositories.drupal composer https://packagist.drupal-composer.org in your Drupal docroot, this will add the https://packagist.drupal-composer.org repository to your drupal composer.json. Now you can install any module from Drupal.org via composer. So going back to the example of Deploy you can run composer require drupal/deploy:8.1.0-alpha5 in your Drupal docroot and it will install Deploy in the modules directory. It will also install key_value, multiversion, and relaxed, which are all Drupal modules required by Deploy. Furthermore it will install relaxedws/replicator as we know is a PHP package needed for Deploy, and doctrine/couchdb which is a PHP package needed for releaxedws/replicator.

I hope this helps those confused what to do with Drupal 8 modules that have composer dependencies. Now go do the smart thing, rebuild your site using Composer!

Please enable JavaScript to view the comments powered by Disqus.

Now that we have an initiative to get get a composer.json file in each contrib module, we cal start getting them all on Packagist.

Back in April the Drupal community decided on a naming convention for projects on Drupal.org. Projects must use the package name drupal/PROJECT where PROJECT is the part from the URL.

Using drupal as the vendor name in the package name allows us to lock down Packagist to a select list of maintainers. Currently this is the people who submitted a package to Packagist under the drupal vendor name before May 7th when the restriction was added.

The Packagist API currently allows us to update existing packages, so we could quite easily add a hook_node_update and hook_node_insert to drupal.org that will do this when project releases are updated or added. The API doesn’t offer a create endpoint, so we can’t automatically create new packages, an issue on Github has been opened for this, and I’ve started work on a pull request.

Once we have the Packagist pull request done, and the hooks added to drupal.org we can use the API key from Packagist for one of the maintainers (maybe Dries?) to push any new module or existing module updates across. Packagist will read directly from the drupal.org git repo and we’ll finally have one canonical place for all Drupal modules and PHP dependencies.

Drupal.org issue: https://www.drupal.org/node/2547617

Please enable JavaScript to view the comments powered by Disqus.

Over the last few weeks I’ve been spending a lot of time with Drupal 8 and Composer. This has lead me building up a PoC for a client and diving into the issue queues and IRC. In this post I wanted to document some of the processes I’ve been looking at.

Creating a project

The people behind drupal-composer have put together a template, a Drupal project can be started from the template using the command composer create-project drupal-composer/drupal-project:8.x-dev drupal --stability dev --no-interaction. This will create a folder called “drupal”, in there you will find “web” directory containing the Drupal installation. It has also downloaded drush, and two modules, devel and token.

Adding modules

If you open up composer.json in your drupal project folder you will see a repository with the URL https://packagist.drupal-composer.org defined. This is a custom version of Packagist setup by the drupal-composer team. If required packages are not found on Packagist they will get pulled from here.

You will also notice in composer.json under the “require” section is where drupal/token and drupal/devel are added. You can add any module on the Drupal Packagist to this then run the command composer update to update your project and download the newly added modules.

In the “extra” section of composer.json you will see a number of installer paths are added, this tells Composer (via the required composer/installers package) where to put things. You will see everything is going in the web directory, Drupal core in “web/core”, modules in “web/modules/contrib” etc. Therefore, when you call the composer update command to add the new modules you required, these automatically went into the correct Directory ready for Drupal to use. Composer knows these are Drupal modules because the modules have a composer.json files too (often dynamically added by the Drupal Packagist because Drupal doesn’t require modules to have a composer.json yet). In this composer.json the type is set to “drupal-module” for modules, “drupl-theme” for themes, etc.

Patching modules

Greg Anderson went into this in a lot of detail on the Pantheon blog earlier this week, but using the cweagans/composer-patches package you can define a patch file to use. For example, add the following to the “extra” section of your composer.json file to patch the token module:

"patches": { "drupal/token": { "Description for reused fields not correct": "https://www.drupal.org/files/issues/token-Fix_description_for_reused_fields-2497251-5.patch" } } }

You’ll see the package to patch is defined, then within that a name or discription of the patch, followed by the URL for the patch file.

Custom modules

There are a number of ways you can handle custom modules here. You could create a folder at web/modules/custom and just put them in there, or you could add them via composer. In the PoC I’m working on we have many custom modules that will be added to multiple projects. The custom modules have their own git repo, and if they’re not on Packagist or the Drupal Packagist (which they shouldn’t be if they’re custom modules) we need to tell Drupal about this repository. In composer.json under the “repositories” section add something like:

{ "type": "vcs", "url": "https://github.com/timmillwood/couchdb_statistics.git" }

In this case we’re adding a repository that contains the “drupal/couchdb_statistics” package. You could add "drupal/couchdb_statistics": "dev-master" to the “require” section of your composer.json to add this Drupal module to your modules directory.

If you have a lot of custom modules you’re adding to multiple sites it might be worth you setting up Toran Proxy. This is a project by Jordi Boggiano, the guy behind Composer and Packagist. It allows you to proxy your git repos and packagist. You then add your Toran Proxy installation to the “repositories” section of you composer.json, then require any packages you have there. Give Toran Proxy a Github token and it can also grab your private repos too.

One thing to note about repositories is:

> Repositories are only available to the root package and the repositories defined in your dependencies will not be loaded.

For the PoC project mentioned earlier I looked at creating a sub-project which just contained a composer.json. This then required all of the common modules, custom and contrib. However the custom ones were not getting pulled in because all the custom repositories were not defined in the root project, they were only defined in the sub-project. Having Toran Proxy as a single source meant that we could add it to the root project and all dependencies could also get pulled from there.

Post install

You may noticed the drupal-composer template has a scripts directory and this is defined in composer.json as a “post-install-cmd”. This is run after composer.install. The script add settings.php, services.yml and the files directory. You could customise this to do a number of other things. Run drush commands, setup a vagrant box, etc

Summary

Composer is here in Drupal 8 and it’s awesome. You can run, develop and deploy your whole Drupal 8 project with Composer. You can add and patch contrib and custom module, as well as themes, profiles and other PHP packages. The drupal/couchdb_statistics module mentioned earlier requires a couchdb client, this will all get pulled in via composer with a single command.

Move over Drush make, this is how you should be running Drupal 8.

Please enable JavaScript to view the comments powered by Disqus.

Doctrine Annotations is a central part of the Drupal 8 Plugin API. One of its conceptual annoyances is the need for a separate autoloading process for annotations, which in some scenarios leads to duplicated path declarations. Let us see how to remedy to it.

The Annotations loading process

Doctrine Annotations are classes, and the whole Doctrine annotations process, like PHP autoloading in general (the single autoload function in plain PHP, the SPL autoload stack otherwise), relies upon a single global, the AnnotationRegistry class, to which annotation classes can be registered.

This registry supports three registration mechanismes, as described in its documentation:

• direct file registration, with the AnnotationRegistry::registerFile($path) method, which is just a wrapper for require_once $path. This is useful to load annotation files containing multiple annotation classes, like Doctrine ORM's own DoctrineAnnotations.php, without going through multiple autoloads
• namespace(s) registration with the AnnotationRegistry::registerNamespace($ns, $path)/AnnotationRegistry::registerNamespaces($namespaces) methods, which register autoload namespaces to the annotation autoloader, implemented in AnnotationRegistry::loadAnnotationClass.
• custom loaders with the <code>AnnotationRegistry::registerLoader($callable), to handle specific scenarios

Implementation in PSR-0/PSR-4 projects with Composer

In most cases, registration will be handled by the namespace-based registration. However, since the annotations autoloader is not tied to a specific autoloading scheme, it needs to receive a path parameter for each namespace it handles. In a typical PSR-0/PSR-4 project with Composer, it can mean something like this:

<?php
AnnotationRegistry::registerAutoloadNamespace("Symfony\Component\Validator\Constraints", "$basePath/vendor/symfony/validator");
AnnotationRegistry::registerAutoloadNamespace("MyProject\Annotations", "$basePath/src");
?>

As evidenced by that fragment, this means having to specify in code the namespace-to-path mapping already specified in the composer.json file, under the PSR-0/PSR-4 specification, which is a source of extra work and possible errors.

Autoloading error handling

Why is it necessary ? To quote the Doctrine doc:

How are these annotations loaded? From looking at the code you could guess that the ORM Mapping, Assert Validation and the fully qualified annotation can just be loaded using the defined PHP autoloaders. This is not the case however: For error handling reasons every check for class existence inside the AnnotationReader sets the second parameter $autoload of class_exists($name, $autoload) to false. To work flawlessly the AnnotationReader requires silent autoloaders which many autoloaders are not. Silent autoloading is NOT part of the PSR-0 specification for autoloading.

Indeed, the requirement for silent autoloaders is lacking in PSR-0 and had been hotly and repeatedly debated in the PSR-4 discussions (see the PSR-4: The registered autoloader MUST NOT throw exceptions thread for an example), so caution is indeed needed in the absolutely general case.

However, in most cases, the autoloader is known to be silent: Composer's findFile() method is silent, which means this extra caution is not needed in a project where Composer is used for autoloading.

Removing duplication via registerLoader()

So if we want to avoid re-declaring our namespace-to-path mapping and rely on the existing declarations, what can we do ? Create a custom loader and use it with AnnotationRegistry::registerLoader($callable). Since we will want to pass it parameters, and loaders only receive the name of the class to load, we will implement it as a class with the __invoke() magic method.

All this pseudo-loader will have to do will be to answer true for classes in the namespace it handles, to claim it actually found the class, and let the default autoloader work its magic later on, when the Annotations DocParser will try to access the annotation classes.

That way we can parse the example given on the Annotations documentation page without repeating the namespace to path mapping:

That's it: annotation parsing, without duplicated mappings.