Jan 30 2019
Jan 30

If you were an early Drupal 8 adopter you've might have downloaded and installed your Drupal 8 sites by downloading a tarball or using Drush. We did as well, but the benefits of using Composer are so great that it's time to convert those in to being Composer-managed.

Luckily, grasmash has built a great Composer plugin called Composerize Drupal which does all the heavy-lifting for us.

Here's how we did it:

Before you even begin, make sure you branch out $ git checkout -b chore/composerize-drupal

And then we installed the Composer plugin globally:

composer global require grasmash/composerize-drupal

Consider the plugin options available:

  • Use the --exact-versions option if the site is big and complex. Especially if you don't have any good test coverage to ensure your site doesn't break. The option sets the constraints of your composer.json to the exact versions of your currently downloaded modules.

Now we run the command:

composer composerize-drupal --composer-root=. --drupal-root=. --exact-versions

Next:

  • Update your .gitignore and ignore the vendor/, core/ and modules/contrib folders. If the files were already commited you also need to remove them: Ignore files that have already been committed to a Git repository
  • Re-apply your patches! Since all the core code and contrib. modules are managed by Composer you'll need to add those patches to your composer.json:
    Something like this:
    "extra": {
        "enable-patching": true,
        "patches": {
            "drupal/core": {
                "2492171 - Adds transliteration to uploaded file and images": "_kodamera/patches/use_new_transliteration-2492171-72.patch"
           }
       }
       ...

When you run composer install they are automatically applied for you. No more manually work here. Yay!

Do some regression testing and if everything looks fine, you're done! Commit and deploy :)

Mar 25 2018
Mar 25

Updating Drupal 8 core with Composer has proven to be a problematic process for many developers. For some, this is nearly as upsetting as the fact that the Composer logo is actually a conductor, and some have abandoned the platform entirely, opting to stick with Drupal 7.

The process isn’t always as simple as running composer update drupal/core and going about your day — the update from 8.3 to 8.4 was notoriously difficult and I recently experienced an issue while updating from 8.4.5 to 8.5.0. In this article, I’ve provided instructions for updating D8 core with Composer, plus some tips for dealing with common issues.

This is especially important now as we await a highly critical security update to all versions of Drupal, to be released on Wednesday, March 28, 2018. This level of security update is quite rare, but the update needs to be implemented on all sites as soon as possible after its release.

As the PSA linked to above notes, the Drupal Security Team will be providing security releases for unsupported minor versions 8.3.x and 8.4.x due to the issues many have encountered when updating from these versions. If you’re still on one of those versions, the update may be more straightforward if you stick with the release for that minor version.

General instructions for updating core

First, let’s cover the steps needed to update Drupal 8 core with Composer.

  1. To update the core package, run:

     composer update drupal/core --with-dependencies -v
    
    

    It’s recommended to run the update command with the --with-dependencies flag to update any of Drupal core’s dependencies as well.

  2. To capture any included database updates, run drush updb -y.
  3. To capture any included configuration changes, run drush config-export -y and commit the changes.

All three of these steps are necessary whenever the core package is updated.

Dealing with errors

Core version doesn’t update

If you run the composer update command but core isn’t updating, edit your composer.json file to include the specific version of core you want, e.g. ^8.5. Then, run the composer update command again.

Composer command outputs errors

Composer may not be able to resolve all of the dependencies of core and will output an error like this:

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Conclusion: don't install drupal/core 8.5.0
    - Conclusion: don't install drupal/core 8.5.0-rc1
    - Conclusion: don't install drupal/core 8.5.0-beta1
    - Conclusion: don't install drupal/core 8.5.0-alpha1
    - Conclusion: don't install drupal/core 8.6.x-dev
    - Conclusion: remove symfony/config v3.2.9
    - Installation request for drupal/core ^8.5 -> satisfiable by drupal/core[8.5.0, 8.5.0-alpha1, 8.5.0-beta1, 8.5.0-rc1, 8.5.x-dev, 8.6.x-dev].
    - Conclusion: don't install symfony/config v3.2.9
    - drupal/core 8.5.x-dev requires symfony/dependency-injection ~3.4.0 -> satisfiable by symfony/dependency-injection[3.4.x-dev, v3.4.0, v3.4.0-BETA1, v3.4.0-BETA2, v3.4.0-BETA3, v3.4.0-BETA4, v3.4.0-RC1, v3.4.0-RC2, v3.4.1, v3.4.2, v3.4.3, v3.4.4, v3.4.5, v3.4.6].
    - symfony/dependency-injection 3.4.x-dev conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-BETA1 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-BETA2 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-BETA3 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-BETA4 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-RC1 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-RC2 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.1 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.2 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.3 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.4 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.5 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.6 conflicts with symfony/config[v3.2.9].
    - Installation request for symfony/config (locked at v3.2.9) -> satisfiable by symfony/config[v3.2.9].

This happens when one of Drupal’s dependencies is updated and the new version requires an updated version of another package. To resolve this, include the dependency package causing the issue in the composer update command. The --with-dependencies flag this will ensure that the dependency’s dependencies are also updated. To fix the error above, I ran:

composer update drupal/core symfony/config --with-dependencies -v

You’re not alone

If you continue to run into problems, the best advice I can give you is to search for the specific update you’re trying to make. Every time I’ve had an issue I’ve been able to find discussions online regarding that specific update and potential resolutions.

In fact, when I got the error above while trying to update to 8.5.0, I found this helpful article by drupal.org user eiriksm and was able to resolve the issue. Check out the article and its comments for more discussion on how to deal with Composer issues when updating Drupal 8 core.

Jan 27 2018
Jan 27

The problem: XDebug doesn't work for Composer scripts

PhpStorm is quite convenient to debug scripts with XDebug (do you support Derick for giving us XDebug ?): just add a "Run/Debug configuration", choosing the "PHP Script" type, give a few parameters, and you can start debugging your PHP CLI scripts, using breakpoints, evaluations, etc.

Wonderful. So now, let's define such a configuration to debug a Composer script, say a Behat configuration generator from site settings for some current Drupal 8 project. Apply the configuration, run it in debug mode, and ....

...PhpStorm doesn't stop, the script runs and ends, and all breakpoints were ignored. How to actually use breakpoints in the IDE ?

Broken configuration to debug a Composer script

The diagnostic

Let's see how Composer actually starts: <?php
#!/usr/bin/env php
<?phpif (PHP_SAPI !== 'cli') {
    echo
'Warning: Composer should be invoked via the CLI version of PHP, not the '.PHP_SAPI.' SAPI'.PHP_EOL;
}

require

__DIR__.'/../src/bootstrap.php';

use

Composer\Factory;
use
Composer\XdebugHandler;
use
Composer\Console\Application;error_reporting(-1);// Create output for XdebugHandler and Application
$output = Factory::createOutput();$xdebug = new XdebugHandler($output);
$xdebug->check();
unset(
$xdebug);// [...more...]
?>

Now, we can step through this, and notice the Composer script actually runs during the $xdebug->check(); line. What's going on ?

<?php
// In vendor/composer/composer/src/Composer/XdebugHandler.php
   
public function check()
    {
       
$args = explode('|', strval(getenv(self::ENV_ALLOW)), 2);

        if (

$this->needsRestart($args[0])) {
            if (
$this->prepareRestart()) {
               
$command = $this->getCommand();
               
$this->restart($command);
            }

            return;
        }
    

// [...more...]
?>

Here's the crux of the problem: in order to alleviate the extreme slowdown caused by XDebug when running Composer commands, any time Composer is run, it checks for the presence of Xdebug in the running PHP configuration, and if it finds it (the needsRestart() check), it rebuilds a command line with a different PHP configuration ($command = $this->getCommand();), which does not include the xdebug extension, and runs it in the $this->restart($command); call, which actually runs the new command using the passthru mechanism.

<?php
   
protected function restart($command)
    {
       
passthru($command, $exitCode);
   
// [...more...]
?>

Now, since this command no longer runs with Xdebug, there is no way a debugging tool can use it. So how does one fix the problem ?

The solution

Let's go back to the beginning of this check() method:

<?php
class XdebugHandler
{
    const
ENV_ALLOW = 'COMPOSER_ALLOW_XDEBUG'; // [...snip...] public function check()
    {
       
$args = explode('|', strval(getenv(self::ENV_ALLOW)), 2);

        if (

$this->needsRestart($args[0])) {
   
// [...more...]
?>

So the call to needsRestart($args[0] does actually depend on the value of the XdebugHandler::ENV_ALLOW constant, i.e. COMPOSER_ALLOW_XDEBUG. Let's see.

<?php
   
private function needsRestart($allow)
    {
        if (
PHP_SAPI !== 'cli' || !defined('PHP_BINARY')) {
            return
false;
        }

        return empty(

$allow) && $this->loaded;
    }
?>

So, if COMPOSER_ALLOW_XDEBUG is not empty, needsRestart($allow) will return FALSE. In which case, as shown above, it won't cause the new command to be built, and Composer will just proceed with our script and allow our debugging to work.

In practice, this means all we need it to pass a COMPOSER_ALLOW_XDEBUG environment variable with a non-empty value in the PhpStorm run configuration, like this.

Debug configuration for Composer command in PhpStorm

Debugging a Composer script in PhpStorm

Problem solved_! BTW, did you notice this was explained in the Composer vendor/composer/composer/doc/articles/troubleshooting.md documentation ?

[...snip...]
## Xdebug impact on Composer

To improve performance when the xdebug extension is enabled, Composer automatically restarts PHP without it.
You can override this behavior by using an environment variable: `COMPOSER_ALLOW_XDEBUG=1`.
[...more...]

Yeah, me neither, until I solved the problem as described. So let's all just keep in mind to RTFM.

Nov 05 2017
Nov 05
If you need to install the latest version of a composer you can use next bash snippet:
EXPECTED_SIGNATURE=$(wget -q -O - https://composer.github.io/installer.sig) && \
php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" && \
php -r "if (hash_file('SHA384', 'composer-setup.php') === '${EXPECTED_SIGNATURE}') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" && \
php composer-setup.php && \
php -r "unlink('composer-setup.php');" && \
mv composer.phar /usr/local/bin/composer
Using EXPECTED_SIGNATURE variable with the latest available signature value you don't have to hardcode a specific one for comparison on 3rd line.
Jan 14 2017
Jan 14

The problem: packages.drupal.org broken

I was starting my weekly work for the Drupal GraphQL module by the customary composer update --prefer-source -vvv command, ready to watch composer spit out some hundred lines sipping my coffee, but this time something turned out to be wrong:

OK, so packages.drupal.org is down for now. How can we work around this ?

The diagnostic

By default, most of my Drupal 8 project (aren't yours ?) are either based on drupal-composer/drupal-project for actual projects, or a slightly modified drupal/drupal when working on Drupal core or contrib modules like GraphQL. This was the case of the latter model, done by adding specific repositories entries to core composer.json, like:

    "repositories": {
        "fgm_graphql": {
            "type": "vcs",
            "url": "https://github.com/FGM/graphql-drupal"
        },
        "drupal": {
            "type": "composer",
            "url": "https://packages.drupal.org/8"
        }
    },

The explicit vcs entry on top is to enable me to work on my custom fork of the module (from which I send pull requests), no problem here. The issue is with the bottom one, used to download the other package in this project, specifically devel.

When working with such composer repositories, what Composer does is fetch a packages.json path below the url parameter, in this case https://packages.drupal.org/8/packages.json. That files is a list of data providers, basically one more level of metadata about available repositories and distributions (releases), looking like this:

{
   "notify-batch":"\/8\/downloads",
   "providers-url":"\/8\/%package%$%hash%.json",
   "search":"\/8\/search.json?s=%query%",
   "provider-includes":{
      "drupal\/provider-2017-1$%hash%.json":{
         "sha256":"67abb1f57bb826754ae5f4f877f92d2b1b58bfd45a56f4de883f2424be6cf8d5"
      },
      "drupal\/provider-2016-4$%hash%.json":{
         "sha256":"a30289dd8394e5271bd77777bb14b361c5938656f1cddad7fae1c00d5d6ba9c6"
      },
     [ ...snip ...]
      }
   }
}

Composer will then download from each of these providers in turn, hence the URL displayed as returning a 404 at the top of this story. Sure enough, manual checking of the URL returned a 404, even with a cache-buster query added.

Sure enough, the issue was already mentioned on IRC on #drupal : what could be done at this point without being able to touch packages.drupal.org itself ?

The solution: skip one layer

Actually, the answer is already present in the existing repositories clause: vcs-type repositories do not need a "directory"-type service like Packagist or packages.drupal.org, because in essence what these directories do is provide a way to locate sources and dists. But vcs provide the same service, with the limitation that they have to be listed for each project. So let us skip the composer directory and list devel's repository directly:

    "repositories": {
        "fgm_graphql": {
            "type": "vcs",
            "url": "https://github.com/FGM/graphql-drupal"
        },
        "devel": {
            "type": "vcs",
            "url": "https://git.drupal.org/project/devel.git"
        }
    },

Now, a composer require drupal/devel --prefer-source -vvv works normally, no longer needing to parse the broken directory:

Reading ./composer.json
Loading config file /Users/fgm/.composer/auth.json
Loading config file ./composer.json
[...snip...]
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git show-ref --tags --dereference
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git branch --no-color --no-abbrev -v
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git branch --no-color
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git show '8.x-1.x':'composer.json'
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git log -1 --format=%at '8.x-1.x'
Reading composer.json of drupal/devel (5.x-0.1)
Skipped tag 5.x-0.1, invalid tag name
[...snip...]
Reading composer.json of drupal/devel (8.x-1.x)
Reading /Users/fgm/.composer/cache/repo/https---git.drupal.org-project-devel.git/70f62fd0773082a1a4305c6a7c2bccc649bc98a2 from cache
Importing branch 8.x-1.x (dev-8.x-1.x)
[...snip...]
(success)

Time to return to actual work :-)

Nov 15 2016
Nov 15

Our clients are often looking to reach their audiences via email campaigns, and MailChimp is one of the solutions we frequently recommend for this. MailChimp makes it easy to create and manage email campaigns while also providing beneficial analytics on user behavior.

Earlier this year I wrote a blog post showing how to use Composer Manager along with the Mailchimp API v2.0 PHP package to subscribe users to mailing lists in a Drupal 6 or 7 custom module without the need for the Mailchimp contributed module.

However, since then, MailChimp API v3.0 was released and Mailchimp announced that v2.0 (and all prior versions) will no longer be supported after 2016.

So in this blog post, I’ll demonstrate how to accomplish the same objective using the new MailChimp API v3.0, and I’ll expand the tutorial to also include some Drupal 8 specifics.

Background

To quickly summarize the key takeaways from my previous blog posts on Composer Manager and subscribing users to MailChimp lists using the old API:

  • Composer is a tool for managing PHP libraries that your project depends on.
  • Challenges arise managing project-wide dependencies when custom and contributed modules specify their own unique dependencies.
  • Composer Manager is a contributed module for Drupal 7 (and formerly Drupal 6) that addresses these challenges and allows contributed and custom modules to depend on PHP libraries managed via Composer.
  • Using a Composer managed PHP package for the MailChimp API, we can easily subscribe users to MailChimp lists in a Drupal custom module without relying on the Mailchimp module.
  • While the Mailchimp contributed module is great, sometimes all you need is a simple, lightweight method for subscribing users to mailing lists.

One important development since my previous posts is that Composer Manager has been deprecated for Drupal 8. Improvements introduced in Drupal 8.1.0 allow modules to rely on Composer managed dependencies without the need for the Composer Manager module.

Implementation

There are a few steps we must take so that we can subscribe users to mailing lists in our custom module. We’ll review each of these steps in detail:

  • Add the MailChimp API v3.0 PHP library as a dependency of our custom module.
  • Ensure that the library is installed for our project.
  • Properly use the library in our custom module to subscribe users to mailing lists.

Specify the dependency

ThinkShout maintains the Mailchimp contributed module and we were very excited to see that as part of the effort to “get Drupal off the island” they also released a PHP library for MailChimp API v3.0.

To use this new library, we must specify it as a dependency of our custom module. We do that in a composer.json file that sits in our custom module’s root directory and requires that library via the following code:

{
  "require": {
    "thinkshout/mailchimp-api-php": ">=1.0.3"
  }
}

Install the library

Composer is intended for projects and therefore requires a Drupal site to have a single composer.json, so things get complicated when individual modules specify their own dependencies.

For Drupal 7 sites (or still active Drupal 6 sites), the Composer Manager contributed module handles this by merging the requirements specified by each custom and contributed module’s composer.json files into a single, consolidated, site-wide composer.json file.

So for Drupal 6/7 projects we’ll need Composer Manager installed and enabled.

Once enabled, we can generate the consolidated composer.json and then install all of the site’s dependencies that file specifies (including the MailChimp API v3.0 PHP library specified by our custom module) in one of two ways:

From the command line, we can run the following drush commands:

$ drush composer-json-rebuild
$ drush composer-manager install

Alternatively, we could include the following lines in an update hook:

// Re-build Composer Manager composer.json and run composer update.
drush_composer_manager_composer_json_rebuild();
drush_composer_manager('update');

For Drupal 8 sites, the process is slightly different. As mentioned previously, as of release 8.1.0, Drupal core directly uses Composer to manage dependencies and the Composer Manager module is no longer necessary. For Drupal 8 sites, we should follow the Drupal.org instructions for managing dependencies for a custom project. Following those instructions ensures that all of the site’s dependencies, including the MailChimp library specified by our custom module, are installed.

Use the library

Once we have the MailChimp API v3.0 PHP library installed, we can use it in our custom module to subscribe users to mailing lists.

We suggest creating a dedicated function for subscribing users to email lists which can then be called throughout the custom module. For our purposes, we modeled that function off of the Mailchimp module (version 7.x-4.6) mailchimp_subscribe_process() function.

We implemented the following function, which can be reviewed and modified for your specific purposes:

<?php
/**
 * Add an email to a MailChimp list.
 *
 * This code is based on the 7.x-4.6 version of the Mailchimp module,
 * specifically the mailchimp_subscribe_process() function. That version of
 * the Mailchimp contrib module makes use of the ThinkShout PHP library for
 * version 3.0 of the MailChimp API. See the following for more detail:
 * https://www.drupal.org/project/mailchimp
 * https://github.com/thinkshout/mailchimp-api-php.
 *
 * @see Mailchimp_Lists::subscribe()
 *
 * @param string $api_key
 *   The MailChimp API key.
 * @param string $list_id
 *   The MailChimp list id that the user should be subscribed to.
 * @param string $email
 *   The email address for the user being subscribed to the mailing list.
 */
function mymodule_subscribe_user($api_key, $list_id, $email) {

  try {
    // Set the timeout to something that won't take down the Drupal site:
    $timeout = 60;
    // Get an instance of the MailchimpLists class.
    $mailchimp = new \Mailchimp\MailchimpLists($api_key, 'apikey', $timeout);

    // Use MEMBER_STATUS_PENDING to require double opt-in for the subscriber. Otherwise, use MEMBER_STATUS_SUBSCRIBED.
    $parameters = array(
      'status' => \Mailchimp\MailchimpLists::MEMBER_STATUS_PENDING,
      'email_type' => 'html',
    );

    // Subscribe user to the list.
    $result = $mailchimp->addOrUpdateMember($list_id, $email, $parameters);

    if (isset($result->id)) {
      watchdog('mymodule', '@email was subscribed to list @list.',
        array('@email' => $email, '@list' => $list_id), WATCHDOG_NOTICE
      );
    }
    else {
      watchdog('mymodule', 'A problem occurred subscribing @email to list @list.', array(
        '@email' => $email,
        '@list' => $list_id,
      ), WATCHDOG_WARNING);
    }
  }
  catch (Exception $e) {
    // The user was not subscribed so log to watchdog.
    watchdog('mymodule', 'An error occurred subscribing @email to list @list. Status code @code. "%message"', array(
      '@email' => $email,
      '@list' => $list_id,
      '%message' => $e->getMessage(),
      '@code' => $e->getCode(),
    ), WATCHDOG_ERROR);
  }
}

With that function defined, we can then subscribe an email address to a specific Mailchimp mailing list through the following function call in our custom module:

mymodule_subscribe_user($api_key, $list_id, $email);

Conclusion

By taking advantage of the modern PHP ecosystem built on reusable Composer managed packages, we can easily build or adapt a custom module to subscribe users to mailing lists without the MailChimp contributed module.

Lastly, a special thanks to ThinkShout for their hard work maintaining the MailChimp module and creating the library, on which this approach depends!

Apr 01 2016
Apr 01

The April 2016 issue of php[architect] magazine is out! This issue we take a look at how Drupal is using tools and techniques from the PHP community.

This issue also includes articles on how to easily generate documentation, advice on learning new frameworks, what it means to be a leader in the PHP community, and how to use PHPStorm to improve your code.

Download a FREE article and get your issue today.

Oscar still remembers downloading an early version of the Apache HTTP server at the end of 1995, and promptly asking "Ok, what's this good for?" He started learning PHP in 2000 and hasn't stopped since. He's worked with Drupal, WordPress, Zend Framework, and bespoke PHP, to name a few. Follow him on Google+.
Feb 18 2016
Feb 18

Drupal 8 logo

The recommended approach to getting started with Drupal 8 is now via Composer. An official project template has been created for this. We will create our project directly using the template, which is also available on Packagist.

To create a new project based on this template we can run the following Composer command:

composer create-project drupal-composer/drupal-project:8.x-dev my_project --stability dev --no-interaction

--ADVERTISEMENT--

This Composer command will pull in the template from Packagist and run a few Drupal specific scripts to prepare our project for installation. The only thing left to do is point our browser to the web/ directory (since that is where the index.php file is) and run the installer as usual.

This template comes with a /web folder that contains, among other things, the main folders of a Drupal installation that are no longer considered part of Drupal core (such as the index.php file or the modules and themes folders). Additionally, it comes with an autoload.php file used by Drupal that simply points to the Composer vendor/ directory, one folder up. So all PHP libraries are now handled from one single place.

The template’s composer.json file requires the latest stable Drupal core + some additional helper tools such as Drush and the Drupal Console. Additionally, it adds the Drupal specific Packagist repository from where we can install Drupal contributed modules, themes and profiles (that get automatically installed in the right place).

If we want to add a Drupal contributed module, we need to find it on the Drupal Packagist and require it in our project via Composer:

composer require drupal/ctools

This will add the Ctools module directly to our web/modules/ directory and update our composer.json file.

The project template also comes with a .gitignore file that keeps Drupal core and all the contributed packages outside of Git, similar to the regular vendor/ packages. So based on an updated composer.json file, we can maintain a smaller Git repository and recreate our project any time. A lot of the benefits of Drush Make have now been incorporated into a Composer flow.

Conclusion

Drupal 8 has come a long way in catching up with other major PHP software. The possibility of fully managing it via Composer, either as a main project or even just as part of a bigger set of applications, is a testament to the community effort that went in.

Daniel Sipos

Meet the author

Daniel Sipos is a Drupal developer who lives in Brussels, Belgium. He works professionally with Drupal but likes to use other PHP frameworks and technologies as well. He runs webomelette.com, a Drupal blog where he writes articles and tutorials about Drupal development, theming and site building.
Feb 03 2016
Feb 03

As of 1:23pm GMT on Febuary 3rd 2016 there are no dependencies in the Drupal 8.1.x git branch.

Why?

There was no reason to have them there. It makes the repository bigger, and therefore takes longer to download. No one else does it.

What does it mean?

Not much really.

If you download Drupal via the zip file or tarball then the drupal.org packager will run composer so the dependencies will be there for you.

If you submit a Drupal core patch on drupal.org then DrupalCi testbot will run composer install.

It’s only if you clone Drupal 8.1.x or higher directly from git, you will need to run composer install in the Drupal root directory

Top tip: composer install --prefer-source --no-interaction can often be quicker.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Jan 22 2016
Jan 22

Drupal 8 is out, Drupal 8.1 will be out before we know it, but it seems contrib is still catching up. One question that seems to keep coming up is around installing a Drupal 8 module. In this post I will look at the different ways to install a module and resolving dependencies.

Deploy, Search API Solr Search, and Commerce are just a few of the modules with Drupal 8 releases that require dependencies loaded via composer. This means you can’t just download the module’s ZIP file, unzip it in your modules directory, and enable it. You need to install the dependencies.

One simple way to do this is the Composer Manager module. This module has extensive documentation on how to use it. Essentially what it does it merge the composer.json that ships with core and the composer.json files from all the contrib module you have, then downloads the dependencies. You may notice that this will also update core dependencies, but this is a good thing! The core composer.json has been written in such a way that it won’t introduce API breaking dependencies and only uses stable releases. So you will benefit from any bug fixes or security fixes rolled out in these dependencies before they’re rolled out with Drupal.

The two other ways we’re going to look at involve directly using Composer.

Just the dependencies
It’s possible to carry on installing Drupal modules exactly as you always have, download the zip or tarball, then unzip it into your modules directory. As mentioned earlier this will not install the dependencies, therefore you will need to look inside the module’s composer.json file, see what the dependencies are, and install them manually. Let’s take Deploy module as an example, this depends on the dev-master version of relaxedws/replicator. So, go to your Drupal docroot and run the command composer require relaxedws/replicator:dev-master. This will add relaxedws/replicator to Drupal’s composer.json, download it, and put it in the vendor directory ready for the module to make use of. This will not change any other other dependencies you have. Then to update the dependencies you can either run composer update to update relaxedws/replicator and all core dependencies or composer update relaxedws/replicator to just update the relaxedws/replicator package.

The module too
If you install all you Drupal modules via composer, all of the dependencies will automatically be installed too. First you will need to add a new repository, so run composer config repositories.drupal composer https://packagist.drupal-composer.org in your Drupal docroot, this will add the https://packagist.drupal-composer.org repository to your drupal composer.json. Now you can install any module from Drupal.org via composer. So going back to the example of Deploy you can run composer require drupal/deploy:8.1.0-alpha5 in your Drupal docroot and it will install Deploy in the modules directory. It will also install key_value, multiversion, and relaxed, which are all Drupal modules required by Deploy. Furthermore it will install relaxedws/replicator as we know is a PHP package needed for Deploy, and doctrine/couchdb which is a PHP package needed for releaxedws/replicator.

I hope this helps those confused what to do with Drupal 8 modules that have composer dependencies. Now go do the smart thing, rebuild your site using Composer!

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Jan 22 2016
Jan 22

Overview

In my last blog post, I wrote about the virtues of Composer Manager and how it allows modules to depend on PHP libraries managed via Composer. Basically, Composer Manager allows us to easily use PHP libraries that exist outside of the Drupal ecosystem within our own projects.

In this post, I’ll show you how we:

Custom vs. Contrib?

But first, why didn’t we just use the MailChimp contributed module? Contributed modules are often a great option and offer many benefits, such as security, maintenance, and flexibility.

But there is a cost to installing all those contributed modules. As The Definitive Guide to Drupal 7 explains “The more modules you install, the worse your web site will perform.”

With each installed module comes more code to load and execute, and more memory consumption. And in some cases, contributed modules add complexity and features that just aren’t necessary for the required task.

In our case, the decision to go with a custom solution was easy:

  • The MailChimp contributed module had many features we didn’t need.
  • We were already using Composer Manager on the project to manage other module dependencies.
  • The custom module we were building already included logic to determine when to subscribe users to mailing lists (don’t worry, we made sure they opted in!)

All we needed was a simple, lightweight method for subscribing a given user to a specific MailChimp mailing list.

Implementation

We were able to achieve this by adding the MailChimp PHP library as a dependency of our custom module. We were then able to make a simple call using the API to subscribe a user to the mailing list. We implemented this via the following code.

First, in our module’s root directory we created a composer.json file that specified the MailChimp PHP library as a dependency:

{
  "require": {
    "mailchimp/mailchimp": "*"
  }
}

We then installed the Mailchimp API using the Composer Manager drush commands:

$ drush composer-json-rebuild
$ drush composer-manager install

As explained in my last post, the first command builds (or rebuilds) the consolidated project wide composer.json file and the second command installs the dependencies.

Next, we created a function in our custom module to subscribe a user to a MailChimp mailing list.

<?php
/**
 * Add an email to a MailChimp list.
 *
 * @param string $api_key
 *   The MailChimp API key.
 * @param string $list_id
 *   The MailChimp list id that the user should be subscribed to.
 * @param string $email
 *   The email address for the user being subscribed to the mailing list.
 */
function my_module_subscribe_user($api_key, $list_id, $email) {

  $mailchimp = new Mailchimp($api_key);

  try {
    $result = $mailchimp->lists->subscribe($list_id, array('email' => $email));
  }
  catch(Exception $e) {
    watchdog('my_module', 'User with email %email not subscribed to list %list_id', array('%email' => $email, '%list_id' => $list_id), WATCHDOG_WARNING);
  }
}

With that function defined, we could then subscribe any user to any mailing list by simply calling

<?php
my_module_subscribe_user($api_key, $list_id, $email);

Conclusion

That’s it! A nice, simple, and clean approach to subscribing users to a MailChimp mailing list that doesn’t require installation of the MailChimp contributed module.

We hope you’re as excited as we are at the opportunities Composer and Composer Manager afford us to take advantage of PHP libraries and projects that exist outside of the Drupal ecosystem.

Aug 10 2015
Aug 10

Now that we have an initiative to get get a composer.json file in each contrib module, we cal start getting them all on Packagist.

Back in April the Drupal community decided on a naming convention for projects on Drupal.org. Projects must use the package name drupal/PROJECT where PROJECT is the part from the URL.

Using drupal as the vendor name in the package name allows us to lock down Packagist to a select list of maintainers. Currently this is the people who submitted a package to Packagist under the drupal vendor name before May 7th when the restriction was added.

The Packagist API currently allows us to update existing packages, so we could quite easily add a hook_node_update and hook_node_insert to drupal.org that will do this when project releases are updated or added. The API doesn’t offer a create endpoint, so we can’t automatically create new packages, an issue on Github has been opened for this, and I’ve started work on a pull request.

Once we have the Packagist pull request done, and the hooks added to drupal.org we can use the API key from Packagist for one of the maintainers (maybe Dries?) to push any new module or existing module updates across. Packagist will read directly from the drupal.org git repo and we’ll finally have one canonical place for all Drupal modules and PHP dependencies.

Drupal.org issue: https://www.drupal.org/node/2547617

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Aug 07 2015
Aug 07

Drupal 8 has seen a lot of love for Composer. As various posts have mentioned, it’s possible to build a whole Drupal site with little more than a composer.json file. However contrib needs the same love too.

In this post I want to talk about step one, and it requires little effort at all.

For module maintainers

Add a composer.json file to your Drupal 8 module. The example below gives everything you need. The name uses “drupal” as the vendor namespace, followed by the module name after the slash. The description provides more information on the module. The type tells composer it is specifically a Drupal module (there are also types for themes, profiles and more), this allows developers to put “drupal-module” packages into the modules directory rather than the vendor directory on composer install. Finally we add Drupal’s license of GPL 2.0+.

{ "name": "drupal/example", "description": "This is an example composer.json for example module.", "type": "drupal-module", "license": "GPL-2.0+" }

For everyone else

  • Tell everyone.
  • Tweet about the issue or this post.
  • See if modules with a Drupal 8 release have a composer.json, if not add an issue for it.

What does it mean

Once all modules have a composer.json we can add modules using composer more easily. We can also start step two, which is to get all Drupal 8 modules on Drupal.org listed on Packagist.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Jul 24 2015
Jul 24

Over the last few weeks I’ve been spending a lot of time with Drupal 8 and Composer. This has lead me building up a PoC for a client and diving into the issue queues and IRC. In this post I wanted to document some of the processes I’ve been looking at.

Creating a project

The people behind drupal-composer have put together a template, a Drupal project can be started from the template using the command composer create-project drupal-composer/drupal-project:8.x-dev drupal --stability dev --no-interaction. This will create a folder called “drupal”, in there you will find “web” directory containing the Drupal installation. It has also downloaded drush, and two modules, devel and token.

Adding modules

If you open up composer.json in your drupal project folder you will see a repository with the URL https://packagist.drupal-composer.org defined. This is a custom version of Packagist setup by the drupal-composer team. If required packages are not found on Packagist they will get pulled from here.

You will also notice in composer.json under the “require” section is where drupal/token and drupal/devel are added. You can add any module on the Drupal Packagist to this then run the command composer update to update your project and download the newly added modules.

In the “extra” section of composer.json you will see a number of installer paths are added, this tells Composer (via the required composer/installers package) where to put things. You will see everything is going in the web directory, Drupal core in “web/core”, modules in “web/modules/contrib” etc. Therefore, when you call the composer update command to add the new modules you required, these automatically went into the correct Directory ready for Drupal to use. Composer knows these are Drupal modules because the modules have a composer.json files too (often dynamically added by the Drupal Packagist because Drupal doesn’t require modules to have a composer.json yet). In this composer.json the type is set to “drupal-module” for modules, “drupl-theme” for themes, etc.

Patching modules

Greg Anderson went into this in a lot of detail on the Pantheon blog earlier this week, but using the cweagans/composer-patches package you can define a patch file to use. For example, add the following to the “extra” section of your composer.json file to patch the token module:

"patches": { "drupal/token": { "Description for reused fields not correct": "https://www.drupal.org/files/issues/token-Fix_description_for_reused_fields-2497251-5.patch" } } }

You’ll see the package to patch is defined, then within that a name or discription of the patch, followed by the URL for the patch file.

Custom modules

There are a number of ways you can handle custom modules here. You could create a folder at web/modules/custom and just put them in there, or you could add them via composer. In the PoC I’m working on we have many custom modules that will be added to multiple projects. The custom modules have their own git repo, and if they’re not on Packagist or the Drupal Packagist (which they shouldn’t be if they’re custom modules) we need to tell Drupal about this repository. In composer.json under the “repositories” section add something like:

{ "type": "vcs", "url": "https://github.com/timmillwood/couchdb_statistics.git" }

In this case we’re adding a repository that contains the “drupal/couchdb_statistics” package. You could add "drupal/couchdb_statistics": "dev-master" to the “require” section of your composer.json to add this Drupal module to your modules directory.

If you have a lot of custom modules you’re adding to multiple sites it might be worth you setting up Toran Proxy. This is a project by Jordi Boggiano, the guy behind Composer and Packagist. It allows you to proxy your git repos and packagist. You then add your Toran Proxy installation to the “repositories” section of you composer.json, then require any packages you have there. Give Toran Proxy a Github token and it can also grab your private repos too.

One thing to note about repositories is:

> Repositories are only available to the root package and the repositories defined in your dependencies will not be loaded.

For the PoC project mentioned earlier I looked at creating a sub-project which just contained a composer.json. This then required all of the common modules, custom and contrib. However the custom ones were not getting pulled in because all the custom repositories were not defined in the root project, they were only defined in the sub-project. Having Toran Proxy as a single source meant that we could add it to the root project and all dependencies could also get pulled from there.

Post install

You may noticed the drupal-composer template has a scripts directory and this is defined in composer.json as a “post-install-cmd”. This is run after composer.install. The script add settings.php, services.yml and the files directory. You could customise this to do a number of other things. Run drush commands, setup a vagrant box, etc

Summary

Composer is here in Drupal 8 and it’s awesome. You can run, develop and deploy your whole Drupal 8 project with Composer. You can add and patch contrib and custom module, as well as themes, profiles and other PHP packages. The drupal/couchdb_statistics module mentioned earlier requires a couchdb client, this will all get pulled in via composer with a single command.

Move over Drush make, this is how you should be running Drupal 8.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Feb 27 2015
Feb 27

There are thousands of situations in which you do not want to reinvent the wheel. It is a well known principle in Software Engineering, but not always well applied/known into the Drupal world.

Let’s say for example, that you have a url that you want to convert from relative to absolute. It is a typical scenario when you are working with Web (but not just Web) crawlers. Well, you could start building your own library to achieve the functionality you are looking for, packaging all in a Drupal module format. It is an interesting challenge indeed but, unless for training or learning purposes, why wasting your time when someone else has already done it instead of just focussing on the real problem? Especially if your main app purpose is not that secondary problem (the url converter).

What’s more, if you reuse libraries and open source code, you’ll probably find yourself in the situation in which you could need an small improvement in that nice library you are using. Contributing your changes back you are closing the circle of the open source, the reason why the open source is here to stay and conquer the world (diabolical laugh here).

That’s another one of the main reasons why lot’s of projects are moving to the Composer/Symfony binomium, stop working as isolated projects and start working as global projects that can share code and knowledge between many other projects. It’s a pattern followed by Drupal, to name but one, and also by projects like like phpBB, ezPublish, Laravel, Magento,Piwik, …

Composer and friends

Coming back to our crawler and the de-relativizer library that we are going to need, at this point we get to know Composer. Composer is a great tool for using third party libraries and, of course, for contributing back those of your own. In our web crawler example, net_url2 does a the job just beautifully.

Nice, but at this point you must be wondering… What does this have to do with Drupal, if any at all? Well, in fact, as everyone knows, Drupal 8 is being (re)built following this same principle (DRY or don’t repeat yourself) with an strong presence of the great Symfony 2 components in the core. Advantages? Lots of them, as we were pointing out, but that’s the purpose of another discussion

The point here is that you don’t need to wait for Drupal 8, and what’s more, you can start applying some of this principles in your Drupal 7 libraries, making your future transition to Drupal 8 even easier.

Let’s rock and roll

So, using a php library or a Symfony component in Drupal 7 is quite simple. Just:

  1. Install composer manager
  2. Create a composer.json file in your custom module folder
  3. Place the content (which by the way, you’ll find quite familiar if you’ve already worked with Symfony / composer yaml’s):
    "require": {
      "pear/net_url2": "2.0.x-dev"
     }
    
    
  4. enable the custom module

And that’s it basically. At this point we simply need to tell drupal to generate the main composer.json. That’s basically a composer file generated from the composer.json found in each one of the modules that include a composer themselves.

Lets generate that file:

drush composer-rebuild

At this point we have the main composer file, normally in a vendor folder (if will depend on the composer manager settings).

Now, let’s make some composer magic :

drush composer update

At this point, inside the vendors folder we should now have a classmap, containing amongst others our newly included library.

Hopefully all has gone well, and just like magic, the class net_url2 is there to be used in our modules. Something like :

$base = new Net_URL2($absoluteURL);

Just remember to add the library to your class. Something like:

use Net_URL2;

In the next post we’ll be doing some more exciting stuff. We will create some code that will live in a php library, completely decoupled but at the same time fully integrated with Drupal. All using Composer magic to allow the integration.

Why? Again, many reasons like:

  1. Being ready for Drupal 8 (just lift libraries from D7 or D6 to D8),
  2. Decoupling things so we code things that are ready to use not just in Drupal, and
  3. Opening the door to other worlds to colaborate with our Drupal world, …
  4. Why not use Dependency Injection in Drupal (as it already happens in D8)? What about using the Symfony Service container? Or something more light like Pimple?
  5. Choose between many other reasons…

See you in my next article about Drupal, Composer and friends, on the meantime, be good :-).

Updated: Clarified that we are talking about PHP Libraries and / or Symfony components instead of bundles. Thanks to @drrotmos and @Ross for your comments.

Feb 16 2014
Feb 16

Doctrine Annotations is a central part of the Drupal 8 Plugin API. One of its conceptual annoyances is the need for a separate autoloading process for annotations, which in some scenarios leads to duplicated path declarations. Let us see how to remedy to it.

The Annotations loading process

Doctrine Annotations are classes, and the whole Doctrine annotations process, like PHP autoloading in general (the single autoload function in plain PHP, the SPL autoload stack otherwise), relies upon a single global, the AnnotationRegistry class, to which annotation classes can be registered.

This registry supports three registration mechanismes, as described in its documentation:

  • direct file registration, with the AnnotationRegistry::registerFile($path) method, which is just a wrapper for require_once $path. This is useful to load annotation files containing multiple annotation classes, like Doctrine ORM's own DoctrineAnnotations.php, without going through multiple autoloads
  • namespace(s) registration with the AnnotationRegistry::registerNamespace($ns, $path)/AnnotationRegistry::registerNamespaces($namespaces) methods, which register autoload namespaces to the annotation autoloader, implemented in AnnotationRegistry::loadAnnotationClass.
  • custom loaders with the <code>AnnotationRegistry::registerLoader($callable), to handle specific scenarios

Implementation in PSR-0/PSR-4 projects with Composer

In most cases, registration will be handled by the namespace-based registration. However, since the annotations autoloader is not tied to a specific autoloading scheme, it needs to receive a path parameter for each namespace it handles. In a typical PSR-0/PSR-4 project with Composer, it can mean something like this:

<?php
AnnotationRegistry
::registerAutoloadNamespace("Symfony\Component\Validator\Constraints", "$basePath/vendor/symfony/validator");
AnnotationRegistry::registerAutoloadNamespace("MyProject\Annotations", "$basePath/src");
?>

As evidenced by that fragment, this means having to specify in code the namespace-to-path mapping already specified in the composer.json file, under the PSR-0/PSR-4 specification, which is a source of extra work and possible errors.

Autoloading error handling

Why is it necessary ? To quote the Doctrine doc:

How are these annotations loaded? From looking at the code you could guess that the ORM Mapping, Assert Validation and the fully qualified annotation can just be loaded using the defined PHP autoloaders. This is not the case however: For error handling reasons every check for class existence inside the AnnotationReader sets the second parameter $autoload of class_exists($name, $autoload) to false. To work flawlessly the AnnotationReader requires silent autoloaders which many autoloaders are not. Silent autoloading is NOT part of the PSR-0 specification for autoloading.

Indeed, the requirement for silent autoloaders is lacking in PSR-0 and had been hotly and repeatedly debated in the PSR-4 discussions (see the PSR-4: The registered autoloader MUST NOT throw exceptions thread for an example), so caution is indeed needed in the absolutely general case.

However, in most cases, the autoloader is known to be silent: Composer's findFile() method is silent, which means this extra caution is not needed in a project where Composer is used for autoloading.

Removing duplication via registerLoader()

So if we want to avoid re-declaring our namespace-to-path mapping and rely on the existing declarations, what can we do ? Create a custom loader and use it with AnnotationRegistry::registerLoader($callable). Since we will want to pass it parameters, and loaders only receive the name of the class to load, we will implement it as a class with the __invoke() magic method.

All this pseudo-loader will have to do will be to answer true for classes in the namespace it handles, to claim it actually found the class, and let the default autoloader work its magic later on, when the Annotations DocParser will try to access the annotation classes.

That way we can parse the example given on the Annotations documentation page without repeating the namespace to path mapping:

That's it: annotation parsing, without duplicated mappings.

Jan 04 2014
Jan 04

Yesterday, Seldaek committed PSR-4 support to Composer, as his New Year's Day gift to the PHP community.

So, since I always supported PSR-4 in FIG discussions, and after the tl;dr discussion about it regarding autoloading in Drupal 8, I jumped on the occasion and converted the code base for PlusVite to PSR-4 to get an idea of how PSR-4 "felt" in practice.

A the documentation explains, to convert from PSR-0 to PSR-4 to shorten paths, the change is extremely simple: just edit your composer.json and change it like this:

  PSR-0 PSR-4 with PSR-0 layout Pure PSR-4 \Foo\Bar\Bazclass is in src/Foo/Bar/Baz.php src/Foo/Bar/Baz.php src/Baz.php composer.json contains "psr-0": { "Foo\\Bar\\": "src/" } "psr-4": { "Foo\\Bar\\": "src/Foo/Bar/" } "psr-4": { "Foo\\Bar\\": "src/" }

Of course, removing two levels of directories provided the expected small degree of instant gratification, making everything seem simpler. But then, doubt appears: maybe this is just because I've been using PSR-0 for too long (3 years, already ?), but now, seeing these namespaced classes/interfaces/traits in short paths like src/Baz.php for a simple project like this seems to be hiding information, degrading DX instead of improving it. Eewww !

Now, in Drupal 8, the situation is different: classes core modules are located under

core/modules/(module)/lib/Drupal/(module)/(subnamespace hierarchy)/someclass.php

while PSR-4 will change this to

core/modules/(module)/lib/(subnamespace hierarchy)/someclass.php

so the "hiding" effect will not be the same, since the name of the module (hence the namespace) will be in plain sight. But still, this is an unexpected feeling and I hope we will not regret it later on.

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web