If you were an early Drupal 8 adopter you've might have downloaded and installed your Drupal 8 sites by downloading a tarball or using Drush. We did as well, but the benefits of using Composer are so great that it's time to convert those in to being Composer-managed.
Luckily, grasmash has built a great Composer plugin called Composerize Drupal which does all the heavy-lifting for us.
Here's how we did it:
Before you even begin, make sure you branch out $ git checkout -b chore/composerize-drupal
And then we installed the Composer plugin globally:
composer global require grasmash/composerize-drupal
Consider the plugin options available:
- Use the
--exact-versions
option if the site is big and complex. Especially if you don't have any good test coverage to ensure your site doesn't break. The option sets the constraints of your composer.json to the exact versions of your currently downloaded modules.
Now we run the command:
composer composerize-drupal --composer-root=. --drupal-root=. --exact-versions
Next:
- Update your
.gitignore
and ignore thevendor/
,core/
andmodules/contrib
folders. If the files were already commited you also need to remove them: Ignore files that have already been committed to a Git repository - Re-apply your patches! Since all the core code and contrib. modules are managed by Composer you'll need to add those patches to your
composer.json
:
Something like this:
"extra": {
"enable-patching": true,
"patches": {
"drupal/core": {
"2492171 - Adds transliteration to uploaded file and images": "_kodamera/patches/use_new_transliteration-2492171-72.patch"
}
}
...
When you run composer install
they are automatically applied for you. No more manually work here. Yay!
Do some regression testing and if everything looks fine, you're done! Commit and deploy :)
Updating Drupal 8 core with Composer has proven to be a problematic process for many developers. For some, this is nearly as upsetting as the fact that the Composer logo is actually a conductor, and some have abandoned the platform entirely, opting to stick with Drupal 7.
The process isn’t always as simple as running composer update drupal/core
and going about your day — the update from 8.3 to 8.4 was notoriously difficult and I recently experienced an issue while updating from 8.4.5 to 8.5.0. In this article, I’ve provided instructions for updating D8 core with Composer, plus some tips for dealing with common issues.
This is especially important now as we await a highly critical security update to all versions of Drupal, to be released on Wednesday, March 28, 2018. This level of security update is quite rare, but the update needs to be implemented on all sites as soon as possible after its release.
As the PSA linked to above notes, the Drupal Security Team will be providing security releases for unsupported minor versions 8.3.x and 8.4.x due to the issues many have encountered when updating from these versions. If you’re still on one of those versions, the update may be more straightforward if you stick with the release for that minor version.
General instructions for updating core
First, let’s cover the steps needed to update Drupal 8 core with Composer.
-
To update the core package, run:
composer update drupal/core --with-dependencies -v
It’s recommended to run the update command with the
--with-dependencies
flag to update any of Drupal core’s dependencies as well. - To capture any included database updates, run
drush updb -y
. - To capture any included configuration changes, run
drush config-export -y
and commit the changes.
All three of these steps are necessary whenever the core package is updated.
Dealing with errors
Core version doesn’t update
If you run the composer update
command but core isn’t updating, edit your composer.json
file to include the specific version of core you want, e.g. ^8.5
. Then, run the composer update
command again.
Composer command outputs errors
Composer may not be able to resolve all of the dependencies of core and will output an error like this:
Your requirements could not be resolved to an installable set of packages.
Problem 1
- Conclusion: don't install drupal/core 8.5.0
- Conclusion: don't install drupal/core 8.5.0-rc1
- Conclusion: don't install drupal/core 8.5.0-beta1
- Conclusion: don't install drupal/core 8.5.0-alpha1
- Conclusion: don't install drupal/core 8.6.x-dev
- Conclusion: remove symfony/config v3.2.9
- Installation request for drupal/core ^8.5 -> satisfiable by drupal/core[8.5.0, 8.5.0-alpha1, 8.5.0-beta1, 8.5.0-rc1, 8.5.x-dev, 8.6.x-dev].
- Conclusion: don't install symfony/config v3.2.9
- drupal/core 8.5.x-dev requires symfony/dependency-injection ~3.4.0 -> satisfiable by symfony/dependency-injection[3.4.x-dev, v3.4.0, v3.4.0-BETA1, v3.4.0-BETA2, v3.4.0-BETA3, v3.4.0-BETA4, v3.4.0-RC1, v3.4.0-RC2, v3.4.1, v3.4.2, v3.4.3, v3.4.4, v3.4.5, v3.4.6].
- symfony/dependency-injection 3.4.x-dev conflicts with symfony/config[v3.2.9].
- symfony/dependency-injection v3.4.0 conflicts with symfony/config[v3.2.9].
- symfony/dependency-injection v3.4.0-BETA1 conflicts with symfony/config[v3.2.9].
- symfony/dependency-injection v3.4.0-BETA2 conflicts with symfony/config[v3.2.9].
- symfony/dependency-injection v3.4.0-BETA3 conflicts with symfony/config[v3.2.9].
- symfony/dependency-injection v3.4.0-BETA4 conflicts with symfony/config[v3.2.9].
- symfony/dependency-injection v3.4.0-RC1 conflicts with symfony/config[v3.2.9].
- symfony/dependency-injection v3.4.0-RC2 conflicts with symfony/config[v3.2.9].
- symfony/dependency-injection v3.4.1 conflicts with symfony/config[v3.2.9].
- symfony/dependency-injection v3.4.2 conflicts with symfony/config[v3.2.9].
- symfony/dependency-injection v3.4.3 conflicts with symfony/config[v3.2.9].
- symfony/dependency-injection v3.4.4 conflicts with symfony/config[v3.2.9].
- symfony/dependency-injection v3.4.5 conflicts with symfony/config[v3.2.9].
- symfony/dependency-injection v3.4.6 conflicts with symfony/config[v3.2.9].
- Installation request for symfony/config (locked at v3.2.9) -> satisfiable by symfony/config[v3.2.9].
This happens when one of Drupal’s dependencies is updated and the new version requires an updated version of another package. To resolve this, include the dependency package causing the issue in the composer update
command. The --with-dependencies
flag this will ensure that the dependency’s dependencies are also updated. To fix the error above, I ran:
composer update drupal/core symfony/config --with-dependencies -v
You’re not alone
If you continue to run into problems, the best advice I can give you is to search for the specific update you’re trying to make. Every time I’ve had an issue I’ve been able to find discussions online regarding that specific update and potential resolutions.
In fact, when I got the error above while trying to update to 8.5.0, I found this helpful article by drupal.org user eiriksm and was able to resolve the issue. Check out the article and its comments for more discussion on how to deal with Composer issues when updating Drupal 8 core.
The problem: XDebug doesn't work for Composer scripts
PhpStorm is quite convenient to debug scripts with XDebug (do you support Derick for giving us XDebug ?): just add a "Run/Debug configuration", choosing the "PHP Script" type, give a few parameters, and you can start debugging your PHP CLI scripts, using breakpoints, evaluations, etc.
Wonderful. So now, let's define such a configuration to debug a Composer script, say a Behat configuration generator from site settings for some current Drupal 8 project. Apply the configuration, run it in debug mode, and ....
...PhpStorm doesn't stop, the script runs and ends, and all breakpoints were ignored. How to actually use breakpoints in the IDE ?
The diagnostic
Let's see how Composer actually starts:<?php
#!/usr/bin/env php
<?phpif (PHP_SAPI !== 'cli') {
echo 'Warning: Composer should be invoked via the CLI version of PHP, not the '.PHP_SAPI.' SAPI'.PHP_EOL;
}require
__DIR__.'/../src/bootstrap.php';use
Composer\Factory;
use Composer\XdebugHandler;
use Composer\Console\Application;error_reporting(-1);// Create output for XdebugHandler and Application
$output = Factory::createOutput();$xdebug = new XdebugHandler($output);
$xdebug->check();
unset($xdebug);// [...more...]
?>
Now, we can step through this, and notice the Composer script actually runs during the $xdebug->check();
line. What's going on ?
<?php
// In vendor/composer/composer/src/Composer/XdebugHandler.php
public function check()
{
$args = explode('|', strval(getenv(self::ENV_ALLOW)), 2); if (
$this->needsRestart($args[0])) {
if ($this->prepareRestart()) {
$command = $this->getCommand();
$this->restart($command);
} return;
}
// [...more...]
?>
Here's the crux of the problem: in order to alleviate the extreme slowdown caused by XDebug when running Composer commands, any time Composer is run, it checks for the presence of Xdebug in the running PHP configuration, and if it finds it (the needsRestart()
check), it rebuilds a command line with a different PHP configuration ($command = $this->getCommand();
), which does not include the xdebug extension, and runs it in the $this->restart($command);
call, which actually runs the new command using the passthru
mechanism.
<?php
protected function restart($command)
{
passthru($command, $exitCode);
// [...more...]
?>
Now, since this command no longer runs with Xdebug, there is no way a debugging tool can use it. So how does one fix the problem ?
The solution
Let's go back to the beginning of this check()
method:
<?php
class XdebugHandler
{
const ENV_ALLOW = 'COMPOSER_ALLOW_XDEBUG'; // [...snip...] public function check()
{
$args = explode('|', strval(getenv(self::ENV_ALLOW)), 2); if (
$this->needsRestart($args[0])) {
// [...more...]
?>
So the call to needsRestart($args[0]
does actually depend on the value of the XdebugHandler::ENV_ALLOW
constant, i.e. COMPOSER_ALLOW_XDEBUG
. Let's see.
<?php
private function needsRestart($allow)
{
if (PHP_SAPI !== 'cli' || !defined('PHP_BINARY')) {
return false;
} return empty(
$allow) && $this->loaded;
}
?>
So, if COMPOSER_ALLOW_XDEBUG
is not empty, needsRestart($allow)
will return FALSE
. In which case, as shown above, it won't cause the new command to be built, and Composer will just proceed with our script and allow our debugging to work.
In practice, this means all we need it to pass a COMPOSER_ALLOW_XDEBUG
environment variable with a non-empty value in the PhpStorm run configuration, like this.
Problem solved_! BTW, did you notice this was explained in the Composer vendor/composer/composer/doc/articles/troubleshooting.md
documentation ?
[...snip...] ## Xdebug impact on Composer To improve performance when the xdebug extension is enabled, Composer automatically restarts PHP without it. You can override this behavior by using an environment variable: `COMPOSER_ALLOW_XDEBUG=1`. [...more...]
Yeah, me neither, until I solved the problem as described. So let's all just keep in mind to RTFM.
EXPECTED_SIGNATURE=$(wget -q -O - https://composer.github.io/installer.sig) && \ php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" && \ php -r "if (hash_file('SHA384', 'composer-setup.php') === '${EXPECTED_SIGNATURE}') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" && \ php composer-setup.php && \ php -r "unlink('composer-setup.php');" && \ mv composer.phar /usr/local/bin/composerUsing EXPECTED_SIGNATURE variable with the latest available signature value you don't have to hardcode a specific one for comparison on 3rd line.
The problem: packages.drupal.org broken
I was starting my weekly work for the Drupal GraphQL module by the customary composer update --prefer-source -vvv
command, ready to watch composer spit out some hundred lines sipping my coffee, but this time something turned out to be wrong:
OK, so packages.drupal.org is down for now. How can we work around this ?
The diagnostic
By default, most of my Drupal 8 project (aren't yours ?) are either based on drupal-composer/drupal-project
for actual projects, or a slightly modified drupal/drupal
when working on Drupal core or contrib modules like GraphQL. This was the case of the latter model, done by adding specific repositories
entries to core composer.json
, like:
"repositories": {
"fgm_graphql": {
"type": "vcs",
"url": "https://github.com/FGM/graphql-drupal"
},
"drupal": {
"type": "composer",
"url": "https://packages.drupal.org/8"
}
},
The explicit vcs
entry on top is to enable me to work on my custom fork of the module (from which I send pull requests), no problem here. The issue is with the bottom one, used to download the other package in this project, specifically devel
.
When working with such composer
repositories, what Composer does is fetch a packages.json
path below the url
parameter, in this case https://packages.drupal.org/8/packages.json. That files is a list of data providers, basically one more level of metadata about available repositories and distributions (releases), looking like this:
{
"notify-batch":"\/8\/downloads",
"providers-url":"\/8\/%package%$%hash%.json",
"search":"\/8\/search.json?s=%query%",
"provider-includes":{
"drupal\/provider-2017-1$%hash%.json":{
"sha256":"67abb1f57bb826754ae5f4f877f92d2b1b58bfd45a56f4de883f2424be6cf8d5"
},
"drupal\/provider-2016-4$%hash%.json":{
"sha256":"a30289dd8394e5271bd77777bb14b361c5938656f1cddad7fae1c00d5d6ba9c6"
},
[ ...snip ...]
}
}
}
Composer will then download from each of these providers in turn, hence the URL displayed as returning a 404 at the top of this story. Sure enough, manual checking of the URL returned a 404, even with a cache-buster query added.
Sure enough, the issue was already mentioned on IRC on #drupal : what could be done at this point without being able to touch packages.drupal.org
itself ?
The solution: skip one layer
Actually, the answer is already present in the existing repositories
clause: vcs
-type repositories do not need a "directory"-type service like Packagist or packages.drupal.org
, because in essence what these directories do is provide a way to locate sources and dists. But vcs
provide the same service, with the limitation that they have to be listed for each project. So let us skip the composer
directory and list devel
's repository directly:
"repositories": {
"fgm_graphql": {
"type": "vcs",
"url": "https://github.com/FGM/graphql-drupal"
},
"devel": {
"type": "vcs",
"url": "https://git.drupal.org/project/devel.git"
}
},
Now, a composer require drupal/devel --prefer-source -vvv
works normally, no longer needing to parse the broken directory:
Reading ./composer.json
Loading config file /Users/fgm/.composer/auth.json
Loading config file ./composer.json
[...snip...]
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git show-ref --tags --dereference
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git branch --no-color --no-abbrev -v
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git branch --no-color
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git show '8.x-1.x':'composer.json'
Executing command (/Users/fgm/.composer/cache/vcs/https---git.drupal.org-project-devel.git/): git log -1 --format=%at '8.x-1.x'
Reading composer.json of drupal/devel (5.x-0.1)
Skipped tag 5.x-0.1, invalid tag name
[...snip...]
Reading composer.json of drupal/devel (8.x-1.x)
Reading /Users/fgm/.composer/cache/repo/https---git.drupal.org-project-devel.git/70f62fd0773082a1a4305c6a7c2bccc649bc98a2 from cache
Importing branch 8.x-1.x (dev-8.x-1.x)
[...snip...]
(success)
Time to return to actual work :-)
Our clients are often looking to reach their audiences via email campaigns, and MailChimp is one of the solutions we frequently recommend for this. MailChimp makes it easy to create and manage email campaigns while also providing beneficial analytics on user behavior.
Earlier this year I wrote a blog post showing how to use Composer Manager along with the Mailchimp API v2.0 PHP package to subscribe users to mailing lists in a Drupal 6 or 7 custom module without the need for the Mailchimp contributed module.
However, since then, MailChimp API v3.0 was released and Mailchimp announced that v2.0 (and all prior versions) will no longer be supported after 2016.
So in this blog post, I’ll demonstrate how to accomplish the same objective using the new MailChimp API v3.0, and I’ll expand the tutorial to also include some Drupal 8 specifics.
Background
To quickly summarize the key takeaways from my previous blog posts on Composer Manager and subscribing users to MailChimp lists using the old API:
- Composer is a tool for managing PHP libraries that your project depends on.
- Challenges arise managing project-wide dependencies when custom and contributed modules specify their own unique dependencies.
- Composer Manager is a contributed module for Drupal 7 (and formerly Drupal 6) that addresses these challenges and allows contributed and custom modules to depend on PHP libraries managed via Composer.
- Using a Composer managed PHP package for the MailChimp API, we can easily subscribe users to MailChimp lists in a Drupal custom module without relying on the Mailchimp module.
- While the Mailchimp contributed module is great, sometimes all you need is a simple, lightweight method for subscribing users to mailing lists.
One important development since my previous posts is that Composer Manager has been deprecated for Drupal 8. Improvements introduced in Drupal 8.1.0 allow modules to rely on Composer managed dependencies without the need for the Composer Manager module.
Implementation
There are a few steps we must take so that we can subscribe users to mailing lists in our custom module. We’ll review each of these steps in detail:
- Add the MailChimp API v3.0 PHP library as a dependency of our custom module.
- Ensure that the library is installed for our project.
- Properly use the library in our custom module to subscribe users to mailing lists.
Specify the dependency
ThinkShout maintains the Mailchimp contributed module and we were very excited to see that as part of the effort to “get Drupal off the island” they also released a PHP library for MailChimp API v3.0.
To use this new library, we must specify it as a dependency of our custom module. We do that in a composer.json
file that sits in our custom module’s root directory and requires that library via the following code:
{
"require": {
"thinkshout/mailchimp-api-php": ">=1.0.3"
}
}
Install the library
Composer is intended for projects and therefore requires a Drupal site to have a single composer.json
, so things get complicated when individual modules specify their own dependencies.
For Drupal 7 sites (or still active Drupal 6 sites), the Composer Manager contributed module handles this by merging the requirements specified by each custom and contributed module’s composer.json
files into a single, consolidated, site-wide composer.json
file.
So for Drupal 6/7 projects we’ll need Composer Manager installed and enabled.
Once enabled, we can generate the consolidated composer.json
and then install all of the site’s dependencies that file specifies (including the MailChimp API v3.0 PHP library specified by our custom module) in one of two ways:
From the command line, we can run the following drush commands:
$ drush composer-json-rebuild
$ drush composer-manager install
Alternatively, we could include the following lines in an update hook:
// Re-build Composer Manager composer.json and run composer update.
drush_composer_manager_composer_json_rebuild();
drush_composer_manager('update');
For Drupal 8 sites, the process is slightly different. As mentioned previously, as of release 8.1.0, Drupal core directly uses Composer to manage dependencies and the Composer Manager module is no longer necessary. For Drupal 8 sites, we should follow the Drupal.org instructions for managing dependencies for a custom project. Following those instructions ensures that all of the site’s dependencies, including the MailChimp library specified by our custom module, are installed.
Use the library
Once we have the MailChimp API v3.0 PHP library installed, we can use it in our custom module to subscribe users to mailing lists.
We suggest creating a dedicated function for subscribing users to email lists which can then be called throughout the custom module. For our purposes, we modeled that function off of the Mailchimp module (version 7.x-4.6) mailchimp_subscribe_process()
function.
We implemented the following function, which can be reviewed and modified for your specific purposes:
<?php
/**
* Add an email to a MailChimp list.
*
* This code is based on the 7.x-4.6 version of the Mailchimp module,
* specifically the mailchimp_subscribe_process() function. That version of
* the Mailchimp contrib module makes use of the ThinkShout PHP library for
* version 3.0 of the MailChimp API. See the following for more detail:
* https://www.drupal.org/project/mailchimp
* https://github.com/thinkshout/mailchimp-api-php.
*
* @see Mailchimp_Lists::subscribe()
*
* @param string $api_key
* The MailChimp API key.
* @param string $list_id
* The MailChimp list id that the user should be subscribed to.
* @param string $email
* The email address for the user being subscribed to the mailing list.
*/
function mymodule_subscribe_user($api_key, $list_id, $email) {
try {
// Set the timeout to something that won't take down the Drupal site:
$timeout = 60;
// Get an instance of the MailchimpLists class.
$mailchimp = new \Mailchimp\MailchimpLists($api_key, 'apikey', $timeout);
// Use MEMBER_STATUS_PENDING to require double opt-in for the subscriber. Otherwise, use MEMBER_STATUS_SUBSCRIBED.
$parameters = array(
'status' => \Mailchimp\MailchimpLists::MEMBER_STATUS_PENDING,
'email_type' => 'html',
);
// Subscribe user to the list.
$result = $mailchimp->addOrUpdateMember($list_id, $email, $parameters);
if (isset($result->id)) {
watchdog('mymodule', '@email was subscribed to list @list.',
array('@email' => $email, '@list' => $list_id), WATCHDOG_NOTICE
);
}
else {
watchdog('mymodule', 'A problem occurred subscribing @email to list @list.', array(
'@email' => $email,
'@list' => $list_id,
), WATCHDOG_WARNING);
}
}
catch (Exception $e) {
// The user was not subscribed so log to watchdog.
watchdog('mymodule', 'An error occurred subscribing @email to list @list. Status code @code. "%message"', array(
'@email' => $email,
'@list' => $list_id,
'%message' => $e->getMessage(),
'@code' => $e->getCode(),
), WATCHDOG_ERROR);
}
}
With that function defined, we can then subscribe an email address to a specific Mailchimp mailing list through the following function call in our custom module:
mymodule_subscribe_user($api_key, $list_id, $email);
Conclusion
By taking advantage of the modern PHP ecosystem built on reusable Composer managed packages, we can easily build or adapt a custom module to subscribe users to mailing lists without the MailChimp contributed module.
Lastly, a special thanks to ThinkShout for their hard work maintaining the MailChimp module and creating the library, on which this approach depends!
The April 2016 issue of php[architect] magazine is out! This issue we take a look at how Drupal is using tools and techniques from the PHP community.
This issue also includes articles on how to easily generate documentation, advice on learning new frameworks, what it means to be a leader in the PHP community, and how to use PHPStorm to improve your code.
Download a FREE article and get your issue today.
The recommended approach to getting started with Drupal 8 is now via Composer. An official project template has been created for this. We will create our project directly using the template, which is also available on Packagist.
To create a new project based on this template we can run the following Composer command:
composer create-project drupal-composer/drupal-project:8.x-dev my_project --stability dev --no-interaction
This Composer command will pull in the template from Packagist and run a few Drupal specific scripts to prepare our project for installation. The only thing left to do is point our browser to the web/
directory (since that is where the index.php
file is) and run the installer as usual.
This template comes with a /web
folder that contains, among other things, the main folders of a Drupal installation that are no longer considered part of Drupal core (such as the index.php
file or the modules and themes folders). Additionally, it comes with an autoload.php
file used by Drupal that simply points to the Composer vendor/
directory, one folder up. So all PHP libraries are now handled from one single place.
The template’s composer.json
file requires the latest stable Drupal core + some additional helper tools such as Drush and the Drupal Console. Additionally, it adds the Drupal specific Packagist repository from where we can install Drupal contributed modules, themes and profiles (that get automatically installed in the right place).
If we want to add a Drupal contributed module, we need to find it on the Drupal Packagist and require it in our project via Composer:
composer require drupal/ctools
This will add the Ctools module directly to our web/modules/
directory and update our composer.json
file.
The project template also comes with a .gitignore
file that keeps Drupal core and all the contributed packages outside of Git, similar to the regular vendor/
packages. So based on an updated composer.json
file, we can maintain a smaller Git repository and recreate our project any time. A lot of the benefits of Drush Make have now been incorporated into a Composer flow.
Conclusion
Drupal 8 has come a long way in catching up with other major PHP software. The possibility of fully managing it via Composer, either as a main project or even just as part of a bigger set of applications, is a testament to the community effort that went in.
Meet the author
Daniel Sipos is a Drupal developer who lives in Brussels, Belgium. He works professionally with Drupal but likes to use other PHP frameworks and technologies as well. He runs webomelette.com, a Drupal blog where he writes articles and tutorials about Drupal development, theming and site building.As of 1:23pm GMT on Febuary 3rd 2016 there are no dependencies in the Drupal 8.1.x git branch.
Why?
There was no reason to have them there. It makes the repository bigger, and therefore takes longer to download. No one else does it.
What does it mean?
Not much really.
If you download Drupal via the zip file or tarball then the drupal.org packager will run composer
so the dependencies will be there for you.
If you submit a Drupal core patch on drupal.org then DrupalCi testbot will run composer install
.
It’s only if you clone Drupal 8.1.x or higher directly from git, you will need to run composer install
in the Drupal root directory
Top tip: composer install --prefer-source --no-interaction
can often be quicker.
Please enable JavaScript to view the comments powered by Disqus.
blog comments powered byDrupal 8 is out, Drupal 8.1 will be out before we know it, but it seems contrib is still catching up. One question that seems to keep coming up is around installing a Drupal 8 module. In this post I will look at the different ways to install a module and resolving dependencies.
Deploy, Search API Solr Search, and Commerce are just a few of the modules with Drupal 8 releases that require dependencies loaded via composer. This means you can’t just download the module’s ZIP file, unzip it in your modules directory, and enable it. You need to install the dependencies.
One simple way to do this is the Composer Manager module. This module has extensive documentation on how to use it. Essentially what it does it merge the composer.json that ships with core and the composer.json files from all the contrib module you have, then downloads the dependencies. You may notice that this will also update core dependencies, but this is a good thing! The core composer.json has been written in such a way that it won’t introduce API breaking dependencies and only uses stable releases. So you will benefit from any bug fixes or security fixes rolled out in these dependencies before they’re rolled out with Drupal.
The two other ways we’re going to look at involve directly using Composer.
Just the dependencies
It’s possible to carry on installing Drupal modules exactly as you always have, download the zip or tarball, then unzip it into your modules directory. As mentioned earlier this will not install the dependencies, therefore you will need to look inside the module’s composer.json file, see what the dependencies are, and install them manually. Let’s take Deploy module as an example, this depends on the dev-master version of relaxedws/replicator. So, go to your Drupal docroot and run the command composer require relaxedws/replicator:dev-master
. This will add relaxedws/replicator to Drupal’s composer.json, download it, and put it in the vendor directory ready for the module to make use of. This will not change any other other dependencies you have. Then to update the dependencies you can either run composer update
to update relaxedws/replicator and all core dependencies or composer update relaxedws/replicator
to just update the relaxedws/replicator package.
The module too
If you install all you Drupal modules via composer, all of the dependencies will automatically be installed too. First you will need to add a new repository, so run composer config repositories.drupal composer https://packagist.drupal-composer.org
in your Drupal docroot, this will add the https://packagist.drupal-composer.org repository to your drupal composer.json. Now you can install any module from Drupal.org via composer. So going back to the example of Deploy you can run composer require drupal/deploy:8.1.0-alpha5
in your Drupal docroot and it will install Deploy in the modules directory. It will also install key_value, multiversion, and relaxed, which are all Drupal modules required by Deploy. Furthermore it will install relaxedws/replicator as we know is a PHP package needed for Deploy, and doctrine/couchdb which is a PHP package needed for releaxedws/replicator.
I hope this helps those confused what to do with Drupal 8 modules that have composer dependencies. Now go do the smart thing, rebuild your site using Composer!
Please enable JavaScript to view the comments powered by Disqus.
blog comments powered byOverview
In my last blog post, I wrote about the virtues of Composer Manager and how it allows modules to depend on PHP libraries managed via Composer. Basically, Composer Manager allows us to easily use PHP libraries that exist outside of the Drupal ecosystem within our own projects.
In this post, I’ll show you how we:
Custom vs. Contrib?
But first, why didn’t we just use the MailChimp contributed module? Contributed modules are often a great option and offer many benefits, such as security, maintenance, and flexibility.
But there is a cost to installing all those contributed modules. As The Definitive Guide to Drupal 7 explains “The more modules you install, the worse your web site will perform.”
With each installed module comes more code to load and execute, and more memory consumption. And in some cases, contributed modules add complexity and features that just aren’t necessary for the required task.
In our case, the decision to go with a custom solution was easy:
- The MailChimp contributed module had many features we didn’t need.
- We were already using Composer Manager on the project to manage other module dependencies.
- The custom module we were building already included logic to determine when to subscribe users to mailing lists (don’t worry, we made sure they opted in!)
All we needed was a simple, lightweight method for subscribing a given user to a specific MailChimp mailing list.
Implementation
We were able to achieve this by adding the MailChimp PHP library as a dependency of our custom module. We were then able to make a simple call using the API to subscribe a user to the mailing list. We implemented this via the following code.
First, in our module’s root directory we created a composer.json
file that specified the MailChimp PHP library as a dependency:
{
"require": {
"mailchimp/mailchimp": "*"
}
}
We then installed the Mailchimp API using the Composer Manager drush commands:
$ drush composer-json-rebuild
$ drush composer-manager install
As explained in my last post, the first command builds (or rebuilds) the consolidated project wide composer.json
file and the second command installs the dependencies.
Next, we created a function in our custom module to subscribe a user to a MailChimp mailing list.
<?php
/**
* Add an email to a MailChimp list.
*
* @param string $api_key
* The MailChimp API key.
* @param string $list_id
* The MailChimp list id that the user should be subscribed to.
* @param string $email
* The email address for the user being subscribed to the mailing list.
*/
function my_module_subscribe_user($api_key, $list_id, $email) {
$mailchimp = new Mailchimp($api_key);
try {
$result = $mailchimp->lists->subscribe($list_id, array('email' => $email));
}
catch(Exception $e) {
watchdog('my_module', 'User with email %email not subscribed to list %list_id', array('%email' => $email, '%list_id' => $list_id), WATCHDOG_WARNING);
}
}
With that function defined, we could then subscribe any user to any mailing list by simply calling
<?php
my_module_subscribe_user($api_key, $list_id, $email);
Conclusion
That’s it! A nice, simple, and clean approach to subscribing users to a MailChimp mailing list that doesn’t require installation of the MailChimp contributed module.
We hope you’re as excited as we are at the opportunities Composer and Composer Manager afford us to take advantage of PHP libraries and projects that exist outside of the Drupal ecosystem.
Now that we have an initiative to get get a composer.json file in each contrib module, we cal start getting them all on Packagist.
Back in April the Drupal community decided on a naming convention for projects on Drupal.org. Projects must use the package name drupal/PROJECT
where PROJECT
is the part from the URL.
Using drupal as the vendor name in the package name allows us to lock down Packagist to a select list of maintainers. Currently this is the people who submitted a package to Packagist under the drupal vendor name before May 7th when the restriction was added.
The Packagist API currently allows us to update existing packages, so we could quite easily add a hook_node_update and hook_node_insert to drupal.org that will do this when project releases are updated or added. The API doesn’t offer a create endpoint, so we can’t automatically create new packages, an issue on Github has been opened for this, and I’ve started work on a pull request.
Once we have the Packagist pull request done, and the hooks added to drupal.org we can use the API key from Packagist for one of the maintainers (maybe Dries?) to push any new module or existing module updates across. Packagist will read directly from the drupal.org git repo and we’ll finally have one canonical place for all Drupal modules and PHP dependencies.
Drupal.org issue: https://www.drupal.org/node/2547617
Please enable JavaScript to view the comments powered by Disqus.
blog comments powered byDrupal 8 has seen a lot of love for Composer. As various posts have mentioned, it’s possible to build a whole Drupal site with little more than a composer.json file. However contrib needs the same love too.
In this post I want to talk about step one, and it requires little effort at all.
For module maintainers
Add a composer.json file to your Drupal 8 module. The example below gives everything you need. The name uses “drupal” as the vendor namespace, followed by the module name after the slash. The description provides more information on the module. The type tells composer it is specifically a Drupal module (there are also types for themes, profiles and more), this allows developers to put “drupal-module” packages into the modules directory rather than the vendor directory on composer install. Finally we add Drupal’s license of GPL 2.0+.
{ "name": "drupal/example", "description": "This is an example composer.json for example module.", "type": "drupal-module", "license": "GPL-2.0+" }
For everyone else
- Tell everyone.
- Tweet about the issue or this post.
- See if modules with a Drupal 8 release have a composer.json, if not add an issue for it.
What does it mean
Once all modules have a composer.json we can add modules using composer more easily. We can also start step two, which is to get all Drupal 8 modules on Drupal.org listed on Packagist.
Please enable JavaScript to view the comments powered by Disqus.
blog comments powered byOver the last few weeks I’ve been spending a lot of time with Drupal 8 and Composer. This has lead me building up a PoC for a client and diving into the issue queues and IRC. In this post I wanted to document some of the processes I’ve been looking at.
Creating a project
The people behind drupal-composer have put together a template, a Drupal project can be started from the template using the command composer create-project drupal-composer/drupal-project:8.x-dev drupal --stability dev --no-interaction
. This will create a folder called “drupal”, in there you will find “web” directory containing the Drupal installation. It has also downloaded drush, and two modules, devel and token.
Adding modules
If you open up composer.json in your drupal project folder you will see a repository with the URL https://packagist.drupal-composer.org defined. This is a custom version of Packagist setup by the drupal-composer team. If required packages are not found on Packagist they will get pulled from here.
You will also notice in composer.json under the “require” section is where drupal/token and drupal/devel are added. You can add any module on the Drupal Packagist to this then run the command composer update
to update your project and download the newly added modules.
In the “extra” section of composer.json you will see a number of installer paths are added, this tells Composer (via the required composer/installers package) where to put things. You will see everything is going in the web directory, Drupal core in “web/core”, modules in “web/modules/contrib” etc. Therefore, when you call the composer update
command to add the new modules you required, these automatically went into the correct Directory ready for Drupal to use. Composer knows these are Drupal modules because the modules have a composer.json files too (often dynamically added by the Drupal Packagist because Drupal doesn’t require modules to have a composer.json yet). In this composer.json the type is set to “drupal-module” for modules, “drupl-theme” for themes, etc.
Patching modules
Greg Anderson went into this in a lot of detail on the Pantheon blog earlier this week, but using the cweagans/composer-patches package you can define a patch file to use. For example, add the following to the “extra” section of your composer.json file to patch the token module:
"patches": { "drupal/token": { "Description for reused fields not correct": "https://www.drupal.org/files/issues/token-Fix_description_for_reused_fields-2497251-5.patch" } } }
You’ll see the package to patch is defined, then within that a name or discription of the patch, followed by the URL for the patch file.
Custom modules
There are a number of ways you can handle custom modules here. You could create a folder at web/modules/custom and just put them in there, or you could add them via composer. In the PoC I’m working on we have many custom modules that will be added to multiple projects. The custom modules have their own git repo, and if they’re not on Packagist or the Drupal Packagist (which they shouldn’t be if they’re custom modules) we need to tell Drupal about this repository. In composer.json under the “repositories” section add something like:
{ "type": "vcs", "url": "https://github.com/timmillwood/couchdb_statistics.git" }
In this case we’re adding a repository that contains the “drupal/couchdb_statistics” package. You could add "drupal/couchdb_statistics": "dev-master"
to the “require” section of your composer.json to add this Drupal module to your modules directory.
If you have a lot of custom modules you’re adding to multiple sites it might be worth you setting up Toran Proxy. This is a project by Jordi Boggiano, the guy behind Composer and Packagist. It allows you to proxy your git repos and packagist. You then add your Toran Proxy installation to the “repositories” section of you composer.json, then require any packages you have there. Give Toran Proxy a Github token and it can also grab your private repos too.
One thing to note about repositories is:
> Repositories are only available to the root package and the repositories defined in your dependencies will not be loaded.
For the PoC project mentioned earlier I looked at creating a sub-project which just contained a composer.json. This then required all of the common modules, custom and contrib. However the custom ones were not getting pulled in because all the custom repositories were not defined in the root project, they were only defined in the sub-project. Having Toran Proxy as a single source meant that we could add it to the root project and all dependencies could also get pulled from there.
Post install
You may noticed the drupal-composer template has a scripts directory and this is defined in composer.json as a “post-install-cmd”. This is run after composer.install
. The script add settings.php, services.yml and the files directory. You could customise this to do a number of other things. Run drush commands, setup a vagrant box, etc
Summary
Composer is here in Drupal 8 and it’s awesome. You can run, develop and deploy your whole Drupal 8 project with Composer. You can add and patch contrib and custom module, as well as themes, profiles and other PHP packages. The drupal/couchdb_statistics module mentioned earlier requires a couchdb client, this will all get pulled in via composer with a single command.
Move over Drush make, this is how you should be running Drupal 8.
Please enable JavaScript to view the comments powered by Disqus.
blog comments powered byThere are thousands of situations in which you do not want to reinvent the wheel. It is a well known principle in Software Engineering, but not always well applied/known into the Drupal world.
Let’s say for example, that you have a url that you want to convert from relative to absolute. It is a typical scenario when you are working with Web (but not just Web) crawlers. Well, you could start building your own library to achieve the functionality you are looking for, packaging all in a Drupal module format. It is an interesting challenge indeed but, unless for training or learning purposes, why wasting your time when someone else has already done it instead of just focussing on the real problem? Especially if your main app purpose is not that secondary problem (the url converter).
What’s more, if you reuse libraries and open source code, you’ll probably find yourself in the situation in which you could need an small improvement in that nice library you are using. Contributing your changes back you are closing the circle of the open source, the reason why the open source is here to stay and conquer the world (diabolical laugh here).
That’s another one of the main reasons why lot’s of projects are moving to the Composer/Symfony binomium, stop working as isolated projects and start working as global projects that can share code and knowledge between many other projects. It’s a pattern followed by Drupal, to name but one, and also by projects like like phpBB, ezPublish, Laravel, Magento,Piwik, …
Composer and friends
Coming back to our crawler and the de-relativizer library that we are going to need, at this point we get to know Composer. Composer is a great tool for using third party libraries and, of course, for contributing back those of your own. In our web crawler example, net_url2 does a the job just beautifully.
Nice, but at this point you must be wondering… What does this have to do with Drupal, if any at all? Well, in fact, as everyone knows, Drupal 8 is being (re)built following this same principle (DRY or don’t repeat yourself) with an strong presence of the great Symfony 2 components in the core. Advantages? Lots of them, as we were pointing out, but that’s the purpose of another discussion
The point here is that you don’t need to wait for Drupal 8, and what’s more, you can start applying some of this principles in your Drupal 7 libraries, making your future transition to Drupal 8 even easier.
Let’s rock and roll
So, using a php library or a Symfony component in Drupal 7 is quite simple. Just:
- Install composer manager
- Create a composer.json file in your custom module folder
- Place the content (which by the way, you’ll find quite familiar if you’ve already worked with Symfony / composer yaml’s):
"require": { "pear/net_url2": "2.0.x-dev" }
- enable the custom module
And that’s it basically. At this point we simply need to tell drupal to generate the main composer.json. That’s basically a composer file generated from the composer.json found in each one of the modules that include a composer themselves.
Lets generate that file:
drush composer-rebuild
At this point we have the main composer file, normally in a vendor folder (if will depend on the composer manager settings).
Now, let’s make some composer magic :
drush composer update
At this point, inside the vendors folder we should now have a classmap, containing amongst others our newly included library.
Hopefully all has gone well, and just like magic, the class net_url2 is there to be used in our modules. Something like :
$base = new Net_URL2($absoluteURL);
Just remember to add the library to your class. Something like:
use Net_URL2;
In the next post we’ll be doing some more exciting stuff. We will create some code that will live in a php library, completely decoupled but at the same time fully integrated with Drupal. All using Composer magic to allow the integration.
Why? Again, many reasons like:
- Being ready for Drupal 8 (just lift libraries from D7 or D6 to D8),
- Decoupling things so we code things that are ready to use not just in Drupal, and
- Opening the door to other worlds to colaborate with our Drupal world, …
- Why not use Dependency Injection in Drupal (as it already happens in D8)? What about using the Symfony Service container? Or something more light like Pimple?
- Choose between many other reasons…
See you in my next article about Drupal, Composer and friends, on the meantime, be good :-).
Updated: Clarified that we are talking about PHP Libraries and / or Symfony components instead of bundles. Thanks to @drrotmos and @Ross for your comments.
Doctrine Annotations is a central part of the Drupal 8 Plugin API. One of its conceptual annoyances is the need for a separate autoloading process for annotations, which in some scenarios leads to duplicated path declarations. Let us see how to remedy to it.
The Annotations loading process
Doctrine Annotations are classes, and the whole Doctrine annotations process, like PHP autoloading in general (the single autoload function in plain PHP, the SPL autoload stack otherwise), relies upon a single global, the AnnotationRegistry
class, to which annotation classes can be registered.
This registry supports three registration mechanismes, as described in its documentation:
- direct file registration, with the
AnnotationRegistry::registerFile($path)
method, which is just a wrapper forrequire_once $path
. This is useful to load annotation files containing multiple annotation classes, like Doctrine ORM's ownDoctrineAnnotations.php
, without going through multiple autoloads - namespace(s) registration with the
AnnotationRegistry::registerNamespace($ns, $path)/AnnotationRegistry::registerNamespaces($namespaces)
methods, which register autoload namespaces to the annotation autoloader, implemented inAnnotationRegistry::loadAnnotationClass
. - custom loaders with the
<code>AnnotationRegistry::registerLoader($callable)
, to handle specific scenarios
Implementation in PSR-0/PSR-4 projects with Composer
In most cases, registration will be handled by the namespace-based registration. However, since the annotations autoloader is not tied to a specific autoloading scheme, it needs to receive a path parameter for each namespace it handles. In a typical PSR-0/PSR-4 project with Composer, it can mean something like this:
<?php
AnnotationRegistry::registerAutoloadNamespace("Symfony\Component\Validator\Constraints", "$basePath/vendor/symfony/validator");
AnnotationRegistry::registerAutoloadNamespace("MyProject\Annotations", "$basePath/src");
?>
As evidenced by that fragment, this means having to specify in code the namespace-to-path mapping already specified in the composer.json
file, under the PSR-0/PSR-4 specification, which is a source of extra work and possible errors.
Autoloading error handling
Why is it necessary ? To quote the Doctrine doc:
How are these annotations loaded? From looking at the code you could guess that the ORM Mapping, Assert Validation and the fully qualified annotation can just be loaded using the defined PHP autoloaders. This is not the case however: For error handling reasons every check for class existence inside the AnnotationReader sets the second parameter $autoload of class_exists($name, $autoload) to false. To work flawlessly the AnnotationReader requires silent autoloaders which many autoloaders are not. Silent autoloading is NOT part of the PSR-0 specification for autoloading.
Indeed, the requirement for silent autoloaders is lacking in PSR-0 and had been hotly and repeatedly debated in the PSR-4 discussions (see the PSR-4: The registered autoloader MUST NOT throw exceptions thread for an example), so caution is indeed needed in the absolutely general case.
However, in most cases, the autoloader is known to be silent: Composer's findFile() method is silent, which means this extra caution is not needed in a project where Composer is used for autoloading.
Removing duplication via registerLoader()
So if we want to avoid re-declaring our namespace-to-path mapping and rely on the existing declarations, what can we do ? Create a custom loader and use it with AnnotationRegistry::registerLoader($callable)
. Since we will want to pass it parameters, and loaders only receive the name of the class to load, we will implement it as a class with the __invoke()
magic method.
All this pseudo-loader will have to do will be to answer true
for classes in the namespace it handles, to claim it actually found the class, and let the default autoloader work its magic later on, when the Annotations DocParser
will try to access the annotation classes.
That way we can parse the example given on the Annotations documentation page without repeating the namespace to path mapping:
That's it: annotation parsing, without duplicated mappings.
Yesterday, Seldaek committed PSR-4 support to Composer, as his New Year's Day gift to the PHP community.
So, since I always supported PSR-4 in FIG discussions, and after the tl;dr discussion about it regarding autoloading in Drupal 8, I jumped on the occasion and converted the code base for PlusVite to PSR-4 to get an idea of how PSR-4 "felt" in practice.
A the documentation explains, to convert from PSR-0 to PSR-4 to shorten paths, the change is extremely simple: just edit your composer.json
and change it like this:
Of course, removing two levels of directories provided the expected small degree of instant gratification, making everything seem simpler. But then, doubt appears: maybe this is just because I've been using PSR-0 for too long (3 years, already ?), but now, seeing these namespaced classes/interfaces/traits in short paths like src/Baz.php
for a simple project like this seems to be hiding information, degrading DX instead of improving it. Eewww !
Now, in Drupal 8, the situation is different: classes core modules are located under
core/modules/(module)/lib/Drupal/(module)/(subnamespace hierarchy)/someclass.php
while PSR-4 will change this to
core/modules/(module)/lib/(subnamespace hierarchy)/someclass.php
so the "hiding" effect will not be the same, since the name of the module (hence the namespace) will be in plain sight. But still, this is an unexpected feeling and I hope we will not regret it later on.
About Drupal Sun
Drupal Sun is an Evolving Web project. It allows you to:
- Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
- Facet based on tags, author, or feed
- Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
- View the entire article text inline, or in the context of the site where it was created
See the blog post at Evolving Web
