Feb 26 2018
Feb 26

With each release of Drupal 8 more and more things are being deprecated, which is awesome. It shows innovation, forward thinking, and a thought for backwards compatibility. However throwing notices or warnings when deprecated code is used can cause tests to fail. We already counter this a little by adding <env name="SYMFONY_DEPRECATIONS_HELPER" value="weak_vendors"> to phpunit.xml.dist in core.

To quote the Symfony documentation:

By using the weak_vendors value, deprecations that are triggered outside the vendors directory will make the test suite fail, while deprecations triggered from a library inside it will not, giving you the best of both worlds.

This shows that deprecations within Drupal will still cause test fails, to combat that simply update the SYMFONY_DEPRECATIONS_HELPER setting to "weak" or "disabled", which would ignore the deprecations or disable the deprecation helper.

However if you're testing with run-test.sh this will interfere with the settings in your phpunit.xml file and use always use weak_vendors unless you have the suppress-deprecations argument set. Therefore use run-tests.sh --suppress-deprecations and the SYMFONY_DEPRECATIONS_HELPER setting will be set to "disabled". See the simpletest_script_run_one_test() function in run-tests.sh for more context.

Hopefully this helps, we found this really useful when testing Drupal modules with Travis, where we needed tests passing for Drupal 8.4 and 8.5.

Jan 10 2018
Jan 10

At 12:54pm on January 15th 2008 (based on the created time stamp of "1200401651") I signed up for my Drupal.org account while working for International Baccalaureate. Before my interview there I had never heard of Drupal, but managed to ace the interview and get the job, no idea why I didn't bother signing up for Drupal.org until after I started the job, but hey, that's just how it was. I was jamesfk who introduced me to Drupal there and who I worked with building a bunch of community sites in Drupal 5, then 6. I got the chance to visit Toronto for training with Lullabot where I got to meet great people such as James Walker, Matt Westgate, Jeff Robbins, Addison Berry, and fellow student Doug Vann. I also landed a trip to DrupalCon DC 2009, where I met even more great people and many people I now knew virtually from Drupal.org, IRC, and Twitter.

Later in 2009 I started work with Mark Boulton who was already well into his work with the Drupal.org redesign, and starting work on the new admin theme for Drupal 7. It was awesome to be a part of these projects, as well as many Drupal projects which were coming in on the back of these. While working here I got to attend DrupalCamp UK, DrupalCon Paris, and DrupalCon Copenhagen.

In 2011 I moved to a position at Acquia which saw me become the 4th Acquia employee in the UK, along with great Drupalists like Hernâni, and the first support engineer based in Europe, shortly followed by Aurelien. Later that year I co-load organization of DrupalCon London (or Croydon) with Jeff. At Acquia I moved to help form the customer success team, assisting clients getting their Drupal sites up and running and launch successfully.

After nearly 4 years I left Acquia to join Appnovation who were looking to setup a UK office in Wales (where I live). This also saw me start working with Pfizer on their Drupal 8 projects, initially I was contributing to a lot of the Composer related issues, then moved on to start the Workflow Initiative.

Outside of "work" in this time I have contributed to many contrib modules, become the core maintainer for Statistics, Content Moderation, and Node modules, been a DrupalCon track chair three times, attended 9 DrupalCons, written Drupal articles for Net magazine and appeared on some Drupal podcasts. Outside of Drupal I have moved house twice, got married, and had two kids.

It's been a fun time, here's to the next 10 years!

Dec 14 2017
Dec 14

Since Drupal 8 we've had services. This also brought the concept of a service collector or tagged services. This allows services to be tagged with a specific tag, then a service collector can collect all services with the a given tag and use whichever service "applies".

As you could imagine loaded all of these tagged services when loading the service collector service can be a performance nightmare, which is why Drupal 8.4.0 brought us service ID collector functionality.

Tagging a service with the service_id_collector tag will pass all services with a given tag as the last parameter in the constructor. This will be an array of service IDs ordered by the priority. You will then need to use the ClassResolver service to lazily instantiate the collected service IDs.

my_module.services.yml
my_module.negotiator:
  class: \Drupal\my_module\MyModuleNegotiator
  arguments: {'@class.resolver'}
  tags:
    - { name: service_id_collector, tag: my_module_negotiator }
my_module.negotiator.foo:
  class:\Drupal\my_module\FooNegotiator
  arguments: {'@entity_type.manager'}
  tags:
    - { name: my_module_negotiator, priority: 100 }
my_module.negotiator.bar:
  class:\Drupal\my_module\BarNegotiator
  arguments: {'@entity_type.manager'}
  tags:
    - { name: my_module_negotiator, priority: -100 }
my_module.negotiator.baz:
  class:\Drupal\my_module\BazNegotiator
  arguments: {'@entity_type.manager'}
  tags:
    - { name: my_module_negotiator, priority: 0 }

MyModuleNegotiator.php
/**
 * The negotiator for my module.
 */
class MyModuleNegotiator {

  /**
   * The class resolver service.
   *
   * @var \Drupal\Core\DependencyInjection\ClassResolver
   */
  protected $classResolver;

  /**
   * The negotiator service IDs.
   *
   * @var array
   */
  protected $negotiatorServiceIds;
 

  /**
   * Constructs the negotiator.
   *
   * @param \Drupal\Core\DependencyInjection\ClassResolver $class_resolver

   *   The class resolver service.
   * @param array $negotiator_service_ids
   *   The negotiator service IDs.
   */

  public function __construct(ClassResolverInterface $class_resolver, array $negotiator_service_ids) {
    $this->classResolver = $class_resolver;
    $this->negotiatorServiceIds = $negotiator_service_ids;
  }

 

  /**
   * Run the negotiators.
   */
  public function runNegotiators() {
    foreach ($this->negotiatorServiceIds as $negotiator_service_id) {
      $negotiator = $this->classResolver->getInstanceFromDefinition($negotiator_service_id);
      if ($negotiator->applies()) {
        $negotiator->run();
      }
    }
  }
}

Nov 23 2017
Nov 23

Last week I switch from years of using Chrome to Firefox 57 because of all the hype about it being fast, and that I'd been suffering from Chrome using up to 10GB of ram. The big issue I hit though was I didn't have Dreditor and there seemed to be no way to install it. I decided to go on using Firefox without Dreditor, and loading Chrome every time I needed to do an in depth patch review.

Then yesterday I saw the latest Commit Strip cartoon, where in a reply @williambl suggested Chrome Store Foxified for converting Chrome plugins to Firefox. First thing I thought was to try the Dreditor Chrome plugin, and it worked.

This morning Berdir suggested "maybe someone will release that thing as a public extension". So I went digging on addons.mozilla.org and found I could download the XPI file Chrome Store Foxified created during the conversion.

So here it is:
Download Dreditor for Firefox now!
MD5SUM: 2b7455e057ac6a84bd01423b0984c21d

Jul 30 2017
Jul 30

Did you know Drupal 8 has a new system to manage config? sure, everyone does by now.
But, did you know you can validate the import of config? Maybe not, that's a bit more of a hidden gem.

If you're writing a module that uses config entities or config for settings, you might want to make sure that the config is valid when imported. This came up recently in the core Content Moderation module. We wanted to make sure that when config for Workflows are imported they don't end up removing moderation states which are in use by other entities. For this we added an event subscriber, ConfigImportSubscriber, which is now part of Drupal 8.4.x.

To add one of these to your modules first you will need to add a service to your modules services.yml file:


content_moderation.config_import_subscriber:
class: Drupal\content_moderation\EventSubscriber\ConfigImportSubscriber
arguments: ['@config.manager', '@entity_type.manager']
tags:
- { name: event_subscriber }

Here you will see, for content moderation, we have the service name, the class, a couple of arguments, and it's tagged as an event subscriber.

You will then need to add you subscriber class within your module add the namespace added in services.yml.

<?php

namespace Drupal\content_moderation\EventSubscriber;

use Drupal\Core\Config\ConfigImporterEvent;
use Drupal\Core\Config\ConfigImportValidateEventSubscriberBase;

/**
* Check moderation states are not being used before updating workflow config.
*/
class ConfigImportSubscriber extends ConfigImportValidateEventSubscriberBase {

/**
* {@inheritdoc}
*/
public function onConfigImporterValidate(ConfigImporterEvent $event) {
if ($this->isWorkflowInUse($event) {
$event->getConfigImporter()->logError($this->t('The workflow @workflow_label is being used, and cannot be deleted.', ['@workflow_label' => $workflow->label()]));
}
}

}

Here, with a lot of the logic removed for simplicity, we're extending ConfigImportValidateEventSubscriberBase, which is the base class for all validation of config at import. Then we override the onConfigImporterValidate() method, which passes in the $event object. From that object we need to determine if the config is valid or not, then log an error if not.

This can also be tested in a kernel test:
$this->config('workflows.workflow.editorial')->get(); will get us the existing config for the editorial workflow entity.
\Drupal::service('config.storage.sync')->write('workflows.workflow.editorial', $config_data); will allow us to update that config.
$this->configImporter()->reset()->import(); will then run an import, imported the updated config. This will need to be wrapped in a try/catch because a failing import will throw an exception. In the catch part of the statement we can pass or fail the test depending on the intended result.

For more information take a look at the full event subscriber: http://cgit.drupalcode.org/drupal/tree/core/modules/content_moderation/…
Also the full test: http://cgit.drupalcode.org/drupal/tree/core/modules/content_moderation/…

Jul 30 2017
Jul 30

As someone who has been using Drupal for over 9 years, attended 8 DrupalCons, spoken at 4 for of them, been on the DrupalCon track team twice, and was co-lead for DrupalCon London I've feel I know Drupal events pretty well.

This year for DrupalCon Vienna things have changed a little. Firstly back in February a few program changes were announced. These included things such as summits, training, sprints, and t-shirts. Some of these are now back on the agenda due to community efforts, but why stop there with community efforts.

This year I was part of the PHP track team with Campbell Vertesi, Michael Cullum, we wanted to try and get an awesome line up, with some top name speakers. As Drupal has started embracing aspects of PHPunit, Doctrine, composer, guzzle etc it'd be great to have people involved with those projects speak at DrupalCon. However the issue is funding! They're not going to get a great deal out of DrupalCon, so won't fund themselves. Their employers have no reason to send them, so they won't fund it. DrupalCon doesn't pay speakers, so that's not an option. So, Campbell had a great idea of crowdfunding. We asked two speakers, Sebastian Bergmann (of PHPunit fame) and Michelle Sanver (of PHPWomen fame). They submitted multiple sessions each, which were equally ranked and chosen along with other sessions. Now comes the time to get their funding.

On GoFundMe there is now a fund running to "help bring (more) PHP to DrupalCon", at the time of writing it's close to 2/3rds of the goal.

This is a very controversial topic, and with the other changes brought in at DrupalCon Vienna has sparked a lot of discussion. After laying off many members of staff over the last few years, and DrupalCon Europe never to make a profit things are tough. Although this sort of experiment is good to see, it's good to see people talking about it, and it's good to see people getting behind it.

https://www.gofundme.com/php-at-drupalcon

See you in Vienna!

May 16 2017
May 16
Programmatically install Drupal 8 module timmillwood Tue, 16/05/2017 - 07:41

There are times, often in tests or upgrade paths, where we want to programmatically install a module. Here's how:

\Drupal::service('module_installer')->install(['workspace']);

Tags drupal-planet drupal drupal 8

Comment

Submitted by Rupert (not verified) on Fri, 19/05/2017 - 00:56

Permalink

Can you say a bit more about this? Where does this code go?

Submitted by timmillwood on Thu, 25/05/2017 - 11:36

In reply to by Rupert (not verified)

Permalink

This code can go pretty much anywhere. My most recent use for it was in a test where I wanted to add some content, then install a module, and test how the module reacted to the content which was there before the module was installed. For this I just added the code in the test method of a functional browser test.

Add new comment

May 03 2017
May 03
Submitted by timmillwood on Wed, 03/05/2017 - 11:40

Place this code within .htaccess underneath RewriteEngine on.


RewriteCond %{HTTPS} off
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

 

Tags drupal-planet drupal drupal 8 ssl

Add new comment

Apr 30 2017
Apr 30

In his keynote at DrupalCon Baltimore 2017, Dries talked for some time about how Drupal is now for Ambitious Digital Experiences. There has been a lot of talk over the last few years, especially since Drupal 8 was release, that Drupal is now an enterprise CMS. With this keynote it seems as though Dries is, in a way, acknowledging this. Ambitious Digital Experiences reads as something more complex than a blog, a brochure site, or sites for SMEs.

With this in mind, and recent discussions about the Future of Drupal, maybe it's time to put together a sort overview of where we are and what we have in Drupal.

Drupal 8 took nearly 5 years to develop and had over 4,000 people contribute code to it. There are now 11 core committers for Drupal 8, with a further 3 committers tasked with Drupal 7. Listed in MAINTAINERS.txt are over 60 subsystem maintainers who oversee the development of parts of Drupal known as "subsystems".

When downloading Drupal 8 the majority of the code sites within the core directory, and from there is split into a number of distinct parts. There are 22 Drupal components, which are independent libraries and don't depend on anything else within Drupal, there are all available via github. On top of are over 70 base subsystems, these may depend on components or other base subsystems, but don't depend on any modules. Finally there's over 70 modules, 5 themes, and 2 profiles, these may depend on any other parts of Drupal core. From these modules there are 12 that are (or were) experimental, experimental modules are here to add new functionality into core for testing purposes, but not yet fully supported.

The documentation for Drupal core is pretty awesome, and well worth a read for those looking for more information on the governance, "gate", or development workflow.

Apr 28 2017
Apr 28

As part of the Drupal Workflow Initiative we have critical issue relating to Content Moderation and translations. This is not actually a Content Moderation issue, but is just surfaced by Content Moderation because it allows you to create forward revisions. The video here should explain the issue:

[embedded content]

Forward revisions + translation UI can result in forked draft revisions and Only having one default revision per entity forces translations to be kept in sync are the related core issues.

Comment

Submitted by Agnar Ødegård (not verified) on Fri, 28/04/2017 - 19:36

Permalink

You have one field (revision) in the node table and you're trying to keep tabs on two or more pieces of information. You may succeed by creating rules to keep revision number across languages to be the same, but as you point out, when reverting revision for one language you really don't know the correct revision for the other language(s).

I think we'd avoid a lot of hurt by introducing a new table translation_revision. The id in this table will replace the revision id in the node table. The new table will know which revision is current for any language.

This will for sure break BC, but in the end I think it will be the cleanest solution.

Add new comment

Apr 23 2017
Apr 23
Submitted by timmillwood on Sun, 23/04/2017 - 22:00

A few weeks ago I recorded an updated screencast of the Workspace, Deploy, and Relaxed Web services module. Here it is:

Please comment, tweet, IRC, or Slack any questions.

Tags drupal-planet drupal 8

Add new comment

Apr 21 2017
Apr 21

Over the past month there has been a lot of focus on Drupal, the community. More recently it seems people are back to thinking about the software. Dave Hall and David Hernandez both posted eye opening posts with thoughts and ideas of what needs doing and how we can more forward.

A one line summary of those posts would be "We should slim down core, make it more modular, and have many distros".

To a degree this makes sense, however it could cause divergence. Core is not great at all following the same pattern, but contrib is even worse. As part of the Workflow Initiative specifically there is a lot of work going on to try and get the Entity API aligned, get many more entity types revisionable and publishable, using common base classes, traits, and interfaces. If we maintained Node, Block Content, Taxonomy, Comment, etc all as separate projects then there's a chance less of this would happen. Also by doing this we are laying out foundations and setting examples to be followed.

One solution to this may be to follow Symfony (yet again), they have a monolithic project but then split this up into the various components, which are "read only" repos. It's be pretty awesome if we could do this with Drupal. From there we could make Drupal downloadable without many of the core modules. People with the right skills can create a composer.json file to pull in exactly what parts of Drupal are needed, others could use a form on d.o to select which parts are wanted, which downloads a compiled zip.

What would be more awesome is if we could abstract more of Drupal out of Drupal. Imagine if the Entity API was a PHP generic library. Imagine if you could create a Laravel or Symfony app with Nodes. This would be really tricky, especially since Dries announced the plans to make Drupal upgrades easy forever, but possible.

Currently most Drupal sites are still on 7, and here we're talking about what would be Drupal 9? Maybe we need to take step back and look at why sites aren't being upgraded. Dave mentions "A factor in this slow uptake is that from a developer's perspective, Drupal 8 is a new application. The upgrade path from Drupal 7 to 8 is another factor." Although another reason is also why would a company spend the tens of thousands upgrading to Drupal 8? It looks at works (from a users point of view) the same as Drupal 7. Drupal is a CMS, a Content Management System, and the management of content is more or less the same. Yes, with initiatives like Workflow and Media this is changing, but even then similar functionality can be achieved in Drupal 7 with contrib modules. Will Drupal 8 be the version to skip? go straight from 7 to 9?

As Drupal is now pretty firmly an enterprise platform we need to look at this from a marketing point of view. What is going to sell Drupal 9? Why are people going to upgrade? What do they really want? Is a slimmed down core and more modular application really the selling feature that's wanted?

Drupal is a CMS, quoting Dave again "do one thing and do it well". We need to focus on making the authoring experience awesome, and the workflows that go along with it awesome too. This should all be done in a consolidated way to make managing Node content, Block content, etc just as intuitive as each other. If during this process we can also make things more modular, and less Drupally, that'd be awesome!

Feb 15 2017
Feb 15

Drupal 8 has two revisionable entity types, Node and Block Content. When working with revisions it's key to understand the different states a revision can have.

If you have a node with the ID 1 you can load it with Node::load(1);, this will load the default revision. You may have 200 revisions of node 1, but the default revision is the one which is loaded by default.

Currently forward revisions are revisions with a higher revision ID than the default revision, and past revisions are revisions with a lower revision ID than the default revision.

Creating a forward revision is as simple as creating a new revision, which is not the default.

$node->setNewRevision(TRUE);
$node->isDefaultRevision(FALSE);
$node->save();

The code above assumes $node is an existing node revision object. First we enforce a new revision created, then set that new revision to not be the default revision, and save. If you were to look in the database node_field_data would have the original revision information as it's still the default, but in node_field_revision you will see a revision with a higher revision ID (vid), this is the forward revision we just created.

Using the same code without the isDefaultRevision(FALSE) line will create a new default revision as this is the standard behaviour when saving an entity. The original revision will then become a past revision.

Dec 08 2016
Dec 08

We have been extremely hard at work this year with the Workflow Initiative bringing some quite big underlying changes to the Entity API and some great advances in core such as the Content Moderation module. There are now around 8 weeks until feature freeze for 8.3.x and there’s still a number of key things we want to get in. Please take a look at our kanban board and see if there’s anything you’d like to take a look at even if it’s just a quick review, a little feedback, or a +1.

At the time of writing this post there are 8 Workflow Initiative issues in Needs Review:
Which core entities get revisions?
We are looking at making more content entities in core revisionable, this will roll out gradually, but which ones should we make revisionable?

Add a content entity form which allows to set revisions
This is mainly an effort by the Media Initiative, but affects the Workflow Initiative too. The idea is to have a generic entity form for revisionable entities.

Add a publishing status field to BlockContent
Shock horror! Block content entities are currently not able to be unpublished. We need to fix this! This will be a pattern for making other things able to be unpublished too.

Upgrade path between revisionable / non-revisionable entities
This patch has had so many iterations but the current one is looking pretty awesome. It will give ups an upgrade path for entity types such as taxonomy terms and menu links to have revisions.

Introduce sorted set key value store
Before we introduce Workspaces in core we want to store a sequence of entity updates, a sorted set key value store will allow us to do this.

Store revision id in originalRevisionId property to use after revision id is updated
There are a number of bugs in core which require us to know what the revision id of an entity object used to be. This issue solves this by adding a simple property.

Introduce a EditorialContentEntityBase class for revisionable and publishable entity types
When we make more entity types revisionable and publishable it’d be great if they all extended the same base class. This issues does this for Nodes, we will then be able to roll it out to Block content once publishable, and other entities once revisionable.

WI: Content Moderation module roadmap
Content moderation is currently in core as an alpha experimental module. How can we get this to beta then stable?

Sep 06 2016
Sep 06

For the last 18 months I have been working full-time on Drupal workflow solutions, first with the Deploy suite of contrib modules, and now with the core Workflow Initiative.

It turns out core is a lot harder than contrib. Everything we have mapped out in the plan for the Workflow Initiative is already done, and working, in the contrib modules. So getting it all in core should be easy, right. Just port over the code from the modules into core APIs or core modules and it’s done. Well… no.

Everything in core needs to be generic, it needs to work for everyone. In contrib you can choose not to use module X if it’s not compatible with module Y, or if it doesn’t work with your use case, in core there is less of that luxury.

Also in core everything has to be done “properly”. We can’t just get a solution that works well enough and run with it, we need to be sure it is the best (or least worse) solution.

Some examples of the issues we are facing so far is with the Upgrade path between revisionable / non-revisionable entities issue. In contrib we use the migrate module to migrate all content out, make entity types revisionable then migrate everything back. For core this is not stable enough so we need a custom set of update hooks. Then Add StatusItem field type to store archived state in contrib we just add a new base field to all entity types to store the archived state. In core there are performance concerns with a new field and new conditions.

Thankfully there are so many people working and thinking about things that will help some of these areas, and as we continue to develop core we should be able to make things easier.

Aug 25 2016
Aug 25

Drupal 8.2.0 will see a bunch of new experimental modules. Once of these is Content Moderation. This is a port of the Drupal 8 release of Workbench Moderation, with a number of updates to make it suitable for core.

Work is currently underway to get Content Moderation working for Workspaces. Workspaces, defined by the Workspace module and it’s dependency Multiversion, are a way of creating different versions of your site’s content and getting the elusive “full site preview” functionality. With Content Moderation you are able to moderate a whole Workspace. Then once published all content will be deployed to the live workspace.

The screencast below shows adding some content on the Stage Workspace and confirming the content doesn’t exist on the Live Workspace. Then publishing the Stage Workspace results in the content being on both Workspaces.

[embedded content]
Jul 11 2016
Jul 11

Throughout the software development world there are many “evangelist” roles who sell the code to the community, but maybe we need the other side? Maybe we need to sell the community to the those who are just there for the code.

Drupal is famed for its community, with the slogan “Come for the code, stay for the community”, but as Drupal starts to evolve into a more enterprise platform it’s expected to see more organisations coming for the code, and staying just for the code. Why should we care?

Drupal Cores lists 3635 contributors to Drupal 8 core. Without these people you wouldn’t have Drupal. If you’re not supporting these people your business model is flawed. Many organisations sell Drupal to clients and / or use Drupal themselves. What would happen if Drupal wasn’t a sustainable platform anymore?

The Drupal Association doesn’t have anything to do with the code itself, but they do run the platform that packages the code, hosts the code, tests the code, markets the code, as well as many other roles within the community. This year and last year the Drupal Association had to lay off a number of staff members due to funding issues. Organisations really need to get behind the Drupal Association otherwise there will be no Drupal. There are a number of ways you can support the Drupal Association, and it’s great to see more and more non-dev-shops listed on https://www.drupal.org/organizations.

At Appnovation we have had a lot of growth over the last 2 years and with this growth community contributions have not kept up. Therefore we’re currently working on a community contributions program to try to inspire the company as a whole to work closer with all the open source communities. We’re also embedding this within the sales and pre-sales process too, so we can ensure our clients know about and understand the open source communities behind the software we’re using with them.

It’d be great to hear your thoughts, ideas, and views. In return there will be more blog posts with progress updates.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Jun 27 2016
Jun 27

The Workflow Initiative was announced just over a month ago and since then I have been working on the first phases full-time. Here’s what I’ve touched:

In Drupal 8.1.x the RevisionLogInterface was introduced, but not used. In 8.2.x the Node entity will use it. Also BlockContent entity will use it.

When we make everything revisionable we’ll need all entities to get their base fields from the base class. So from 8.2.x all content entities inherit their base fields.

To help people make sensible decisions Node revisions will now be enabled by default.

We got into a pretty annoying issue when trying to add revision fields to entities which already had data because the revision field couldn’t be set to NOT NULL. Andrei wrote an awesome patch to allow an initial from field to be set. We therefore set the revision id as the entity id.

When we make everything revisionable we’ll need an upgrade path, to test this the Shortcut entity is being upgraded to revisionable by a new service. This has still not been committed, so reviews are welcome.

We’re trying to get Workbench Moderation into core as Content Moderation. Still lots to do to make this happen, but the patch is there with passing tests.

Initial work has also started to get an archive field into all entities. This will allow us to have CRAP (create, read, archive, purge) workflow, create a trash system, and replicate entity deletion through environments.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by Disqus
May 16 2016
May 16
Submitted by timmillwood on Mon, 16/05/2016 - 09:36

Last week I presented the plan for the Drupal Workflow Initiative at DrupalCon New Orleans. Please review the ever evolving plan at: https://www.drupal.org/node/2721129.

Workflow Initiative from timmillwood Tags drupal drupal planet drupal-planet DrupalCon

Add new comment

May 06 2016
May 06

Single site content staging with Deploy

[embedded content]

This demo shows creating content on a stage workspace then deploying it to live. Once deployed an edit is made on the live workspace and an update is done on stage to pull from live.

Cross site content staging with Deploy

[embedded content]

Now with RELAXed Web Services installed cross-site deployments can be done. First replicator users are setup with the permissions to replicate content. These users are added to the Relaxed settings on Drupal 1. A remote is added to Drupal 1 for Drupal 2. The live workspace is updated to set the upstream workspace to Live on Drupal 2. Content is created on Drupal 1 and deployed to Drupal 2. A change is then made on Drupal 2, and Drupal 1 is updated to pull the changes from Drupal 2.

Feb 29 2016
Feb 29

This year I have proposed three sessions for the North American DrupalCon based on the topics I have been working on for the last 9 months.

Please take a look, ask questions, make comments, and share.

Building sites with Composer
If you have been keeping up to date with by blog you will notice I have been working a lot with Drupal and Composer. In this session I will share this knowledge and look from a site builders perspective.
https://events.drupal.org/neworleans2016/sessions/building-sites-composer

Managing and staging your content
In Barcelona and Mumbai, Dick Olsson and I, presented on content management and content staging solutions. We have been continuing to work on improving these solutions and wish to give an updated session in New Orleans.
https://events.drupal.org/neworleans2016/sessions/managing-and-staging-your-content

Drupal is a CMS, so how can we better manage content?
This session will discuss many of the same topic as the earlier session “Managing and staging your content” however from a much more technical point of view, and with the goal of, how do we get this into Drupal core. Drupal 8 now has configuration management, it’s time we also focussed on content management.
https://events.drupal.org/neworleans2016/sessions/drupal-cms-so-how-can-we-better-manage-content

Feb 03 2016
Feb 03

As of 1:23pm GMT on Febuary 3rd 2016 there are no dependencies in the Drupal 8.1.x git branch.

Why?

There was no reason to have them there. It makes the repository bigger, and therefore takes longer to download. No one else does it.

What does it mean?

Not much really.

If you download Drupal via the zip file or tarball then the drupal.org packager will run composer so the dependencies will be there for you.

If you submit a Drupal core patch on drupal.org then DrupalCi testbot will run composer install.

It’s only if you clone Drupal 8.1.x or higher directly from git, you will need to run composer install in the Drupal root directory

Top tip: composer install --prefer-source --no-interaction can often be quicker.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Jan 22 2016
Jan 22

Drupal 8 is out, Drupal 8.1 will be out before we know it, but it seems contrib is still catching up. One question that seems to keep coming up is around installing a Drupal 8 module. In this post I will look at the different ways to install a module and resolving dependencies.

Deploy, Search API Solr Search, and Commerce are just a few of the modules with Drupal 8 releases that require dependencies loaded via composer. This means you can’t just download the module’s ZIP file, unzip it in your modules directory, and enable it. You need to install the dependencies.

One simple way to do this is the Composer Manager module. This module has extensive documentation on how to use it. Essentially what it does it merge the composer.json that ships with core and the composer.json files from all the contrib module you have, then downloads the dependencies. You may notice that this will also update core dependencies, but this is a good thing! The core composer.json has been written in such a way that it won’t introduce API breaking dependencies and only uses stable releases. So you will benefit from any bug fixes or security fixes rolled out in these dependencies before they’re rolled out with Drupal.

The two other ways we’re going to look at involve directly using Composer.

Just the dependencies
It’s possible to carry on installing Drupal modules exactly as you always have, download the zip or tarball, then unzip it into your modules directory. As mentioned earlier this will not install the dependencies, therefore you will need to look inside the module’s composer.json file, see what the dependencies are, and install them manually. Let’s take Deploy module as an example, this depends on the dev-master version of relaxedws/replicator. So, go to your Drupal docroot and run the command composer require relaxedws/replicator:dev-master. This will add relaxedws/replicator to Drupal’s composer.json, download it, and put it in the vendor directory ready for the module to make use of. This will not change any other other dependencies you have. Then to update the dependencies you can either run composer update to update relaxedws/replicator and all core dependencies or composer update relaxedws/replicator to just update the relaxedws/replicator package.

The module too
If you install all you Drupal modules via composer, all of the dependencies will automatically be installed too. First you will need to add a new repository, so run composer config repositories.drupal composer https://packagist.drupal-composer.org in your Drupal docroot, this will add the https://packagist.drupal-composer.org repository to your drupal composer.json. Now you can install any module from Drupal.org via composer. So going back to the example of Deploy you can run composer require drupal/deploy:8.1.0-alpha5 in your Drupal docroot and it will install Deploy in the modules directory. It will also install key_value, multiversion, and relaxed, which are all Drupal modules required by Deploy. Furthermore it will install relaxedws/replicator as we know is a PHP package needed for Deploy, and doctrine/couchdb which is a PHP package needed for releaxedws/replicator.

I hope this helps those confused what to do with Drupal 8 modules that have composer dependencies. Now go do the smart thing, rebuild your site using Composer!

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Dec 21 2015
Dec 21

For the last 6 months I’ve been helping Dick and Andrei with a number of Drupal modules to enhance the management of content.

Multiversion

This module enhances the Drupal core Entity API by making all content entities revisionable. Revisions are enabled by default and not optional. This means that edits to users, comments, taxonomy terms etc are created as new revisions.

Another ground breaking advance is that deleting any of these entities now just archives it. Delete is a flag in the entity object, and just like any other update, it creates a new revision.

The concept of workspaces has also been added, this allows for a new instance of the site, from a content perspective, can be created. An example use case for workspaces would be to have a dev, stage and production workspace, and move content between them as it gets promoted through the workflow.

Trash

Now that deleting content entities just means a new revision marked as deleted we need a way to recover or purge them. The trash module is a UI on top of Multiversion allowing users to do just this.

Relaxed Web Services

Drupal 8 has always been about getting off the island, Relaxed Web Services furthers this by getting content off the island. It uses Drupal core’s REST API to expose CouchDB compatible endpoints. This means that replicating content is just a case of using CouchDBs replicator. Then creating a decoupled Drupal site is as simple as using PouchDB.

This works really well with Multiversion’s Workspaces, where each workspace is exposed as a separate CouchDB database.

CouchDB Replicator

So that we don’t need to depend on CouchDB for replication, the replicator has been rewritten in PHP. This will allow us replicator content from within Drupal or even via Drush.

Deploy

There is a long history for Deploy in Drupal, but now in Drupal 8 it’s little more than a UI for the PHP based CouchDB replicator. It allows replication of content between workspaces, between Drupal sites, and between CouchDB databases.

Mango

Something we’re currently working on is Mango, inspired by MongoDB and based on Cloudant’s implementation for CouchDB. Mango will allow querying for content entities over the Relaxed Web Services API. This is going to be very interesting to those creating decoupled sites because PouchDB supports the same querying API.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Dec 18 2015
Dec 18

Drupal Content Hub

Creating a Content Hub using CouchDB and Drupal via the Deploy module.

[embedded content] Please enable JavaScript to view the comments powered by Disqus. blog comments powered by Disqus
Dec 18 2015
Dec 18

Deploy suite for Drupal 8

I just uploaded a screencast of how to use Deploy module in Drupal 8. It covers a few of the different possible use cases.

[embedded content] Please enable JavaScript to view the comments powered by Disqus. blog comments powered by Disqus
Oct 20 2015
Oct 20

Over the last 7+ years working with Drupal, one question always asked by clients is, how do I copy my content from staging to production? Generally the answer has been, “Don’t! Just add it on production”. Another option is to use the deploy module, which allows for the deployment of content between environments.

Drupal core has changed greatly in Drupal 8, the deploy module is following suit.

Multiversion
The multiversion module will be the foundation of all this change. It introduces three big changes to the way content is managed in Drupal.
Workspaces - All content is associated with a workspace, you can have multiple workspaces and switch between them. Think of them much like branches in version control systems such as git.
CRAP - Create, Read, Archive, and Purge. This is an alternative to CRUD (Create, Read, Update, and Delete). It means that in multiversion no content is deleted and everything is a new revision. Deleting an entity will just archive it, flagging it as deleted.
Revisions for everything - All content entities are revisionable with multiversion module. This means blocks, user, comments etc all have revisions.

Relaxed
The relaxed module is a RESTful API which is aligned to the CouchDB API. It exposes each workspace as a Couch database. This allows it to be replicated to another Drupal site running relaxed, to a native CouchDB database, or to something else that uses the same API, such as PouchDB.

Deploy
Finally the deploy module. This is now just a UI module as all the the content deployment is handed with external projects. The relaxed module exposes the API needed, a PHP based CouchDB replicator uses a PHP based CouchDB client to replicate / deploy the content.

The current plan is to have a “Deploy” button in the Drupal toolbar, this will open a modal where you can choose which site you want to deploy to and which workspace you want to merge into. It will be possible to deploy to a workspace on the current site. For example, a staging site may have two content editors, both working on different workspaces, then can then deploy / merge their content to the default workspace. This default workspace can then be merged up to the production site’s default workspace.

Tagging is also an idea being worked on. It will allow you to tag a workspace at a current point in time, then continue to edit content. When deploying from a tag it will only merge up changed until the tag was created.

These are big changes for the deploy module, but something that will make the content workflow a lot easier.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Oct 02 2015
Oct 02

This was a question I got from a client. So I set to work on finding a solution to alert the team who needed to know when a page has changed.

TL;DR: CacheTags.

Drupal has an awesome caching layer. It makes use of cache tags, these tags can be invalidated when something changes. For example if this is a view called “frontpage”, it would output the cache tag “view:frontpage”. This view lists a bunch of nodes, each with their own cache tag, “node:1”, “node:2” etc. When node 2 is edited, it’s cache tag and the frontpage view cache tag would be invalidated.

Taking this into account if we want to know what pages have changed, we need to know what cache tags the page has, then what cache tags have been invalidated.

The client already has a crawler to check their sites, this can look at the X-Drupal-Cache-Tags header and capture the cache tags the page has. So all we need now is a way of telling the crawler what cache tags have been invalidated.

Welcome to the CacheTag Notify module. This has a simple settings page where an endpoint URL can be added. Then every time a cache tag is invalidated it gets POSTed to the endpoint as a JSON string. The crawler will then need to to lookup which pages are using the invalidated cache tags, then it knows which have changed.

The CacheTag Notify module works by adding CacheTagsInvalidator service. The invalidateTags method is passed an array of invalited tags, this is then POSTed to the endpoint url using Guzzle. Overall a very, very simple, but effective solution.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Sep 23 2015
Sep 23
[embedded content]

At DrupalCon Barcelona this year I presented with Dick Olsson outlining a plan for CRAP (Create Read Archive Purge) and revisions (on all content entities) in core.

Phase 0

For Drupal 8.0.0
Enable revisions by default (https://www.drupal.org/node/2490136) on content types in the standard install profile and when creating new content types.

Phase 1

For Drupal 8.1.0

  • Improve the Revisions API performance, some of this will come from moving elements from the multiversion module into the entity API.
  • Enable revisions by default for all content entity types. So not just nodes anymore but blocks, comments, taxonomy terms etc.
  • Introduce a revision hash, parents and tree. Each revision needs to have a parent so you know where it’s come from, each parent can have multiple child revisions.
  • Data migration - Moving all 8.0.0 sites to 8.1.0 will mean moving their data to the new revision system.

Phase 2

For Drupal 8.2.0

  • Remove the ability to not have revisions. To simplify the API and the data stored it makes sense to remove the ability to disable revisions. This will allow us to remove all the conditional code around if an entity has a revision or not.
  • Delete is a new flagged revision. When deleting an entity a new revision will be created and this revision will be flagged as deleted. This is the archive element of the CRAP workflow.
  • Introduce purge functionality. There may be times when an entity needs to be completely deleted.
  • Commit trash module to core. Trash is just a UI for the delete flag. It displays all entities marked as deleted. It then allows these to be restored by creating a new revision not flagged deleted, or purged by removing the entity.

Simple right?

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Sep 11 2015
Sep 11

At heart Drupal is an awesome CMS. The reason I would advise going for Drupal over any other framework is it’s ability to manage a scalable amount of content entities.

Earlier this week I discussed revisions in Drupal 8. Specifically the use of Multiversion module to allow Drupal to use a CRAP (Create Read Archive Purge) workflow for content management.

Since then I have been working on the archive and purge aspects of this workflow. The Multiversion module is pretty astonishing in how it revolutionises the way revisions are handled and content is managed but so far there has been little focus on the archive and purge process. When entities are deleted they are simply flagged as “deleted”, they exist in the Drupal database but are gone from any where on the site’s UI. Until now…

The all-new Trash module builds on the Multiversion module provide a place where users can see all their deleted entities, listed by entity type, then restore them along with all their revisions. The first commit on the 8.x-1.x branch was only done 3 days ago on September 8th, so it’s still early days, but the MVP is available as a dev release to have a play with.

Now, this is more or less the archive process working, next up is purge. So what do we purge? Well, ideally, nothing. It’d be great if we never need to truly delete any entities, but instead purge specific revisions. This is something that will be getting a lot of thought over the coming weeks.

Another step to look at is compaction. Multiversion has been following CouchDB’s API and coupled with the Relaxed web services module it allows entities to be replicated to CouchDB and other platforms following the same API. CouchDB has a notion of compaction via the “_compact” endpoint. How harshly should the content be compacted? remove all revisions apart from the current one?

All of this will be discussed by Dick Olsson and I, in our session Planning for CRAP and entity revisions everywhere in core at Drupalcon Barcelona.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Sep 07 2015
Sep 07

I’m sorry to say that on the surface little has changed with revisions in Drupal 8, but there is some work still being done.

The first step, for me, is to get revisions enabled by default. I was interested to hear from webchick that this came up indirectly during user testing. Most Drupal sites I have worked on have been large enterprise organisations. These types of companies all want an audit trail and all want to effectively manage their content. This is easily achievable by using revisions. It looks as though many new to Drupal are in a way scared by revisions. Enabling revisions by default will take away the reasoning for being scared and give this awesome feature out of the box.

The current revisions system isn’t perfect, it’s pretty good, but not perfect. There’s a lot of work being done to improve upon in. In Drupal 7 we had the deploy module, which allowed users to move content from one site to another, such as from staging to production, this built upon the existing revisioning platform. In Drupal 8 we have the multiversion and relaxed web services modules. These were both demoed by dixon_ at Drupalcon Los Angeles.

Although multiversion builds upon Drupal 8?s core revision system it’s vastly different with an awesome direction. One key feature is it enables revisions for every entity, users, comments, everything. The way it displays revisions as a tree also makes it clear and understandable as to how the revisons work and how the relate.

The relaxed web services module builds upon a similar ethos as the deploy module, but it makes use of an API based on the replicatio.io protocol to handle bi-directional content replication. This means it can replicate content to any other replicatio.io protocol source such as couchdb or pouchdb.

Step by step Drupal 8 is getting more amazing.

To find out more about all this head along to the Planning for CRAP and entity revisions everywhere in core session at Drupalcon Barcelona on Tuesday 22nd September at 11am in room 122-123 “Interoute”.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Sep 02 2015
Sep 02

In a post last week I discussed versioning in Drupal and briefly touched on version numbers in info files. A lot of my focus over the last few months has been around composer and Drupal, one issue for Drupal when using composer for contrib modules is that the info file is pulled from git and therefore doesn’t have the version number, project name or datestamp in the info.yml file that is added by the drupal.org packager.

I am currently working on a patch for the update module which will get the project name from the module’s composer.json. This is just the first step because the project name is what’s needed for the module to show on the update status page.

The patch adds a composer parser service, which like the info parser service that parses the info.yml file of a module, this parses the composer.json file of a module. From here we can get the project name such as “drupal/devel”, then by exploding that we can get the project name from the second element in the array.

Composer.json files sometimes include a version number so we could use that too, but this is vary rare. The only real option for the version number is the git branch or git tag, this is how the git deploy module works.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Aug 25 2015
Aug 25

Currently Drupal has naming conventions for branches and tags in git for contrib module. These are based on the core version then the module version, for example 7.x-1.x would create a dev version 1 for Drupal 7, 7.x-2.3 would create a stable release version 2.3 for Drupal 7.

As we head towards Drupal 8 there has been a lot of talk about versioning for contrib. Core has already moved to using semantic versioning (semver), which is widely adopted by a lot of software now. Contrib is still on the old version numbering format. There is a lot of discussion about switching to something like semantic versioning for contrib. It’d be ideal to keep the core version somewhere in the number, there have been many suggestions, some of which are:

My preferred, and it seems the favourite so far is 8.2.3.0, this is much like semver but the major version has been bumped down to make way for the core version, therefore core.major.minor.patch. The only possible issue here is if using composer for contib modules, composer will think the core version number is the major version number, and it will think the major version number is the minor version number. I feel that this is an ok compromise and we as developers can take this into account when using the syntax in composer.json.

Another versioning related issue is that we currently add the version number into the contrib info file as part of the packager that creates the module zip or tarball files. If more people pull modules straight from git (via composer) they won’t have this version number, causing the core update module not to work.

I say we should start asking module maintainers to add the version number to the info file. They already have to add the version number to the tag or branch, surely it’s no extra effort to add it to the info file, and saves so much effort trying to find a fancy programatic solution.

Update:
As part of this I have opened another issue on d.o to discuss how we can make update module work for modules installed via git (including via composer).

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Aug 10 2015
Aug 10

Now that we have an initiative to get get a composer.json file in each contrib module, we cal start getting them all on Packagist.

Back in April the Drupal community decided on a naming convention for projects on Drupal.org. Projects must use the package name drupal/PROJECT where PROJECT is the part from the URL.

Using drupal as the vendor name in the package name allows us to lock down Packagist to a select list of maintainers. Currently this is the people who submitted a package to Packagist under the drupal vendor name before May 7th when the restriction was added.

The Packagist API currently allows us to update existing packages, so we could quite easily add a hook_node_update and hook_node_insert to drupal.org that will do this when project releases are updated or added. The API doesn’t offer a create endpoint, so we can’t automatically create new packages, an issue on Github has been opened for this, and I’ve started work on a pull request.

Once we have the Packagist pull request done, and the hooks added to drupal.org we can use the API key from Packagist for one of the maintainers (maybe Dries?) to push any new module or existing module updates across. Packagist will read directly from the drupal.org git repo and we’ll finally have one canonical place for all Drupal modules and PHP dependencies.

Drupal.org issue: https://www.drupal.org/node/2547617

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Aug 07 2015
Aug 07

Drupal 8 has seen a lot of love for Composer. As various posts have mentioned, it’s possible to build a whole Drupal site with little more than a composer.json file. However contrib needs the same love too.

In this post I want to talk about step one, and it requires little effort at all.

For module maintainers

Add a composer.json file to your Drupal 8 module. The example below gives everything you need. The name uses “drupal” as the vendor namespace, followed by the module name after the slash. The description provides more information on the module. The type tells composer it is specifically a Drupal module (there are also types for themes, profiles and more), this allows developers to put “drupal-module” packages into the modules directory rather than the vendor directory on composer install. Finally we add Drupal’s license of GPL 2.0+.

{ "name": "drupal/example", "description": "This is an example composer.json for example module.", "type": "drupal-module", "license": "GPL-2.0+" }

For everyone else

  • Tell everyone.
  • Tweet about the issue or this post.
  • See if modules with a Drupal 8 release have a composer.json, if not add an issue for it.

What does it mean

Once all modules have a composer.json we can add modules using composer more easily. We can also start step two, which is to get all Drupal 8 modules on Drupal.org listed on Packagist.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Jul 27 2015
Jul 27

Since July 2014 there’s been a feature in Drupal 8 to override backend specific services. There are over 25 services in Drupal core that are backend overridable.

In this post we’ll look at a simple way to override services using aliases, which comes downstream fro symfony.

A simple service is the “user.data” service. It allows data to be stored, fetched and deleted, relating to a user account. For this there is an interface, UserDataInterface and this is implemented by UserData. We need to create a module that adds a new service that implements UserDataInterface.

Firstly create a folder for your module, something like alternative_userdata, then the file alternative_userdata.info.yml, in there we can define the module’s info:

name: Alternative UserData type: module description: Adds alternative storage for user data. core: 8.x dependencies: - user

So we are setting the name of the module, the type, a description of it the core version and that it depends on the user module.

Next we need to define our service, for this you wil need to create the file alternative_userdata.services.yml. In here we can add the service called alternative_userdata.user.data and set the class.

services: alternative_userdata.user.data: class: Drupal\alternative_userdata\AlternativeUserData

Next is to create the class, add a folder named “src” (following PSR-4 autoloading standards) and within this a file called AlternativeUserData.php.

<?php /** * @file * Contains \Drupal\alternative_userdata\AlternativeUserData. */ namespace Drupal\alternative_userdata; // Here you may need to add the use operator to pull in any backend client. /** * Defines the alternative user data service. */ class AlternativeUserData implements UserDataInterface { // You may need to add a protected variable here for your backend client. /** * Constructs a new user data service. */ public function __construct() { // Here you can create a new instance of your backend client. } /** * Implements \Drupal\user\UserDataInterface::get(). */ public function get($module, $uid = NULL, $name = NULL) { // This needs to return the user data from your backend. } /** * Implements \Drupal\user\UserDataInterface::set(). */ public function set($module, $uid, $name, $value) { // This needs to save the user data to your backend. } /** * Implements \Drupal\user\UserDataInterface::delete(). */ public function delete($module = NULL, $uid = NULL, $name = NULL) { // This needs to save your user data to your backend. } }

Now that we have a service defined in our module, the module can be enabled, however it won’t do anything until we set an alias in your site’s services.yml file.

Edit services.yml (normally at sites/default/services.yml in your Drupal 8 codebase) and add the following:

services: user.data: alias: alternative_userdata.user.data

Now when Drupal looks to use the user.data service it will actually use the alternative_userdata.user.data service from your module.

Simple, right?

Edit: Lee Rowlands expands on this in his post Overriding services in Drupal 8 - advanced cases.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Jul 24 2015
Jul 24

Since February 2012 I have been the maintainer of the Statistics module in Drupal core. Since then I have overseen two pretty big changes to the module.

Admittedly this was after in 2011 I tried to get the module removed from Drupal core.

The first big change was to remove a good chunk of the module, the access log. This tracked things like referrers and visitors, which most people do in a service like Google Analytics. This was committed by Dries to Drupal 8 in early 2013.

The second big change was to use an AJAX call to count a node view. This allowed the module to work even when a site was using a reverse proxy cache such as Varnish. As well as getting committed to Drupal 8, this was also back ported to Drupal 7.

Now I am on a mission to breath new live into the module.

A patch I’ve been working on for the last 3 years is to give the statistics module a swappable or overridable backend. Since July 2014 it has been possible to define a service as being “backend_overridable”, and this is exactly what I’m proposing for the statistics module. The patch waiting to be committed creates a service and moves all database queries to there, this can (and already is) be overridden by a contrib module.

This will bring a huge performance boost of allowing statistics to be written to CouchDB, MongoDB, Redis or anywhere.

The next step then is to get the statistics module to count views for all entity types and not just nodes. This is a legacy feature that is left over from Drupal 6 which didn’t have entities. Hopefully this can be committed to Drupal 8.1.x.

If you want to help out, please join the issue queue.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Jul 24 2015
Jul 24

Over the last few weeks I’ve been spending a lot of time with Drupal 8 and Composer. This has lead me building up a PoC for a client and diving into the issue queues and IRC. In this post I wanted to document some of the processes I’ve been looking at.

Creating a project

The people behind drupal-composer have put together a template, a Drupal project can be started from the template using the command composer create-project drupal-composer/drupal-project:8.x-dev drupal --stability dev --no-interaction. This will create a folder called “drupal”, in there you will find “web” directory containing the Drupal installation. It has also downloaded drush, and two modules, devel and token.

Adding modules

If you open up composer.json in your drupal project folder you will see a repository with the URL https://packagist.drupal-composer.org defined. This is a custom version of Packagist setup by the drupal-composer team. If required packages are not found on Packagist they will get pulled from here.

You will also notice in composer.json under the “require” section is where drupal/token and drupal/devel are added. You can add any module on the Drupal Packagist to this then run the command composer update to update your project and download the newly added modules.

In the “extra” section of composer.json you will see a number of installer paths are added, this tells Composer (via the required composer/installers package) where to put things. You will see everything is going in the web directory, Drupal core in “web/core”, modules in “web/modules/contrib” etc. Therefore, when you call the composer update command to add the new modules you required, these automatically went into the correct Directory ready for Drupal to use. Composer knows these are Drupal modules because the modules have a composer.json files too (often dynamically added by the Drupal Packagist because Drupal doesn’t require modules to have a composer.json yet). In this composer.json the type is set to “drupal-module” for modules, “drupl-theme” for themes, etc.

Patching modules

Greg Anderson went into this in a lot of detail on the Pantheon blog earlier this week, but using the cweagans/composer-patches package you can define a patch file to use. For example, add the following to the “extra” section of your composer.json file to patch the token module:

"patches": { "drupal/token": { "Description for reused fields not correct": "https://www.drupal.org/files/issues/token-Fix_description_for_reused_fields-2497251-5.patch" } } }

You’ll see the package to patch is defined, then within that a name or discription of the patch, followed by the URL for the patch file.

Custom modules

There are a number of ways you can handle custom modules here. You could create a folder at web/modules/custom and just put them in there, or you could add them via composer. In the PoC I’m working on we have many custom modules that will be added to multiple projects. The custom modules have their own git repo, and if they’re not on Packagist or the Drupal Packagist (which they shouldn’t be if they’re custom modules) we need to tell Drupal about this repository. In composer.json under the “repositories” section add something like:

{ "type": "vcs", "url": "https://github.com/timmillwood/couchdb_statistics.git" }

In this case we’re adding a repository that contains the “drupal/couchdb_statistics” package. You could add "drupal/couchdb_statistics": "dev-master" to the “require” section of your composer.json to add this Drupal module to your modules directory.

If you have a lot of custom modules you’re adding to multiple sites it might be worth you setting up Toran Proxy. This is a project by Jordi Boggiano, the guy behind Composer and Packagist. It allows you to proxy your git repos and packagist. You then add your Toran Proxy installation to the “repositories” section of you composer.json, then require any packages you have there. Give Toran Proxy a Github token and it can also grab your private repos too.

One thing to note about repositories is:

> Repositories are only available to the root package and the repositories defined in your dependencies will not be loaded.

For the PoC project mentioned earlier I looked at creating a sub-project which just contained a composer.json. This then required all of the common modules, custom and contrib. However the custom ones were not getting pulled in because all the custom repositories were not defined in the root project, they were only defined in the sub-project. Having Toran Proxy as a single source meant that we could add it to the root project and all dependencies could also get pulled from there.

Post install

You may noticed the drupal-composer template has a scripts directory and this is defined in composer.json as a “post-install-cmd”. This is run after composer.install. The script add settings.php, services.yml and the files directory. You could customise this to do a number of other things. Run drush commands, setup a vagrant box, etc

Summary

Composer is here in Drupal 8 and it’s awesome. You can run, develop and deploy your whole Drupal 8 project with Composer. You can add and patch contrib and custom module, as well as themes, profiles and other PHP packages. The drupal/couchdb_statistics module mentioned earlier requires a couchdb client, this will all get pulled in via composer with a single command.

Move over Drush make, this is how you should be running Drupal 8.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Jan 24 2014
Jan 24

Although not advised, it's actually possible to get Drupal up and running on Heroku.

First download Drupal, this is easy with drush

drush dl drupal

and install the Heroku Toolbelt. Then cd into the Drupal directory, make it a git repo and push to heroku

git init
git add .
git commit -m "initial commit"
heroku create
git push heroku master

The heroku open command will open your app in the browser, you should see the drupal installer, but we don't need this, we need to manually add settings.php and commit it to git.

heroku addons:add heroku-postgresql
heroku config

This will give you the database details you need to add to settings.php. We also need to remove (or edit) the gitignore file to let us commit settings.php.

rm .gitignore
cd sites/default
cp default.settings.php settings.php

Edit settings.php to add the database details, then commit and push to Heroku.

git add -u
git add *
git commit -m "adding settings.php and removing gitignore"
git push heroku master

Running heroku open will open your app with an error page. Add /install.php to the end of the URL and run through the Drupal installer. On the third step you will see a warning about "Unicode library". This is because the default Heroku PHP buildpack doesn't have mbstring, however you can roll your own buildpack or use a third-party one to add mbstring. For now, just click "proceed with the installation". After completing the installer you will get logged into your site and can start using Drupal.

Due to Heroku's ephemeral filesystem all files Drupal creates will not be stored if the dyno is stopped or restarted. Also some pages such as "/admin/reports/status" and "/admin/config" cause timeout errors. This issues could be resolved by creating a custom buildpack and copying all Drupal generated files to Amazon S3. Although, there are better solutions for Drupal hosting, so maybe not worth the effort.

Have fun!

Jul 04 2013
Jul 04

When doing any sort of creative work picking the right tool for the job is important. The right brush for a painting, the right brick for a house, or the right programming language or framework for web development.

Clients often try to pick the language or framework they wish their site to be developed in without a clear understanding of the benefits and constraints. It should not be the clients role to choose the platform, this should be down to a developer or a technical project lead. Although putting this at the developers feet will often see your site developed by whatever the developer is most familiar with or a fan of.

When I first started using Drupal I must admit entering fanboy status and thinking that all sites could and should be built using Drupal. It turns out that although all sites could be built using Drupal it is not always the best option. Also similarly a framework such as a Ruby on Rails may not be the option if all that is needed is a Blog or content managed site.

As a personal preference the following frameworks would be used in these situations:

  • Basic fairly static site - Plain HTML or static site builder Jekyll
  • Basic Blog - Jekyll or Wordpress
  • Content rich dynamic site - Drupal
  • E-commerce - Drupal commerce or SaaS app such as Shopify
  • Custom workflow or SaaS app - Ruby on Rails
  • API - Sinatra

Many of these choices come down to personal experience and each developer may say different. So when looking for someone to build your new site or web app don't just pick a framework. Ask around, see what developers would use and why.

What do you use and why?

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web