Jul 20 2018
Jul 20

Illustration of workflow concept

Did you know that setting up a content workflow is included in Drupal 8 core? It can be easily set up by simply turning on the Workflow and Content Moderation modules. The Workflow module gives you the ability to define a workflow and the Content Moderation module sets up a simple workflow for drafts and the ability to create more content moderation workflows.

Introduction to Content Moderation

If you aren’t familiar with content moderation in Drupal, let’s fix that with a quick overview of what that means. Without this module, Drupal content can only be in one of two states, published or unpublished. The content is either available to the public or it isn’t. Different users with different permissions can manage the publishing status certain types of content. All of this combines to make a pretty useful and normal experience. This should also seem like a normal experience for anyone that has ever used content management system.

Where the Content Moderation module comes in is when this workflow needs to be more than just on or off. This can allow for content to be placed into different statuses or states. With built-in states available, like Draft, content can be staged by one editor and then approved by a user with different permissions. In the publishing world, this matches workflows that exist for paper content, so this is an attractive feature for those in that vertical.

Set up in 5 minutes

Content Moderation in core provides a very simple but useful workflow. Simply turning on the Workflows and Content Moderation modules adds the Editorial workflow. The Editorial workflow adds the Draft state and sets up the content workflow to go from Draft to Published and provides an admin view for drafts that need to be moderated. Using permissions you can restrict authors to only be able to create and edit drafts. Then grant the "Transition drafts to published" permission to your editors and boom!—you have content moderation set up in a matter of minutes.

View of content moderation in Drupal core

 

If you are running a Drupal 8 site and have multiple content contributors that need feedback, there is little reason to not use moderation. The out-of-the-box Content Moderation should be able to handle most situations. If that doesn’t quite fit the needs of your content workflow then there is still good news, you can create a custom workflow.

Custom workflows

If you need a more complex workflow, you probably still don’t need to write any custom code. If it’s just content type, block, or paragraph based entities that you are building a workflow for, you can just create a new workflow based on the "Content" moderation type. Different states can be defined, for example let's say you have the states draft, ready for review, ready for second review, reviewed, and published. Next you need to define the transitions, for example one transition for the above states would be “move ready for review to ready for second review”. These transitions are going to be what you give different user roles permission to do. After that is set up, you and your team are ready to roll with your new workflow.

Setting up a complex workflow in Drupal 8 Core

 

Another example of a custom moderation workflow could be for a site that publishes to multiple platforms. A workflow could be set up to allow the editors of different platforms to approve content onto the system they manage. Let’s say you have a front-facing site, a native app, and an internal portal. You can create a workflow that goes through moderation for the front-facing site and then adds content to a queue to be published or declined for each of the other outlets. This is just one of the many possible use cases for a custom workflow.

Illustration of workflow in Drupal 8 Core for publishing to multiple platforms

If you need to extend Workflow further, maybe for a custom entity, you can write a WorkflowType plugin that covers your needs. This can be used for any entity that needs to change states, so think beyond the idea of content moderation. It could be used for steps in a manufacturing process, or steps for ordering in a restaurant app, the possibilities are limitless.

Do you need it?

Workflows are super powerful and moderation comes mostly ready to go with Drupal core but does that mean you should always use them? On some sites with only a handful of admins and not a lot of roles, it may be more cumbersome than useful. Just because workflows are an option, it shouldn’t be implemented unless your users understand the human element of a workflow. Moderators should know their role in content moderation. If most of your authors have admin privileges and can just push content straight through the flow, then your workflow is mostly standing in the way of being efficient. Every good workflow should start with a meeting of your entire editorial team, and have things worked out on a whiteboard or Slack channel first. Workflows are amazing as long as everyone understands their role in the publishing chain. 

New Call-to-action

Jul 18 2018
Jay
Jul 18

drupal 8 logo featured alongside drupal logos showing the growth of Drupal

Now that Drupal 8 has gained some momentum, it is time to start planning out your upgrade strategy. You want to upgrade to get the latest benefits and take advantage of the future stability that comes with the direction that Drupal will be taking from here on out. Before upgrading you will want to consider some things about what your current site has. In this article we will be covering some of those questions with some context to assist in the decision making process. Let’s determine if you website is adequately serving the current needs of your business and which content will need to be brought over to the new Drupal 8 site. There may be a difficulty in the switch, but being prepared will put you in position to handle whatever comes up.

So, you have a nice Drupal 7 site that's been running happily for years, and maybe you're thinking it's time to upgrade – or, if you haven't given it much thought yet, there's certainly plenty of reasons to do so now. But, Drupal 8 is a pretty big step forward from Drupal 7. What exactly is an upgrade going to entail? What do you need to consider before you do so? We've upgraded numerous sites now, and today I'd like to go over a few things that I think are important to know before starting on such a project.

Before You Start

Just because you've decided to upgrade, doesn't mean that you can do it right away and be done with it by dinner. Unfortunately, there isn't any one-click solution to take your site from Drupal 7 to Drupal 8 (although future upgrades should be closer to reaching that level of simplicity). Here are a few things to consider before you touch the first line of code or content:

Do you still want the same site? Doing a large upgrade such as this is an excellent time to re-evaluate your site's functionality and appearance. Now that you've had it for some time, is there anything you want to change? Any new features you want to add that maybe weren't feasible before because of how the old site was built? Although upgrading can be done in such a way that everything stays almost exactly the same for your site's visitors, it's good to at least consider what potential improvements you could make, perhaps by leveraging some of Drupal 8's great new features. Maybe your site just needs a facelift, or has some long-standing bugs you want to fix, or you want to take advantage of new CSS or JavaScript technologies which weren't available when you first built the site. Upgrading is the perfect time to take a fresh look at things.

How are you migrating? One critical part of most upgrades is to migrate content and user information from the old site to the new one. Databases for Drupal 8 are structured differently than they were in Drupal 7, which means that to upgrade the site you can't just swap out the code while keeping the same database… you have to actually move the content into a new database as well and adjust it to fit Drupal 8's slightly different structure. This is a pretty big departure from how Drupal upgrades worked in the past.

There are two main ways to do this migration. Drupal 8 includes a Migrate API, which is perfect for some simple sites. However, not all contributed modules have the plugins necessary to migrate their data in this way, and any data you might be storing in some customized way certainly doesn't. Although it is possible to write your own plugins, in our experience it's easier to take a different approach by writing an entirely custom migration. This does however require having some more familiarity with Drupal coding than the Migrate API route does. Generally, what a custom migration involves is writing a script and appropriate queries to first retrieve your information from the old database, and then to write it to the new Drupal 8 database in the appropriate format.

Do you know enough Drupal 8? If your site is mostly built using core and contributed modules and themes, you'll probably be good to go. Drupal 8's admin UI does have some changes from Drupal 7's, but none of them are very difficult to get used to and most introduce new ways to navigate and improve your site. However, if your site uses much in the way of custom modules or if it has a custom theme, then before upgrading you should make sure your developers and themers are familiar with Drupal 8, perhaps by working on some other simple Drupal 8 project first or by getting some training on the changes, either in-person or through an online course. The changes in how you code custom modules and themes in Drupal 8 are significant, so they may take some getting used to, but ultimately they are the foundation of what makes Drupal 8 leaps and bounds better than its predecessor. Knowing how to use them effectively is key to having a Drupal 8 site that continues to be maintainable for years to come.

Still Not Sure You're Ready to Upgrade?  Learn more about the benefits and our upgrade process. 

What to Expect

Alright, so you've gotten everything figured out. All that's left to do now is to, well, actually upgrade the site. Here are a couple of things you should plan for when starting most any upgrade.

Expect to have a few difficulties: Even if the upgrade seems like it will be a simple one, you should go into it expecting to have a few unexpected challenges come up, because they almost certainly will. Maybe it's one particular type of field that doesn't have the same settings in 8 as it does in 7. Maybe it's a combination of contrib modules which used to work together one way and don't anymore (one example: The Context module can no longer be used in conjunction with Metatags to set metadata in contextual circumstances). Maybe it's that your site uses a lot of different webforms, and although the submission data can get migrated without too much trouble, recreating the forms themselves can't be done automatically. 

Regardless of what it ends up being, plan to have some unexpected difficulties during the upgrade. In my experience, these challenges tend to be fairly easy to fix or work around through some combination of contributed modules and custom code, so although they can add some complexity to the upgrade it's rare to find a dealbreaker so late in the process.

Expect to have two sites for a little while: Your new Drupal 8 site is completely separate from your old Drupal 7 site, and that means that it needs to be installed somewhere other than where your old site is, and it probably has to have its own domain so you can access it during the upgrade process. You'll be able to keep your old domain though - once the upgrade is done and your Drupal 8 site is ready to go, all you should have to do is change your main domain's settings to point to the new website rather than the old one.

When you first start the upgrade, your Drupal 8 site is likely to be almost empty. If you have lots of custom modules to recreate or anything and the migration itself isn't done yet, then maybe it has some placeholder content to start with. Then, once the migration is done, all the relevant content from the old site gets migrated into the new one. At this point, your content and user information is being stored in both sites. It also means that, from now until the new version of the site launches, any changes you make to the content of the old site won't be automatically reflected in the new site and will be lost when you deploy the upgrade. For many sites, this problem can be mitigated by having a period of time in which changes get made to both sites, instead of just to one of them.

Still, it isn't ideal to have to enter new content twice for an extended period of time, so to solve this, one thing we like to do is to actually have two migrations. The first migration happens early on in the development of the Drupal 8 site, and we use the content from that migration while finishing most of the upgrade. Then, shortly before the upgrade goes live, we do a "final migration" to get any changes which have been made on the Drupal 7 site since the original migration. If all goes well, this results in only a day or two (or zero!) during which changes have to be made in both sites.

Once the upgrade is finished and your domain name settings adjusted so that the new site is live, it's worthwhile to do one last check of the old site to see if there are any recent changes or user registrations that may have been missed. Depending on how your migration script works, these could be migrated again with that, or they could be replicated manually. Once that's done though, huzzah! Your newly upgraded Drupal 8 site is finally ready for the world to see.

 Offer for a free consultation with an Ashday expert

Jun 15 2018
Jun 15

As you may or may not have noticed, we’re having a lot of fun over here at Ashday building Drupal sites with React. Check out our own site, for example. We are really digging this new direction for front-end and you can learn more about why we did it how we approached it in other articles, but here we are going to talk about how we approached the Drupal editorial experience, because honestly - we just didn’t find a lot of great resources out there discussing how this might be done well in a decoupled experience.

Something about Gift-Horses

While we can’t speak with authority on the potential detriment of literally looking gift-horses in the mouth, we can speak to the idea that we should be grateful for what we’ve been given, and Drupal has given us a lot! If you were in this industry when most of us built everything from scratch, you’ll know that it’s a crazy pile of work to do the most basic things and nothing is taken for granted. Need a login form? Sure thing. Oh, now you need flood control? Um, ok. Captcha? Boy. Password reset? Ugh. Ok, I need a week just to get user authentication in place.

Drupal does so much out of the box and we aren’t about to throw it all away so we can call ourselves fully decoupled, which is the idea that Drupal isn’t providing any of the front-end at all. It’s worth noting that some are pursuing the concept of “Progressively Decoupled,” where only select components of the front-end site are managed with React, but we don’t prefer that approach in most cases because we don’t want the overhead of taking on traditional Drupal theming and a React build out, yet with less obvious design constraints, potentially extra hosting, multiple development workflows, duplicated styles, etc - leaving us short of many of the benefits of going decoupled at all.

We prefer to an approach that we’ve, for the moment, dubbed “Deliberately Decoupled.”

Deliberately Decoupled

What we mean by “Deliberately Decoupled” is that we aren’t decoupling purists, who see ourselves just as much evangelists of a particular approach as we are just software engineers, and we also aren’t operating by a FOMO (fear of missing out), where anxiety about not being on a bandwagon drives our decisions. We prefer to leverage what we think is beneficial for our clients and the site, and secondarily abide by our preferences. A good example is the Open Source philosophy. We love open source! But we aren’t for a minute going to bypass a proprietary library that does something really cool just because it’s not open source. It’s the same with decoupling. We want what gives us good bang for the buck - either in the deliverable, or the cost, or the UX, or whatever else is impactful - and it helps far more than it hurts. So, for the largely public facing ends of our sites, we hands-down love what React is giving us and rarely find Drupal’s out of the box front-end solution to be easier or more flexible. On the back-end? For the editorial and admin experience, we really have no interest in trying to replace everything Drupal provides unless there is a major overarching requirement or clear benefit that can justify hundreds or thousands of hours of additional work. There are some projects that do merit it, such as when you have highly dynamic and interactive forms, but if that’s not the case Drupal can do the job on its own.

Considering Outsourcing? Consider Ashday!  Request your free consultation today. 

One prime example of leveraging cool stuff in Drupal is the use of Paragraphs for content. If you haven’t heard of Paragraphs, you really must check it out. It’s been our favorite way to give editors the ability to create interesting and dynamic content creation without the cringe-worthy experience of complex WYSIWYG and HTML source editing, especially when it comes to the mobile experience. Nearly all of the Ashday.com pages are built with paragraphs, so that means an editor can do parallax, add related content boxes, throw in some CTAs, etc. Pretty cool! And as you’ll see later, we leverage React for our pages, but Drupal’s admin for editing the paragraphs and it creates a clean and intuitive editorial experience.

So does all this mean for editing we just hand over Drupal Admin accounts to our users so they can swim through the Sea Nettle-like admin experience say good luck? Hardly. Let’s get into the nitty gritty.

Our Philosophy - Less is More

Generally speaking, Drupal doesn’t matter much to the user. Of course Drupal matters, but not really to the end user just trying to do their editorial work. Not as far as they are aware. And they shouldn’t have to be “aware”. Do you remember when you first looked into Drupal? Do you remember how weird the world was as you learned that a “node” is “content”, but then so is a “block” (kinda), and a “view” is a query but maybe with a page attached, or a “taxonomy” is hierarchical data, but with display pages? All frameworks have to be abstracted sufficiently, which means making up generic terms and concepts that only make sense when you’ve spent time with them. Well, guess what? Most users would really rather not just learn all of those concepts and instead just easily write content and update their site. The key to that is to reduce, simplify and obfuscate.

Drupal’s admin is very friendly to developers, but unnecessarily verbose when it comes to editors. They don’t need half the contextual menus, vertical tabs, sidebars, etc most of the time. And the stuff they do need should be stripped down to the obvious and helpful. So let’s hide tons of help text on a WYSIWYG with multiple text format options and use Paragraphs + a simple WYSIWYG instead. Let’s rename any buttons or links that have the word “node” in them because really, who cares if it’s a node? Let’s put our field labels inline to save vertical space and use something like Field Group to organize them cleanly. And let’s take away that rats’ nest default Drupal menu and create an alternative that just gives them what they care about. Here is an example of what we’re talking about.

From this…

Example of Drupal admin showing contextual menus, vertical tabs, and sidebars.

To this. 

Example of streamlined Drupal Menu. 

In addition, let’s take the coolness of contextual admin, like Drupal provides, and make it more intuitive as well. If you hover over a block you can edit it or configure it. If you’re on a node page, you get view and edit links in the tab bar. And you can even sometimes use the new inline editing features in Drupal 8. The problem, though, is that once again the user has to understand what each of these elements are and why they’re all different in terms of the triggering UX. We are having a lot of fun in React finding more consistent and creative ways to manage content without understanding it.

So here’s a rudimentary example using Ashday.com.

Example of Drupal contextual admin, showing node page.

You see some sorting icons and some edit icons. That’s it. Technically, those sortable sections are paragraphs, which - thanks to React - can be sorted inline - but are also editable individually apart from the others. The “Free Consultation” button in the header is a React button tied to a few settings stored in some custom Drupal configuration in the back-end. Further, that edit icon in the page title area can be used to edit the entire page at once so you can change the page title and things like meta tags, although the next step is to probably provide more direct access to edit those things so that you don’t have to know what all is buried in a node form. So the goal here is for the user to have just some basic concepts to think about, like edit and sort, instead of Edit Node vs Configure Block vs Edit Block vs Sort Paragraph vs Some Other Drupal Configuration Buried Deep In the Admin or Form.

And by the way, here’s what happens when you click the edit link on the header button, vs the first paragraph. 

 

Edit link on header button

Intuitive Drupal 8 admin

  

Edit link on first paragraph

Screen shot of admin interface for Drupal page built with React.

 

So despite varied back-end architectural implementations, the user has a very similar experience contextually editing something. A simple page mask + offcanvas + clean form. Its cool stuff, all made easier with the ability to re-use display elements in React all while still getting a ton out of Drupal’s back-end for content management and forms. Simple, kinda easy, and totally user friendly.

Looking to use react for your next digital project? Find out how Ashday can use React to make your project a success.

Jun 08 2018
Jun 08

Background

I am the sysadmin and developer for Art & Object, a Drupal 8 website built with the Drupal Composer project. The version pin in composer for Drupal was 8, which in hindsight was too broad for our usage. Meaning Drupal point releases (8.3 to 8.4) require study to ensure you understand all the implications, which wasn't something I did. I just blindly did a composer update, thinking everything would be handled automatically.

This really bit me when 8.4 came out because my server was running Debian Jessie, which runs PHP 5.6 and my composer didn't have a platform PHP configuration, so a lot of the underlying Symfony code updated to PHP 7. So I ended up doing a backgrade until I figured it out.

Then there were the critical security Drupal updates (SA-CORE-2018-002 and SA-CORE-2018-004) earlier this year that would not be released for 8.3, so I had to upgrade (or at least, at the time, I felt I had to, though I see now they have a patch for older 8.x releases). By that time, 8.5 was released, so I updated the composer to 8.5 and ran update and after some basic testing, moved on.

Then a few months later, I noticed the status error messages about running the contrib media module alongside the core and I knew I missed something and there was a problem.

I then started down a wicked rabbit hole of getting a local copy running and following the upgrade instructions, running into problem and going back to getting a local copy running fresh again and trying again. Lots of trail and error (mostly errors) and head-banging-on-the-desk. I looked for help on the #media IRC channel, but the best advice came from posting on Stack Overflow, where @sonfd pointed out that the media module needs to be uninstalled first. I thought I had tried that and ran into an error message that mentioned you can't uninstall the media module with media items already created.

The Fix

So after lots and lots (and lots) of local refreshes and trials and errors, here's the list I finally followed when it came time to upgrade production:
  1. First, put the site in maintenance mode. Then take a database backup and make a tarball of your project directory. Don't skip over this.
  2. drush pmu media crop_media_entity: pmu = pm-uninstall. Remove the media module (and crop_media_entity, if you have that, too). This was the tip from @sonfd that opened the rest of this process for me.
  3. composer remove drupal/media: Remove the contrib media module from the filesystem. I should add that I prefixed all my composer commands with /usr/bin/php -d memory_limit=-1 /usr/local/bin/ because I often ran into memory limits when running composer.
  4. composer require drupal/inline_entity_form drupal/crop:1.x-dev drupal/media_entity_instagram:2.x-dev drupal/media_entity:2.x-dev drupal/media_entity_slideshow:2.x-dev drupal/media_entity_twitter:2.x-dev drupal/slick_media:2.x-dev drupal/media_entity_actions: These modules are temporary to help upgrade the database records.
  5. composer remove drupal/video_embed_field: For some reason, I couldn't require video_embed_field:2.x-dev, so I removed it and then...
  6. composer update: When I ran this, it updated video_embed_field to 2.x-dev.
  7. composer require drupal/media_entity_image drupal/media_entity_document drupal/image_widget_crop: More temporary modules to help the upgrade process.
  8. drush cr: Clear cache to make sure Drupal picks up new modules and paths.
  9. drush updb: Run the database updates.
  10. drush pmu entity media_entity: Uninstall these modules (these were the old contrib modules)
  11. composer remove drupal/media_entity drupal/media_entity_image drupal/media_entity_document drupal/crop drupal/image_widget_crop
    /usr/bin/php -d memory_limit=-1 /usr/local/bin/composer require drupal/crop:2.x-dev drupal/image_widget_crop drupal/empty_page:2
    : Clean out the temporary modules from the filesystem.
  12. drush cr: Clear caches
  13. drush updb: Run database updates
  14. drush cex: Export the configuration (so you can commit it later).
  15. The blazy module had an error with the core media and hasn't been updated (as of this writing), but there is a patch to fix that. So I learned how to add patches to a composer file - turned out pretty simple. Add this to composer.json in the extra section:
            "patches": {
                "drupal/blazy": {
                    "Gets Blazy to work with Drupal Core Media": "https://www.drupal.org/files/issues/2881849-8.patch"
                }
            }
  16. composer update: This was odd, but I had to do an update, which picked up the patch, but didn't really install it. I can't remember exactly now, but I believe this actually deleted the blazy folder.
  17. composer remove drupal/blazy: So removing this actually installed it. Who knew? Whatever ... it's still in my composer.json and now the filesystem has the module and the patch.
  18. drush cr: Clear caches!
  19. For some reason, this upgrade created a new field called field_media_image_1 and assigned that as the source for the image media type, which broke some of the images on the site. So I edited media.type.image.yml file to revert source_field back to my original field_image.
  20. drush cim: Import my hack to get my media image type to work.
  21. I had a custom field formatter that I had to edit to change the namespace from media_entity to media.
  22. drush cr: Final cache clear!
  23. Test and make sure all is well. If so, take the site out of maintenance mode and commit your repo changes.

Advice / Conclusion

A lot of this pain could be negated by studying the release notes better. I own that and this counts as one of my many scars of lessons learned. I hope others can learn from my lesson, too. Someone may end up writing a meta post about this post to point out the high cost of maintaining a Drupal site and I don't think they'd be wrong about that, but that's the price you pay for running servers that are publicly accessible.
Jun 01 2018
Jay
Jun 01

Illustration of a person signing a document electronically.

Previously, I wrote a bit about the HelloSign eSignature platform and how it can be integrated into a Drupal 7 website. As promised, a Drupal 8 version of the integration is now available and ready for use on cutting-edge websites everywhere. But this new version is much more than a one-to-one upgrade of the original module— we've leveraged some of Drupal 8's great new features to make using HelloSign with your site even easier than it was before. Here are just some of the highlights of the new release:

Composing Your Site

Installing the old HelloSign module could be a bit of a pain. In addition to needing to download the HelloSign module for Drupal, it was also necessary to download a separate HelloSign PHP SDK and put it in the right place in your site's code. Now, the HelloSign module can be installed easily using Composer. Instead of downloading two files from two different sites, unzipping them, and carefully making sure to put them in the right place in your site code, all you have to do is run one simple Composer command:

composer require drupal/hellosign

That one command will install the module in the right place, and it also installs all of the module's dependencies, including the HelloSign PHP SDK and the Encryption module (more on that in a moment), making it even easier to get started with HelloSign in Drupal 8 than it was in Drupal 7.

Need to Integrate Drupal with HelloSign, or with Anything Else?  Request your free consultation with an Ashday Drupal integrations expert today. 

Security by Default

Many Drupal modules are available which, like the HelloSign module, help make it simpler to integrate with other services. One thing common to most such modules is that they add a configuration form on the Drupal site, which administrators can use to set up the integration by entering essential information such as credentials and API keys. The HelloSign module is no different; integrating with HelloSign requires the use of an API Key and a Client ID, and the module provides an easy way to set these (and other useful settings) through the administration interface.

One thing that often gets overlooked by integration modules, however, is the security of this saved information. Once the API key is configured, it is stored as plain text both in the database and (if you are using Drupal 8's configuration management feature) in the code of the site. Unfortunately, that means that if any ill-intentioned person somehow gained access to most any part of your site, they'd have access to your secret API key.

To mitigate this common issue, the HelloSign module in Drupal 8 makes use of the Encryption module to store this private information in a secure, encrypted format, decrypting it only when necessary to interact with the HelloSign API. This feature is on and enabled by default and, in fact, can't be disabled, because having such an option would make for inherently weaker security.

Built for the Modern Web

One issue with Drupal 7 was that it was not, on the whole, built with modern object-oriented PHP coding practices, and the modules built for Drupal 7 often followed suit. Drupal 8, however, is a whole new object-oriented world, and that's as true of the HelloSign module as it is of Drupal Core.

The Drupal 7 version of the HelloSign module had most of its features buried in a number of disparate functions, many of which were ultimately just wrappers for functionality made available in an object-oriented fashion by the HelloSign PHP SDK. For Drupal 8, we've simplified this. The HelloSign module provides a

service which, in addition to allowing the creation of signature requests, also provides direct access to the rest of the HelloSign API. Instead of making assumptions about what bits and pieces of the API's many features a site might need, now, all of it is available. The service also automatically establishes the necessary connection info, so that your developers never have to worry about how to connect to HelloSign.

In Conclusion

The HelloSign Module for Drupal 8 has all these improvements (the use of composer, enhanced security, and a modern object-oriented design) and many more, including vastly expanded documentation and more granular permission handling. We hope that it will be of use to you on your next Drupal project which needs to support eSignatures.New Call-to-action

May 25 2018
May 25

Illustration of search function on a website.

Search is an important facet of any large website these days. We’d talked previously about why you want to take full control of your site search. Bombarding your users with a mess of links won’t do anyone any favors. One of our favorite solutions for this problem is Apache Solr and recently we had the opportunity to set it up on Drupal 8. Let’s take a moment to go through a bit of what that solution looked like and some thoughts along the way.

Setting the stage

Before we dive too far into the how, we really ought to give a bit of time to the why. More specifically we need to cover why we didn’t simply use one of the existing modules contributed to Drupal by the community. At the time, there was one prominent module group for implementing Solr search in Drupal 8. The Search API module in tandem with the Search API Solr Search module were really the most direct way to implement this advanced search on your site. These are great modules and for a different situation would have worked just fine. Unfortunately, the requirements we were working with for the project were more specific than these modules were equipped to handle.

There were three key things that we needed control over and we aren’t keen on hacking a module to get something like this done. We needed to have specific control over what was indexed into Solr. The Search API module allows for you to generically specify how fields are translated to the Solr index, but if you need some different handling you would either need multiple indexes or you would need to sacrifice some of that customization. The site also needed to make use of a fairly complicated feature of Solr, the more like this query. (Warning, incoming search jargon!) This query allows you to search the index for content relevant to another indexed piece of content. This relevancy is determined by fields you specify in the query and results can be limited to content that meets a certain relevancy score threshold.

The last thing we had to have in this was the ability to manage how often content was indexed. The existing modules allowed for this action to happen on a periodic cron, but wasn’t able to have the index updated as soon as changes were made to content. This project was going to have a lot of content updated each day and that meant we couldn’t afford to wait for things to be indexed and updated. With these three things creating hurdles to getting Solr implemented in this project it seemed like we were going to have to go another way, but after looking at some documentation we determined that creating our own implementation would not be so difficult.

Brainstorm your next development project with  an Ashday expert!  Request your free session today. 

Solr search with Solarium

Before we get too far ahead of ourselves, we should note that this wasn’t done with a contributable module in mind. That isn’t because we don’t like giving back the the community, we totally do, it was because it was created for a very specific client need. There will likely be a more generic version of this coming out down the road if demand is high enough. Also, we are under the impression that most use cases are covered by the modules mentioned above, so that would be where most would start. Enough with the disclaimers; let’s talk Solarium.

We went with Solarium as the Solr client to use for this. That is what most of the existing Drupal modules use and it seemed to be the most direct way to do this with PHP. Installing Solarium is pretty simple with Composer and Drupal 8. (If you aren’t using Composer yet, you really should be.) Using a client for communicating with a Solr instance isn’t specifically required. Ultimately, the requests are just simple HTTP calls, but the client saves you from having to memorize all of the admittedly confusing query language that comes with using Solr.

Installing Solarium can be done as simply as composer install "solarium/solarium". You could also do this by adding a line to your composer.json file in the require section for "solarium/solarium": “3.6.0”. Your approach on this part may vary, but this should be done from the root of your Drupal site so that this library goes into the global dependencies for the project. These instructions are also detailed a bit more in the official docs for Solarium, here. The official docs also have a bunch of example code that will help if you dive into this like we did.

For this implementation, we opted to create a Solr PHP class to do the heavy lifting and made use of a Drupal service for calls to it from the rest of the app.


namespace Drupal\my_module\Solr;
use Solarium\Core\Client\Client;


class SolrExample {

  /**
   * Connection to the solr server
   *
   * @var Client
   */

  protected $solr;
}

The heart of the class is going to be the connection to Solr which is done through the Solarium client. We will make use of this client in our constructor by setting it up with the credentials and default settings for connection to our Solr instance. In our case, we used a config form to get the connection details and are passing those to the client. We wanted to use the configuration management system so that we could keep those settings consistent between environments. This allowed more accurate testing and fewer settings for developers to keep track of.


/**
* Solr constructor.
*/
public function __construct() {
 $config = \Drupal::config(‘example.solr_config’); //Normally we’d inject this, but for this example we’ll ignore that
 $settings = [
   'endpoint' => [
     'default' => [
       'host' => $config->get('host'),
       'port' => $config->get('port'),
       'path' => $config->get('path'),
       'scheme' => $config->get('protocol'),
       'http_method' => 'AUTO',
       'site_hash' => TRUE,
      ]
    ]
  ];
 $this->solr = new Client($settings);
}


We are doing this in the constructor so that we don’t have to create a new client connection multiple times during a given call. In our case, we ended up using this as a Drupal service which allows us to only have the Client object created once per call and gives a simple way to use this class throughout the app.

The next part is the actual search method. This does a lot and may not be clear from the code below. In this method, we take parameters passed in and build a Solr query. We have a helper function in this that does some specific formatting of the search terms to put it in the right query syntax. For most sites, this code would serve fine for doing generic searching of the whole index or having multiple versions for searching with specific filters

/**
* General Search functionality
*
* @param array $params
*
* @return mixed
*/
public function search($params = []) {

 $query = $this->solr->createSelect();

 $default_params = [
   'start' => 0,
   'rows' => 20,
   'sort' => 'score',
   'sort_direction' => 'DESC',
   'search' => '*:*',
   'time' => '*'
 ];

 $params = array_merge($default_params, $params);

// Building a proper solr search query with the search params
 $search_string = $this->getTextSearchString($params['search'], $params['time']);

 $query->setQuery($params['search']);
 $query->setStart($params['start'])->setRows($params['rows']);
 $query->addSort($params['sort'], $params['sort_direction'] == 'ASC' ? $query::SORT_ASC : $query::SORT_DESC);

 try {
   $results = $this->solr->select($query);
   return ['status' => 1, 'docs' => $results->getData()['response']['docs']];
 }
 catch (HttpException $e) {
   \Drupal::logger('custom_solr')->warning('Error connecting to solr while searching content. Message: @message',['@message' => $e->getMessage()]);
   return ['status' => 0, 'docs' => [], 'message' => 'Unable to reach search at this time. Try again later.'];
 }
}

The code we’ve presented so far isn’t breaking new ground and for the most part does a similar job to the existing search modules available from the Drupal community. What really made us do something custom was the more like this feature of Solr. At the time that we were implementing this, we found that piece to be not quite working in one module and impossible to figure out in another, so we put our own together. 

Thankfully with Solarium, this was a pretty simple query to tackle and we were able to have related content on the site without much other setup. We can create a new more like this query and submit an id so Solr knows which content to compare against for similarity. The rest of it behaves very similar to the search method presented previously. The results are still returned the same and we are able to do some other filtering to change the minimum relevancy score or number of rows.

$query = $this->solr->createMoreLikeThis();
$helper = $query->getHelper();
$query->setQuery('id:' . $id);
$query->setRows($params['rows']);
$query->setMltFields($params['mltfields']);
$query->setMinimumDocumentFrequency(1);
$query->setMinimumTermFrequency(1);
$query->createFilterQuery('status')->setQuery($params['queryfields']);

We didn’t share all of the code used for this here, obviously. The point of this post isn’t to help others create an exact duplicate of this custom implementation of Solarium in Drupal 8. At the time of this writing, it seems that the existing Solr modules might be in great shape for most use cases. We wanted to point out that if you have to dip into code for something like this, it can certainly be done and without an insane amount of custom code.

MIKE OUT

New Call-to-action

May 18 2018
Jay
May 18

Illustration of finger selecting 5 stars

Drupal 8 has been available now for more than two years, but if your site is up and running on Drupal 6 or 7, you may be wondering… why should I upgrade? And why now?

Well, if you're on Drupal 6, the answer is easy: Drupal 6 is no longer an officially supported platform, which means that in addition to not getting the latest and greatest Drupal features, your site may be vulnerable to serious security issues and you will have to find backports of the latest security patches from the community or attempt to create your own.

But if you're already on Drupal 7, then security updates and occasional small new features are still being made available for your version. So… why upgrade? Well, I can think of five great reasons to do so, and for why now is a better time than ever.

Contrib Is Ready

One common hesitation about upgrading to Drupal 8 has been that many key contributed modules from Drupal 7 were not yet upgraded, and many sites depend on having such modules. It's taken a little while, but especially over the past year we've seen a huge increase in the number of modules ready for use on Drupal 8… and many of them are even better than they were on Drupal 7. Consider, for instance, the popular Webform module. Not only is it available for Drupal 8, but it's been rebuilt and redesigned from the ground up. The interface for managing your webforms is much more intuitive than it used to be, while also having many more options. At this point, if there's some feature you want your form to have, then Webform can probably handle it.

Of course, Webform isn't the only major module that's ready. Popular modules such as Drupal Commerce, Domain Access, Paragraphs, and Metatag are ready to go. And let's not forget that many vital Drupal 7 contributed modules, such as Views and Media, are now included in Drupal 8 by default, so you don't even have to install anything extra to use them.

Considering an Upgrade to Drupal 8?  Claim your free consultation with a Drupal expert!

Easier Migrations Than Ever Before

One thing that may have had some people holding off on upgrading to Drupal 8 was that Drupal's new migration features weren't quite ready, and writing a custom data migration or working with a buggy migration tool can be quite a challenge. Well no more! The Drupal Migrate module is finally stable. This module is included with Drupal and is designed to make it as easy as possible to move all of your content from a Drupal 6 or 7 site to your shiny new Drupal 8 site.

One caveat: If your site is multilingual, then you might have to wait another version or two for the last few bugs with multilingual migrations to get worked out. But, that doesn't mean you have to hold off on building your new Drupal 8 site! Now would be a great time to get the new site built, and once the multilingual migration bugs are fixed, moving your content over at that point should be a comparatively trivial task.

New Features in Drupal Core

Using the most recent version of Drupal means that your site can easily start making use of the many new features that are always being added to Drupal core. For instance, Drupal 8.5 includes a whole host of new features, such as the Media module being included by default, a new Layout Builder module, and enhanced interfaces and settings for managing content and site configuration… and that's not even talking about the many significant bug fixes and other minor enhancements which are detailed in the full release notes. With feature releases such as these, sites built with Drupal 8 get fantastic improvements twice each year essentially for free.

By comparison, although Drupal 7 sites still get security fixes and the occasional other bugfix, the last time that there was a significant update for Drupal 7 was back in June 2017 with Drupal 7.55, and even then, the handful of improvements made were still small compared to the rate of improvement Drupal 8 has been seeing. Upgrading to Drupal 8 means you get more new features, more frequently, in addition to being able to use all of the great new features added to Drupal 8 in the five feature releases it has already had.

Ease of Upgrades

I've written a bit about this before, but I think it bears repeating: Once you've upgraded to Drupal 8, the greatest upgrade challenges should be over. Because of how Drupal 8 is designed, future upgrades to Drupal 9 and beyond should be a much simpler transition than moving to 8 from older versions.

This is important to realize because, prior to Drupal 8, many people would "skip" a Drupal version… going from 5 to 7, or from 6 to 8, since the older version would be supported with bug fixes until the new version was released. Many people may be in a similar state now, waiting to upgrade from 7 until 9 is released, but at this point, holding off like that really shouldn't be necessary. If the upgrade from 8 to 9 should be a simple change, like Drupal creator Dries Buytaert has said, then there's no need to skip 8 entirely and miss out on all the great improvements that it can bring to your site today.

Upgrade While There's Time

There's a balance to be struck when deciding when to upgrade to Drupal 8. Doing it right after Drupal 8's release may have been tricky for many sites, due both to some bugs present in the new version and to the many important contrib modules not yet having been upgraded themselves. At the same time though… the longer you wait, the closer you get to your Drupal 7 no longer receiving critical security fixes. Although it's not yet clear when official support for Drupal 7 support will end, it's easy to see that it will happen at some point, and you'll want to already have upgraded by the time it does.

To me, right now feels like the ideal time. Drupal 8 is ready (and better than ever!), and there's still enough time left for Drupal 7 that if you have a complex site, you can take the necessary amount of time to upgrade without being rushed. The worst situation to be in would be to start upgrading a complex website when there's only a month or two of Drupal 7 support remaining, since then you might have to rush the upgrade and end up with more bugs (or things missing from your site) than you'd have if you took the time to upgrade now.

Conclusion

All of this is to say: It's time. Even if you weren't able to upgrade before, due to lacking contrib modules or the need for the Migrate module, those should no longer pose a problem. You can upgrade today and get countless major improvements over previous versions of Drupal (with more coming all the time) and you can even rest easy knowing that the next "big" upgrade shouldn't be anywhere near as daunting as they have been in the past.

It's been more than two years, but finally, Drupal 8 is ready for your site. The question now shouldn't be "Why upgrade?"... it should be "Why wait?"

Is there another reason why you are waiting to upgrade that we didn't mention? We can probably help, schedule your free consultation today.Offer for a free one-hour consultation, make you next project a success

Mar 25 2018
Mar 25

Updating Drupal 8 core with Composer has proven to be a problematic process for many developers. For some, this is nearly as upsetting as the fact that the Composer logo is actually a conductor, and some have abandoned the platform entirely, opting to stick with Drupal 7.

The process isn’t always as simple as running composer update drupal/core and going about your day — the update from 8.3 to 8.4 was notoriously difficult and I recently experienced an issue while updating from 8.4.5 to 8.5.0. In this article, I’ve provided instructions for updating D8 core with Composer, plus some tips for dealing with common issues.

This is especially important now as we await a highly critical security update to all versions of Drupal, to be released on Wednesday, March 28, 2018. This level of security update is quite rare, but the update needs to be implemented on all sites as soon as possible after its release.

As the PSA linked to above notes, the Drupal Security Team will be providing security releases for unsupported minor versions 8.3.x and 8.4.x due to the issues many have encountered when updating from these versions. If you’re still on one of those versions, the update may be more straightforward if you stick with the release for that minor version.

General instructions for updating core

First, let’s cover the steps needed to update Drupal 8 core with Composer.

  1. To update the core package, run:

     composer update drupal/core --with-dependencies -v
    
    

    It’s recommended to run the update command with the --with-dependencies flag to update any of Drupal core’s dependencies as well.

  2. To capture any included database updates, run drush updb -y.
  3. To capture any included configuration changes, run drush config-export -y and commit the changes.

All three of these steps are necessary whenever the core package is updated.

Dealing with errors

Core version doesn’t update

If you run the composer update command but core isn’t updating, edit your composer.json file to include the specific version of core you want, e.g. ^8.5. Then, run the composer update command again.

Composer command outputs errors

Composer may not be able to resolve all of the dependencies of core and will output an error like this:

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Conclusion: don't install drupal/core 8.5.0
    - Conclusion: don't install drupal/core 8.5.0-rc1
    - Conclusion: don't install drupal/core 8.5.0-beta1
    - Conclusion: don't install drupal/core 8.5.0-alpha1
    - Conclusion: don't install drupal/core 8.6.x-dev
    - Conclusion: remove symfony/config v3.2.9
    - Installation request for drupal/core ^8.5 -> satisfiable by drupal/core[8.5.0, 8.5.0-alpha1, 8.5.0-beta1, 8.5.0-rc1, 8.5.x-dev, 8.6.x-dev].
    - Conclusion: don't install symfony/config v3.2.9
    - drupal/core 8.5.x-dev requires symfony/dependency-injection ~3.4.0 -> satisfiable by symfony/dependency-injection[3.4.x-dev, v3.4.0, v3.4.0-BETA1, v3.4.0-BETA2, v3.4.0-BETA3, v3.4.0-BETA4, v3.4.0-RC1, v3.4.0-RC2, v3.4.1, v3.4.2, v3.4.3, v3.4.4, v3.4.5, v3.4.6].
    - symfony/dependency-injection 3.4.x-dev conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-BETA1 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-BETA2 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-BETA3 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-BETA4 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-RC1 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.0-RC2 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.1 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.2 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.3 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.4 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.5 conflicts with symfony/config[v3.2.9].
    - symfony/dependency-injection v3.4.6 conflicts with symfony/config[v3.2.9].
    - Installation request for symfony/config (locked at v3.2.9) -> satisfiable by symfony/config[v3.2.9].

This happens when one of Drupal’s dependencies is updated and the new version requires an updated version of another package. To resolve this, include the dependency package causing the issue in the composer update command. The --with-dependencies flag this will ensure that the dependency’s dependencies are also updated. To fix the error above, I ran:

composer update drupal/core symfony/config --with-dependencies -v

You’re not alone

If you continue to run into problems, the best advice I can give you is to search for the specific update you’re trying to make. Every time I’ve had an issue I’ve been able to find discussions online regarding that specific update and potential resolutions.

In fact, when I got the error above while trying to update to 8.5.0, I found this helpful article by drupal.org user eiriksm and was able to resolve the issue. Check out the article and its comments for more discussion on how to deal with Composer issues when updating Drupal 8 core.

Nov 08 2017
Nov 08

diet-Drupal.pngDrupal and lightweight aren’t words that have historically gone hand in hand. Drupal 6 and 7 had some hoops to jump through to get something not included in the box. If you wanted to use an external service for displaying content or adding features you would feel that pain. Drupal had apparently not heard the phrase, “Less is more”. Does using Drupal really mean that you have to always deal with every Drupal layer to accomplish seemingly simple things? Luckily with Drupal 8 this isn’t the case any longer.

Hooks, hooks, and more hooks

Drupal is an awesome box of legos, you can build just about anything you want with it, that’s always been part of the draw for the system. The problem is that you had to have a lot of Drupal around to do it. You could do without all of the Drupal in the way by getting around it, but very quickly you would end up with a site that is in a bit of a Frankenstein’s monster situation and inheriting it would be a nightmare. If you needed something installed through some sort of package manager, you would likely have to do a whole lot of work to get it to play nice with Drupal, or you would have to sacrifice some of the benefits of using that system.

If you aren't interested in writing your own code, you could make use of the many contributed modules that are available. These little nuggets of awesome are great for adding a variety of features to your site, but they are bound by the same rules as above. Everything had to be added in a specific Drupal way and that caused a bit of overhead. Not to mention that if you have to use four modules to accomplish the feature you needed, then you also have four more to keep updated for security and stability. Not a bad solution, but it requires a lot more investment than should be necessary to allow for what should be simple, right?

Server shown with many cables tangled together.

I’m sure you’ll remember how this all went together later.

Your hooks can’t hold me!

Drupal 8 is a bit different in how you can extend it. This time around Drupal has partnered up with other industry experts, like Symfony, to make the system even more extendable and with less Drupal involved. That PDF converter library your friend likes that is installed with Composer can now be part of the project much more easily. You can even install Drupal itself that way if your hosting provider allows for that.

You can use many other libraries from a variety of sources, and they can integrate very simply while using very little Drupal code. This means you can easily add some other PHP, JavaScript, Python (if you’re into that), library and get the functionality you want without adding more reliance on other community contributed modules or deeper Drupal hooks. This opens the doors for using external services you might already be familiar with for things that Drupal already does as well. Want to use Solr or Google for searching on your site? Easy. Need your files to be stored somewhere separate from the website? Done. Want React to replace the display of your site? You get the idea.

More community contributed modules are becoming available for Drupal 8 and they are also taking advantage of the ‘less is more’ approach that comes with the upgrade. If you aren’t as technically inclined or don’t have a team that can deep dive into Drupal easily, then these will be a great way to add more features to your site. You may still need a custom solution for your needs, but these may get you closer before you get to that point. Alternatively, you could find an outside vendor that already has this all down and could expertly judge when contributed or custom solutions will work. Where would you find one of those you ask?

Brainstorm your next development project with an Ashday Drupal expert! Request your free session today. 

Want to get more out of Drupal? If you don’t know your Symfony from your Composer and you are stuck on these hooks, then I would suggest you try working with a team of Drupal integration experts. Ashday happens to be pretty good at this sort of thing and we aren’t shy to talk about it either. We have been working with Drupal 8 since before it was officially released and we are pretty big fans of all the great things to come out of it.

MIKE OUT

Offer for a free consultation with an Ashday expert

Nov 08 2017
Nov 08

This is part two of a two-part series.

In part one, we discussed how Drupal 8’s adoption in its first two years was a bit lackluster compared to what many expected. Grounded in a better understanding of the shortcomings of the past two years, we’ll try to equip those of you considering Drupal 8 with the information you need to make the best decision for your organization as you continue to invest in the powerful framework of Drupal.

Image of Drupal 8 adoption curve
Credit: Angie Byron AGAIN on “Everything you need to know about the top 8 changes in Drupal 8” from May 2015

Drupal 8 adoption is certainly no longer in the “early adopter” phase, yet we still haven’t entered the “majority” phase. For most organizations not yet powered by Drupal 8, our stance is: it’s probably time to upgrade. The value you’re missing out on with the newer software is real. Perhaps less obviously, if you’re investing in your Drupal 7 site beyond passive maintenance, you may well be doubling your long-term costs by deferring and exacerbating what will need to be refactored later. For organizations who have web staff, work with an agency, or do any non-trivial customization to their Drupal website, this applies to you.

A disclaimer upfront

Drupal agencies like ours benefit from upgrades in the short-term because they are usually a substantial undertaking. This fact, in part, is why over the past two years people have written more often about encouraging an upgrade to Drupal 8 rather offering a more holistic and measured perspective. A small dose of healthy skepticism typically serves site owners best. If Savas Labs is to live into its values we must factor in the needs of two other stakeholder groups when advising on an upgrade: our clients, and the collective Drupal community. Given the substantial effort to upgrade, if we focus solely on the short-term, we do our clients a disservice. In doing that, the next time those site owners and admins have the option to select a tool to power their web systems, they may look elsewhere remembering their pain and disappointment in recent experience. This ripple effect has the potential to create many former Drupal users. Imbued with the open source ethos, we believe we owe it to the broader Drupal community from which we’ve gained so much to consult with honesty and integrity.

What you’re missing out on

As we discussed in part 1, we lived through the challenges of the complete re-architecture of the Drupal application from 7 to 8.

Angie Byron, the person I apparently can’t stop referencing, said in 2013:

For people who grew up learning PHP on Drupal, and there are a lot of people for whom that’s true, I think Drupal 8 will be kind of a big adjustment for them.

Though it wasn’t easy, at Savas Labs we feel strongly that it was a wise investment that’s just beginning to pay off. At this point in its maturation, we believe now (as other Drupal leaders have felt for some time), that Drupal 8 is superior to previous versions in nearly all use cases for which organizations currently use Drupal. Some argue that Drupal 8 has become too complex and left smaller sites behind, but it’s important to consider their incentives for a well-rounded perspective. Via Acquia, Pantheon and other hosting providers, you can serve up a Drupal 8 website within minutes equipped with more features and a superior user experience to previous versions on a free tier to boot! While simultaneously catering better to those not writing code, the engineers who have always pushed Drupal to its physical limits have more power to build sophisticated tools and integrations that can do more for their clients than ever before.

In exploring this deeper, let’s start with the technical, and dig into the more nuanced to answer the Drupal 8 question: “What’s in it for me?” (WIIFM?), for you.

WIIFM? Features.

It’s fairly easy to find information touting Drupal 8’s strengths around the web, and it’s pretty straightforward that software we write today (and have been writing for 4 years) is superior to software written 8.5 years ago (or 10.5 years ago with Drupal 6). Let’s look briefly at some high-impact improved features for site owners and admins.

Design/UX/Usability Improvements with Drupal 8

In developing Drupal 8, perhaps for the first time, the Drupal leadership took user experience work seriously and developed a cohesive strategy to improve UX for Drupal 8. The results paid off.

  1. Responsive out of the box: Given that Drupal 8’s release came long after responsive web design became popular enough to garner its own acronym, naturally, all themes (administrative and otherwise) were developed to be responsive. RWD has been a must for years, but it took heavy lifting to achieve in Drupal 7.
  2. Better content authoring: Drupal 8 has adopted a more Wordpress-like UX for editors, which for many years had been cited as a distinction between the two, rightfully favoring Wordpress. Content authoring layout improvements coupled with responsiveness have made administration from a phone a pleasant experience.
  3. Accessibility at lower cost: Accessibility efforts, though not prioritized by all, continue to gain traction as we continue to expand our ability to be inclusive. We’ve seen clients threatened with lawsuits over not adhering to accessibility standards. Whether motivated by benevolence or risk-aversion, accessibility should be on your radar, and it’s easier in Drupal 8.
  4. Multilingual in core: With a cohesive system now in core, we have been able to build a couple of multilingual sites with relative ease, not having to dedicate substantial additional time to the translation component.

Drupal 8 multilingual is a world of difference. What would take 22 or more modules in Drupal 7 you would do with 4 (and all in core). - @kristen_pol

RESTful possibilities

One of the developmental focuses of Drupal 8 we believe has tremendous impact on how organizations can maximize the value of their content is the API-first Initiative. We will likely write an entire post about this in the future, but in short the initiative makes Drupal 8 much better equipped to serve as a central content repository that can expose content to many types of devices in the formats they require for display. Historically, Drupal has been pretty exclusively focused on producing HTML (one format) for a web browser (the device/software). Drupal 8 now treats Roku, iOS & Android Apps, video game systems and the web browser all as first-rate citizens for content consumption. As the number and variety of devices that connect to the web continues to rapidly grow, Drupal 8 can serve as a powerful hub that provides relevant content and experiences to end-users. You’d be remiss to snooze on this one. To get an idea of the possibilities check out Contenta CMS, a Drupal 8 distribution built by some of the people behind the initiative.

WIIFM? Performance.

If you take performance seriously, which you should, there’s a lot to like about Drupal 8. Sticking with the theme of sophistication, Drupal 8 provides a much more granular ability to cache specific components than its predecessors. And as we know in the high-performance web world, cache is king. When Drupal 8 first came out, a leading Acquia engineer showed some mixed results on Drupal 8 performance. The heavier codebase invariably means having to swim upstream to make it outperform the lighter codebase in Drupal 7, but I’m happy to say the architects had their flippers on when working through these challenges. Take these two fundamental points:

  1. Regardless of how fast the underlying code executes, what matters to users is perceived performance, i.e. how long they have to wait to interact with the page. Perceived delay has been drastically reduced by an experimental-turned-core module (more on that later) called BigPipe. BigPipe loads components of a page in the order in which a user is expected to interact with them while delivering more expensive components as they’re available. This breaks with the tradition of all-or-nothing webpages served by Drupal that were either in the cache or not, lending itself to a Facebook-like experience, which is where BigPipe came from.

    GIF of Drupal 8 BigPipe Video
    Slower video here

  2. Modern performance tactics derive the largest gains from outside of the application leveraging services like a Content Delivery Network (CDN), and/or a web application accelerator, like Varnish, to serve up resources to anonymous traffic (users not signed-in) as quickly as possible. For most sites, anonymous traffic comprises a majority of overall traffic. Traditionally, there have been limitations to improving performance for authenticated traffic, and that’s where Drupal 8 shines. With BigPipe and a more granular caching system, Drupal 8 can substantially outperform Drupal 7’s authenticated user experience, so it’s a win-win.

If you’re made of time today, check out our other articles we’ve written about performance for a deeper dive into this broad and complex topic.

WIIFM? People.

We know that behind any powerful movement are powerful people. To quote Dries, as I did in my Drupalcon talk in New Orleans:

fostering the Drupal community is actually more important than just managing the code base.

Also, atop the Drupal.org homepage used to read

Come for the code, stay for the community.

Without needing to resonate with all the warm and fuzzies that many within the community do, these sentiments show the richness and value of the Drupal community. And that rich community, not out of neglect, but rather necessity, has moved on from Drupal 7. Top designers, developers, and strategists are working on few Drupal 7 projects these days, and most would prefer to move on. For those who work with web designers and engineers (or used to be one like me), you know that they often have an insatiable appetite for learning, and want to do that with increasingly relevant tools to their growth and output. Sticking to dated software is an effective way to weed out the best and brightest.

The Improved Developer Experience (DX) of Drupal 8

Just like happy customers tend to be repeat customers, happier developers also produce returns; they’re more productive.

There are a few improvements in Drupal 8 that make life substantially better for developers. The Configuration Management Initiative was a boon to developers who struggled with a module called “features” which was not designed to do what most of us used it for. The CMI addresses the previous workaround, rife with inconsistencies, of moving site configuration from development to staging and production environments. Although it may seem trivial, developers love this better system in Drupal 8, and it means more efficient development, therefore higher ROI.

Proudly found elsewhere / not invented here / getting off the island

A primary Drupal 8 philosophy that has largely been successful, but yet to fully bear fruit, is the concept to drastically reduce “Drupalisms” that had proven a challenge for newcomers to the system who had to learn a suite of things specific to only Drupal. The proudly found elsewhere paradigm seeks to mitigate this by leveraging the best of other open source tools when possible rather than reinvent the wheel. A few of the tools Drupal 8 now exploits are Symfony components, Twig templating engine, and Composer Dependency Manager. This “borrowing” has two positive consequences: 1) it reduces the workload for Drupal core contributors by utilizing what’s freely available and well vetted through other communities, 2) it allows people familiar with those other frameworks a smoother onramp to productivity in Drupal. I believe we haven’t yet seen a majority of the benefit to the Drupal 8 project from the many people who were already versed in Symfony and TWIG before working with Drupal.

To quote Angie Byron for the thousandth time (full video here):

For people who are classically trained or have experience in other languages, Drupal 8 is going to make a lot more sense to them than Drupal 7 did. We’re just falling more in line with what the larger people are doing… within the broader PHP community.

WIIFM? Cost savings.

The active decision to upgrade or passive indecision to wait both have cost implications. Perhaps this is the most useful section for readers whose primary responsibilities aren’t technical.

Continuing to invest in Drupal 7 (or earlier) can be costly in ways that may not be abundantly apparent on the surface. For most organizations who work with an agency, custom development is where a brunt of the efforts are spent, and therefore is the primary cost driver. “Custom development” occurs when the functionality a client requests is either not freely available on the open-source market or the agency is unaware of its existence and a developer will write code for the specific use case to “extend” the out-of-the-box functionality. The 80-20 rule applies well to software development in Drupal: roughly 20% of the functionality a client requests accounts for 80% of the effort of a project since that 20% must be built from scratch. When site owners request various functionality, it can be difficult for them to differentiate what may constitute custom development efforts vs. freely available from the contributed community. Given the high effort of customization and related technical debt accumulated, site owners should request a high degree of transparency to understand what requires custom development when establishing project budgets. This way, the organization can do a cost/benefit analysis on a granular, per-feature basis. The goal for developers should be to always start with exploring what already-made wheels are out there for the turning before crafting their own. Be wary of alternative thinking. Yet, as extensible and rich the Drupal community is, nearly all of our engagements require customization.

Easy Drupal upgrades forever

A happy Drupal sunrise
Image from Dries’s blog post

To the surprise of the community, in an abrupt departure from business-as-usual in early 2017, Dries committed to “easy upgrades forever”, starting with Drupal 8 of course. The short of it is Drupal 8 to 9 upgrades should be far easier (and less expensive) than any previous major version upgrade, and so will be from here on in. That means for those not yet on Drupal 8, you only have one final difficult upgrade left in your Drupal journey until the end of time.

This is a fairly natural outcome given the possibilities afforded by a more structured, object-oriented architecture coupled with the growing desire to ease upgrade pain that has been building for some time. Although difficult technical work is needed to flesh out exactly how this will be done, the commitment from the top is worth putting stock in and the community is on the way to making this grand proclamation a reality. When upgrades are far easier, they will help rectify some of the sentiment of leaving the smaller sites behind, since major version upgrades will be a much less daunting task with Drupal 8 and beyond.

Drupal 6 or 7 custom development is especially expensive

However, perhaps the most important point is that you may be doubling your efforts for the final time if you’re doing custom development on Drupal 6 or 7, since it will invariably need to be rewritten to work on Drupal 8 with the same 80-20 rate we mentioned earlier. Given the commitment to easy upgrades and guidelines for backwards compatibility, it’s quite likely that custom code written for Drupal 8 will be highly portable to Drupal 9 and 10 that won’t require an arduous rewrite.

A Drupal 6 house
Not our actual house, and it’s not this bad.

I live in the equivalent of a Drupal 6 house. My partner and I keep putting off things we’d like to do now in prep for a more substantial renovation “on the horizon.” We’re not going to get solar panels before replacing the roof, and we won’t upgrade to a high energy efficiency HVAC system until we restructure some of the foundation. We’re being mindful of mitigating our overall costs, which makes sense, but this all sets up a perverse incentive to make no improvements in the immediate. The same can be true for an older Drupal site. As she frequently reminds me, I’ll remind you: it’s probably time to take the plunge and build your Drupal 8 house.

Cost to upgrade is going down

While there remains one final decidedly not easy upgrade if you’re not yet on Drupal 8, the good news is the cost to upgrade has gone down and will continue to. As of the release of 8.4.0, migrating from Drupal 6 is nearly all the way there:

Core provides migrations for most Drupal 6 data and can be used for migrating Drupal 6 sites to Drupal 8, and the Drupal 6 to 8 migration path is nearing beta stability.

The sentiment on 7, expectedly so, is not as far along:

The Drupal 7 to Drupal 8 migration is incomplete but is suitable for developers who would like to help improve the migration and can be used to test upgrades especially for simple Drupal 7 sites. Most high-priority migrations are available.

So migration from 7 still requires some work. More on this ahead.

Early adopters paved the way

We all owe a debt of gratitude to those who were willing to take the risk of building on Drupal 8 in its earlier days. We commend both organizations and agencies who were ambitious and willing to incur some risk to help push the rest of the project forward. We’re proud to put ourselves on that list, starting 2.5 years ago, but it unsurprisingly came with challenges and lessons learned. Mistakes that come with experience are virtually entirely positive for the future since we’ve learned what to do and what to avoid. It’s time for you to benefit from the work of the early adopters.

WIIFM? The Future.

The future is uncertain; the only things guaranteed are death and taxes. Actually, even those I’m not so sure :wink: Regardless, the future for Drupal 6 and 7 are a known entity not likely to get much better. The upside of Drupal 8, while partially known, is largely still in the making and will keep getting better over time.

Continuous innovation with experimental modules! Who doesn’t want that?

Among the suite of other firsts, Drupal 8 has updated its minor version approach to accommodate for innovations in core and this is another game changer. Previously, the first version of Drupal 7 (7.0) was essentially functionally the same as the latest (currently 7.56). Now new minor releases introduce experimental modules which are driven by agreed-upon priorities and are then vetted over time to see if they’ll graduate from the “experimental” label and be fully baked into core. Therefore Drupal 8 can and will adapt; Drupal 7 cannot. The transparent structure leadership established provides a good balance of innovation and predictability with two minor version releases a year. To track these for your own planning, at any time you can check the development roadmap.

Javascript

The Drupal community has been abuzz with “headless” or “decoupled” Drupal since the advent of Drupal 8. The basic idea is that Drupal can lean on its strength of being an excellent tool for highly structured and organized data in the backend while allowing freedom and flexibility of choice on the presentation layer (front end). Though discussed two years ago to no formal conclusion, Dries has recently cited React as the go-to presentation layer for Drupal administrative interfaces come early 2018. This is a fairly big deal, and formally moving forward to more tightly link with React has many implications we haven’t yet fully explored. As the lines between websites and web applications continue to blur, this proudly-found-elsewhere addition may prove to be a powerful one that will not be possible for Drupal 6 or 7. We see this as another wise move to be more in-sync with the rapidly growing impact of JS frameworks.

Access to complementary tools

The re-architecture towards object-orientation helped Drupal join the modern PHP community’s framework (Symfony, Laravel, Cake, Phalcon, Zend, Slim, CodeIgniter, Yii, and Fuel to name the most popular) development practices. One subtle yet substantial value to that move is now many tools that are built to help support these other frameworks are available to Drupal as well. As the toolkit for modern web development grows richer and more robust, the more Drupal can utilize, the better. A couple examples we’ve recently used to help inform project quality and future maintenance costs are Code Climate and Scrutinizer. These tools have much less value analyzing a Drupal 6 or 7 site.

Our advice

So we’ve dumped a lot of information on you at this point, but it may still not be entirely clear what you should do with your outdated Drupal site. Ahead we provide general suggestions as well as what is pertinent to site owners for each version separately (Drupal 6 and 7).

To everyone on Drupal <8

We still have <3 for Drupal <8. Drupal 5, 6, and 7 got us to where we are. But here’s what we think you should consider about where you’re going.

  1. Plan 3 years out if possible. A stitch in time saves nine, and every minute of planning saves 10 in execution. They’re clichés, but true. Planning well requires a real dedication to strategic and investigative work; there’s no way around it. The upside is it allows you to be intentional about when to incorporate an upgrade, rather than being at the mercy of expiring security support. Organizational stakeholders are usually not compelled by upgrading for the sake of upgrading without other bells and whistles that come with it. An experienced partner can help shepherd the long-term planning process to provide guidance on efforts and things to consider. If you’re not working with an agency, do it yourself. Expect to redesign and do a software upgrade every 3-5 years and time those together if possible. Factor in upgrades to other systems that integrate with your website as well as any initiatives that may require functional improvements. Put all of these larger investments on a roadmap with a timeline and be clear about what components are dependent on or impacted by other components. With technical work, the devil is in the details, so a thorough assessment or “discovery project” is usually the best next step. Discovery work is light on upfront investment yet thorough enough to guide your organization through the many choices in your roadmap. This is really the best way to use your resources most efficiently. If your organization hasn’t historically done this, it handicaps you a bit at the moment, but if you’ll excuse one final cliché: there’s no time like the present.
  2. Be mindful of what you don’t need. We all get excited about the possibilities of new functionality. However, when things we’ve built have outlived their purpose, let them go. Given the complexity and interdependence of the tools we build, customizations take the form of mounting, insidious and potentially crippling technical debt if left unaddressed. The cost to upgrade the technical debt is likely the major cause for most of those who have not yet upgraded; it is certainly the case for all of our partners who haven’t. This debt can be hard to track, and it’s not something most agencies proactively share since they have a hand in creating it and can also be shortsighted. Ask for answers as to how your partner is managing your technical debt. If you don’t get good answers, keep asking. Another subset of this concept is that even if you want to maintain certain functionality, it needn’t be done in the with the same modules on the Drupal 8 platform. So don’t take a given module’s absence in Drupal 8 as a certainty that it cannot be efficiently achieved in Drupal 8. In many cases, it can.
  3. Training will be required. If you plan to build with the same team that built your <8 site, and they have not worked on any other Drupal 8 or object-oriented PHP projects, make sure you dedicate time and budget resources for substantial training.

To those with production Drupal 6 sites.

I wrote a Drupal 6 series as Drupal 8 had announced its release (the overview, the risks, the options, Drupal 7 or 8) which is a good reference for both what was true then and what has changed now. Then, I certainly encouraged a conversation with your partner about what is right for you. That has not changed. What has changed is the maturity of the migration system, making it easier to port your content from 6 to 8. An upgrade to Drupal 8 by way of migration should be where you start based on all of the above, and the job before committing to that path is to well vet all requirements to migrate. Given that migrations are high-effort, you should explore alternatives with your development team. How much it’s worth to invest in a migration depends on how valuable your old content is to preserve, which varies widely among organizations.

If through research you uncover that you’re still not ready for Drupal 8, you should make an action plan to follow up on the components that will allow you to upgrade to Drupal 8 and track those over time. You should look into efforts to upgrade to Drupal 7, being mindful of how you can mitigate the costs to the Drupal 8 upgrade. You should consider support with the MyDropWizard team in the immediate. It’s lead by David Snopek who is on the core security team and has an impressive Drupal resume. It’s hard to assess how the support provided by MDW compares to the core security team, but it’s much better than not having any security support. I would also caution those to not use the relief from having coverage through MDW as a reason to rest on your laurels. If Drupal is still working for you, you should be thinking about how to get to Drupal 8. Additionally, as we agreed earlier there’s no longer even certainty to death and taxes, it’s possible that things could change for MDW, and you’d be without support again.

To those with production Drupal 7 sites.

As far as building new on Drupal 7, I have a hard time conceptualizing for whom that is the right choice. MDW wants to keep the door open to building new in Drupal 7, but others point out incentives again. Much like the hat tip to Dries for accepting criticism, I must commend MDW accepting these comments on their blog.

If you have a high degree of customization and technical debt, keep track of the development of Drupal 9. Upgrade now cannot be prescriptive for the 900,000+ sites still on Drupal 7. We generally agree with Angie’s recent presentation at Acquia Engage called Drupal 9 and Backwards Compatibility: Why now is the time to upgrade to Drupal 8 for those on Drupal 7:

If it’s working for you that’s fine! (Until Drupal 9) But if D8 offers features you want, consider earlier adoption.

So, if you’ve determined that you’ll remain on Drupal 7 for some time, your development team should be aware of a couple Drupal modules (xautoload and service_container) that make writing Drupal-8 like code possible in Drupal 7. These tools will help familiarize developers with Drupal 8 paradigms and possibly reduce substantial technical debt in the future.

To those not on Drupal, but considering it

If you’re in the market for a CMS and have ambitious web goals, you should at least check out Drupal. It’s been holding fairly steadily in the CMS market, and given some of the problems of the past with versions before Drupal 8, we think this speaks very promisingly of the future for Drupal. This is not to say that it is the right fit for all websites. It is not. However, with free hosting options, an improved and simplified admin experience, and the most powerful backend of the open source CMSes, it does fit a lot of needs.

We want to hear from you

We want to hear from your experience, whether or not it resonates with what we’ve presented here. Are you having challenges to upgrading that you feel went unaddressed here? Notice anything we overlooked? Comment away, or write us privately if that’s appropriate. I’m also often on Drupal slack (@chrisarusso) checking in on our local #TriDUG meetup conversations.

Nov 01 2017
Jay
Nov 01

drupal-upgrade-crisis.jpg

The Drupal Upgrade Crisis is Over

Back in March 2017, Dries Buytaert, the creator of Drupal, published a blog post entitled "Making Drupal upgrades easy forever" in which he confirmed what we here at Ashday had already suspected: Drupal 8 has laid the groundwork for seamless upgrades to future Drupal versions. But what exactly does that mean, and what benefits could this have for your website?

The Dark Age of Upgrades

Prior to the release of Drupal 8, Drupal releases broadly followed a policy that included minor releases and major releases. Minor releases (such as upgrading from Drupal 7.18 to 7.19) were usually fairly simple updates. They would fix bugs, add in some small new features, and resolve security issues, but ultimately, it was rare for a minor update to cause any problems for a well-built Drupal site.

However, major releases (such as upgrading from Drupal 6 to 7) were a whole 'nother matter; they're basically different systems entirely.. Standard policy with major updates was that most anything that needed to be changed could be, even if doing so would make it impossible to seamlessly update a site from one version to the next. For instance, Drupal 6 included a feature by default that allowed each of a site's users to select what theme they saw the site in (changing the site's overall look and feel). In practice, not many sites made use of this feature, so it was removed in Drupal 7 so that developers could focus on the more frequently-used parts of Drupal. Of course, if a site used this feature, it couldn't be upgraded to Drupal 7 without either losing that functionality or having new development done to add a replacement. The Drupal way to make progress was to not tie the potential of the future to the decisions of the past.

What this has lead to, however, is that oftentimes, once a site has been built on one version of Drupal, it doesn't get upgraded to the next version without good reason. In fact, there are many sites still out there running on Drupal 6 simply because upgrading to Drupal 8 would be too expensive and time-intensive of a process.

Fortunately, the dark age of upgrades may be at an end.

Enter: Drupal 8

When Drupal 8 was released, the paradigm shifted. Drupal 8 is very different from Drupal 7 (even more so than 7 was from 6), and so upgrading a site to Drupal 8 presents many of the same difficulties as upgrades in the past. But, thanks to the hard work of the Drupal developers and everyone else involved in the project, future upgrades should be much simpler. Drupal 8's code has been written to make good use of web development standards in ways that past versions never even tried. Because of this, Drupal 8 is fundamentally a foundation that can be built on by future versions without needing a complete overhaul.

What that means is that any site which is kept up to date as minor releases come out should continue to work on Drupal 9 with minimal effort needed to upgrade it. Instead of everything changing at once like it did going from Drupal 7 to Drupal 8, old code and little-used features can be phased out slowly as better alternatives are developed and implemented.

And that leads us to the final result of this change of approach: Prior to Drupal 8, a site could be built and then be minimally maintained with security updates until the next version of Drupal came out, at which point it had to be rebuilt entirely. Now, a Drupal 8 site can be built, and if it is minimally maintained until Drupal 9, there might be a few adjustments needed to remove deprecated features, but there will certainly be fewer than in the old way. But what's more, if that Drupal 8 site is actively maintained, it may not even be using any deprecated features at all by the time Drupal 9 comes out, allowing that upgrade to be made with ease.

What Does This All Mean for You?

From a technical perspective, these are all great changes, but what does it actually mean for your website? Several things:

If you are on an older version of Drupal: Now is the time to upgrade. Upgrading from 6 is long overdue (it doesn't even get official security updates anymore!), but upgrading from 7 makes sense now as well. Before Drupal 8, it was common to skip versions… for instance, a Drupal 5 site would skip Drupal 6 entirely and get rebuilt on Drupal 7. But upgrading a Drupal 7 site to 9 shouldn't really be too different from upgrading it to 8… so, why wait? Upgrading now will help avoid any last-minute scrambling to update the site when Drupal 7 eventually stops getting security updates.

From a business and marketing standpoint, there's another hidden advantage as well. Before, redesigning a website often got tied to the need to upgrade Drupal. There wasn't much reason to do a big redesign, if you'd have to upgrade from one major release of Drupal to another just a year later, so the changes would get bundled together to avoid having to do two rebuilds. But what's more, taking on all the concerns of a site redesign alongside the technological challenges of a Drupal upgrade could make doing both at once a challenging task in itself.

Once your site is on Drupal 8, major releases should no longer require rebuilding the site… which frees up your Drupal developers to be able to do a redesign whenever it makes sense to from a branding or business perspective. Drupal 8 gives you the power to decide when a website overhaul should be done, rather than tying such upgrades to the technology the site happens to be built with.

Offer for a free consultation with an Ashday expert

Oct 27 2017
Oct 27

Drupal 8’s official release was nearly two years ago, and many ask how is it doing? Has it lived up to its ambition to revolutionize Drupal websites?

In the first of a two-part series, we’ll provide our insight into the evolution of Drupal 8 over its first two years in the wild. In part two, we’ll look at important factors to consider in your Drupal investments going forward.

(Drupal) Change is hard

To launch a website (much like to rock a rhyme that’s right on time) is tricky; to operate a web system in a way that uplifts your organization is just plain hard. Although keeping up with the most current software is typically advisable, there are costs to doing so, even for free software like Drupal. Without the vendor lock-in that comes with proprietary Content Management Systems, Drupal site owners have a high degree of freedom to consider how best to invest their web resources. However, this freedom also has a price (see a pattern emerging?) in the form of time and stress incurred from the responsibility to select the best digital tools to drive your organization for years with limited information to evaluate the nearly limitless options.

Of the 1 million+ organizations whose main window to the digital world is powered by Drupal, many have priorities that compete in time and budget with web system investments. When considering these priorities, determining the right time to invest in an improved user experience, design, feature-set, or software upgrade can be difficult.

With the ever-increasing complexity and interconnectedness of the software systems we build, even the world’s most prominent organizations, often with legions of engineers, have had colossal mishaps with upgrades. By default, upgrades are not easy.

To upgrade or not? That is the question.

Like any decision, whether you’re building new or upgrading, the fundamental question is: do the benefits outweigh the costs? In the specific case of investing in Drupal the question becomes: when does making the leap to Drupal 8 rather than continuing to invest in Drupal 7 (or possibly 6… don’t tell me it’s 5 :wink: ) outweigh the costs to take that leap? For any organization to properly answer that question, it’s necessary to look 3-5 years out with regard to budget and organizational goals. It’s also helpful to better understand how the broader community has approached this same decision over the past two years. Let’s take a look.

Taking stock of Drupal 8’s adoption

stock market image

After nearly two years since its public release, how has the adoption of Drupal 8 gone?

Analysis from the top

Before DrupalCon North America in May 2016 in New Orleans, Drupal founder and current project lead Dries Buytaert blogged that Drupal 8 was doing “outstanding,” citing statistics to substantiate his optimistic view.

Based on my past experience, I am confident that Drupal 8 will be adopted at “full-force” by the end of 2016.

Many in the community contested the veracity of his optimism in the article’s comments and I commended Dries (yes that’s me and not him, and definitely not him) for facilitating an open conversation that elicited a broad perspective.

About a month later, some six months after Drupal 8 was released, Savas Labs attended DrupalCon NOLA.

During the perennial “Driesnote,” Dries continued to present Drupal 8 as well on its way to match if not exceed the success of Drupal 7.

I really truly believe, Drupal 8 will take off. My guess is that by the end of this year [2016] Drupal 8 will serve an escape velocity… it will become the de facto standard.

and

The new architecture, features, as well as frequent releases: all of these things make me feel really, really optimistic and bullish about Drupal 8.

Adoption by the numbers

According to the usage statistics available on the Drupal website, when writing this nearly 80% of the world’s Drupal websites were powered by version 7.

drupal stats d8A graph started by Angie Byron of Acquia that I updated to present.

When Drupal 7 was released on January 5, 2011, there were already more Drupal 7 sites than sites powered by the major version two releases prior: Drupal 5 (A). The same feat for Drupal 8 took over nine months after its release to achieve (C). Total Drupal 7 sites eclipsed total sites of its predecessor version (6) about 13 months after the release of Drupal 7 (B). After nearly 2 years from the release of Drupal 8, it has not yet eclipsed Drupal 7 installations, and at present there are over 700,000 more Drupal 7 sites than Drupal 8.

Our take on Dries’s bullish-ness

To his credit, the future is notoriously difficult to predict, and even when predicting it, Dries spoke of the significant work that lay ahead to see his vision come to fruition. He also made the referenced comments well over a year ago, and I’ll concede speaking in hindsight is infinitely easier. Having said that, comparing the total number of upgraded Drupal 8 sites to Drupal 7 sites over the same period from release in a community that had grown ~220% since Drupal 7’s release, while factually indisputable, was probably not as accurate as using adoption percentages to analyze overall trends.

Even the most conservative interpretations of “escape velocity” or “full-force” would have to concede that we’re at least a year behind Dries’s hopes when he was reporting from DrupalCons Barcelona and New Orleans on impending rapid Drupal 8 adoption. But, what’s a dictator worth his salt to do, benevolent or not, other than to stretch the stats a bit to show what he would like to be true for his beloved community, from which he also profits?

Our assessment

After two years, the data unequivocally show, as I began discussing at DrupalCon New Orleans, the rate of Drupal 8 adoption is objectively slower than Drupal 7. At this point, a majority of organizations have not yet upgraded from 7 to 8, though likely many have begun efforts. Taking a simplistic view, this means Drupal 8 has either been more costly to upgrade, a comparatively less valuable product, or perhaps both.

Regardless, since it matters to our partners, we found it important to explore the reasons behind the slow adoption rather than to pretend it’s not happening. After architecting Drupal 8 web systems for 2.5 years, we have gained insight into the relatively slow adoption.

Drupal 8 adoption challenges

Drupalers haven’t written much about the retrospective analysis of the Drupal 8 adoption challenges. But without being able to take a real, honest look inward, we cannot improve. We must know thyself because the examined Drupal problems are worth fixing! We highlight here the most prominent challenges that have slowed Drupal 8 adoption.

1. Complete code re-architecture

The massive shift of the underpinnings of the Drupal code is a decision that has long been debated within the community. There’s no question it has proven a challenge for proficient Drupal 7 developers to develop on Drupal 8: for most, substantial training and learning is required. Training takes time, and time can often mean money. The loss in short-term efficiency for seasoned Drupal developers made early adoption riskier, and typically added to a project’s expense. Joining with other prominent frameworks known outside of Drupal like Twig and Symfony (colloquially referred to as “getting off the island”) was a collective decision by wise Drupal leadership with the long-term value of the product in mind, but in the short-term, for the average Drupal developer, it meant more new things to learn.

2. Slow contributed module porting

Historically Drupal has derived much of its usefulness from the rich contributed module ecosystem that extends the features of Drupal core. Contributed modules, although crucial to most live Drupal websites, by definition are not directly driven by those that oversee Drupal core development. This disconnect invariably leads to some important modules not having a usable upgraded version when a new major version of Drupal core is released. This is well-known within the Drupal community, explained at great length by Angie Byron (second reference), and not unique to the Drupal 8 release. Tremendous amounts of individual and community efforts are required to upgrade modules to the latest major version. Due to #1 from above, these efforts were further exacerbated by the re-architecture. Costs to upgrade even one module (it’s common for a Drupal 7 site to use 100) are often greater than clients or agencies are willing to absorb on a given project.

3. Incomplete upgrade path

We often describe websites as comprised of three main asset groups: the code (Drupal core, contributed and custom modules), files (think media assets like images), and the database where content and site configuration lives. When upgrading, you download the new Drupal code, which has a set of instructions that must be run to apply complex updates to the database. Files remain unchanged. A well-oiled upgrade process is required to update the content and configuration from the site being upgraded into a format intelligible to the new system. The approach to perform those upgrades has also changed in Drupal 8 to what is now referred to as “a migration”. As of the most recent minor release of Drupal 8 in October states:

…Drupal 6 to 8 migration path is nearing beta stability. Some gaps remain, such as for some internationalization data. The Drupal 7 to Drupal 8 migration is incomplete but is suitable for developers who would like to help improve the migration and can be used to test upgrades especially for simple Drupal 7 sites. Most high-priority migrations are available.

“Nearing beta stability” after two years out from release is not ideal though it is reality since perfecting these migration tasks is hard work. One can discern from the Drupal 7 -> 8 migration snippet that it’s clearly further afield, and for those who need to preserve their content, perhaps a non-starter for a 7 -> 8 upgrade. The inability to efficiently update database structures adds to project expense. Whatever doesn’t come over “for free” with the migration will need to be manually replicated by a human, and humans are costly, as our time is precious.

4. Stance on backwards compatibility

Drupal’s approach to backwards compatibility is famously “for data, not code”. Briefly put, in their words: “While the upgrade path will reliably preserve your data, there is no backward compatibility with the previous Drupal code.” If you want to dig deeper, there’s a lot of good discussion on this topic.

WordPress’s approach, perhaps more than anything, explains its ubiquity and ability to better keep sites on the latest version. In their words:

Major releases add new user features and developer APIs. Though typically a “major” version means you can break backwards compatibility (and indeed, it normally means that you have), WordPress strives to never break backwards compatibility. It’s one of our most important philosophies, and makes updates much easier on users and developers alike.

Albeit a bit confusing, even for the non-technologist, you get the sense they’re more worried about breaking stuff and want upgrades to Just Work™. The strength of the Drupal approach is it allows for more innovation, and in some ways, less baggage since preserving backwards compatibility often means hanging on to outdated code. The trade-off is, once old code is determined to be holding innovation back, it’s cast to the side, and new structures must be implemented in the updated version. Historically, this paradigm has caused many to get stuck in an outdated Drupal version for longer than they’d like because they cannot afford an upgrade.

5. Inertia, perceived value, and expense

A modern organization is focused on more than just their website, and for investments that don’t deliver direct, visual, tangible change, stakeholders often overlook them, even when they may present value. Examples where the value is oft-invisible to clients are investing in an automated testing framework that ensures perpetual site integrity, or vigilantly applying security updates as they become available. In either case, the client may perceive them as optional, but foregoing them is likely to cost the organization in the long-run.

Since Drupal only provides security support for two major versions at a time (presently 7 and 8), for many, the prime motive for a new release, often framed as a mandate, is to upgrade from the version two major releases prior, which has fallen out of support. When Drupal 8 came out, Drupal 6 fell out of support after a grace period of three months, generously extended from the day of release given some of the community’s recognition of some of the challenges we’ve documented here.

If an organization doesn’t heed the security warnings, and doesn’t find enough value in the new features, they may choose to ignore the upgrade completely. The truth is it’s hard to estimate the future risk of using outdated software. However that future risk is very real, and digital security compromises show no signs of slowing down. Savas Labs always advocates for timely security coverage, but it has not always been a budgetary possibility for our partners to upgrade from Drupal 6 to Drupal 8 upon release of 8.

An answer to the Drupal 6 problem

In addition to our experience, the usage data show many organizations did not plan sufficiently to upgrade from Drupal 6 to 7 or 8 upon Drupal 8’s release. Recognizing that, a Drupal agency My Drop Wizard set up long-term security support for the many Drupal 6 sites that were not ready to upgrade to Drupal 8. It’s debatable whether or not this was a good thing for the community. People forced to change, often will change sooner than they would otherwise, but they may resent you for it. Conversely, you’d be hard-pressed to find an MDW client who didn’t experience anxiety relief when offered an inertia-compliant alternative.

Organizations that don’t perceive opportunity in the value the new software provides will look at an upgrade strictly as an expense to avoid, likely citing topics we’ve covered here.

Takeaways

Through experience and analysis, we see there are many understandable and justifiable reasons why many organizations haven’t yet upgraded to Drupal 8. Now that we’ve done the hard reflection, the good news is that the present is a much brighter place for not only Drupal 8 but all future versions of Drupal. We have made it through most of the difficult growing pains, and there’s great reason to believe that the community has invested wisely in the future. In part two, we cover the costs of investing in Drupal 7, and why it’s probably time to move to Drupal 8.

Oct 11 2017
Oct 11
I was using a Drupal composer template and a few days ago, it upgraded from 8.3 to 8.4 automatically. I noticed and didn't really think much of it, so I applied the database updates and went on my way. Today, two days afterwards, I ran across this article which pointed out that PHP 7 is required for the underlying Symfony framework. Our site is running on Debian 8, which has PHP 5.6 and powers other PHP applications, so I wasn't looking to update the underlying OS and possibly break my other PHP things.

So I downloaded a db snap from 8.3 and then re-applied the 8.4 update and took another database snapshot. Then I diff'd the database snapshots. The biggest changes seemed to be in the cache tables and removing and adding some revision columns. So I reverse-engineered a backgrade SQL script. With that, I updated the composer.json file from this:
        "drupal/core": "~8.0",


to this:
        "drupal/core": "8.3.*",

Then I took a precautionary database backup and then did a composer update, which took care of the code backgrade. Then I ran my script (drush sqlc < backgrade.sql) and then I did a drush entity-updates to actually update the database schemas to match the backgraded code.

Now I just need to ignore Drupal telling me about 8.4 until I'm ready to fully embrace PHP 7. I feel like something bigger needed to get my attention to the update in system requirements when I ran the initial composer update.

May 24 2017
May 24
Drupal logo used in Ashday Blog

In Drupal 7, site deployments could be rather difficult on ambitious sites. Some database level elements were worth programming out in hook_updates (turning on modules, reverting views, etc) and some usually weren't (block placement, contrib module configuration). I remember days where a deployment involved following a three page long Google doc of clicks that had to be carefully replicated. Ugh.

A New Hope

So if you've taken the dive into Drupal 8, you'll quickly discover one of it's most prominent features - Configuration Management. Drupal 8's ability to manage configuration with yml files is absolutely amazing! It's nearly akin to watching Star Wars and thinking "Hey, I can do anything with a lightsaber! Fight bad guys, cut holes in doors, remove my hand cuffs. Sweet!"

The Empire Strikes Back

Here's the rub. Managing Drupal 8 configuration in complex real world apps is akin to building a real world laser sword after watching Star Wars only to promptly burn your face off and lose two limbs as soon as you try to fight with it. "Ambitious digital experiences" essentially equates to "arduous development concerns" and even config management can't save the day simply by existing. You must use it for good. You must unlearn what you have learned. I blogged a bit on this shortly after Drupal 8 released, but oh how much I learned since then!

We've been doing Drupal 8 pretty heavy for about a year and a half here at Ashday and had both the fortune and misfortune of needing to manage a more complex set up which quickly revealed our deficiencies in understanding how to properly manage config.

Here's the scenario: A client needs a site that will become the model for many sites, but they don't want them to be a single site with multiple domains and they also don't want it to be costly or complicated to keep them mostly similar from a functional perspective. Given that our preferred hosting solution is Pantheon, this quickly turned into an obvious Upstream project. And that means figuring out a new way to manage D8 config other than just import/export of the whole site.

If you aren't familiar, a Pantheon Upstream works nearly identical to their core updates - you have a remote repository that, upon code getting pushed to it, notifies you through the dashboard of your updates where you can apply them in the same way you do Pantheon core updates. It's pretty slick because it provides an easy way to have a big shared chunk of code and apply updates to many sites with a few clicks (well, except when nearly every update is major and requires hands-on management - but I'm not bitter).

The Phantom Menace

Our first try at this was to give the Features module a go, but at the time the interface was just too buggy to give us enough confidence to rely on it, it auto-selected what we didn't want and didn't select what we did, and it didn't support some key things we needed like permissions. As a result we decided to home brew our own solution. We knew these sites were going to have a lot of config in common, and a lot of config unique, and we needed to deploy to many of them all in different states without tragedy striking. So to accomplish this, we concocted the following procedure that we would run at deployment time, all from a single drush command.

  • Export the current site's live config (using drush) to the config sync folder
  • Copy all config files (with uuids removed) in our cross-site custom module over top of the config sync directory
  • Copy all config files (with uuids removed) in our site-specific custom module over top of the config sync directory
  • Import all config.

What this allowed us was the ability to allow each upstream site to stray a bit as they needed to, but we could be assured the config we cared about was prioritized in the proper stacking order. The approach ultimately wasn't that different than the goal of Features, but we were in control of the process, it was all live and it was relatively quick. And you know what? It worked! For a while...

And then it didn't. You see, the method we used caused Drupal to see every config file we were tracking (upwards of 300) as changed simply because of the missing uuid. So if only 8 config files changed in a deployment, Drupal was attempting to import hundreds of config files every time. This meant that it started to slow significantly over time as the site grew in complexity and eventually, we started having timeout issues and long deployments. We also started to run into issues when there was a significant core update (ie: 8.3) because so much config was being imported unnecessarily that wasn't compatible in that moment with the new code because db updates hadn't run yet. Not good. It was time for something else.

Return of the Jedi

The Jedi in question here is again the Features module. Or maybe it's Mike Potter. At least it's not me anyways. At DrupalCon Baltimore, I was set on speaking with Mike about how we were handling config because I simply knew there was a better way. If you don't know, Mike is one of the founders of the Features module and ran a great BOF on config management in Baltimore.

So I found this delightful man and laid out what we were doing and he reacted exactly as I had hoped. He didn't say that what we were doing was terribly wrong, but it made him visibly uneasy. After a chat, I discovered that Features had come a long way since we initially tried to use it and we should really give it another shot. He also explained some of the configuration of Features to help me better understand how to use it.

So we returned to Features now and are much happier for it. The thing is though that I don't think I would have really known how to manage it if we hadn't taken the deep dive into config and figured out how it needed to work. It all helped us a lot to decide how to incorporate Features properly for this particular situation so that I actually feel good about relying on it again. And that's how most good Drupal development goes. You really should know how something works before simply relying on a contrib module or someone else's code to take care of everything because otherwise you won't really know how to deal with problems - heck, you might not even know you have a problem! I personally don't prefer spending weeks writing code and then depending at a critical moment on a mysterious piece to make it all successfully roll out to production.

So as it all played out, we now understand what Drupal puts in config, what we care about and don't, what belongs in the upstream vs our site-specific modules vs no where, etc. Here is our current process after this 6 month long journey.

  • Revert the global base feature
  • If needed, revert the site specific feature
  • Run our previous script outlined above, but now on only the 5 or 6 role config files so we handle the permissions in the same fashion

So there you have it! For how long-winded this turned out to be, I'm glossing over a lot of details that are pretty critical to understanding Drupal 8 configuration (ex: blocks are a mix of config and content), but I recommend you do the same thing we did and really get your hands dirty and understand what's going on so that you don't get bit at rollout. After all of this, we feel even moreso that Configuration Management is an astoundingly useful component of Drupal 8 and now we find ourselves a bit sad when we update our Drupal 7 sites (a version we absolutely loved!) where we don't have this amazing tool. 

So good luck and don't hesitate to drop us a note if you have any questions or thoughts on this stuff. I'll probably change my mind on all of it anyways tomorrow. That's why this job is awesome.

P.S. I apologize that I didn't find room to incorporate Attack of the Clones, Revenge of the Sith, The Force Awakens or Rogue One, but the reality is that I just didn't have time to modify our whole approach to configuration in order to make this blog post more cohesive.

Offer for a free consultation with an Ashday expert

Mar 06 2017
Mar 06

Creating and publishing quality content within time constraints is a common challenge for many content authors. As web engineers, we are focused on helping our clients overcome this challenge by delivering systems that are intuitive, stable, and a pleasure to operate.

During the architectural phase, it’s critical to craft the editorial experience to the specific needs of content authors to ensure the best content editing experience possible. Drupal 8 makes it even easier than previous versions for digital agencies to empower content creators and editors with the right tools to get the job done efficiently, and more enjoyably.

Our five tips to enhance the content editing experience with Drupal 8 are:

1. Don’t make authors guess - use structured content

2. Configure the WYSIWYG editor responsibly

3. Empower your editorial team with Quick-Edit

4. Enrich content with Media Embeds

5. Simplify content linking with LinkIt

1. Don’t make authors guess - use structured content

The abundance of different devices, screen sizes and form factors warrants the use of structured content. Structured content is content separated into distinct parts, each of which has a clearly defined purpose and can be edited and presented independently from one another according to context.

“How does that relate to a content editor’s experience?” - you may ask.

In years past, it was very popular to give content editors an ability to create “pages” using one big “MS Word-like” text box for writing their articles, news releases, product descriptions, etc. This approach produced content that was not reusable and was presented in one strict way. Who wants to navigate within one enormous text area to move images around?

Though those days are long behind us, and even though we all know about the importance of structured content, sometimes we still fail to utilize the concept correctly.

Drupal was one of the first Content Management Systems (CMS) to introduce the concept of structured content (node system - Drupal 3 in 2001). In fact, Drupal is no-doubt the best CMS for implementing the concept of structured content, but its ability to provide a good content authoring experience lagged behind this solid foundation.

Today, in Drupal 8, editing structured content is a joy!

With the WYSIWYG (What You See Is What You Get) editor and Quick Edit functionality in Drupal core, we can equip our content editors with the best of class authoring experience and workflow!

You can see the difference between unstructured and structured D8 content below. Instead of only one field containing all text, images, etc., the structured content stores each definitive piece of information in it’s own field, making content entry fast and presentation flexible!

Structured vs unstructured content

The benefits of Drupal 8 structured content approach:

  • The author clearly understands where each piece of information should reside and does not have to factor in markup, layout, and design while editing (see tip #2). Content entry becomes remarkably efficient and allows the author to concentrate on the essence of their message instead of format.
  • The publishing platform is easier to maintain while supporting system scalability.
  • The modular nature of structured content makes migrations between CMS versions or to a completely different CMS much more streamlined. A huge plus for those long-term thinkers!

2. Configure the WYSIWYG editor responsibly

Drupal 8 ships with WYSIWYG text editor in core. The editor even works great on mobile! In a society that is so dependent on our mobile devices - who wouldn’t like to be able to quickly edit a missed typo right from your phone?

Drupal 8 provides superior enhancements to the UX (User Experience) for content authors and editors out of the box. However, with a little configuration, things can be further improved.

When establishing the UI (User Interface) for content authors, site builders should focus on refining rather than whole-sale adoption of the available features. Customizing the WYSIWYG editor is the perfect example of subtle improvements that can immediately make a big difference.

The WYSIWYG text editor is an effective tool for simple content entry since it does not require the end user to be aware of HTML markup or CSS styles. Many great functions like text formatting options (font family, size, color, and background color), source code viewing, and indentation are available at our fingertips, but as site builders we should think twice before adding all those options to the text editor toolbar!

With great power comes great responsibility! Sometimes, when you give content editors control over the final appearance of the published content (e.g. text color, font family and size, image resizing, etc.), it can lead to an inconsistent color schemes, skewed image ratios, and unpredictable typography choices.

How do we help our content authors in avoiding common design / formatting mistakes? Simple!

Use a minimalist approach when configuring the WYSIWYG text editor. Give authors access to the most essential text formatting options that they will need for the type of content they create and nothing more. If the piece of content edited should not contain images or tables - do not include those buttons in the editor. The text editor should be used only for sections of text, not for the page layout.

A properly configured CMS should not allow content editors the ability to change the size of the text or play with image positioning within the text section or the ability to add H1 headers within auxiliary content.

Below is an example of a bad vs. good WYSIWYG configuration.

WYSIWYG editor configuration compared

Benefits of the minimal (thoughtful) WYSIWYG configuration:

  • Easy to use
  • Less confusion (though there are edge cases, most editors don’t use all the buttons)
  • Better usability on mobile devices
  • Less risk of breaking established website design

Let’s keep our content editors happy and not overcrowd their interfaces when it’s absolutely not necessary. It is our duty as software engineers to deliver systems that are easy to use, intuitive, scalable and uphold design consistency.

3. Empower your editorial team with Quick-Edit

The Quick Edit module is one of the most exciting new features that is included in Drupal 8 core. It allows content authors and editors to make updates to their content without ever leaving the page.

The days of clicking “Edit” and waiting for a separate page to load just to fix a tiny typo are gone! The Quick Edit module eliminates that extra step and allows content editors to save a great deal of time on updating content. As an added bonus - content editors can instantly see how updated content will look within the page flow.

Here’s the Quick Edit functionality in action.

Quick Edit module demo

Quick Edit configuration tip for back-end and front-end developers

To make use of the Quick Edit functionality within the website pages, entities have to be rendered on the page via View Modes and not as separate fields.

This restriction presents a challenge when there’s a needs to provide Quick Edit functionality for a page constructed by the Views module. More often than not, Views are used to single out and output individual fields from the entities. The most used Views formats are “Table” and “Grid”. They currently do not support Quick Edit functionality for usability reasons.

A workaround for this issue is to use the custom View modes for Entities and create custom Twig templates for each View mode that should be outputted by Views in order to accommodate custom layout options.

4. Enrich content with Media Embeds

In the era of social media, content editors can’t imagine their daily routine without being able to embed their Tweets or videos into the stories they publish on their sites. In Drupal 6 and the early days of Drupal 7, it was pretty challenging to provide this functionality within the WYSIWYG editor. Developers had to configure many different plugins and modules and ask them politely to cooperate.

The Drupal 8 Media initiative has placed the content author’s experience and needs at the forefront of community efforts. As a result, we have access to a great solution for handling external media - CKEditor Media Embed Module. It allows content editors to embed external resources such as videos, images, tweets, etc. via WYSIWYG editor. Here’s an example of the Tweet embed – the end result looks beautiful and requires minimal effort.

"If you're going to build a new site, build it in D8." - someone who knows what they're talking about quotes @jrbeaton @TriDUG pic.twitter.com/8w9GAuuARu

— Savas Labs (@Savas_Labs) January 27, 2017

With all this media goodness available to us, there is no reason why we shouldn’t go the extra mile and configure the CKEditor Media Embed module for our content authors!

5. Simplify content linking with LinkIt

Linking to content has always been a clumsy experience for content editors, especially when linking internally within the same site.

There was always the risk of accidentally navigating away from the page that you were actively editing (and losing any unsaved information) while searching for the page to link to. Also, the default CKEditor link button allowed editors to insert a link, assign it a target value, title, maybe an anchor name, but that was about it. If the link to the internal content changed, there was no way for the page to update and links throughout the website would end up broken.

Let’s not put our content editors through that horrible experience again. LinkIt module for Drupal 8 to the rescue!

With the LinkIt module the user does not have to copy / paste the URL or remember it. LinkIt provides a search for internal content with autocomplete field. Users can link not only to pages, but also to files that are stored within Drupal CMS.

The new and improved linking method is much more sustainable, as it recognizes when the URL of the linked content changes, and automatically produces the correct link within the page without the need to update that content manually.

LinkIt File link demo

Linking to files with LinkIt

My personal favorite feature of the LinkIt module is the flexible configuration options. The LinkIt module makes it possible to limit the type of entities (pages, posts, files) that are searchable via the link field. You can also create a custom configuration of the LinkIt autocomplete dialog for each WYSIWYG editor profile configured on your site. Plus, it is fully integrated with Drupal 8 configuration synchronization.

Final Thoughts

As site builders, there are many improvements that we can make in order to streamline the process of content authoring.

With the right mix of forethought and understanding, Drupal 8 allows web engineers to deliver content publishing platforms that are unique to the client’s specific needs, while making web authoring a productive and satisfying experience.

Feb 20 2017
Feb 20

Overview

Savas Labs has been using Docker for our local development and CI environments for some time to streamline our systems. On a recent project, we chose to integrate Phase 2’s Pattern Lab Starter theme to incorporate more front-end components into our standard build. This required building a new Docker image for running applications that the theme depends on. In this post, I’ll share:

  • A Dockerfile used to build an image with Node, npm, PHP, and Composer installed
  • A docker-compose.yml configuration and Docker commands for running theme commands such as npm start from within the container

Along the way, I’ll also provide:

  • A quick overview of why we use Docker for local development
    • This is part of a Docker series we’re publishing, so be on the lookout for more!
  • Tips for building custom images and running common front-end applications inside containers.

Background

We switched to using Docker for local development last year and we love it - so much so that we even proposed a Drupalcon session on our approach and experience we hope to deliver. Using Docker makes it easy for developers to quickly spin up consistent local development environments that match production. In the past we used Vagrant and virtual machines, even a Drupal-specific flavor DrupalVM, for these purposes, but we’ve found Docker to be faster when switching between multiple projects, which we often do on any given Sunworkday.

Usually we build our Docker images from scratch to closely match production environments. However, for agile development and rapid prototyping, we often make use of public Docker images. In these cases we’ve relied on Wodby’s Docker4Drupal project, which is “a set of docker containers optimized for Drupal.”

We’re also fans of the atomic design methodology and present our clients interactive style guides early to facilitate better collaboration throughout. Real interaction with the design is necessary from the get-go; gone are the days of the static Photoshop file at the outset that “magically” translates to a living design at the end. So when we heard of the Pattern Lab Starter Drupal theme which leverages Pattern Lab (a tool for building pattern-driven user interfaces using atomic design), we were excited to bake the front-end components in to our Docker world. Oh, the beauty of open source!

Building the Docker image

To experiment with the Pattern Lab Starter theme we began with a vanilla Drupal 8 installation, and then quickly spun up our local Docker development environment using Docker4Drupal. We then copied the Pattern Lab Starter code to a new custom/theme/patter_lab_starter directory in our Drupal project.

Running the Phase 2 Pattern Lab Starter theme requires Node.js, the node package manager npm, PHP, and the PHP dependency manager Composer. Node and npm are required for managing the theme’s node dependencies (such as Gulp, Bower, etc.), while PHP and Composer are required by the theme to run and serve Pattern Lab.

While we could install these applications on the host machine, outside of the Docker image, that defeats the purpose of using Docker. One of the great advantages of virtualization, be it Docker or a full VM, is that you don’t have to rely on installing global dependencies on your local machine. One of the many benefits of this is that it ensures each team member is developing in the same environment.

Unfortunately, while Docker4Drupal provides public images for many applications (such as Nginx, PHP, MariaDB, Mailhog, Redis, Apache Solr, and Varnish), it does not provide images for running the applications required by the Pattern Lab Starter theme.

One of the nice features of Docker though is that it is relatively easy to create a new image that builds upon other images. This is done via a Dockerfile which specifies the commands for creating the image.

To build an image with the applications required by our theme we created a Dockerfile with the following contents:

FROM node:7.1
MAINTAINER Dan Murphy <[email protected]>

RUN apt-get update && \
    apt-get install -y php5-dev  && \
    curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer && \

    # Directory required by Yeoman to run.
    mkdir -p /root/.config/configstore \

    # Clean up.
    apt-get clean && \
    rm -rf \
      /root/.composer \
      /tmp/* \
      /usr/include/php \
      /usr/lib/php5/build \
      /var/lib/apt/lists/*

# Permissions required by Yeoman to run: https://github.com/keystonejs/keystone/issues/1566#issuecomment-217736880
RUN chmod g+rwx /root /root/.config /root/.config/configstore

EXPOSE 3001 3050

The commands in this Dockerfile:

  • Set the official Node 7 image as the base image. This base image includes Node and npm.
  • Install PHP 5 and Composer.
  • Make configuration changes necessary for running Yeoman, a popular Node scaffolding system used to create new component folders in Pattern Lab.
  • Expose ports 3001 and 3050 which are necessary for serving the Pattern Lab style guide.

From this Dockerfile we built the image savaslabs/node-php-composer and made it publicly available on DockerHub. Please check it out and use it to your delight!

One piece of advice I have for building images for local development is that while Alpine Linux based images may be much smaller in size, the bare-bones nature and lack of common packages brings with it some trade-offs that make it more difficult to build upon. For that reason, we based our image on the standard DebianJessie Node image rather than the Alpine variant.

This is also why we didn’t just simply start from the wodby/drupal-php:7.0 image and install Node and npm on it. Unfortunately, the wodby/drupal-php image is built from alpine:edge which lacks many of the dependencies required to install Node and npm.

Now a Docker purist might critique this image and recommend only “one process per container”. This is a drawback of this approach, especially since Wodby already provides a PHP image with Composer installed. Ideally, we’d use that in conjunction with separate images that run Node and npm.

However, the theme’s setup makes that difficult. Essentially PHP scripts and Composer commands are baked into the theme’s npm scripts and gulp tasks, making it difficult to untangle them. For example, the npm start command runs Gulp tasks that depend on PHP to generate and serve the Pattern Lab style guide.

Due to these constraints, and since this image is for local development, isn’t being used to deploy a production app, and encapsulates all of the applications required by the Pattern Lab Starter theme, we felt comfortable with this approach.

Using the image

To use this image, we specified it in our project’s docker-compose.yml file (see full file here) by adding the following lines to the services section:

node-php-composer:
 image: savaslabs/node-php-composer:1.2
 ports:
   - "3050:3050"
   - "3001:3001"
 volumes_from:
   - php

This defines the configuration that is applied to a node-php-composer container when spun up. This configuration:

  • Specifies that the container should be created from the savaslabs/node-php-composer image that we built and referenced previously
  • Maps the container ports to our host ports so that we can access the Pattern Labs style guide locally
  • Mounts the project files (that are mounted to the php container) so that they are accessible to the container.

With this service defined in the docker-compose.yml we can start using the theme!

First we spin up the Docker containers by running docker-compose up -d.

Once the containers are running, we can open a Bash shell in the theme directory of the node-php-composer container by running the command:

docker-compose run --rm --service-ports -w /var/www/html/web/themes/custom/pattern_lab_starter node-php-composer /bin/bash

We use the --service-ports option to ensure the ports used for serving the style guide are mapped to the host.

Once inside the container in the theme directory, we install the theme’s dependencies and serve the style guide by running the following commands:

npm install --unsafe-perm
npm start

Voila! Once npm start is running we can access the Pattern Lab style guide at the URL’s that are outputted, for example http://localhost:3050/pattern-lab/public/.

Note: Docker runs containers as root, so we use the --unsafe-perm flag to run npm install with root privileges. This is okay for local development, but would be a security risk if deploying the container to production. For information on running the container as an unprivileged user, see this documentation.

Gulp and Bower are installed as theme dependencies during npm install, therefore we don’t need either installed globally in the container. However, to run these commands we must shell into the theme directory in the container (just as we did before), and then run Gulp and Bower commands as follows:

  • To install Bower libraries run $(npm bin)/bower install --allow-root {project-name} --save
  • To run arbitrary Gulp commands run $(npm bin)/gulp {command}

Other commands listed in the Pattern Lab Starter theme README can be run in similar ways from within the node-php-composer container.

Conclusion

Using Docker for local development has many benefits, one of which is that developers can run applications required by their project inside containers rather than having to install them globally on their local machines. While we typically think of this in terms of the web stack, it also extends to running applications required for front-end development. The Docker image described in this post allows several commonly used front-end applications to run within a container like the rest of the web stack.

While this blog post demonstrates how to build and use a Docker image specifically for use with the Pattern Lab Starter theme, the methodology can be adapted for other uses. A similar approach could be used with Zivtech’s Bear Skin theme, which is another Pattern Lab based theme, or with other contributed or custom themes that rely on npm, Gulp, Bower, or Composer.

If you have any questions or comments, please post them below!

Jan 13 2017
Jan 13
I realize I haven't updated my blog in awhile, because I've been in the thick of developing and rolling out our new site (check it out!). This was taking our existing Drupal 7 site for FSR Magazine and migrating it to this new site, which is responsive and better catered to deal with all food service news. Unfortunately, we ended up killing the Drupal 8 upgrade because it was just too complex, given our custom module codebase. D8 has a mindset change, where everything is an Entity and it takes a lot of OOP and YAML files to get things done.

I figure it would cost us hundreds of hours to upgrade to D8 and it was tempting to upgrade to D8 for the configuration management, Big Pipe, and better caching features. But that's time and money we didn't want to spend on the upgrade.

I've talked with some folks at my local Drupal User Group (shoutout TriDUG!) and several others are in the same boat, where existing sites aren't upgrading, but rather, they're doing new sites in D8.

At some point, we'll need to do something, but we're able to do what we need with D7. If pressed, it may be easier to port to Backdrop vs. upgrade to D8.

Nov 17 2016
Nov 17

This latest release for [RNG](RNG brings two major features: The ability for anonymous users to register for events. And the ability to create, and associate non-users with events.

RNG is an event management module for Drupal 8 created in the spirit of Entity Registration (Drupal 7) and Signup (Drupal 6). Users can create registrations for events, and event managers can manage these registrations.

Note: This post discusses updates to the RNG project which are available in a beta release. See this issue for how to get RNG 1.3 beta.

The event registration form has been reworked into a re-usable Drupal element, whilst making heavy use of AJAX. The registrant selector now accepts multiple registrants. Registrants can also be modified after the registration is created.

*Associate multiple registrants with a registration.* *Create new registrants within the registration form.* *Modify the meta registrant form within the registration form.*

Access control has been reworked to permit anonymous users to register for events.

RNG requires that all registrants for a registration are a Drupal entity. Since anonymous users do not correspond to a user entity, the RNG Contact project provides a way to create non-user registants in a similar fashion to how contacts work on your phone.

See main RNG Contact article: RNG Contact: Anonymous registrants for RNG.

  • Registrant entities now have bundles
  • Added registrant type configuration entity
  • Added control over which identity types can be referenced or created within each event type.
  • Added ability to specify minimum and maximum registrants per registration.
  • Added an interface to view and add RNG related fields.
  • Event settings pages now use the admin theme.
  • And many other behind the scene changes.

Cover photo: B&W Crowd by whoohoo120. License CC BY 2.0

Sep 23 2016
Sep 23

Here at Savas Labs, we listen to our clients needs, and what many of our clients need is to reach their target audiences effectively. Let’s be honest here - perfectly coded, a pretty looking website will be of little use if it doesn’t produce leads / increase brand awareness / facilitate conversions or generate revenue! So how do we help our clients achieve their goals? We get there by balancing website objectives while giving priority to lead generation via SEO. The more quality traffic that comes to the website - the more conversions we can achieve. It’s that simple!

There are multiple ways of generating traffic to a website. The most popular methods are SEO (Search Engine Optimization) and PPC (Pay Per Click). Both bring traffic through search engines. The difference between the two is that SEO brings long-term results boosting organic traffic while PPC helps marketers achieve short-term goals by gaining instant exposure throughout the duration of an ad campaign.

In this post I’ll share some insight about current SEO trends. I’ll also describe new features of Drupal 8 that make it the most SEO-friendly content management framework available today.

Let’s start by taking a look at what exactly Search Engine Optimization is.

What is SEO?

Search Engine Optimization (SEO) is a marketing discipline focused on optimizing a website’s architecture and content so that it performs well (read “ranks high”) in organic search engine results.

Search engines (Google, Bing, etc.) change their search algorithms many times throughout the year. There are over 200 ranking factors that become updated with every algorithmic change. It is worth noting that Google, one of the leading search engines, is steadily growing its market share in the world and the United States for both desktop and mobile search (see chart below).

Search Engine Market Share 2016

Search Engine Market Share 2016

Google’s dominance in web search makes it clear that in 2017 marketers should pay close attention to Google’s algorithmic updates in order to stay ahead of the curve. Standards introduced to SEO by Google are likely to satisfy all other search engines. Given this reality, there are many techniques that website owners can use to optimize their digital property for search engine consumption. So where do we start? How do we know what efforts will bring us the best Return on Investment (ROI) and let our marketers do their job effectively in the long run?

SEO Outlook 2017

The Savas Labs team stays dialed in on the current trends of Search Engines Optimization. By leveraging aggregated research data and first-hand experience, we’ve developed a solid, yet constantly evolving, foundation of currently effective marketing methods.

Here are four ranking factors that we’ve identified as being most important as of Q4 2016. Our forecast is that these four factors will likely remain at the top of the list throughout 2017.

1. Content

Content is still king! Yes, that’s right. It is and it always will be! You’ve got to be relevant in order to even appear in search. And nothing will make you more relevant than carefully crafted, practical, awesome, juicy, shareable, actionable (you name it) CONTENT! It is important to note that marketers should stop thinking of content as purely text and focus their efforts on providing visual content that supports storytelling, is engaging and matches user intent.

Not just any good ‘ol link to your website. Good backlinks come from high authority domains that are in the same niche as your website. Strong backlinks bring quality traffic and are therefore considered highly desirable to your SEO cause.

3. Responsive Design

With more people using their handheld devices to browse the internet, it has become increasingly important to make a website look good across multiple platforms (smartphone, tablet, etc.). It is not an option in 2017 - it is a necessity! While we won’t get into the notoriously labeled Google algorithm update “Mobilegeddon” that happened in April 2015, we will provide some interesting statistics to back up the importance of responsive design.

There are more mobile internet users than desktop internet users; 52.7% of global internet users access the internet via mobile, and 75.1% of U.S. internet users access the internet via mobile.

4 out of 5 consumers use a Smartphone to shop.

4. Page Speed

In response to the substantial increase in mobile traffic growth, search engines have acknowledged the importance of page speed and the effect it has on user experience (UX) and now give more weight to fast-loading websites.

40% of people abandon a website that takes more than 3 seconds to load.

a 2-second delay in load time during a transaction resulted in abandonment rates of up to 87%. This is significantly higher than the baseline abandonment rate of 67%.

Can your business handle the loss in revenue that may occur from slow page load speed?

Drupal 8 - Built with SEO in Mind

The base of all our SEO efforts lies within the website’s architecture. There are many website engines and CMS’s to choose from and most of them will claim that they are SEO optimized. Don’t be fooled! No CMS will come search engine optimized out of the box. It may have some features, which, if configured correctly, may bring you some SEO benefits. SEO is not only about code, though it does start there. SEO is also about the continuous efforts of your marketing team. We all know that time = money. The more efficient your marketing team is in performing tasks within your CMS - the more ROI you get!

A good CMS must provide means for your marketing team to work independently from your development team. Drupal 8 does just that! It provides a solid framework that can be tuned to become a powerful marketing-machine.

Let’s take a look at some of the new features in core that make search engines love Drupal 8.

Drupal 8 is Responsive out of the Box

Drupal 8 comes with responsive themes in core. Now both public facing and admin facing themes are responsive and make user experience great on any device.

Drupal 8 Page Load is Fast

There has been a lot of debate about Drupal 8 vs. Drupal 7 performance / page load since the Drupal 8’s release. It is a fact that vanilla Drupal 8 is running much more code than vanilla Drupal 7. It runs vendor code like Symfony, which adds some overhead. However, Drupal 8 has a significant number of performance improvements that are making up for that overhead:

  • Javascript files are now loading in the footer. Due to this change pages build up faster and user can see and use them earlier.

  • Pluggable CSS/JS aggregation and minification to support more optimal optimization algorithms.

  • Highly improved caching. Drupal 8 uses “cache tags” that makes caching more efficient and includes Cache Context API which provides context-based caching. This means pages load faster while ensuring that visitors always see the latest version of your site.

  • BigPipe render pipeline. Sends pages in a way that allows browsers to show them much faster. First sends the cacheable parts of the page, then the dynamic/uncacheable parts. Uses the BigPipe technique.

These improvements have the potential to make your Drupal 8 website fly! And if after all that it is not “flying” - than you need someone to review the code that powers your website’s features. Contact us.

Semantic Markup

Search engines appreciate clean markup that explicitly describes the purpose of on-page elements. Thanks to the HTML5 Initiative for Drupal 8 development, we now have a number of great markup improvements right in Drupal core:

  • HTML5 themes with new semantic elements in core templates

  • Support for the new form elements to Drupal’s Form API

  • Rich media handling with <video> and <audio> elements

  • ARIA roles in markup to improve accessibility

  • Resource Description Framework (RDF) support that provides a standardized model for data interchange and facilitates Schema.org mappings

  • Twig theming engine - makes it harder for developers to create messy, non-semantic code

Content-as-a-Service

Another exciting new feature of Drupal 8 is a flexible content delivery.

Today, content owners want to get their content to as many platforms and channels as possible: web, mobile, social networks, smart devices, etc. It is expensive to have a separate solution for every channel. It is much more efficient to have a single editorial team and single software platform that allows for well-organized content management. Drupal 8 and its content-as-a-service capability provides a one-stop solution where content is created and managed via unified web-interface and then consumed by other channels with minimal effort.

Drupal 8 Multilingual Capabilities

To reach audiences from around the world, companies need to speak to users in their native language. In 2017, producing content in English language is not enough, even if English is considered an internationally accepted language. The United States is now the world’s second largest Spanish-speaking country after Mexico, which amplifies the necessity of serving multilingual content for U.S. based audience. To help put things in perspective we checked recent statistics.

English is a #1 language used in the Web, but it only amounts to 26.3% of the online market.

There are 41 million native Spanish speakers in the U.S. Around 79% of them using search engines on a daily basis for gathering information about a future purchase.

Reaching a global audience with Drupal has never been this easy! Previous versions of Drupal had partial support for multilingual websites. Luckily, Drupal 8 had a fundamental overhaul of its multilingual system. Every single component is translatable out of the box in Drupal core without any additional modules. Drupal core natively supports 94 languages. Also, the administration interface is now entirely translatable. Media assets (files or images), can now be assigned to a language or shared between languages. This gives a huge advantage to businesses that aim to reach a global audience.

SEO for Drupal 8 is off to a good start with just the core features! Drupal 8 also has a growing number of contributed modules that can amplify your SEO efforts. Just to name a few: Metatag, Google Analytics, Pathauto, Redirect, and more.

Drupal 8 satisfies current SEO trends enabling marketers do their job effectively and efficiently! Even with minimal configuration, Drupal 8 lays a solid base for future marketing performance.

Jun 27 2016
Jun 27

The Workflow Initiative was announced just over a month ago and since then I have been working on the first phases full-time. Here’s what I’ve touched:

In Drupal 8.1.x the RevisionLogInterface was introduced, but not used. In 8.2.x the Node entity will use it. Also BlockContent entity will use it.

When we make everything revisionable we’ll need all entities to get their base fields from the base class. So from 8.2.x all content entities inherit their base fields.

To help people make sensible decisions Node revisions will now be enabled by default.

We got into a pretty annoying issue when trying to add revision fields to entities which already had data because the revision field couldn’t be set to NOT NULL. Andrei wrote an awesome patch to allow an initial from field to be set. We therefore set the revision id as the entity id.

When we make everything revisionable we’ll need an upgrade path, to test this the Shortcut entity is being upgraded to revisionable by a new service. This has still not been committed, so reviews are welcome.

We’re trying to get Workbench Moderation into core as Content Moderation. Still lots to do to make this happen, but the patch is there with passing tests.

Initial work has also started to get an archive field into all entities. This will allow us to have CRAP (create, read, archive, purge) workflow, create a trash system, and replicate entity deletion through environments.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by Disqus
Jun 20 2016
Jun 20

Preface

Wow. Entities. What a mess. I really struggled with this. There's an example of a database-based module not using them and one that does use them. I think it boiled down to me that what I was representing were reports, which seemed very similar to nodes (in hindsight, perhaps this should have been a content type all along), so I should use entities.

Entities

As I look through examples, trying to figure out how to get my menu routing to work with my database entries, it appears that I need to set them up as entities. I say that, looking at the xmlsitemap and dbtng_example and content_entity_example from the examples module. So, let me back up and learn about D8 Entities. It looks like entities were introduced in Drupal 7 and there is a guide for that, but it's based on D7, so the concepts stuff is ok, but the implementation guidance is outdated. There's a link at the top to take you to a D8-specific guide. The guide says it's just a holder, but I'm going to give it a go. If nothing else, perhaps this post will help others.


First, entity types. Either you're setting up a configuration entity or a content entity. I think content entity fits for me. Next, looking at requirements section of the content type doc, I need to setup src/Entity/GatedContentEntity.php file with docblock stuff that will give the loader the information it needs. Hey, checkout this helpful hint about double quotes in the annotations doc (post #1 reference)! That annotations doc doesn't provide what all you need, but I'll take those 3 lines to start off with. I think this doc may list all of the properties you can use with ContentEntityType, but not sure which ones I really need and how to use them. This is when the examples/content_entity_example/src/Entity/Contact.php file helped. I first just used the core form & access handlers for now.

Random Aside: Sublime Text 3

I use Sublime Text 3 as my IDE and there is a PHPDoc plugin that makes life editing these annotations easier. Package Manager makes it easy to install plugins: just follow the installation instructions (and restart ST), then follow the usage instructions to pull up the Install Package option and then type phpdoc and select it to install. Then restart ST again and you'll be much happier. I think the same applies for Sublime Text 2, too. This Drupal doc has some other helpful Sublime Text tips.

Back to it...

So I found this great doc on how to create your own content entity. And whoa ... this comment pointed me to the Drupal Console project, which can jumpstart this a LOT. So I tucked my handcrafted code away and ran this command:

drupal --target="fnmd8.local" gect --module="gated_content" --entity-class="GatedContent" --entity-name="gated_content" --label="Gated Content"

The end result is about 10 files with lots of boilerplate fields that don't match up with what I'm using. Maybe I'm going about this all wrong. Maybe I don't need Entities? Time to take a break...

After the weekend

So I went back and added the preface above and I feel I need to stick the course with Entities. Perhaps this post will be totally rewritten after I totally understand them and and write a better guide that flows better. I need to just release this post because I'm about to flip back to some D7 work and will be taking another break. I'll pick this post thread back up again shortly.

Jun 17 2016
Jun 17
This is the next steps in getting my module working in Drupal 8. In the previous post, I got the block setup working, but it's just a generic "Hello World!" output. The block gets the most recent entries from a database table, so my next step is getting the database setup.

Databases

This doesn't seem to have changed from Drupal 7 to 8. You basically copy over the .install file. The hook_schema() takes care of it. BTW, one thing that is different is if you need to uninstall/reinstall the modules, you just need to drush pm-uninstall MODULE vs. drush dis MODULE ; drush pm-uninstall MODULE.

Once I had the database tables in place, I dumped my database tables from D7 and imported them into D8. NOTE: I probably need to either codify this (perhaps there's a way to hook into the migration system?) or keep notes about what tables to dump/import once everything's in place.

Now I needed to update my block to query the most recent entries in the table. db_query() is still supported in D8, however it looks like it's going away for D9, so if I wanted to future-proof my module even more, I could convert it to the new injection technique. So I created a storage class in my src folder. I downloaded the examples module and looked at the dbtng_example submodule and its DbtngExampleStorage class. I also found this answer on StackOverflow and the example posted in the select doc helpful. I ended up with a hybrid approach:

So a few things there. First, my post_date field was a legacy datetime field, so I had to convert the current time from a timestamp to the date format that mysql expected. format_date() is deprecated, so they recommend using Drupal::service('date.formatter'). Also, I make my __construct() method a little dynamic so that I don't have to pass in a connection, but it can support it if it is passed in.

Then my block build() method uses my storage class and calls the get_reports() method, passing in the block's configuration array. First, I add use Drupal\gated_content\GCStorage; to the top of my block plugin. Then my build() looks like this:

  public function build() {
    $storage = new GCStorage;
    $reports = $storage->get_reports($this->configuration);
    $build = array();
    kint($reports, $build);
    return $build;
  }

Block View

So now that my block has the right set of reports to show, I need to show them. This takes me to the whole theming and templates stuff. The D7 version called theme() directly and there was a hook_theme and I had a template. The end result was a simple
and foreach report, a
and l(); call to link to the report. Kind of reminds me of item_list a bit, so I think I'll start there.

Ok, it looks like item_list looks about the same. My render array just needs to '#theme' => 'item_list'. But l() is gone. This comment pointed me in the right direction. So now it looks like this:

    $items = array();
    foreach ($reports as $report) {
      $url = Url::fromRoute('');
      $items[] = \Drupal::l(t($report['title']), $url);
    }
    $build = array(
      'report_list' => array(
        '#theme' => 'item_list',
        '#items' => $items,
        '#attributes' => array(
          'class' => 'report-block',
        ),
      ),
    );

Success!

Of course, these all link to just the homepage, which isn't very helpful.

Menu

So now I'm looking down another rabbit hole. The URL to my report is reports/[ID#]. But I use this module on separate domains, so the reports part is configurable. So I need to get that configuration setting to get the first bit of the URL. Actually, I think I can skip that since we'll be doing something different in our migration plans, so I can skip that part of the module. [postnote: if I do end up needing this, it looks like this shows how]

Also, I had this module all setup with pathauto to get a nice SEO-friendly public URL. Ok, so I need to setup the menu routing so the user can get to /gated_content and /gated_content/[ID#] and /gated_content/[ID#]/download. And I need to setup pathauto again.

First, all of those URL's were setup in D7 with type of MENU_CALLBACK, so according to this doc, I need to set them up in a .routing.yml file. This wasn't something DMU setup for me, so I did this from scratch. This was a great doc that showed some examples going from D7 to D8.

My first question is that I was familiar with named placeholders and how the name would be used to call a NAME_load() function to put together the argument that gets passed into the callback. But these examples are using book, which uses node and entity, so I need something that will use my own data type.

Stuff for the next post!

Jun 15 2016
Jun 15
We are looking into porting our site to Drupal 8 and we have over 30 custom modules, so it's a pretty big undertaking. I thought I'd blog about the adventures in the hopes of helping others tackle porting to D8 as well as notes for myself.

Back in April, I took one of our sites and used Drupal Upgrade module to get it moved into D8, following these instructions. I really haven't touched it since then, so I can't really remember how that went. So I just blew the dust off and upgraded from 8.0 to 8.1 in the process. One of the first things I noticed is that comments were turned off, so I had to navigate into the various content types and edit them and edit the comments field to change it from Closed to Open.

Then I was looking through our custom modules and trying to figure out where to start first. Some of our stuff builds on top of each other, so I need to look at the hierarchy to determine what is something that isn't dependent on other things.

One of my first steps was to create a custom modules folder and copy my D7 custom module into that. I then copied the folder as a backup. Then I used the Drupal Module Upgrader (DMU) to generate an upgrade info report as well as attempt the upgrade process. Then I renamed that folder to append -auto and then I created a new folder where I will start work in earnest.

cd /path/to/drupal8 ;
mkdir -p modules/custom/gated_content ;
cp -r /path/to/drupal7/sites/all/modules/custom/gated_content modules/custom/gated_content ;
cp -r /path/to/drupal7/sites/all/modules/custom/gated_content modules/custom/gated_content-orig ;
drush dmu-analyze gated_content --path=modules/custom/gated_content ;
drush dmu-upgrade gated_content --path=modules/custom/gated_content ;
mv modules/custom/gated_content modules/custom/gated_content-auto ;
mkdir modules/custom/gated_content ;

So my first custom module handles what we call "gated content," which is a collection of PDF files that our website users can request to download, which takes them to a request form and after they fill it out, it emails them a link to the PDF.

The module also has a block that shows the most recent gated content entries, which we include in our sidebar.

So I just wanted to start with the block and the block's settings.

The Block

One of the reasons I wanted to start from scratch is that this module is pretty big with lots of code that needs addressing to work in D8 and I'd like to incrementally get stuff working without having all the old stuff in there. So I copied the module's .info.yml and .permissions.yml files into my blank folder to start. No .module file. At least for now.
Blocks have a different way of doing things. You create a plugin. You don't need to point to your plugin through some .yml file. You create a src/Plugin/Block folder in your module and then create a php file for your plugin class. The filename needs to match PSR-4 specifications, which is the magic that autoloads all of this. So I called mine GCListBlock.php. One of my first lessons is that my namespace has to match the module name. My module name is gated_content and I initially used gatedcontent, which resulted in the block showing up in the list, but once you tried to place it in a region, it would result in an error.
Another odd thing is that the phpdoc is very specific with double quotes vs. single quotes. Use double quotes.

Using single quotes resulted in this error:

Doctrine\Common\Annotations\AnnotationException: [Syntax Error] Expected PlainValue, got ''' at position 18 in class Drupal\gated_content\Plugin\Block\GCListBlock. in Doctrine\Common\Annotations\AnnotationException::syntaxError() (line 42 of /Users/jason/Sites/devdesktop/fnmd8-dev/docroot/vendor/doctrine/annotations/lib/Doctrine/Common/Annotations/AnnotationException.php).

Another hiccup I ran across is I had t() calls through my block code that I had to change to $this->t().
I think there was some sort of 8.1 update, which wasn't reflected in the Blocks doc I was pointed to from the DMU report (I added a comment), that says $form_state is now a FormStateInterface. This also means you need to update accessing the form_state values to use $form_state->getValue('FORMFIELDNAME').

Finally, blocks don't just show up in the list of blocks on the blocks page like you might expect from Drupal 6/7 days. You have to click the Place Block button to pull up a list of available blocks and you should see it there. This is pretty neat because you can add a block more than once to a region or across multiple regions. I'm sure there's some way to differentiate the build based on its own order and region, but I don't have to worry about that for now.

The Settings

I initially thought I needed to create a settings yml file, but actually, since the settings aren't used outside of the block code, they can be all specified and used in the block's class. You can provide the defaultConfiguration() method to specify the default values (return an array of keys to values).

Bonus: Devel Module

One of the hiccups I faced was how to get the devel and dpm() calls working like I was used to in D7. One of the cool things I undercovered while getting devel setup is the webprofiler submodule, which adds a nifty toolbar at the bottom of the page with lots of development information and options.


I couldn't get dpm() (or dsm()) calls to show anything, so I stumbled across the kint submodule, which looks even better. You basically enable it and then use kint() vs. dpm(). One bonus is that kint() allows for multiple variables in the same call. It outputted the variable in a narrow region, but there's right arrow you can click on to open it in a new tab. And there's a useful stack trace beneath the output. One other thing I read is that dpm() doesn't have access to protected data, but kint() does.

Also, if you're not seeing output from either dpm() or kint() calls, try doing a cache-rebuild (drush cr). If that fixes it, you may want to setup local development configuration overrides so you don't have to rebuild manually every time.
Mar 31 2016
Mar 31

In the previous article of this series we’ve started our dive into the Entity Validation and Typed Data APIs. We’ve seen how DataType plugins interact with data definitions and how various constraints can be added to the latter at multiple levels and extension points.

Drupal 8 logo

In this part, we will cover the aspect of actual validation and violation handling. In addition, we will write our own constraint and validator so that we can use custom behaviors in the data validation process.

Validation and Violation Handling

Even though we don’t yet know exactly how constraints are built, we’ve seen how they can be added to Typed Data definitions, including entity fields. Let us now see how we can validate the entities and handle possible violations we find.

When talking about Typed Data we’ve already seen how the validate() method can be called on the DataType plugin instance which holds a data definition. When it comes to entities, this can happen both at entity and field levels.

For instance, we can validate the entire entity using the validate() method:

$entity->set('title', 'this is too long of a title');
$violations = $entity->validate();

In our previous article, we added the Length constraint to Node titles to prevent title strings longer than 5 characters. If that is still in place and we run the code above, the validation should obviously fail. The $violations object is now, however, an EntityConstraintViolationListInterface instance which provides some helper methods for accessing violation data specific to Drupal content entities. It’s worth looking into that interface for all the helper methods available.

To get a list of Entity level violations we can use the getEntityViolations() method but we can also loop through all of them. Once we have our individual ConstraintViolationInterface instances, we can inspect them for what went wrong. For instance, we can get the error message with getMessage(), the property path that failed with getPropertyPath() and the invalid value with getInvalidValue(), among other useful things.

When it comes to fields, the property path is in the following format: title.0.value. This includes the field name, the key (delta) of the individual field item in the list and the actual property name. This represents the property path of our violation above.

Apart from calling validation on the entire entity (which may be superfluous at times), we can also do so directly on each field:

$entity->set('title', 'this is too long of a title');
$violations = $entity->get('title')->validate();

In this case, $violations is again an instance of ConstraintViolationListInterface and can be looped over to inspect each violation. This time, though, the property path changes to no longer include the field name: 0.value.

And lastly, we can even validate the individual items in the list:

$violations = $entity->get('title')->get(0)->validate();

As we can expect, the difference now is that the property path will only show value in the violation since we know exactly what we are validating: the first data definition in the list.

Constraints and Validators

We only touched upon the aspect of constraints and validators but let us better understand how they work by creating one ourselves. We will create one single constraint and validator but that can be used for both Node entities and their fields. It’s always better to have constraints targeted to the data definition we want but for the sake of brevity, we’ll do it all in one constraint to see how all these options could be handled.

The business case of our example is to have a constraint that we can apply to any string-based content entity field that would force the string to contain a certain alphanumeric code. The latter will be passed on as an option so it can be reused. If it’s applied to a Node entity, we want to make sure the title of the node contains the code. So let’s get started.

First, inside our demo module, which can be found in this git repository, we need to create the Validation folder inside the Plugin folder of our namespace (the src/ directory). Inside that, we need the Constraint folder. This is because constraints are plugins expected to be defined in there.

Second, inside Constraint/, we can create our constraint class:

<?php



namespace Drupal\demo\Plugin\Validation\Constraint;

use Symfony\Component\Validator\Constraint;
use Symfony\Component\Validator\Exception\MissingOptionsException;



class HasCodeConstraint extends Constraint {

  
  public $messageNoCode = 'The string <em>%string</em> does not contain the necessary code: %code.';

  
  public $code;

  
  public function __construct($options = NULL) {
    if ($options !== NULL && is_string($options)) {
      parent::__construct(['code' => $options]);
    }

    if ($options !== NULL && is_array($options) && isset($options['code'])) {
      parent::__construct($options);
    }

    if ($this->code === NULL) {
      throw new MissingOptionsException('The code option is required', __CLASS__);
    }
  }
}

As we mentioned earlier, the constraint plugin class is relatively simple. It has the expected annotation which, among boilerplate metadata, also specifies what type of data this constraint can be applied to. We chose both a simple string and the Node entity type. We then declare two public properties: a message to be used when the constraint fails (with placeholders being populated inside the validator) and the actual code to be checked for (populated by the constructor of the parent Constraint class). Inside our constructor, we check if the options passed are a string or array (just to be a bit flexible) and throw an exception if the code parameter is not passed as an option in any shape or form.

One of the tasks of the parent Constraint class is to specify which class will be used to validate this constraint. By default, it is a class named like the constraint itself but with Validate appended at the end (HasCodeConstraintValidator). If we wanted to create a differently named and/or namespaced class, we would have to override the validatedBy() method and return the fully qualified name of the class we want.

Let’s now see the HasCodeConstraintValidator class since for us the default is fine:

<?php



namespace Drupal\demo\Plugin\Validation\Constraint;

use Drupal\Core\Field\FieldItemListInterface;
use Drupal\node\NodeInterface;
use Symfony\Component\Validator\Constraint;
use Symfony\Component\Validator\ConstraintValidator;
use Symfony\Component\Validator\Exception\UnexpectedTypeException;
use Symfony\Component\Validator\Violation\ConstraintViolationBuilderInterface;


class HasCodeConstraintValidator extends ConstraintValidator {

  
  protected $context;

  
  public function validate($data, Constraint $constraint) {
    if (!$constraint instanceof HasCodeConstraint) {
      throw new UnexpectedTypeException($constraint, __NAMESPACE__.'\HasCodeConstraint');
    }

       if ($data instanceof NodeInterface) {
      $this->validateNodeEntity($data, $constraint);
      return;
    }

       if ($data instanceof FieldItemListInterface) {
      foreach ($data as $key => $item) {
        $violation = $this->validateString($item->value, $constraint);
        if ($violation instanceof ConstraintViolationBuilderInterface) {
          $violation->atPath('title')->addViolation();
        }
      }
      return;
    }

       if (is_string($data)) {
      $violation = $this->validateString($data, $constraint);
      if ($violation instanceof ConstraintViolationBuilderInterface) {
        $violation->addViolation();
        return;
      }
    }
  }

  
  protected function validateNodeEntity($node, $constraint) {
    foreach ($node->title as $item) {
      $violation = $this->validateString($item->value, $constraint);
      if ($violation instanceof ConstraintViolationBuilderInterface) {
        $violation->atPath('title')
          ->addViolation();
      }
    }
  }

  
  protected function validateString($string, $constraint) {
    if (strpos($string, $constraint->code) === FALSE) {
      return $this->context->buildViolation($constraint->messageNoCode, array('%string' => $string, '%code' => $constraint->code));
    }
  }

}

The main job of this class is to implement the validate() method and build violations onto the current execution context if the data passed to it is somehow invalid. The latter can be more than one type of data, depending on what the constraint is applied to.

In our example, we use the constraint for entities, field item lists and primitive data as well. This is just to save us some space. But the logic is that if a Node entity is passed, the code is checked in its title while for the the field items an iteration is performed to check the values of the fields. And of course, the constraint can also be added to individual field items in which case the $data variable would be the value of the data definition.

The $context property has a handy buildViolation() method which allows us to specify a number of things related to what actually failed (path, message, etc).

So if we want to make use of this constraint, we can apply what we learned earlier and do one of the following 3 things:

Inside hook_entity_type_alter():

$node = $entity_types['node'];
$node->addConstraint('HasCode', ['code' => 'UPK']);

This adds the constraint to the Node entities so all node titles now have to contain the code UPK.

Inside hook_entity_base_field_info_alter() or hook_entity_bundle_field_info_alter(), one of the following two lines:

$title->addConstraint('HasCode', ['code' => 'UPK']);
$title->addPropertyConstraints('value', ['HasCode' => ['code' => 'UPK']]);

Where $title is a field definition and the first case adds the constraint to the entire list of items while the latter adds it straight to the individual item.

These three possibilities cover the three cases the constraint validator handles in our example. And that is pretty much it. Not too difficult.

In this article, we’ve continued our dive into the Entity Validation API by looking at two things: validation and violation handling, and the creation of custom constraints. We’ve seen that once applied onto entities or field data definitions, constraints can be very easily validated and inspected for violations. This API borrows a lot from Symfony and makes it a breeze to decouple validation from the Form API.

Also easy, as we’ve seen, is to extend the existing pool of validation criteria available. This is done via constraint plugins, all coupled with their own validators and which can be written in very reusable ways. It’s very interesting to play around with these and see how all the pieces interact. And it is also a great learning experience.

Daniel Sipos

Meet the author

Daniel Sipos is a Drupal developer who lives in Brussels, Belgium. He works professionally with Drupal but likes to use other PHP frameworks and technologies as well. He runs webomelette.com, a Drupal blog where he writes articles and tutorials about Drupal development, theming and site building.
Mar 29 2016
Mar 29

Data validation is a very important part of any application. Drupal 7 has a great Form API that can handle complex validation of submitted data, which can then be turned into entities. However, form level validation is problematic. For example, it becomes difficult to handle entity (data) validation programmatically. We have to either reimplement validation logic or mimic form submissions in code. This makes any data interaction dependent on the form system and this is a bad idea.

Drupal 8 logo

With the introduction of subsystems such as the REST API, Drupal 8 needed something better to handle this problem. The Symfony Validation component was chosen and Drupal 8 builds on top of it to tailor for the realties of the Typed Data and plugin based Entity system. So now, form submissions validate entities just like REST calls do or any other programmatic interaction with the entities easily can.

In this article, and its followup, we will explore the Drupal 8 Entity Validation API, see how it works, and how it can be extended. To better understand this, we will also take a look at the Typed Data API which underpins the entity system in Drupal 8. There will be some code examples that already exist in core but we will also write a bit of our own code which can be found in this git repository within the demo module.

Typed Data

Typed Data is the Drupal 8 API that was built to provide a consistent way of interacting with data or metadata about the data itself. Why is this important for our topic? Because validation is defined and invoked on typed data objects.

Two important components of this API stand out: the data definition and the DataType plugins. The role of the former is to define data and how interaction works with it (including things like settings or validation constraints). The role of the latter is to provide a way to get and set values from that type of data. When they are instantiated, data type plugins make use of data definition instances passed on by the plugin manager. The latter can also infer which data definition needs to be used by a DataType plugin type.

Let’s see an example:

$definition = DataDefinition::create('string')
    ->addConstraint('Length', array('max' => 20));

We created a string data definition and applied the Length constraint to it. Constraints are a key part of the validation API and they define the type of validation that will run on the data. They are coupled with a validator that actually performs the task, but we will see more about constraints and validators in the second part of this series.

Next, we use the DataType plugin manager to create the actual data type plugin instance:

$string_typed_data = \Drupal::typedDataManager()->create($definition, 'my string');

We loaded the manager service statically here for brevity but you should use dependency injection in your project if you are in a class context. The create() method on the TypedDataManager takes the data definition as the first parameter, the actual value as its second and returns a DataType plugin instance of the type that matches the definition: in our case StringData. For complex data, DataType plugins can specify in their annotations which data definition class they need to use. And for more information about plugins in Drupal 8, make sure you check out my previous articles on the topic.

One of the methods on this plugin instance is validate(), which triggers data validation against all the constraints that have been applied to the definition and returns an instance of Symfony’s ConstraintViolationList. By iterating over it or using methods like count() or get() we can check if the data is valid and which constraints have failed if not.

In our example, the violation list should have no validation errors because the string we used is under 20 characters. Had we used a longer string when creating the plugin, we would have had one violation represented by a Symfony ConstraintViolationInterface instance with a message, offending value and even a property path.

Typed Data and Content Entities

Now that we know a bit about the Typed Data API, let’s see how this applies to content entities.

Entity data in Drupal 7 is split between entity properties (usually the columns on the entity table) and Field API fields that are configured through the UI. In Drupal 8, they have been brought under the same umbrella so the old properties become fields as well. However, a difference still remains in that some fields (mainly the old entity properties) are defined as base fields while the rest are configurable fields.

The data definitions for these fields are BaseFieldDefinition and FieldConfig, but they are both implementors of the same FieldDefinitionInterface (which is a complex extender of the DataDefinitionInterface – the interface directly implemented by DataDefinition we saw earlier).

Each individual field holds data in a special FieldItemListInterface implementation and is always a list of individual FieldItem plugins (even if there is only one real value in the field). Each of these plugins extends a DataType plugin and uses a type of DataDefinitionInterface implementation itself (usually FieldItemDataDefinition). Things are quite complex at this level so it’s well worth taking a closer look at the makeup of entity fields to better understand this architecture.

Adding Entity and Field Constraints

Constraints in Drupal 8 are also plugins which usually hold a small amount of information about how data is actually being validated, what error message should be used in case of failure and any additional options the validator needs. The validator class (which is referenced by the constraint) is responsible for checking the data. We’ve seen one example, Length, which is in fact the LengthConstraint class validated directly by Symfony’s LengthValidator class. We will see in the second part how to create our own constraint and validator. For now, though, let’s see how we can add existing constraints to content entities and fields.

Entity level constraints

Entity level constraints are added in the annotation of the entity class itself. For example, this is how the Comment entity has defined a constraint for its name/author fields combination:

...
  constraints = {
    "CommentName" = {}
  }
...

In this example, CommentName is the plugin ID of the constraint that will be validated against when saving a comment entity. The opening and closing braces means that the plugin takes no options.

If we wanted to add to or remove a constraint from an existing entity, we’d have to implement hook_entity_type_alter():

function demo_entity_type_alter(array &$entity_types) {
  
  $node = $entity_types['node'];
  $node->addConstraint('ConstraintPluginName', ['array', 'of', 'options']);
}

In this example, we are adding a fictitious constraint to the Node entity and passing an array of options to it.

Field level constraints

There is more than one way to add field level constraints depending on whether the content entity type is defined in our module and whether the type of field we are talking about is a base field or configurable.

If we are defining our own entity type, one of the methods on the actual entity class that we have to implement is baseFieldDefinitions(). This is where we return an array of BaseFieldDefinition field definitions and we can easily add our constraints there. For example, this is how the Node ID base field is being defined:

$fields['nid'] = BaseFieldDefinition::create('integer')
  ->setLabel(t('Node ID'))
  ->setDescription(t('The node ID.'))
  ->setReadOnly(TRUE)
  ->setSetting('unsigned', TRUE);

Similarly to how we added constraints to the DataDefinition instance earlier, the Node entity could also add constraints to the Node ID field, more specifically:

  • to the BaseFieldDefiniton itself, which is, as we saw, the definition for the FieldItemListInterface implementation (the list)
  • to the individual FieldItemDataDefinition items, which are, as we saw, a type of complex data definition for the FieldItemInterface implementations (the items)

If we want to add a constraint to a Node base field (or of any other content entity type not defined by our module), we have to implement hook_entity_base_field_info_alter() and add our constraint there:

function demo_entity_base_field_info_alter(&$fields, \Drupal\Core\Entity\EntityTypeInterface $entity_type) {
  if ($entity_type->id() === 'node') {
    
    $title = $fields['title'];
    $title->addPropertyConstraints('value', ['Length' => ['max' => 5]]);
  }
}

In the example above, we are adding a constraint to the Node title field to make sure that no title can be longer than 5 characters. To notice is that we are using the addPropertyConstraint() method instead of the addConstraint() one we saw earlier. This is because we are not targeting the definition for the list of items but the individual item definitions themselves (FieldItemDataDefinition).

By using the addConstraint() method, we are adding a constraint to the entire list of items. This means that when the constraint gets validated, the validator receives the entire list not just the value of the individual item.

If we want to add a constraint to a configurable field, the process is quite similar. The difference is that we need to implement hook_entity_bundle_field_info_alter() and work with FieldConfig instances instead of BaseFieldDefinition.

If we want to inspect the constraints that are already set on the field, we can do something like this:

$title = $fields['title'];
$constraints = $title->getConstraints();
$property_constraints = $title->getItemDefinition()->getConstraints();

In this example, $title is either an instance of BaseFieldDefinition or FieldConfig. The $constraints array is, as expected, a list of constraints applied to the entire list of field items while the $property_constraints is an array of constraints applied to the individual field items themselves. We can notice, though, that if we run this code after we’ve applied the Length constraint, we will find inside $property_constraints a ComplexData constraint which wraps over the individual constraints applied to the field values. This is because there can be multiple individual data definitions for a single field item and Drupal by default groups them like so.

Conclusion

In this article, we’ve started looking at the Entity Validation API in Drupal 8. To this end, we also had to get a sense of the Typed Data API which is used as a foundation for the entity system. Once that became a bit clearer, we’ve seen how constraints can be added to various types of data definitions, including those used by entities and fields.

In the next part, we will look at how the actual validation works and how handling the violations that may occur should be done. Additionally, we will create our own constraint and validator and apply our knowledge from this part to various data definition types. Fun!

Daniel Sipos

Meet the author

Daniel Sipos is a Drupal developer who lives in Brussels, Belgium. He works professionally with Drupal but likes to use other PHP frameworks and technologies as well. He runs webomelette.com, a Drupal blog where he writes articles and tutorials about Drupal development, theming and site building.
Mar 09 2016
Mar 09

Migrate is one of the most established modules in the Drupal ecosystem. So much so that with Drupal 8, a decision has been made to get some of its functionality ported and added to Drupal core. An important reason was that the traditional upgrade between major releases was replaced with a migration of Drupal 6 or 7 content and configuration to Drupal 8.

Drupal 8 logo

Not all the functionality of the Migrate module has been moved into core though. Rather, we have the basic framework within the core migrate module and the functionality serving the upgrade path from Drupal 6 and 7 within the core migrate_drupal module. What’s left can be found in a host of contrib modules. The most important is Migrate Tools which contains, among other things, drush commands and the UI for running migrations. Additionally, the Migrate Source CSV, Migrate Source XML and Migrate Source JSON modules provide plugins for the most used types of migration sources.

In this article we are going to look at how migration works in Drupal 8 by migrating some content into node entities. For simplicity, the data we play with resides in tables in the same database as our Drupal installation. If you want to see the final code, you can check it out in this repository.

Drupal 8 Migration Theory

The structure of a migration in Drupal 8 is made up of three main parts: the source, the process and the destination, all three being built using the new Drupal 8 plugin system. These three parts form a pipeline. The source plugin represents the raw data we are migrating and is responsible for delivering individual rows of it. This data is passed to one or more process plugins that perform any needed manipulation on each row field. Finally, once the process plugins finish preparing the data, the destination plugins save it into Drupal entities (either content or configuration).

Creating a migration involves using such plugins (by either extending or directly using core classes). All of these are then brought together into a special migration configuration entity. This is shipped as module config that gets imported when the module is first enabled or can be constructed using a template (a case we won’t be exploring today). The migration configuration entity provides references to the main plugins used + additional metadata about the migration.

Movie Migration

Let us now get down to it and write a couple of migrations. Our data model is the following: we have two tables in our database: movies and movies_genres. The former has an ID, name and description while the latter has an ID, name and movie ID that maps to a movie from the first table. For the sake of simplicity, this mapping is one on one to avoid the complication of a third table. Here is the MySQL script that you can use to create these tables and populate with a few test records (if you want to follow along):

CREATE TABLE `movies` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `name` varchar(255) DEFAULT NULL,
  `description` text,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8;

CREATE TABLE `movies_genres` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `movie_id` int(11) DEFAULT NULL,
  `name` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8;

INSERT INTO `movies` (`id`, `name`, `description`)
VALUES
        (1, 'Big Lebowsky', 'My favorite movie, hands down.'),
        (2, 'Pulp fiction', 'Or this is my favorite movie?');

INSERT INTO `movies_genres` (`id`, `movie_id`, `name`)
VALUES
        (1, 1, 'Comedy'),
        (2, 1, 'Noir'),
        (3, 2, 'Crime');
        

What we want to achieve is migrate the movies into basic Drupal Article nodes and the genres into taxonomy terms of the Tags vocabulary (which the Article nodes reference via a field). Naturally, we also want the migration to mirror the association between the movies and the genres.

Genres

Let us first take care of the genre migration because in Drupal, the movies will depend on them (they will reference them).

Inside the config/install folder of our module, we need the following configuration entity file called migrate.migration.genres.yml:

id: genres
label: Genres
migration_group: demo
source:
  plugin: genres
  key: default
destination:
  plugin: entity:taxonomy_term
process:
  vid:
    plugin: default_value
    default_value: tags
  name: name

The first three elements of this configuration are the migration ID, label and group it belongs to. The following three keys are responsible for the three main parts of the migration pipeline mentioned earlier. Let’s talk about the latter three separately.

Source

Under the source key we specify which plugin the migration should use to represent and deliver the source data (in our case the plugin with the ID of genres). The key key is used to specify which database should our source query use (that is where our data is).

So in the Plugin/migrate/source folder of our module, let’s create our SQL based source plugin for our genres:

Genres.php

namespace Drupal\demo\Plugin\migrate\source;

use Drupal\migrate\Plugin\migrate\source\SqlBase;


class Genres extends SqlBase {

  
  public function query() {
    $query = $this->select('movies_genres', 'g')
      ->fields('g', ['id', 'movie_id', 'name']);
    return $query;
  }

  
  public function fields() {
    $fields = [
      'id' => $this->t('Genre ID'),
      'movie_id' => $this->t('Movie ID'),
      'name' => $this->t('Genre name'),
    ];

    return $fields;
  }

  
  public function getIds() {
    return [
      'id' => [
        'type' => 'integer',
        'alias' => 'g',
      ],
    ];
  }
}

Since we are using a SQL source, we need to have our own source plugin class to provide some information as to how the data needs to be read. It’s not enough to use an existing one from core. The query() method creates the query for the genre data, the fields() method defines each individual row field and the getIds() method specifies the source row field that acts as the unique ID. Nothing complicated happening here.

We are extending from SqlBase which, among other things, looks for the plugin configuration element named key to learn which database it should run the query on. In our case, the default one, as we detailed in the migration configuration entity.

And this is all we need for our simple source plugin.

Destination

For the migration destination, we use the default core taxonomy term destination plugin which handles everything for us. No need to specify anything more.

Process

Under the process key of the migration, we list each destination field we want populated and one or more process plugins that transform the source row fields into data for the destination fields. Since we want the genres to be all terms of the Tags vocabulary, for the vid field we use the default_value plugin which accepts a default_value key that indicates the value each record will have. Since all will be in the same vocabulary, this works well for us.

Lastly, for the term name field we can simply specify the source row field name without an explicit plugin name. This will, under the hood, use the get plugin that simply takes the data from the source and copies it over unaltered to the destination.

For more information on how you can chain multiple process plugins in the pipeline or what other such plugins you have available from core, I recommend you check out the documentation.

Movies

Now that our genres are importable, let’s take a look at the movies migration configuration that resides in the same folder as the previous (config/install):

migrate.migration.movies.yml

id: movies
label: Movies
migration_group: demo
source:
  plugin: movies
  key: default
destination:
  plugin: entity:node
process:
  type:
    plugin: default_value
    default_value: article
  title: name
  body: description
  field_tags:
    plugin: migration
    migration: genres
    source: genres
migration_dependencies:
  required:
    - genres

We notice the same metadata as before, the three main parts of the migration (source, process and destination) but also the explicit dependency which needs to be met before this migration can be successfully run.

Source

Like before, let’s take a look at the movies source plugin, located in the same place as the genres source plugin (Plugin/migrate/source ):

Movies.php:

namespace Drupal\demo\Plugin\migrate\source;

use Drupal\migrate\Plugin\migrate\source\SqlBase;
use Drupal\migrate\Row;


class Movies extends SqlBase {

  
  public function query() {
    $query = $this->select('movies', 'd')
      ->fields('d', ['id', 'name', 'description']);
    return $query;
  }

  
  public function fields() {
    $fields = [
      'id' => $this->t('Movie ID'),
      'name' => $this->t('Movie Name'),
      'description' => $this->t('Movie Description'),
      'genres' => $this->t('Movie Genres'),
    ];

    return $fields;
  }

  
  public function getIds() {
    return [
      'id' => [
        'type' => 'integer',
        'alias' => 'd',
      ],
    ];
  }

  
  public function prepareRow(Row $row) {
    $genres = $this->select('movies_genres', 'g')
      ->fields('g', ['id'])
      ->condition('movie_id', $row->getSourceProperty('id'))
      ->execute()
      ->fetchCol();
    $row->setSourceProperty('genres', $genres);
    return parent::prepareRow($row);
  }
}

We have the same three required methods as before, that do the same thing: query for and define the data. However, here we also use the prepareRow() method in order to alter the row data and available fields. The purpose is to select the ID of the movie genre that matches the current row (movie). That value is populated into a new source field called genres, which we will see in a minute how it’s used to save the Tags taxonomy term reference.

Destination

In this case, we use the node entity destination plugin and we need nothing more.

Process

There are four fields on the Article node we want populated with movie data. First, for the node type we use the same technique as before for the taxonomy vocabulary and set article to be the default value. Second and third, for the title and body fields we map the movie name and description source fields unaltered.

Lastly, for the tags field we use the migration process plugin that allows us to translate the ID of the genre (that we added earlier to the genres source row field) into the ID of its corresponding taxonomy term. This plugin does this for us by checking the migration mapping of the genres and reading these IDs. And this is why the genres migration is also marked as a dependency for the movies import.

Activating and Running the Migration

Now that we have our two migration configuration entities and all the relevant plugins, it’s time to enable our module for the first time and have the configuration imported by Drupal. If your module was already enabled, uninstall it and then enable it again. This will make the config import happen.

Additionally, in order to run the migrations via Drush (which is the recommended way of doing it), install the Migrate Tools module. Then all that’s left to do is to use the commands to migrate or rollback the movies and genres.

To see the available migrations and their status:

drush migrate-status

To import all migrations:

drush migrate-import --all

To roll all migrations back:

drush migrate-rollback --all

Conclusion

And there we have it – a simple migration to illustrate how we can now import, track and roll back migrations in Drupal 8. We’ve seen how the plugin system is used to represent all these different components of functionality, and how the migration definition has been turned into configuration that brings these elements together.

There is much more to learn, though. You can use different source plugins, such as for data in CSV or JSON, complex process plugins (or write your own), or even custom destination plugins for whatever data structure you may have. Good luck!

Daniel Sipos

Meet the author

Daniel Sipos is a Drupal developer who lives in Brussels, Belgium. He works professionally with Drupal but likes to use other PHP frameworks and technologies as well. He runs webomelette.com, a Drupal blog where he writes articles and tutorials about Drupal development, theming and site building.
Feb 24 2016
Feb 24

This is part 4 of a series investigating what to do with your Drupal 6 site as EOL has now come.

Part 1 - overview | Part 2 - the risks | Part 3 - the options | Part 4 - Drupal 7 or Drupal 8?

Should I upgrade my Drupal 6 site to 7 or 8?

Today Drupal 6 reaches retirement, or end of life (EOL) which means the Drupal community and security team will no longer officially support active development, security and bug fixing for the platform. We instead must focus our resources on maintaining Drupal 7 and 8.

Of course, since Drupal is open source software, even though EOL for Drupal 6 is today, some organizations will continue to operate their live websites on Drupal 6 for some time. Upgrading to a current, supported version of Drupal is recommended, but whether Drupal 7 or Drupal 8 is the best fit for your needs, is conditional. Drupal 8 provides enhanced multi-lingual capabilities, responsiveness out-of-the-box, improved UX with WYSIWYG out-of-the-box, in-place content editing, improved UX AND DX (developer experience) with staging and deployment improvements to name just a few advantages over Drupal 7. However, there are cases in which Drupal 8 may not be able to satisfy your organization’s needs cost-effectively in the short-term, before the community has matured around it. You may liken this comparison to deliberating on your next car purchase: a Tesla vs. the Honda Civic you’re accustomed to. The Tesla promises new and shiny features, but you may have to adjust to things like refueling with electricity, not gas, and automatic steering. With the Civic, you know you’re getting a reliable, solid machine that will get the job done, though it may not turn heads. Drupal 7 is the Civic; Drupal 8, the Tesla.

The if it ain’t broke don’t fix it perspective is valid as the enterprise-level websites that put Drupal on the map such as Whitehouse.gov and Weather.com are Drupal 7 sites.

Having said that, we’ll share the main factors to weigh in deciding which platform makes the most sense for your organization.

Top considerations in selecting Drupal 7 or 8

How familiar with OO PHP is the development team?

The mantra of Drupal 8 is getting off the island which means leveraging well-vetted systems and processes that already exist in the broader PHP and web development community. This is in contrast to the preceding Drupal approach which relied on custom, esoteric design decisions familiar only to the Drupal community, contributing to the steep learning curve for Drupal development. To that end, Drupal 8 was rewritten in object oriented PHP, a major architectural change. Although this point may be a bit Greek to non-technical site owners it is important to know how versed the team working on the upgrade is in object oriented programming, and is worth asking when vetting prospective development teams.

How complex is the site?

One of Drupal’s greatest strengths is its extensibility, and robust design to facilitate writing and maintaining custom code for specific client needs. Relatedly, the very large contributed module repository (33,289 as I write this) enables developers a wide array of “plug-and-play” functionality for free which enables fairly complex sites at a relatively low cost.

If your site has fairly complex functionality, it is likely that the development team(s) who have worked on it have installed many contributed modules, and/or written custom modules themselves.

A quick peek into your Drupal website files should shed some light on this:

 /sites/all/modules

Ideally previous developers would have had the foresight to distinguish directories between community contributed modules and custom modules written by them with the by implementing this simple directory structure:

 /sites/all/modules/contrib/
 /sites/all/modules/custom/

E.g. Tilthy Rich Compost

The fewer modules in those directories, the more you should lean toward Drupal 8 as the stronger candidate. If there are many, which there often are, the decision gets more complicated. You’ll likely need an estimate from a development team on the custom module functionality and you’ll want to consider if the previous customization you’ve requested is still relevant. In general, upgrading is a good time to trim the fat and focus on true organizational needs to simplify the process and limit future costs.

As far as contributed modules go, the reality is in the early months of Drupal 8 the quality and coverage of contributed modules will lag Drupal 7. The good news is there are public resources that can help you understand the status and availability of each module you require for your site which will help you make the determination of Drupal 8’s readiness for your needs.

Here are a few

In some cases a mission critical contributed module will not have an official Drupal 8 release, which may force the decision for Drupal 7 for budget reasons. We recently bid on a project that we recommended Drupal 7 since the project heavily relied on the Salesforce suite which Kosta has played a role in the development of.

The shibboleth authentication module is another module that some universities rely on, which has no official Drupal 8 path at the moment partially due to how complex the module is.

A final important point is that some of the most popular contributed Drupal 7 modules have been folded into Drupal 8 core. Therefore, a much higher threshold of sophistication can be achieved by out-of-the-box functionality with Drupal 8 than was previously possible with Drupal 7

What is my timeline to upgrade?

If you take the security concerns of unsupported software seriously, you want to upgrade as soon as possible. However, sometimes website upgrades are more beholden to internal budgeting than security threats. A general rule of thumb is the more time that goes by from today, the more you should lean toward Drupal 8. The contributed module community will continue to improve in the weeks, and months to come, and you’ll want to take advantage of the more mature and modern platform that is Drupal 8. Don’t forget, Drupal 7 was released in early 2011 so its foundation is over 5 years old as I write this.

What are my future website goals and budget?

The less your organization will budget for future site improvements, the stronger the pull for Drupal 8 to ensure your maximizing the time before you have to read an article like this again. The truth is, by the time we’re honestly talking about Drupal 10 (that which would phase out Drupal 8 if things continued as they have thus far) the upgrade path for a Drupal site, and the web in general will look very different. Predicting the web 5-7 years ahead of time is a tall task. Having said that, the total cost of ownership, and specifically upgrade burden for Drupal projects is something the community is looking at closely. A Drupal 6 to Drupal 8 upgrade path does exist and a lot of thought has been given to simplifying the release cycle which should ease the upgrade process as well. If your typical redesign cycle is within 2 to 4 years, you’re likely safe with a Drupal 7 site for at least that long.

Should I even be using Drupal at all?

A worthy partner should be knowledgeable enough to know when Drupal is not the right fit, and honest enough to share that information for you. We pride ourselves on being one of those. As broadly applicable as the Drupal platform is for modern web projects, it isn’t always the right choice. It’s worth asking the question of prospective teams if you should even be on Drupal.

Where does that leave me?

Ultimately, site owners should communicate clearly their needs and desires for the new and improved vision of their website, and lean on their web partner to guide them in the right direction with open and honest conversations. On new site builds leading up to and after the official release of Drupal 8.0.0 we have deferred the decision to the tail-end of our discovery phase. We have found making the most informed decision requires thorough analysis which takes time. I encourage you to take some time with your partners weighing the pros and cons.

Part 1 - overview | Part 2 - the risks | Part 3 - the options | Part 4 - Drupal 7 or Drupal 8?

Feb 18 2016
Feb 18

Drupal 8 logo

The recommended approach to getting started with Drupal 8 is now via Composer. An official project template has been created for this. We will create our project directly using the template, which is also available on Packagist.

To create a new project based on this template we can run the following Composer command:

composer create-project drupal-composer/drupal-project:8.x-dev my_project --stability dev --no-interaction

This Composer command will pull in the template from Packagist and run a few Drupal specific scripts to prepare our project for installation. The only thing left to do is point our browser to the web/ directory (since that is where the index.php file is) and run the installer as usual.

This template comes with a /web folder that contains, among other things, the main folders of a Drupal installation that are no longer considered part of Drupal core (such as the index.php file or the modules and themes folders). Additionally, it comes with an autoload.php file used by Drupal that simply points to the Composer vendor/ directory, one folder up. So all PHP libraries are now handled from one single place.

The template’s composer.json file requires the latest stable Drupal core + some additional helper tools such as Drush and the Drupal Console. Additionally, it adds the Drupal specific Packagist repository from where we can install Drupal contributed modules, themes and profiles (that get automatically installed in the right place).

If we want to add a Drupal contributed module, we need to find it on the Drupal Packagist and require it in our project via Composer:

composer require drupal/ctools

This will add the Ctools module directly to our web/modules/ directory and update our composer.json file.

The project template also comes with a .gitignore file that keeps Drupal core and all the contributed packages outside of Git, similar to the regular vendor/ packages. So based on an updated composer.json file, we can maintain a smaller Git repository and recreate our project any time. A lot of the benefits of Drush Make have now been incorporated into a Composer flow.

Conclusion

Drupal 8 has come a long way in catching up with other major PHP software. The possibility of fully managing it via Composer, either as a main project or even just as part of a bigger set of applications, is a testament to the community effort that went in.

Daniel Sipos

Meet the author

Daniel Sipos is a Drupal developer who lives in Brussels, Belgium. He works professionally with Drupal but likes to use other PHP frameworks and technologies as well. He runs webomelette.com, a Drupal blog where he writes articles and tutorials about Drupal development, theming and site building.
Feb 03 2016
Feb 03

As of 1:23pm GMT on Febuary 3rd 2016 there are no dependencies in the Drupal 8.1.x git branch.

Why?

There was no reason to have them there. It makes the repository bigger, and therefore takes longer to download. No one else does it.

What does it mean?

Not much really.

If you download Drupal via the zip file or tarball then the drupal.org packager will run composer so the dependencies will be there for you.

If you submit a Drupal core patch on drupal.org then DrupalCi testbot will run composer install.

It’s only if you clone Drupal 8.1.x or higher directly from git, you will need to run composer install in the Drupal root directory

Top tip: composer install --prefer-source --no-interaction can often be quicker.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Jan 25 2016
Jan 25

This is part 3 of a series investigating what to do with your Drupal 6 site as EOL approaches.

Part 1 - overview | Part 2 - the risks | Part 3 - the options | Part 4 - Drupal 7 or 8?

The options of operating Drupal 6 after EOL

Though some advisers (solicited or not) might tell you otherwise, you always have options as to what to do with your website if you’re running Drupal. One of them is of course nothing, and the others require more thought.

What is the right thing to do can be a hot-button topic. Some technologists are purists, and believe it sacrosanct to even consider running software beyond EOL. However, we appreciate that although ideally site owners would always have been prepared and budgeted for software upgrades, sometimes life happens, and other business priorities and budget availability don’t perfectly align with timing for a software upgrade.

Ben Affleck in the Hollywood movie Armageddon

One end of the spectrum is well captured by the following quotation from the Drupal security team that some (certainly not us :wink: alarmist reaction to one of the most prolific Drupal security vulnerabilities to-date colloquially referred to as “Drupalgeddon.”

You should proceed under the assumption that every Drupal 7 website was compromised unless updated or patched before Oct 15th, 11pm UTC, that is 7 hours after the announcement.

This is of course meant to err on the side of caution, but can also be very anxiety provoking. Similarly, when hearing of the 3 month window for Drupal 6 support after Drupal 8.0.0 was released “Le dendrite” is not wrong to say:

someone please settle my stomach

I was shocked to read this. it might seriously wreck me, i really hope i’m misunderstanding this

will my sites just die?

That entire thread is a pretty good exposé into the various perspectives and preparedness on upgrading Drupal 6 sites. If you’ve got a spare half-day, give it a once-over.

On the other hand, we have had at least two clients in 2015 who continued to run Drupal 5 (not a typo) websites in production successfully.

Neither end is entirely wrong, they just represent difference of opinion and available options.

The primary options for Drupal 6 site owners are

  1. Do not upgrade your website to a newer Drupal version
  2. Do upgrade your website to a newer Drupal version
  3. Use the time to seriously revision your website altogether

But how do you decide which option is best for you, the site owner?

That decision mostly rests on your organization’s

  • risk tolerance
  • likelihood to be targeted by hackers
  • budget to improve your website

Considerations to not upgrade

Choosing not to upgrade your website is typically dictated by your organization having neither internal technical expertise or sufficient funds to hire a partner. When this is the case, if a small budget can be afforded, a site audit is usually the next best option to have some peace of mind that you’re taking the measures you’re able to mitigate the risk of running outdated, unsupported software. A qualified partner can perform a thorough assessment of risk and make budget-conscious recommendations on how to best harden the site against outside threats short of doing the more costly full-site upgrade.

Primary attack vectors that we focus on lie in custom code, and under-supported contributed modules since core code represents much lower risk having been well vetted by years in the wild.

wolves in the wild

One other consideration independent of your specific website configuration is your server autonomy dictated by your hosting provider. Running old software is usually tolerated in a shared hosting environment for only so long before upgrades are enforced. Warning emails are often (not always) sent out in advance of server-level upgrades, however, it’s easy to miss those emails, and it hurts to be caught by surprise that some PHP version upgrade renders your site completely useless. Understanding the level of control you as the site owner has over your server is crucial.

Considerations to upgrade

Going through a typical Drupal upgrade (the update.php method) is usually the right option for you if you

  • Want to mostly preserve the site to be upgraded as-is.
  • Have a fairly straightforward site using few, and mainly popular contributed modules .
  • Decide you ought to be using supported software because you
    • have sensitive information within your site that would be disastrous if compromised.
    • have financial transactions or information housed within your site.
  • Have sufficient budget to do the work.

Upgrading a Drupal website is a non-trivial investment. We’ve definitely had conversations with clients to the tune of

“oh, upgrading is not a click of a button? Phooey!”

which is very understandable. The Drupal community’s stance of not retaining backwards compatibility for the sake of keeping pace with modern web development is solid and quite defensible as well. Acquia does a good job of addressing that in the “Support” section (unfortunately not directly linkable) in this informative article about total cost of ownership of a Drupal site, which is in itself a topic worth a lot of consideration and analysis.

When prospective clients are committed to the time-line and investment of an upgrade, we always advise them to seriously consider a comprehensive rebuild/redesign as it can often present the path of most value. It is uncommon that a client desires to invest significantly to create an exact replica of the site they built 3, 4, or 5 years ago.

In many cases, legacy content, functionality and style has outlived it’s relevancy after 3 years and it becomes more burdensome than valuable, in the same way a poorly maintained house on a nice lot may be less valuable than if the lot were empty.

Another consideration is whether or not Drupal is the right platform for your organization’s needs. The answer to this is usually yes, especially after there is organizational investment and familiarity, but for less complicated sites that are open to revisioning, there are other options that, for example, are free to host and have no upgrade requirements like the static site generator Jekyll from which our own web site is built. An honest consultant will tell you when Drupal is a good fit and when it’s not.

But alas, if you’re determined to stick with Drupal and you’d like to upgrade, the final consideration is what platform to upgrade to, Drupal 7 or 8?

Ah-ah-ah…

Dennis Nedry from Jurassic Park saying ah ah ah

that’s for the next episode!

Part 4 and beyond

Next, we’ll discuss Drupal 7 vs. Drupal 8 decision-making (or not Drupal at all!).

Part 1 - overview | Part 2 - the risks | Part 3 - the options | Part 4 - Drupal 7 or 8?

Jan 22 2016
Jan 22

Drupal 8 is out, Drupal 8.1 will be out before we know it, but it seems contrib is still catching up. One question that seems to keep coming up is around installing a Drupal 8 module. In this post I will look at the different ways to install a module and resolving dependencies.

Deploy, Search API Solr Search, and Commerce are just a few of the modules with Drupal 8 releases that require dependencies loaded via composer. This means you can’t just download the module’s ZIP file, unzip it in your modules directory, and enable it. You need to install the dependencies.

One simple way to do this is the Composer Manager module. This module has extensive documentation on how to use it. Essentially what it does it merge the composer.json that ships with core and the composer.json files from all the contrib module you have, then downloads the dependencies. You may notice that this will also update core dependencies, but this is a good thing! The core composer.json has been written in such a way that it won’t introduce API breaking dependencies and only uses stable releases. So you will benefit from any bug fixes or security fixes rolled out in these dependencies before they’re rolled out with Drupal.

The two other ways we’re going to look at involve directly using Composer.

Just the dependencies
It’s possible to carry on installing Drupal modules exactly as you always have, download the zip or tarball, then unzip it into your modules directory. As mentioned earlier this will not install the dependencies, therefore you will need to look inside the module’s composer.json file, see what the dependencies are, and install them manually. Let’s take Deploy module as an example, this depends on the dev-master version of relaxedws/replicator. So, go to your Drupal docroot and run the command composer require relaxedws/replicator:dev-master. This will add relaxedws/replicator to Drupal’s composer.json, download it, and put it in the vendor directory ready for the module to make use of. This will not change any other other dependencies you have. Then to update the dependencies you can either run composer update to update relaxedws/replicator and all core dependencies or composer update relaxedws/replicator to just update the relaxedws/replicator package.

The module too
If you install all you Drupal modules via composer, all of the dependencies will automatically be installed too. First you will need to add a new repository, so run composer config repositories.drupal composer https://packagist.drupal-composer.org in your Drupal docroot, this will add the https://packagist.drupal-composer.org repository to your drupal composer.json. Now you can install any module from Drupal.org via composer. So going back to the example of Deploy you can run composer require drupal/deploy:8.1.0-alpha5 in your Drupal docroot and it will install Deploy in the modules directory. It will also install key_value, multiversion, and relaxed, which are all Drupal modules required by Deploy. Furthermore it will install relaxedws/replicator as we know is a PHP package needed for Deploy, and doctrine/couchdb which is a PHP package needed for releaxedws/replicator.

I hope this helps those confused what to do with Drupal 8 modules that have composer dependencies. Now go do the smart thing, rebuild your site using Composer!

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by
Jan 19 2016
Jan 19

In an earlier tutorial, we looked at the Drupal 8 plugin system and how to create our very own custom plugin type. We’ve seen that much of the functionality declared via _info hooks in Drupal 7 has been replaced by these plugins. Our use case was very basic and it allowed each instance of such functionality to be declared manually via a new plugin class and associated form.

Drupal 8 logo

But what if we needed such instances declared dynamically depending on some factors external to our little subsystem? For example, when declaring _info hooks in Drupal 7, we can get a list of something, loop over it and declare a new item in the returned array for each individual something. The menu system does this in order to provide a new block for each menu that either comes with Drupal core or is later created through the UI.

So what about Drupal 8? We’ve seen that for each plugin of a certain type we need to declare a different PHP class. To create a new block, we need a new class. To create another block, we need another class. So where would that looping we see in Drupal 7 take place? The short answer to this is: within a plugin derivative.

In this article we will explore the long answer to that and learn what derivates are and how we can use them. For the latter, we will build an example inside the demo module that can be found in this git repository and which should hopefully help us better understand what’s going on. For a slightly more complex example, the Menu system is great as it provides an individual block for each of its menus (similar to Drupal 7 but using plugins).

What we are going to do is actually very simple. We are going to implement basic Node Block functionality by which for all the article nodes on our site we will have a block. Ridiculous? Sure. Should we be doing this for all the nodes on our site? Definitely not! But it’s a very basic implementation meant to keep things short and demonstrate the use of the plugin derivatives.

Plugin Derivatives

Plugin derivatives are the way through which a plugin of a certain type can be represented in the system as multiple instances of itself. In other words, a plugin can reference a deriver class which is responsible for providing a list of plugin definitions that are based on the initial plugin (start from the same base definition) but have slightly different configuration or definition data. The SystemMenuBlock we referred to above is a great example. It’s a single plugin which has as many derivatives as there are menus on the site.

To go a bit deeper, when a list of all the plugins of a certain type is requested, the plugin manager uses its discovery mechanism to load all the plugins of this type. If that mechanism is decorated with the DerivativeDiscoveryDecorator, the manager will be able to also retrieve derivatives. In order to do this, the derivative discovery looks for a deriver class on each plugin and, if it finds one, asks it for this list.

Plugin type managers that extend the DefaultPluginManager base class should normally have the derivative discovery mechanism decorating the default discovery (annotations). This is the most common pattern in the Drupal core plugin system: annotated discovery wrapped by derivatives.

The Derivative Class

Now that we know what the role of plugin derivatives is, let’s create our first deriver class that will be used by our block plugin (which we will create in a minute).

Inside src/Plugin/Derivative/NodeBlock.php of the demo module we have the following:

<?php

/**
 * @file
 * Contains \Drupal\demo\Plugin\Derivative\NodeBlock.
 */

namespace Drupal\demo\Plugin\Derivative;

use Drupal\Component\Plugin\Derivative\DeriverBase;
use Drupal\Core\Entity\EntityStorageInterface;
use Drupal\Core\Plugin\Discovery\ContainerDeriverInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;

/**
 * Provides block plugin definitions for nodes.
 *
 * @see \Drupal\demo\Plugin\Block\NodeBlock
 */
class NodeBlock extends DeriverBase implements ContainerDeriverInterface {

  /**
   * The node storage.
   *
   * @var \Drupal\Core\Entity\EntityStorageInterface
   */
  protected $nodeStorage;

  /**
   * Constructs new NodeBlock.
   *
   * @param \Drupal\Core\Entity\EntityStorageInterface $node_storage
   *   The node storage.
   */
  public function __construct(EntityStorageInterface $node_storage) {
    $this->nodeStorage = $node_storage;
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container, $base_plugin_id) {
    return new static(
      $container->get('entity.manager')->getStorage('node')
    );
  }

  /**
   * {@inheritdoc}
   */
  public function getDerivativeDefinitions($base_plugin_definition) {
    $nodes = $this->nodeStorage->loadByProperties(['type' => 'article']);
    foreach ($nodes as $node) {
      $this->derivatives[$node->id()] = $base_plugin_definition;
      $this->derivatives[$node->id()]['admin_label'] = t('Node block: ') . $node->label();
    }
    return $this->derivatives;
  }
}

All our class needs to implement is the DeriverInterface and implement its two methods. We use the ContainerDeriverInterface instead because we want to make our deriver container aware. Why? Because we use dependency injection to load Drupal’s entity manager so that we can access the Node storage (this is what the constructor and the create() method do). Additionally, our deriver class extends from the DeriverBase class because that already takes care of one of the required methods (getDerivativeDefinition()).

Finally, getDerivativeDefinitions() is the method responsible for providing an array of plugin definitions that derive from the plugin which uses this class. It receives the $base_plugin_definition as an argument (the definition of the actual plugin which uses this deriver) and we use that to build up our derivative definitions. In our case, we indiscriminately load all the Article nodes and, for each of them, create a separate definition which differs only by having a different admin_label (this is a property on the Drupal\Core\Block\Annotation\Block annotation class). The array of derivatives is keyed by the ID of the derivative (in our case the Node ID which we will use later).

A very important point we need to make here is that loading all the nodes and creating plugins out of them is never a good idea. What would be maybe interesting is to implement functionality by which individual nodes can be exposed as blocks via a checkbox or something like that.

The Block Plugin

Now that we have our deriver class, let’s create a simple block plugin that uses it to generate multiple instances of itself (one for each Article node).

Inside src/Plugin/Block/NodeBlock.php:

<?php

namespace Drupal\demo\Plugin\Block;

use Drupal\Core\Block\BlockBase;
use Drupal\Core\Entity\EntityManagerInterface;
use Drupal\Core\Entity\EntityViewBuilderInterface;
use Drupal\Core\Plugin\ContainerFactoryPluginInterface;
use Drupal\Core\Session\AccountInterface;
use Drupal\node\NodeInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;

/**
 * Provides a 'NodeBlock' block plugin.
 *
 * @Block(
 *   id = "node_block",
 *   admin_label = @Translation("Node block"),
 *   deriver = "Drupal\demo\Plugin\Derivative\NodeBlock"
 * )
 */

class NodeBlock extends BlockBase implements ContainerFactoryPluginInterface {

  /**
   * @var EntityViewBuilderInterface.
   */
  private $viewBuilder;

  /**
   * @var NodeInterface.
   */
  private $node;

  /**
   * Creates a NodeBlock instance.
   *
   * @param array $configuration
   * @param string $plugin_id
   * @param array $plugin_definition
   * @param EntityManagerInterface $entity_manager
   */
  public function __construct(array $configuration, $plugin_id, $plugin_definition, EntityManagerInterface $entity_manager) {
    parent::__construct($configuration, $plugin_id, $plugin_definition);
    $this->viewBuilder = $entity_manager->getViewBuilder('node');
    $this->nodeStorage = $entity_manager->getStorage('node');
    $this->node = $entity_manager->getStorage('node')->load($this->getDerivativeId());
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition) {
    return new static(
      $configuration,
      $plugin_id,
      $plugin_definition,
      $container->get('entity.manager')
    );
  }
  
  /**
   * {@inheritdoc}
   */
  public function build() {
    if (!$this->node instanceof NodeInterface) {
      return;
    }
    $build = $this->viewBuilder->view($this->node, 'full');
    return $build;
  }
  
  /**
   * {@inheritdoc}
   */
  public function blockAccess(AccountInterface $account, $return_as_object = FALSE) {
    return $this->node->access('view', NULL, TRUE);
  }
}

The first thing we notice in this plugin’s annotation is the deriver key which points to the class we created before. And that is basically all we need to couple the two. The derivative discovery decorator handles the heavy lifting.

Much of the rest is basic block building we should be familiar with. What’s interesting is that we can use the getDerivativeId() method to retrieve the node ID we used also as the ID of the derivative being displayed and, using that, we load the node object and build the block as the actual node output. Lastly, inside the blockAccess() method we make sure that this block has the same access checks as the actual node itself. So if the current user doesn’t have access to view the current node, the block won’t even show up.

Now if we clear the caches and navigate to the Block Layout interface we should see some blocks called Node Block: [Node title]. You can place these where you want and they will render the relevant node.

Conclusion

In this article, we’ve looked at plugin derivatives and seen a simple example of how they work. The key take away on this topic is that plugin derivatives are the way we dynamically declare multiple instances of the same plugin. They usually help us transform user configured functionality (e.g. menus) into plugins (e.g. menu blocks).

To illustrate the use of derivatives, we’ve seen a very simple technique which allows us to render Article nodes as blocks. We should remember though not to try this out on a website with many Article nodes but rather implement additional functionality that limits the number of nodes that get exposed. You know, so we don’t crash our site.

Questions? Comments? Anything you’d like explained further? Let us know!

Daniel Sipos

Meet the author

Daniel Sipos is a Drupal developer who lives in Brussels, Belgium. He works professionally with Drupal but likes to use other PHP frameworks and technologies as well. He runs webomelette.com, a Drupal blog where he writes articles and tutorials about Drupal development, theming and site building.
Dec 11 2015
Dec 11

Drupal 8 logo

The Queue API in Drupal allows us to handle a number of tasks at a later stage. What this means is that we can place items into a queue which will run some time in the future and process each individual item at that point and at least once. Usually, this happens on CRON runs, and Drupal 8 allows for a quick set up for cronjob based queues. It doesn’t necessarily have to be CRON, however.

In this article, we will look at using the Queue API in Drupal 8 by exploring two simple examples. The first will see the queue triggered by Cron while the second will allow us to manually do so ourselves. However, the actual processing will be handled by a similar worker. If you want to follow along, clone this git repository where you can find the npq module we will write in this article.

The module we’ll work with is called Node Publisher Queue and it automatically adds newly created nodes that are saved unpublished to a queue to be published later on. We will see how later on can be the next CRON run or a manual action triggered by the site’s administrator. First, let’s understand some basic concepts about queues in Drupal 8.

The theory

There are a few components that make up the Queue API in Drupal 8.

The most important role in this API is played by the QueueInterface implementation which represents the queue. The default queue type Drupal 8 ships with is currently the DatabaseQueue which is a type of reliable queue that makes sure all its items are processed at least once and in their original order (FIFO). This is in contrast to unreliable queues which only do their best to achieve this (something for which valid use cases do exist).

The typical role of the queue object is to create items, later claim them from the queue and delete them when they have been processed. In addition, it can release items if processing is either not finished or another worker needs to process them again before deletion.

The QueueInterface implementation is instantiated with the help of a general QueueFactory. In the case of the DatabaseQueue, the former uses the DatabaseQueueFactory as well. Queues also need to be created before they can be used. However, the DatabaseQueue is already created when Drupal is first installed so no additional setup is required.

The Queue Workers are responsible for processing queue items as they receive them. In Drupal 8 these are QueueWorker plugins that implement the QueueWorkerInterface. Using the QueueWorkerManager, we create instances of these plugins and process the items whenever the queue needs to be run.

The Node Publish Queue module

Now that we’ve covered the basic concepts of the Queue API in Drupal 8, let’s get our hands dirty and create the functionality described in the introduction. Our npq.info.yml file can be simple:

name: Node Publish Queue
description: Demo module illustrating the Queue API in Drupal 8
core: 8.x
type: module

Queue item creation

Inside the npq.module file we take care of the logic for creating queue items whenever a node is saved and not published:

use Drupal\Core\Entity\EntityInterface;
use Drupal\Core\Queue\QueueFactory;
use Drupal\Core\Queue\QueueInterface;

/**
 * Implements hook_entity_insert().
 */
function npq_entity_insert(EntityInterface $entity) {
  if ($entity->getEntityTypeId() !== 'node') {
    return;
  }

  if ($entity->isPublished()) {
    return;
  }

  /** @var QueueFactory $queue_factory */
  $queue_factory = \Drupal::service('queue');
  /** @var QueueInterface $queue */
  $queue = $queue_factory->get('cron_node_publisher');
  $item = new \stdClass();
  $item->nid = $entity->id();
  $queue->createItem($item);
}

Inside this basic hook_entity_insert() implementation we do a very simple task. We first retrieve the QueueFactoryInterface object from the service container and use it to get a queue called cron_node_publisher. If we track things down, we notice that the get() method on the DatabaseQueueFactory simply creates a new DatabaseQueue instance with the name we pass to it.

Lastly, we create a small PHP object containing the node ID and create an item in the queue with that data. Simple.

The CRON queue worker

Next, let’s create a QueueWorker plugin that will process the queue items whenever Cron is run. However, because we know that we will also need one for manual processing that does the same thing, we will add most of the logic in a base abstract class. So inside the Plugin/QueueWorker namespace of our module we can have the NodePublishBase class:

/**
 * @file
 * Contains Drupal\npq\Plugin\QueueWorker\NodePublishBase.php
 */

namespace Drupal\npq\Plugin\QueueWorker;

use Drupal\Core\Entity\EntityStorageInterface;
use Drupal\Core\Plugin\ContainerFactoryPluginInterface;
use Drupal\Core\Queue\QueueWorkerBase;
use Drupal\node\NodeInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;


/**
 * Provides base functionality for the NodePublish Queue Workers.
 */
abstract class NodePublishBase extends QueueWorkerBase implements ContainerFactoryPluginInterface {

  /**
   * The node storage.
   *
   * @var \Drupal\Core\Entity\EntityStorageInterface
   */
  protected $nodeStorage;

  /**
   * Creates a new NodePublishBase object.
   *
   * @param \Drupal\Core\Entity\EntityStorageInterface $node_storage
   *   The node storage.
   */
  public function __construct(EntityStorageInterface $node_storage) {
    $this->nodeStorage = $node_storage;
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition) {
    return new static(
      $container->get('entity.manager')->getStorage('node')
    );
  }

  /**
   * Publishes a node.
   *
   * @param NodeInterface $node
   * @return int
   */
  protected function publishNode($node) {
    $node->setPublished(TRUE);
    return $node->save();
  }

  /**
   * {@inheritdoc}
   */
  public function processItem($data) {
    /** @var NodeInterface $node */
    $node = $this->nodeStorage->load($data->nid);
    if (!$node->isPublished() && $node instanceof NodeInterface) {
      return $this->publishNode($node);
    }
  }
}

Right off the bat we can see that we are using dependency injection to inject the NodeStorage into our class. For more information about dependency injection and the service container, feel free to check out my article on the topic.

In this base class we have two methods: publishNode() and the obligatory processItem(). The former publishes and saves a node that is passed to it. The latter loads the node using the node ID contained in the $data object and publishes it if it’s unpublished.

Now, let’s create a CronNodePublisher plugin that will use this logic on Cron runs:

namespace Drupal\npq\Plugin\QueueWorker;

/**
 * A Node Publisher that publishes nodes on CRON run.
 *
 * @QueueWorker(
 *   id = "cron_node_publisher",
 *   title = @Translation("Cron Node Publisher"),
 *   cron = {"time" = 10}
 * )
 */
class CronNodePublisher extends NodePublishBase {}

And that is all. We don’t need any other logic than what already is in our base class. Notice that, in the annotation, we are telling Drupal that this worker needs to be used by Cron to process as many items as it can within 10 seconds. How does this happen?

Whenever Cron runs, it uses the QueueWorkerManager to load all its plugin definitions. Then, if any of them have the cron key in their annotation, a Queue with the same name as the ID of the worker is loaded for processing. Lastly, each item in the queue is claimed and processed by the worker until the specified time has elapsed.

If we now save an unpublished node, it will most likely become published at the next Cron run.

The manual worker

Let’s create also the possibility for the Queue to be processed manually. First, let’s adapt the hook_entity_insert() implementation from before and change this line:

$queue = $queue_factory->get('cron_node_publisher');

to this:

$queue = $queue_factory->get('manual_node_publisher');

You can of course provide an admin screen for configuring which type of node publisher the application should use.

Second, let’s create our ManualNodePublisher plugin:

namespace Drupal\npq\Plugin\QueueWorker;

/**
 * A Node Publisher that publishes nodes via a manual action triggered by an admin.
 *
 * @QueueWorker(
 *   id = "manual_node_publisher",
 *   title = @Translation("Manual Node Publisher"),
 * )
 */
class ManualNodePublisher extends NodePublishBase {}

This is almost the same as with the CRON example but without the cron key.

Third, let’s create a form where we can see how many items are in the manual_node_publisher queue and process them all by the press of a button. Inside npq.routing.yml in the module root folder:

demo.form:
  path: '/npq'
  defaults:
    _form: '\Drupal\npq\Form\NodePublisherQueueForm'
    _title: 'Node Publisher'
  requirements:
    _permission: 'administer site configuration'

We define a path at /npq which should use the specified form that lives in that namespace and that we can define as such:

/**
 * @file
 * Contains \Drupal\npq\Form\NodePublisherQueueForm.
 */

namespace Drupal\npq\Form;

use Drupal\Core\Form\FormBase;
use Drupal\Core\Form\FormStateInterface;
use Drupal\Core\Queue\QueueFactory;
use Drupal\Core\Queue\QueueInterface;
use Drupal\Core\Queue\QueueWorkerInterface;
use Drupal\Core\Queue\QueueWorkerManagerInterface;
use Drupal\Core\Queue\SuspendQueueException;
use Symfony\Component\DependencyInjection\ContainerInterface;

class NodePublisherQueueForm extends FormBase {

  /**
   * @var QueueFactory
   */
  protected $queueFactory;

  /**
   * @var QueueWorkerManagerInterface
   */
  protected $queueManager;


  /**
   * {@inheritdoc}
   */
  public function __construct(QueueFactory $queue, QueueWorkerManagerInterface $queue_manager) {
    $this->queueFactory = $queue;
    $this->queueManager = $queue_manager;
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container) {
    return new static(
      $container->get('queue'),
      $container->get('plugin.manager.queue_worker')
    );
  }
  
  /**
   * {@inheritdoc}.
   */
  public function getFormId() {
    return 'demo_form';
  }
  
  /**
   * {@inheritdoc}.
   */
  public function buildForm(array $form, FormStateInterface $form_state) {
    /** @var QueueInterface $queue */
    $queue = $this->queueFactory->get('node_publisher');

    $form['help'] = array(
      '#type' => 'markup',
      '#markup' => $this->t('Submitting this form will process the Manual Queue which contains @number items.', array('@number' => $queue->numberOfItems())),
    );
    $form['actions']['#type'] = 'actions';
    $form['actions']['submit'] = array(
      '#type' => 'submit',
      '#value' => $this->t('Process queue'),
      '#button_type' => 'primary',
    );
    
    return $form;
  }
  
  /**
   * {@inheritdoc}
   */
  public function submitForm(array &$form, FormStateInterface $form_state) {
    /** @var QueueInterface $queue */
    $queue = $this->queueFactory->get('manual_node_publisher');
    /** @var QueueWorkerInterface $queue_worker */
    $queue_worker = $this->queueManager->createInstance('manual_node_publisher');

    while($item = $queue->claimItem()) {
      try {
        $queue_worker->processItem($item->data);
        $queue->deleteItem($item);
      }
      catch (SuspendQueueException $e) {
        $queue->releaseItem($item);
        break;
      }
      catch (\Exception $e) {
        watchdog_exception('npq', $e);
      }
    }
  }
}


We are again using dependency injection to inject the QueueFactory and the manager for QueueWorker plugins. Inside buildForm() we are creating a basic form structure and using the numberOfItems() method on the queue to tell the user how many items they are about to process. And finally, inside the submitForm() method we take care of the processing. But how do we do that?

First, we load the Queue and instantiate a Queue worker (in both cases we use the manual_node_publisher id). Then we run a while loop until all the items have been processed. The claimItem() method is responsible for blocking a queue item from being claimed by another queue and returning it for processing. After it gets processed by the worker, we delete it. In the next iteration, the next item is returned and on like this until no items are left.

Although we have not used it, the SuspendQueueException is meant to indicate that during the processing of the item, the worker found a problem that would most likely make all other items in the queue fail as well. And for this reason it is pointless to continue to the next item so we break out of the loop. However, we also release the item so that when we try again later, the item is available. Other exceptions are also caught and logged to the watchdog.

Now if we create a couple of nodes and don’t publish them, we’ll see their count inside the message if we navigate to /npq. By clicking the submit button we process (publish) them all one by one.

This has been a demonstration example only. It’s always important to take into account the potential load of processing a large number of items and either limit that so your request doesn’t time out or use the Batch API to split them into multiple requests.

Conclusion

In this article we’ve looked at the Queue API in Drupal 8. We’ve learned some basic concepts about how it is built and how it works, but we’ve also seen some examples of how we can work with it. Namely, we’ve played with two use cases by which we can publish unpublished nodes either during Cron runs or manually via an action executed by the user.

Have you tried out the Queue API in Drupal 8? Let us know how it went!

Daniel Sipos

Meet the author

Daniel Sipos is a Drupal developer who lives in Brussels, Belgium. He works professionally with Drupal but likes to use other PHP frameworks and technologies as well. He runs webomelette.com, a Drupal blog where he writes articles and tutorials about Drupal development, theming and site building.
Dec 10 2015
Dec 10

This is part 2 of a series investigating what to do with your Drupal 6 site as EOL approaches.

Part 1 - overview | Part 2 - the risks | Part 3 - the options | Part 4 - Drupal 7 or 8?

The risks of operating Drupal 6 after EOL

risk

It is helpful to think of the risks of running publicly-accessible software, like a Drupal website, after end-of-life (EOL) from two distinct perspectives.

  1. Bad things that can happen to your organization due to known security vulnerabilities that may be exploited by the public.
    • These are the kinds of risks we typically think of when running outdated software. Hackers of the worst kind… the one’s that are after you!
  2. Good things that are less likely to happen (or even be possible) to your site when you run old software.
    • Though easy, it’s important not to underestimate this opportunity cost, as it is more subtle, often less pressing, harder to quantify, but ultimately often more costly than the former.

Bad things that can happen

hacker… Scary, huh? …

In the worst of scenarios, it is possible to experience

  • Complete site compromise and escalated server access due to a vulnerability exposed by website code.

Yes, it is possible that with outdated and insecure code, especially when the exploit is publicly disseminated, your site is exposed to complete control by a nefarious (or simply bored) hacker. The best and decreasingly recent example in the Drupal world is a security vulnerability (for sites running Drupal 7.31 and earlier and Drupal 8.0.0-beta2 and earlier) affectionately referred to as “Drupalgeddon” (yes, the world is coming to an end if your website is compromised). You can read until you’re blue in the face druplicon about this significant exploit, but suffice it to summarize that a visitor without an account on your site could completely control it with a sophisticated exploit of this vulnerability. Given the hacker will have access to nearly everything on the server, he may choose his means to best exploit your site. He may

  • Add spam marketing links throughout the site.
  • Defame the site if you’re a high-trafficked, reputable organization.
  • Use your server to send spam email.
  • Use any sensitive data especially financial he’s able to access to his own mean. It’s been done.

Unfortunately, we’ve had a client return to us to rehab a site that has undergone this vulnerability, suffering at least 2 of these un-pleasantries.

Having said that, given that Drupal 6 was released in February 2008, it is less likely (though not impossible) that such a large vulnerability still exists in the core code of the project. Higher risk vectors are contributed modules (i.e. plugins/extensions), especially those that haven’t been maintained, and certainly custom modules, which we’ve seen at Savas Labs vary widely in adhering to best security practices depending on the agency and/or developer who wrote it.

Therefore assessing risk factors for a Drupal 6 site as a whole is nuanced, and requires a comprehensive site audit.

More subtle risks

Outdated software tends to present unpredictable obstacles based on the fact that most of the world is no longer using the software. While your Drupal 6 site may be fairly static from a code standpoint, the world around it continues to adapt, update, and improve. This is the world of PHP, MySQL (the database software used by most Drupal sites, others are Post) and Apache (the web server used for most Drupal sites).

Hosting providers have an interest in keeping their servers and systems up to date for the same reasons we’re discussing here: security, performance, and functionality.

After we had finished a project with no maintenance contract with a former client they authorized routine server maintenance, and promptly rendered their site to the fabled White Screen of Death due to fatal errors caused by a PHP version update incompatibility with their code. Though we were happy to step in during the emergency, we appreciated how difficult it was to see coming for the client.

Much akin to the long-awaited Drupal 8 release, PHP 7 just came out and boasts, like Drupal, functionality and performance improvements. According to a d.o post Drupal 8 is 100% PHP 7 compatible.

Drupal 6 recommends a decidedly small range of PHP versions – between 5.25 and 5.3. This is potentially problematic for Drupal 6 sites since PHP 5.3 is itself over 6.5 years old. Hosting providers don’t like that, and as time goes on there will be upward pressure on the version number providers are willing to support.

Good things that can’t (are unlikely to) happen?

Given exploitative risks vary by installation and implementation of your Drupal 6 site, let’s look more definitively about what you won’t have access to with outdated software.

Your organization cannot reap the benefits of the modern web.

One example here is the concept of responsive web design. Pulling off a responsive design for Drupal 6 is very difficult. The concept was quite fresh when Drupal 7 was officially released (2011). Even though most Drupal 7 development took place before RWD, the community was able to evolve with it and provide solid offerings for responsive themes and frameworks in the Drupal 7 repository. Drupal 8 was built mobile-first and therefore is responsive out of the box. Another example would be the much improved, native content editing experience in Drupal 8. Content editing improved from Drupal 6 to 7 and made a leap from 7 to 8 with inline-editing and wysiwyg in core.

You expose yourself to a shrinking market of developers that are able to serve you.

small pool

Each of the last two major releases of Drupal entail significant architectural shifts from the former. Requisite skills in Drupal 6 for example look fairly different than those for Drupal 7. Even greater disparities exist when comparing the object-oriented Drupal 8 to the mostly procedural Drupal 6. We as developers like to build with what’s new, and most of the market does as well, so the longer you hold out past EOL the smaller the pool of accessible talent which could also drive up the cost to contract due to scarcity. Given the limited pool, you may have to settle for an inexperienced developer, which can result in poor code quality and the associated costs that come with it. Not least of which is the risk of increased technical debt that you may have to deal with down the line and has been estimated to cost $3-$5 per line of code written.

We have a client whose undocumented and non-standard code written by several former developers still provides the occasional production surprise and we’ve been working with it for 1.5 years. This has caused our client hundreds of hours of lost productivity spent debugging, commenting and improving legacy code rather than creating new functionality and features for marketing or other business needs.

The opportunity costs of using old software means more limited feature-set, and likely poorer performance.

A diminishing feature-set might mean you cannot easily or affordably access a hero image homepage carousel (all the rage these days) for example. Or it might mean you’re unable to provide an editorial workflow for content publication for different roles in your organization. Whatever the need may be, ultimately using EOL software means you will have access to what was popular while that software was in active development, which is likely many years prior. Upgrading to current and actively developed software means access to the web’s current needs.

And what about site speed; how important is that? According to many different sources, it’s VERY important.

Suffice it (once again) to say that site load speed is critical and says something about your organization’s credibility. One example of an advanced web development feature available in Drupal 8 is the impressive bigpipe project which leverages the power of a performance tool designed at Facebook to the betterment of the Drupal community! It’s a game-changer for caching and page responsiveness.

Your site looks/feels outdated.

Let’s face(lift) it: We can usually tell when a website is … aging. It’s somewhat unavoidable with a Drupal 6 site as it was likely designed many years ago using practices that were the norm then. Your website is your first and most important means to convey trust and credibility. Much of the research on users’ assessment of credibility from your website references the (ironically poorly designed) work of BJ Fogg of Stanford. Not surprisingly web design tops the list as most important credibility marker, and your Drupal 6 design and feature set is likely not going to cut it any longer.

Part 3 and beyond

Given we have described the risk of operating a Drupal 6 site after EOL, we’ll explore your options as dictated by your tolerance for risk, self-assessed attractiveness to exploit (are you a large retailer?), your budget to upgrade, your time line to upgrade, and other competing business priorities. In a follow up to that we’ll discuss Drupal 7 vs. Drupal 8 decision-making (or not Drupal at all!). If we make it to part 5, we’ll wrap it up with a bang!

Stay tuned

Part 1 - overview | Part 2 - the risks | Part 3 - the options | Part 4 - Drupal 7 or 8?

Dec 09 2015
Dec 09
DrupalCamp is the distribution, for spinning up camp sites.  Lately, as always :), I was part of the site building team at a local camp in India and after working with the team, found that all the sites were/are having similar content architecture e.g. content types, listing pages etc., except their themes and designs.
While sprinting at DrupalCamp Pune, we decided to build a site for DrupalCamp Delhi. While sprinting we realised we need  a common code/feature base for camp sites. And there we started building distribution in D8. The time we started making this distribution profile Drupal 8 was in beta 16 and we are hoping to make first release candidate soon.
Drupalcamp comes up with the 2 contributed modules fb_likebox, twitter_block for the facebook and twitter share blocks in sidebar.
Configurations that this installation profile provides are:
  1. Content types
    1. Basic Page :  For creating basic pages like about us,etc. on the site.
    2. Session : This is used for creating/submitting sessions.
    3. Sponsor : This is used to keep information about the sponsors.
  2. Listings (Views):
    1. Accepted Sessions
    2. Proposed Sessions
    3. Sponsors
    4. Students.
  3. Social sharing buttons:
    1. facebook
    2. twitter
  4. Blocks:
    1. facebook
    2. twitter

We need to just use the drupalcamp profile and theme the site.
Challenges that came up while building this profile :
  1. Contributed modules : The contributed modules stable release was not out so ported the fb_likebox, twitter_block module to drupal 8 and created the stable release of fb_likebox. Special thanks to the maintainer baekelandt for quick promptness.
  2. There was no tool available which will provide the boilerplate code, so we started with forking the standard profile and used the config-export to export the configurations. Now with the drupalconsole 0.9.9 we have generate profile command available to  generate the boilerplate code. Used the phing to automate the testing as sam specified in its blog.

Thanks to the entire Drupalcamp Distribution Team!
What's Next ?
Please join hands to release the stable version for twitter_block and include this stable release in the drupalcamp distribution.

Nov 26 2015
Nov 26

In this blog post we will take you though all the components required to provision a high availability Drupal 8 stack on Microsoft Azure. This is an extract from the demonstration given at Microsoft Ignite on the Gold Coast in November 2015.

What is Drupal?

Drupal is content management software. It's used to make many of the websites and applications you use every day. Drupal has great standard features, like easy content authoring, reliable performance, and excellent security. But what sets it apart is its flexibility; modularity is one of its core principles. Its tools help you build the versatile, structured content that dynamic web experiences need.

More information

aGov for Drupal 8

aGov is a totally free, open source Drupal CMS software distribution developed specifically for Australian Government organisations. Easy to install and configure, this pre-packaged Drupal CMS complies with all Australian Government standards and provides a full suite of essential website management features. It can be used out of the box for basic websites, or customised with any standard Drupal 8 module to deliver large scale, complex government platforms.

More information

Preparation

To replicate this demo you will need to following tools installed:

Architecture

So why do we want a high availability (HA) architecture? Why can’t we just deploy a single Virtual Machine (VM) application? Both of these questions come down to Azure’s service level agreements (SLA), to get a 99.95% SLA on Azure you will need to implement an architecture which implements 2 hosts at a minimum. But don’t fear, because our site is deployed in a high availability architecture doesn’t mean we have to make things difficult. For this demo we are going to stick to the following 3 principles:

  • Use services instead of custom deployments of services eg. Databases and file storage.
  • Keep our complex areas as simple as possible eg. Scripts over tools. You may choose a tool that suites your need in the future eg. Deployment tools, but for now we will keep it as high level as possible.
  • Implement as much open source as possible.

The diagram below illustrates the areas which we are going to focus on for this demo and how they interact.

  • Resource groups
  • Database
  • File storage
  • VM’s
  • Networking (endpoints)
  • Scaling and Availability sets

Resource Groups

Azure is very focused on providing the high level tooling for deploying applications, without the requirement for the end user to know anything about the low level architecture eg. networking and replication. The core fundamentals that resource groups provide us are:

  • Access control
  • Billing management
  • Global architecture management

So how do I create a resource group? These are easily created via the Azure UI when creating new resources. For the purpose of this demo we will create one at the same time we create our database backend, then add resources to it via the same UI as we create additional resources.

Database

The database is the primary storage backend for persistent data of the application, this component is responsible for storing all our content. Drupal 8 core ships with database drivers for Mysql, Postgres and Sqlite. In this demo we are going to use a Mysql backend implementation, on Azure we have 2 options:

  • ClearDB
  • MariaDB Enterprise
  • Install your own Mysql backend

For this demo we are going to use ClearDB, the reasons behind this are:

  • Going back to our principles for this demo, we want to rely on services where possible
  • Quick to provision
  • Simple interface
  • Unlike MariaDB, VM’s are abstracted away behind the ClearDB service on Azure

[embedded content]

As your site grows you should have a discussion about which is a better backend for you. In the following video we are provisioning an Azure ClearDB Mysql backend. Here is some further reading for the database backends:

File Storage

In conjunction with database storage, we also need file storage to make uploaded files available to all our application servers. If only we had a service which we could use as a drop in file storage backend, actually, we do! Azure Files to the rescue. Azure files provides us with a service for mounting scalable backend storage over the SMB 3.0 protocol. To setup Azure Files we need to:

  • Setup an Azure Storage account for our Resource Group
  • Create a file share on the account
  • Mount the storage onto our hosts

The video below demonstrates the setup of a file share, these details will get used later on in the demo.

[embedded content]

aGov image

PreviousNext have done all the hard work for deploying aGov on Azure, we have done this by shipping images to the VM Depot [a][b](https://vmdepot.msopentech.com/List/Index). To achieve this, PreviousNext leverage Packer by HashiCorp, and a custom provider from Microsoft:

With these tools combined we can provision and “bake” aGov images for you to leverage in your deployments. If you wish to extend the image that can be done by forking our open source project.

For your convenience we ship 2 flavours of this image:

  • Full stack - The entire LAMP stack, meaning you do not have to worry about provisioning mysql or files backends. This is only intended for aGov demo purposes before moving to a HA stack.
  • HA - This image provides all the same configuration as the above, minus the mysql backend. It also comes with a suite of scripts for assisting with mounting remote storage and connecting to the mysql backend.

For this demo we will only be using the HA image. In the following video we take you behind the scenes and provision a new image which we can add to the VM Depot.

[embedded content]

So how do we deploy our image and wire them to our backend services? Custom Data to the rescue!

Custom Data

Custom Data allows us to inject a script that will be run at the time of provision of each host. Here we have an example script which:

  • Sets up our database connection backend details
  • Mounts the remote file storage

Create a new script with the name custom-data.sh with the following contents:

#!/bin/bash

# Changing The Storage Location of the Sync Directory: https://www.drupal.org/node/2431247
setenv AGOV_DIR_CONFIG_SYNC "sites/default/files/config_/sync"
setenv AGOV_HASH_SALT ""
setenv AGOV_DB_HOST ""
setenv AGOV_DB_NAME ""
setenv AGOV_DB_USER ""
setenv AGOV_DB_PASS ""
# Environment variables will take effect now that Apache has been restarted.
sudo /etc/init.d/apache2 restart

# Persistent storage for sharing between hosts.
mount-smb "" "/data/app/sites/default/files" 33 33 0777 0777 "" ""

You will reference this script in the next section, so take note of it’s location. So now we are at the stage where we want to deploy these applications, this is where the Azure CLI comes in.

Azure CLI

We chose to use the Azure CLI for this demo so you could reproduce the same results via drop in scripts. If you still need to install the Azure CLI tools you can install these by the following 2 methods:

Using the Azure CLI and the following script we can provision 3 hosts and wire them to our backends. Here is a breakdown of the variables:

  • IMAGE - An image built by PreviousNext, available on the VM Depot.
  • REGION - The region which our site should reside in.
  • USER - Username for access to the cli on the host
  • PASS - Password for access to the cli on the host
  • GROUP - This is the ID of our resource group which we created above
  • AVAIL - Sets up an availability set with the name provided
  • CUSTOM - Path to the Custom Data, this is loaded and associated with the machine once the command is run

Create a new file called create-vms.sh with the following contents:

#!/bin/bash

IMAGE=”vmdepot-61257-1-512”
REGION='Australia East'
USER=””
PASS=””
GROUP=”agov8-ha”
AVAIL=”agov8-ha”
# The location of your custom-data script.
CUSTOM=”/home/nick/.azure/custom-data.sh”

azure vm create --connect "$GROUP" -o "$IMAGE" -l "$REGION" "$USER" "$PASS" --availability-set="$AVAIL" --ssh 12345 --custom-data=$CUSTOM
azure vm create --connect "$GROUP" -o "$IMAGE" -l "$REGION" "$USER" "$PASS" --availability-set="$AVAIL" --ssh 12346 --custom-data=$CUSTOM
azure vm create --connect "$GROUP" -o "$IMAGE" -l "$REGION" "$USER" "$PASS" --availability-set="$AVAIL" --ssh 12347 --custom-data=$CUSTOM

When you run this script, you should see three new VMs being created in the output. In this video we are provisioning our hosts with the above script.

[embedded content]

Install

Now that we have provisioned our instances, we need to install the site. Here is an example set of command which will:

  • Log you into the remote host
  • Change to the applications directory
  • Install the site with a Drupal CLI tool called Drush

In contrast to the other scripts, the following are commands which you run on the command line one after the other.

# Connect to one of the hosts on the cluster.
ssh @ -p 12345

# This is where the application is stored.
$ cd /data/app

# Install the application.
$ CMD=”drush site-install agov -y --site-name=’aGov HA demo’ --account-pass=’’ agov_install_additional_options.install=1”
$ sudo -E -u www-data /bin/bash -c "$CMD"

When you run this script, you will see a message showing Drupal was installed.

Endpoints

Endpoints are a networking abstraction which allow us to “poke holes” in the public ip and forward connections through to our VM instances running in the Resource Group.

Endpoints come in 2 types:

  • Single - Forwarding rule for exposing a single hosts port. Good for debugging single instances.
  • Balanced - Forwarding rules to balance traffic across multiple instances. Great for distributing traffic.

Here are some examples of using endpoints via the Azure CLI:

Single endpoint to access the “agov8-ha” host on port 8080

$ azure vm endpoint create agov8-ha 8080 $LOCAL

Balanced endpoint to access the “agov8-ha”, “agov8-ha-2” and “agov8-ha-3” hosts on port 80

 Create a new script create-endpoints.sh with the following contents:

#!/bin/bash

RULES=’80:80:tcp:10::tcp:80:/robots.txt:60:120:agov8-ha::’

azure vm endpoint create-multiple agov8-ha $RULES
azure vm endpoint create-multiple agov8-ha-2 $RULES
azure vm endpoint create-multiple agov8-ha-3 $RULES

The RULES variable is quite complex, here is a breakdown of the options we have passed into our balanced endpoint.

Option Value public-port 80 local-port 80 protocol tcp idle-timeout 10 direct-server-return probe-protocol tcp probe-port 80 probe-path /robots.txt probe-interval 60 probe-timeout 120 load-balanced-set-name agov8-ha internal-load-balancer-name load-balancer-distribution


When you run this script, you should see the endpoints being created successfully in the output. To verify this has run successfully, run the following command against one of the VMs:

$ azure vm endpoints list agov8-ha

This will return details on the endpoints which have been setup for the “agov8-ha” VM instance.

In this demo we setup a balanced endpoint on our application.

[embedded content]

Scaling

Congratulations! You have deployed a highly available Drupal 8 application on Microsoft Azure! But, your job is not complete, time to save costs with scaling. HA architectures on Azure are all about getting the 99.95% uptime and to get that SLA we require a minimum of 2 instances. In our current situation let’s assume are getting low traffic and we deployed 3 VM’s, we are losing money! To get our “bang for buck” we want our application to inherit the rules of:

  • Scale up in high traffic situations
  • Scale down in low traffic situations, but a minimum of 2 instances

Azure scaling gives us 2 options applicable for our application, those options are:

  • Time based - Turn off VMs at night, turn on extra VMs during the day (or any times you see fit)
  • Metrics based - If the CPU is greater than X, turn on VM. If CPU is less than Y, turn off a VM.

Notice I phases “turn on” and “turn off”, this is because your maximum and minimum application size is dictated by how many instances you have provisioned. However, don’t fear, you are not being charged for instances which are turned off, you are just saving them for later when you need them, preconfigured and ready to go.

In this demo we will scale our cluster down to 2 instances, while keeping a max size of 3.

[embedded content]

Additional Considerations

Now that we have our application deployed we can start to think about how we manage it on an ongoing basis. Here are the conversations that you need to start having now that you are running a HA architecture. These are topics which warrant their own blog posts, for this post here are some productions as part of the Azure market place which will allow you do get a feel for these implementations:

Conclusion

In this demo blog post we have covered many topics which relate to deploying high availability architecture on Microsoft Azure, but this is not the end. As a follow up to this blog post I strongly recommend you look at how this type of architecture fits into your current Drupal deployments and where some of the gaps might be. If you have any further questions please reach out in the comments and let’s keep the conversation going.

Drupal Drupal8 Azure aGov
Nov 24 2015
Nov 24

This is part 1 of a series investigating what to do with your Drupal 6 site as EOL approaches.

Part 1 - overview | Part 2 - the risks | Part 3 - the options | Part 4 - Drupal 7 or 8?

The issue at hand

As most Drupal 6 site owners are aware, after a prolonged development period, Drupal 8 was officially released (8.0.0) last week on November 19th, 2015 Dries’s birthday with a corresponding many, many a lively party: Drupal 8 celebration #celebr8d8Like this fancy one in downtown Durham atop the rooftop bar of The Durham Hotel.

Drupal 8.0.0 is a BIG DEAL and generally speaking is great for the community of Drupal site owners and site builders.

However (there’s always a but), with the official release of Drupal 8, support for Drupal 6 will end on February 24th, 2016. Given the U.S. holiday season has begun, there is little productive time remaining to undertake a site upgrade before Drupal 6 End Of Life (EOL). If you are a site owner fortunate enough to have survived a Drupal site upgrade in the past, you are well aware that the upgrade process can be time-intensive for complex sites. It is never as easy as the click of a button. For most Drupal 6 site owners, it is the fact that their sites are so complex that they have avoided going through the upgrade process for as long as possible.

This presents responsible yet practical site owners who don’t have unlimited budgets with difficult decisions, each with associated pros and cons to weigh. In part 1 of this series, we’ll help walk you through the following topics at a high-level, with a follow-up post examining each topic in finer detail.

  • What does Drupal 6 EOL mean for me?
  • What are the risks to not upgrading?
  • What are my options?
  • Should we upgrade to Drupal 7 or Drupal 8?
  • How do we decide what to do?

N.B.: if you don’t fit into the aforementioned category of having budgetary constraints, please contact us immediately. ;)

What does Drupal 6 EOL mean for me?

Like any good (and probably the bad too) Drupal advisor will tell you, it all depends. Helpful, right? But truly, it’s necessary to understand the organization and its technical requirements very well to assess the risk of operating a Drupal 6 site after EOL. As an agency that leverages open source technology to build modern web applications, on a daily basis Savas Labs relies on the Newtonian

…shoulders of giants…

to perform sophisticated tasks with the click of a button (or more likely a command in the terminal). That Drupal community that we access and contribute to for features is the same one that provides security maintenance. After EOL for Drupal 6, that click of a button access goes away both for features, but more importantly for security fixes. In other words, it means (almost) no one is watching Drupal 6 after February 24th 2015. For sustainability purposes, that huge community (~100,000 active contributors) must use its time and energy to support the newer platforms.

State of Drupal 6 sites in production

Drupal 6 has been around for a long time. As of mid November 2015 there are at least ~125,000 reported instances (likely underrepresented) of Drupal 6 sites in the wild. So you are not alone (…I am here with you…) and there is some comfort in that. If you’re reading this and have not begun your upgrade process yet, it is very likely you will be spending at least some time outside of support for your Drupal 6 site. We dive into this at a deeper level in part 2, but some of the factors that are worth taking into consideration as you strategize the upgrade are:

Considerations to assess risk
  • How well-known is your organization?
    • Larger organizations with high public profiles are systematically targeted more frequently than smaller, lesser-known organizations.
  • How many contributed modules does your site utilize and how well supported are those modules?
    • Attack vectors that remain for Drupal 6 are likely to be modules that have not received a lot of historical support, but are in some way identifiable to the public when they are in use on a site.
  • How much does your site rely on custom code?
    • Custom code has the advantage of not being publicly known, but the large disadvantage of only being vetted by one site.

What are the risks?

High-level risks, more closely examined in part 2 are as follows from most severe to least.

  • Complete site compromise and control with consequences dictated by the whim of a hacker.
  • Site incompatibility with mandatory server security upgrades that fall out of sync with Drupal 6 (PHP 7 comes out in late 2015).
  • You do not keep up with modern web development practices. After all, Drupal 6 came out January 1st, 1970 (I just checked) and given that makes it older than 6 months on the web, it’s ancient.
  • You expose yourself to a decreasing market of developers that are able to serve you. With each major release, especially two in row (7 and 8) with significant architectural modifications to the former, skills honed in development for the current version of the software, provide diminishing returns the further back in versions you go.

What are my options?

In considering a Drupal 6 upgrade you have a few simple options.

  • Do nothing, and keep your fingers crossed.
  • Upgrade Drupal 6 core to a supported version (probably Drupal 7) and match existing functionality.
  • Engage a robust redesign/rebuild (Drupal 7 or 8).
    • Simultaneously harden the site to best mitigate attack vulnerabilities as the rebuild may take 6-18 months to complete.
  • Select a different solution than Drupal, and migrate to that.

Drupal 7 or Drupal 8… heck, what about Drupal 9?

This is another one that, I know…shocker, depends. The factors that effect this choice we discuss more in part 2 are:

  • Organizational tolerance for risk: Drupal 8 is less tested, and is inherently riskier earlier on in the life cycle of your site.
  • Willingness to support community: In some cases Drupal 8 contributed modules will need extra polish to be up to production snuff.
  • Complexity of site: Drupal 8 core has many more bells and whistles, but the contributed module landscape has a long way to catch up to Drupal 7.
  • What is the future/life of the site: Drupal 8 is much more forward-thinking in its approach, whereas though vetted, Drupal 7 is over 4 years old.
  • Existing developer’s skill set: Drupal 8 architecture, coding style (object oriented) and PFE (proudly found elsewhere) approach that leverages strengths from the rest of the PHP community all mark substantial shifts from Drupal 7. Therefore the skills required to succeed in these two realms differ.
  • Get out of here with that Drupal 9 talk! It’s neither prime nor even!

What is our recommendation?

If you feel lost in these concepts or with answering some of these questions on your own, it’s best that you speak with professionals who have years of experience maintaining and upgrading Drupal sites. The upgrade process is a highly variable one, and is not especially easy to estimate as it is much more nuanced than typical feature development.

Reaching EOL for your existing Drupal site is a time that we encourage site owners to look at the process like moving into a new and better home. It’s best to take the time to envision and create what you want in the new space, rather than thoughtlessly replicate what you had in the old. Why make a carbon copy when you had good reasons to make the move after all (even if you were technologically strong-armed by volunteers)? It’s very common to have features and custom development that have outlived their usefulness to your organization’s mission; so it’s a good time to purge. Out with the old, in with the new!

Having said that, the desire to preserve content from the existing site is very common and often necessary. There are advanced migration techniques available from Drupal 6 to Drupal 7 or Drupal 8 that may be entirely separated from the rest of the rebuild, so porting content and matching site functionality can be completely decoupled.

We love talking through this process with site owners. We analyze what makes the most sense for your organization while addressing priorities for both short and long term goals. We have been building sites in Drupal 8 since May 2015 and sites in Drupal 7 since 2010 so we are well versed to the pros and cons of each. Reach out to further discuss, continue on to part 2 and stay tuned for part 3.

Part 1 - overview | Part 2 - the risks | Part 3 - the options | Part 4 - Drupal 7 or 8?

Nov 23 2015
Nov 23

Drupal Public SectorDrupal Public Sector

As a tax payer I want companies who provide frontline public services to make the data they gather in the provision of such services available through Open APIs to other actors in the ecosystem, so that such actors can utilise that data to provide new and innovative services to the public.

News from Australia shared with the DPSX BoF at DrupalCon Barcelona back in September was both good and bad, whilst the Australian government is spearheading adoption with the GovCMS distribution, efforts are we were told loosing steam as the distribution in its current form is quite restrictive requiring significant customisation by individual departments adopting it. The question is why fork it? why not modularise it? To get some answers that may help us in the UK I reached out to my Acquia colleagues in Australia to shed some light on the matter and this is what they had to say:

govCMS is growing and maturing at a good rate after a slow start. The fundamentals have been adjusted to allow for greater openness and govCMS_logo (3)_1govCMS_logo (3)_1 community interaction with the distro via github. The Australian Government is delivering on their aim to work with the community. We are expecting to see further iterations of the govCMS product over the next 12 months that will improve the ‘out of the box’ experience and allow clients to get sites up and running quickly/cheaply. Expect to see improvements that either reduce the need for customisation and make custom work easier.

The restrictions you mention are by design – they largely relate to the modules available to developers. By providing best of breed modules, the distro can avoid bloat/cost. The other effect is to focus expertise on those modules so they become easier to implement via experience and documentation that are relevant to this client space. That said, the process of requesting changes has also moved to github in the last few months and is less a ‘black box’ than it was initially.

As to why govCMS was forked – this largely related to the need Australian Government had to work with a stable platform where they could apply their own governance controls. This will probably mean a distro that evolves at a slow and steady pace, with a focus on security and compatibility. Australian Government is also sensitive to appearance that they are endorsing one vendor over another, or becoming subject to vendor lock in, taking ownership of the distro removes this issue and makes further broad investment by a range of departments more likely.” Gavin Tapp | Program Director | Acquia

There are valuable lessons for local councils in the UK seeking to create a local government distribution, which ought to make for engaging conversations at the 13th January 2016 meetup of the DPSX in London. If you’d like to a part of the conversations please register for the free event here:

One more option to look on her packing at it levitra vardenafil it of course to take not so simply because a form another and there is a wish to hold in hand her not so strongly. You can carry by me on a wide field.

Nov 17 2015
Nov 17

We just launched our first Drupal 8 website for the Northwest Atlantic Marine Alliance (NAMA). During our project retrospective, a few of us brought up how nice it was that so many contrib modules that were part of the D6 site weren’t necessary in Drupal 8 – this was a major factor in making it possible to launch this project before the actual D8 release.

The graphic below compares contrib modules that were running on NAMA’s Drupal 6 site compared to the modules needed to achieve the same functionality in Drupal 8.

Having so many of the modules that we always install built into core gave us more time to focus on ways to optimize the site for NAMA’s specific needs, and it should also save us time down the road when it’s time to handle maintenance tasks (like security updates) or add on new features.

What are you most excited about having in core?

Which modules are you waiting to see in D8 before upgrading?

Nov 06 2015
Nov 06

The Northwest Atlantic Marine Alliance (NAMA) advocates for healthy marine ecosystems. Through their site, namanet.org, they organize and promote a movement toward a thriving ocean and a just seafood system.

Having helped create the original Drupal 6 site for NAMA in 2008, it’s been a rewarding experience re-architecting it in Drupal 8 (beta). The main purpose of the project, at least initially, was to help NAMA plan for how to deal with their soon-to-be-end-of-lifed D6 site by upgrading it to D7. As a fully matured platform, this upgrade (or migration) path has long been the recommended approach, even since talk of a release candidate began in the summer of ’15, when the number of critical issues neared zero.

Choosing an upgrade path – Drupal 7 vs Drupal 8

NAMA was the perfect client to test out a Drupal 8 upgrade. They were flexible about their design and functional requirements, and the original site was straightforward in its architecture. As the betas ticked up in number, it became feasible to consider skipping over D7 to go straight to D8 instead. Once the D6->D8 migration path appeared ready to use, we discussed the pros and cons with NAMA and, together, decided to go for it.

By structuring the project to partially include “internal” and “personal development” time, we were able to deliver the new site on a tight budget while allowing the entire team to gain crucial experience with the upcoming version of our CMS/framework of choice.

The Drupal 8 advantage

While NAMA will enjoy many secondary benefits to having a D8 site, like a responsive front-end and in-place WYSIWYG editing, the most meaningful advantage of NAMA skipping D7 for D8 is the significantly extended lifespan of their redeveloped site. For Advomatic, apart from getting to work with a shiny new Drupal toy, we’re now prepared to hit the ground running when the full release of D8 is ready, and to support the sea of Drupal 6 sites that will soon be in need of migration help.

NAMA's coordinating director, Niaz Dorry.

NAMA’s coordinating director, Niaz Dorry.

As we wrapped up our final tickets and NAMA began working with their new site, I asked Coordinating Director Niaz Dorry to chat with me about their experience with Drupal 8 so far.

When asked about her favorite features of the new site, Niaz noted that it was, “clean, open, uncluttered, more modern, and (so far) appears to be easier to edit.” She also detailed some of the new configurable components we added to the homepage, which leverage the fieldability of D8’s upgraded Block module.

Niaz added that the move to the new site has been fairly seamless from an administrator’s perspective. The ease of this process is largely due to content creation and usability improvements that have been incorporated into D8 core: 

“The transition seems smooth so far. Generally speaking, the two sites feel similar. What is different is not THAT different, and we seem to be learning it pretty quickly.”

And what about the tradeoffs of working with not-quite-ready software, when a robust ecosystem has been developing around its predecessor for years? At the outset of the project, we’d discussed – in theory – what sacrifices might need to be made. But in practice, what diminished functionality did they end up having to live with?

“Not much, really. Some of the things that are not ready yet – like a better date module – don’t appear to be showstoppers. We knew what we were going into, and that there would be some adjustments to things as Drupal 8 goes from Beta to fully operational. So nothing has been a surprise.”

Niaz eloquently summarizes how this project has been a big win for everyone involved:

“We’re glad to have had this opportunity to work with Advomatic. The results – an updated website for us, experience with Drupal 8 for Advomatic, and contributing to the broader cyber community as we learn – are truly a win-win-win, and that’s the kind of effort we like to engage with, whether it’s our program work, our policy work, the way we run our organization, and now even the way we build our website.”

Have you had a chance to work with a client on a D8 project yet? What do they think so far?

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web