Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Feb 11 2021
Feb 11

Last week one of our clients was asking me about how they should think about the myriad of options for website hosting, and it inspired me to share a few thoughts. 

The different kinds of hosting

I think about hosting for WordPress and Drupal websites as falling into one of three groups. We’re going to compare the options using an example of a fairly common size of website — one with traffic (as reported by Google Analytics) in the range of 50,000–100,000 visitors per month. Adjust accordingly for your situation. 

  • “Low cost/low frills” hosting — Inexpensive website hosting would cost in the range of $50–$1,000/yr for a site with our example amount of traffic. Examples of lower cost hosts include GoDaddyBluehost, etc.  Though inexpensive, these kinds of hosts have none of the infrastructure that’s needed to do ongoing web development in a safe/controlled way such as the ability to spin up a copy of the website at the click of a button, make a change, get approval from stakeholders, then deploy to the live site. Also, if you get a traffic spike, you will likely see much slower page loads. 
  • “Unmanaged”, “Bare metal”, or “DIY” hosting — Our example website will likely cost in the range of $500–$2,500/yr. Examples of this type of hosting include: AWSRackspaceLinode, etc. or just a computer in your closet. Here you get a server, but that’s it. You have to set up all the software, put security measures in place, and set up the workflow so that you can get stuff done. Then it’s your responsibility to keep that all maintained year over year, perhaps even to install and maintain firewalls for security purposes. 
  • “Serverless” hosting¹ — It’s not that there aren’t servers, they’re just transparent to you. Our example website would likely cost in the range of $2500–5000/yr. Examples of this kind of hosting: PantheonWP EngineAcquiaPlatform.sh. These hosts are very specialized for WordPress and/or Drupal websites. You just plug in your code and database, and you’re off. Because they’re highly specialized, they have all the security/performance/workflow/operations in place that 90% of Drupal/WordPress websites need.

How to decide?

I recommend two guiding principles when it comes to these kinds of decisions:

  1. The cost of services (like hosting) are much cheaper than the cost of people. Whether that’s the time that your staff is spending maintaining a server, or if you’re working with an agency like Four Kitchens, then your monthly subscription with us. Maybe even 10x.  So saving $1,000/yr on hosting is only worth it if it costs less than a handful of hours per year of someone’s time. 
  2. Prioritize putting as much of your budget towards advancing your organization’s mission as possible. If two options have a similar cost, we should go with the option that will burn fewer brain cells doing “maintenance” and other manual tasks, and instead choose the option where we can spend more of our time thinking strategically and advancing the mission.

This means that you should probably disregard the “unmanaged/bare/DIY” group. Whoever manages the site will spend too much time running security updates, and doing other maintenance and monitoring tasks. 

We also encourage you to disregard the “low cost” group. Your team will waste too much time tripping over the limitations, and cleaning up mistakes that could be prevented on a more robust platform.

So that leaves the “serverless” group. With these, you’ll get the tools that will help streamline every change made to your website. Many of the rote tasks are also taken care of as part of the package. 

Doing vs. Thinking

It’s easy to get caught up in doing stuff. And it’s easy to make little decisions over time that mean you spend all your days just trying to keep up with the doing. The decision you make about hosting is one way that you can get things back on track to be more focused on the strategy of how to make your website better.

¹ The more technical members of the audience will know that “serverless” is technically a bit different.  You’d instead call this “platform-as-a-service” or “infrastructure-as-a-service”. But we said we’d avoid buzzwords.

Making the web a better place to teach, learn, and advocate starts here...

When you subscribe to our newsletter!

Oct 22 2020
Oct 22

A common frustration for Drupal 8 (and 9) site builders is the inability to change text fields from plain text to filtered text through the administrative interface. This was something that was easy to do in Drupal 7 by editing the field’s settings and changing the value for Text processing.

Sometimes requirements for a field change, both during the build phase and long after a site has been in production, and it would be convenient to toggle on a rich text editor and text filtering with minimal effort.

In Drupal 8, there are five types of text fields in core:

  • Text (plain)
  • Text (plain, long)
  • Text (formatted)
  • Text (formatted, long)
  • Text (formatted, long, with summary)

The first two are actually string fields and don’t allow any formatting. If a user enters HTML tags, they are ignored and displayed as plain text.

The last three will allow HTML tags, depending on the settings for the Text Format that the user chooses when entering content. However, only the last two will show the WYSIWYG editor (if it’s associated with the selected text format).

But what happens if midway through your build process, or months after your site has launched, the requirements for that text field change? Your client or designer decides they now want to allow some formatting in a field that was originally Text (plain) or Text (plain, long). And they want their editors to be able to use a WYSIWYG, so they don’t have to deal with HTML code. What do you do?

The long way: Create new field, migrate data, reconfigure

One solution is to write an update hook that will create a new field, migrate existing data to it, and delete the old field. If the field is renamed, you also have to consider reconfiguring any views, entity references, display modes, etc. that referred to the old field. Changing a field type this way is entirely possible, but more time consuming and error prone.

Wouldn’t it be nice if you could simply enable a WYSIWYG on that plain text field and be done with it? Especially if your client is on a tight budget. It’s actually possible to do this in a custom module, with a few lines of code.

The short way part 1: Add a form alter

First you’ll need to add a form_alter function in your custom module, most likely in the .module file. There are many ways to add a form_alter in Drupal 8, and those are documented elsewhere. See the entry about hook_form_alter in the Drupal API.

You may also need to add some conditions so that the field is only changed on certain forms or for certain content types–this is also beyond the scope of this article.

In this example, I’m adding a general hook_form_alter() that will apply to all forms regardless of entity type. If you have a Text (formatted) field, you may want to enable a WYSIWYG on it to make it easier for editors to create the content. Because it’s already a formatted text field, the form alter is very simple.

use Drupal\Core\Form\FormStateInterface;

function mymodule_form_alter(&$form, FormStateInterface $form_state, $form_id) {
  if(isset($form['field_myformattedtextfield'])) {
    $form['field_myformattedtextfield']['widget']['0']['#base_type'] = 'textarea';
  }
}

We are only changing one value associated with the widget: we change the base_type to textarea. The editors will see a WYSIWYG, and the data will be saved and displayed as formatted text.

If you want to add a WYSIWYG widget on a Text (plain) or Text (plain, long) field, it’s a little trickier. There are a few more widget attributes to alter.

use Drupal\Core\Form\FormStateInterface;

function mymodule_form_alter(&$form, FormStateInterface $form_state, $form_id) {
  if(isset($form['field_myplaintextfield'])) {
    // Fetch the entity object.
    $entity = $form_state->getFormObject()->getEntity();
    // Get the current value stored for the field.
    $value = $entity->field_myplaintextfield->getString();
    // Change the base type for this field to a textarea.
    $form['field_myplaintextfield']['widget']['0']['#base_type'] = 'textarea';
    // Change the type of field to formatted text.
    $form['field_myplaintextfield']['widget']['0']['#type'] = 'text_format';
    // Recommended: set a default text format. When rendering 
    // you’ll have to manually set this to make the field use 
    // formatting (see next section).
    $form['field_myplaintextfield']['widget']['0']['#format'] = 'full_html';
    // Set the default value to the currently stored value.
    $form['field_myplaintextfield']['widget']['0']['#default_value'] = $value;
  }
}

This will give us the WYSIWYG where we want it, but the value is still stored and displayed as plain text. We need to add another function to transform it for output.

The short way part 2: Let the fields render as formatted text

For Text (plain) or Text (plain, long) fields, we have to tell Drupal to run the stored value through one of the Text Format filters and render it as formatted text. This involves two functions and a configuration setting.

In your custom module, add a hook_field_formatter_info_alter to allow the plain text field types to use the default text formatter:

function mymodule_field_formatter_info_alter(array &$info) {
  // Let the string field types use the text formatter.
  $info['text_default']['field_types'][] = 'string';
  $info['text_default']['field_types'][] = 'string_long';
}

Then, add a template_preprocess_field function to tell Drupal which text format to use when that field is displayed as filtered text. Since this isn’t a regular formatted text field, Drupal doesn’t store that information the way it does for the standard formatted text field types. Be sure to use the same text format that you used in the hook_form_alter().

function mymodule_preprocess_field(&$variables, $hook) {
  if ($variables['field_name'] == 'field_myplaintextfield') {
    $variables['items']['0']['content']['#format'] = 'full_html';
  }
}

Lastly, we go to the “Manage display” configuration for the field in question, and tell Drupal to use the “Default” format for that field. If you are using Configuration Synchronization, you’ll notice this affects the “type” in the “Entity view display” configuration for this field.

...
  field_myplaintextfield:
    weight: 100
    label: above
    settings: {  }
    third_party_settings: {  }
    type: text_default
    region: content
...

Voila! That should be it. View your content and check that the plain text field is now being rendered with HTML formatting.

Conclusion

With just a few lines of code, we added a WYSIWYG editor to a plain text field and enabled Drupal to display its contents as formatted text. This technique will work for both Drupal 8 and Drupal 9. You can refine this approach depending on your needs to display formatted text only for certain view modes, entity types, field instances, etc.

Want to add this functionality to your site? Contact us if you’d like to hear more about our services.

Making the web a better place to teach, learn, and advocate starts here...

When you subscribe to our newsletter!

Jul 30 2020
Jul 30

argument-open-source Over 500,000 businesses leverage Drupal to launch their websites and projects. From NASA to Tesla, public and private institutions regularly rely on Drupal to launch large-scale websites capable of handling their development and visual needs. But, starting a Drupal project doesn’t guarantee success. In fact, 14% of all IT projects outright fail, 43% exceed their initial budgets, and 31% fail to meet their original goals! In other words, if you want to create a successful Drupal project, you need to prepare. Don’t worry! We’ve got your back. Here are 5 things to keep in mind when starting a Drupal-based project.

1. GATHER REQUIREMENTS FROM STAKEHOLDERS EARLY AND OFTEN

According to PMI, 39% of projects fail due to inadequate requirements. Believe it or not, requirement gathering is the single most important stage of project development. In fact, it’s the first step Drupal itself takes when pushing out new projects (see this scope document for their technical document project). Gathering requirements may sound easy, but it can be a time-consuming process. We recommend using SMART (Specific, Measurable, Agreed Upon, Realistic, Time-based) to map out your specific needs. If possible, involve the end-user during this stage. Don’t assume you know what users want; ask them directly. Internally, requirements gathering should rally nearly every stakeholder with hefty amounts of cross-collaboration between departments. You want to lean heavily on data, establish your benchmarks and KPIs early, and try to involve everyone regularly. The single biggest project mistake is acting like requirements are set-in-stone. If you just follow the initial requirements to a “T,” you may push out a poor project. You want to regularly ask questions, communicate issues, and rely on guidance from stakeholders and subject matter experts (SMEs) to guide your project to completion.

2. PLAN YOUR SDLC/WORKFLOW PIPELINE

We all have different development strategies. You may leverage freelancers, a best-in-class agency, or internal devs to execute your Drupal projects. Typically, we see a combination of two of the above. Either way, you have to set some software development lifecycle and workflow standards. This gets complex. On the surface, you should think about coding standards, code flow, databases, and repositories, and all of the other development needs that should be in sync across devs. But there’s also the deeper, more holistic components to consider. Are you going to use agile? Do you have a DevOps strategy? Are you SCRUM-based? Do you practice design and dev sprints? At Mobomo, we use an agile-hybrid development cycle to fail early, iterate regularly, and deploy rapidly. But that’s how we do things. You need to figure out how you want to execute your project. We’ve seen successful Drupal projects using virtually every workflow system out there. The way you work matters, sure. But getting everyone aligned under a specific way of working is more important. You can use the “old-school” waterfall methodology and still push out great projects. However, to do that, you need everyone on the same page.

3. USE SHIFT-LEFT TESTING FOR BUG AND VULNERABILITY DETECTION

Drupal is a secure platform. Of the four most popular content management systems, Drupal is the least hacked. But that doesn’t mean it’s impenetrable. You want to shift-left test (i.e., automate testing early and often in the development cycle). Drupal 8+ has PHPUnit built-in — taking the place of SimpleTest. You can use this to quickly test out code. You can perform unit tests, kernel tests, and functional tests with and without JavaScript. You can also use Nightwatch.js to run tests. Of course, you may opt for third-party automation solutions (e.g., RUM, synthetic user monitoring, etc.) The important thing is that you test continuously. There are three primary reasons that shift-left testing needs to be part of your development arsenal.

  • It helps prevent vulnerabilities. The average cost of a data breach is over $3 million. And it takes around 300 days to identify and contain website breaches.
  • It bolsters the user experience. A 100-millisecond delay in page load speed drops conversions by 7%. Meanwhile, 75% of users judge your credibility by your website’s design and performance, and 39% of users will stop engaging with your website if your images take too long to load. In other words, simple glitches can result in massive issues.
  • It reduces development headaches. Nothing is worse than developing out completely new features only to discover an error that takes you back to step 1.

4. GET HYPER-FAMILIAR WITH DRUPAL’S API

If you want to build amazing Drupal projects, you need to familiarize yourself with the Drupal REST API. This may sound like obvious advice. But understanding how Drupal’s built-in features, architecture, and coding flow can help you minimize mistakes and maximize your time-to-launch. The last thing you want to do is code redundantly when Drupal may automate some of that coding on its end. For more information on Drupal’s API and taxonomy, see Drupal API. We know! If you’re using Drupal, you probably have a decent idea of what its API looks like. But make sure that you understand all of its core features to avoid headaches and redundancies.

5. SET STANDARDS

Every development project needs standards. There are a million ways to build a website or app. But you can’t use all of those million ways together. You don’t want half of your team using Drupal’s built-in content builder and the other half using Gutenberg. Everyone should be on the same page. This goes for blocks, taxonomy, and every other coding need and task you’re going to accomplish. You need coding standards, software standards, and process standards to align your team to a specific framework. You can develop standards incrementally, but they should be shared consistently across teams. Ideally, you’ll build a standard for everything. From communication to development, testing, launching, and patching, you should have set-in-stone processes. In the past, this was less of an issue. But, with every developer rushing to agile, sprint-driven methodologies, it can be easy to lose sight of standards in favor of speed. Don’t let that happen. Agile doesn’t mean “willy-nilly” coding and development for the fastest possible launch. It still has to be systematic. Standards allow you to execute faster and smarter across your development pipeline.

NEED SOME HELP?

At Mobomo, we build best-in-class Drupal projects for brands across the globe. From NASA to UGS, we’ve helped private, and public entities launch safe, secure, and exciting Drupal solutions. Are you looking for a partner with fresh strategies and best-of-breed agile-driven development practices?

Contact us. Let’s build your dream project — together.

Jul 28 2020
Jul 28

argument-open-source

DRUPAL MIGRATION PREPARATION AUDIT

All good things must come to an end. Drupal 7 will soon meet its end. Does your organization have your migration plan to Drupal 9 in order? Here’s what you need to know to keep your Drupal site running and supported. Talk to Our Drupal Migration Experts Now!

OUR APPROUCH TO DRUPAL MIGRATION.

  • Analyze 
  • Inventory
  • Migration
  • Revision
  • SEO

OVERVIEW

Staying up to date with Drupal versions is vital to maintaining performance to your site:

  • Future-proofing
  • Avoiding the end-of-life cut-off
  • Performance
  • Security

GOALS

  1. Catalog existing community contributed modules necessary to the project
  • Do these modules have a corresponding Drupal 8 version?
  • If the answer to the above question is no, is there an alternative?
  • Is there an opportunity to optimize or upgrade the site’s usage of contributed modules?
  1. Catalog existing custom built modules
  • Do these modules rely on community contributed modules that may not have a migration path to Drupal 8?
  • Do these modules contain deprecated function calls?
  • Are there any newer community contributed modules that may replace the functionality of the custom modules?
  1. Review existing content models.
  • How complex is the content currently—field, taxonomy, media?
  • What specific integrations need to be researched so content will have feature parity?
  1. Catalog and examine 3rd party integrations.
  • Is there any kind of e-commerce involved?
  • Do these 3rd party integrations have any Drupal 8 community modules?
  1. Catalog User roles and permissions
  • Do user accounts use any type of SSO?
  • Is there an opportunity to update permissions and clean up roles?

PRE-AUDIT REQUIREMENTS

  • Access to the codebase
  • Access to the database
  • Access to a live environment (optional)
  • Access to integrations in order to evaluate level of effort

DELIVERABLES

Module Report The module report should contain an outline of the existing Drupal 7 modules with the corresponding path to Drupal 8, whether that’s an upgraded version of the existing module or a similar module. This report should also contain a sheet outlining any deprecated function usage for the custom modules that will need to be ported to Drupal 8.

Content Model Report The Content Model report should contain an overview of the existing site’s content types, users, roles, permissions and taxonomic vocabularies with each field given special consideration. Recommendations should be made in the report to improve the model when migrating to Drupal 8.

Integration Report The integration report contains a catalog of the third party integrations currently in use and marks those with an existing contributed module from the community and those that will require custom work to integrate with the Drupal 8 system.

Our Insights on Drupal Our latest thoughts, musings and successes.

Contact us. We’ll help you expand your reach.

Mar 20 2020
Mar 20

On June 24, 2020 Drupal.org announced that Drupal 7’s end of life has been extended until November 2023 because of the impact of COVID-19 on budgets and capacity. This article still remains relevant — but please note that the dates have been pushed back a year

If you have a Drupal 7 website, you might have already heard that the official end-of-life date for Drupal 7 has been officially set for November 2021. Many organizations should upgrade their Drupal 7 sites before then. But that might not be required. Here’s how you figure out what you need to do.

“What does Drupal 7 end-of-life mean?”

First let’s talk about what EOL means for Drupal. The main thing is security updates. 

Drupal has a highly regarded security team who manage security for both core Drupal and thousands of public modules, themes and distributions that add additional features. When a security problem is found with Drupal core, the team fixes the problem and publishes advisories that explain vulnerabilities, along with steps to mitigate them. All of this is contributed publicly and freely, just like you would expect from open source software. 

The security team supports versions of Drupal until they reach their end-of-life. 

But after the EOL, the baton is passed along to an Extended Security Support team. This team is composed of pre-vetted Drupal agencies, and they are commercially funded by those clients who want to pay for the extended security support. They are mandated to publicly release fixes for most of the security vulnerabilities that they find. 

“Hold on. What level of security support do I need?”

Before we talk about what you should do about D7 EOL, you first need to think about how important security is for your website.

  • Are there people who are actively trying to attack your website (maybe because of your strong stance on a particular issue)?
  • Does your website process commercial transactions? (Most non-profit websites these days use third-party websites to process donations and event registrations.)
  • Does your website collect a lot of personally identifiable information (PII)? This relates back to the first point: if there’s lots of valuable PII, an attacker will be more interested in trying to steal it. 

If you answered “yes” to any of these questions, then security is of extra importance for you.

“I won’t have the budget for a big website rebuild before November 2021” 

It’s going to be okay, we’ve got a few options for you. You’ll fall into one of the following categories:

1. “Security is really important for our website, we need Extended Security Support”

Regardless of whether you are an existing client, or someone we’ve never worked with before, please reach out to us and let us know if we can help.

2. “Security is just as important to our website as it is for every other website, but not in an extra special way”

If your website does not have a reason for someone to actively try to attack it, then you only need to be guarded from publicly known security vulnerabilities. That way, you’re protected against the automated attacks that hit every website. Typically those kinds of automated attacks are either trying to use your web servers to mine bitcoin, or lock up your website and demand a ransom. 

When Drupal 6 reached end-of-life in 2016 we continued to support our Drupal 6 clients using the publicly released updates from the Extended Security Support team. Our last Drupal 6 client just got a new website a few months ago! 

We’ll do the same when Drupal 7 reaches end-of-life. When a Drupal 7 update is released, we’ll update your website, just like we already do for all of our Drupal and WordPress support and maintenance clients.

3. “Help, I have no idea what I need!”

No problem. We can help here too. Regardless of where you’re at — or where you’re going next — we’re here to help. Drop us a line.

Making the web a better place to teach, learn, and advocate starts here...

When you subscribe to our newsletter!

Dec 06 2019
Dec 06

You may have read our previous articles about how to plan for Drupal 6 or Drupal 7 End-of-Life. The important thing to know is that the Drupal 8 End-of-Life is nothing like those. In fact, “End of Life” is completely the wrong idea. Instead, it’s more like one of those spa treatments where you get a full body scrub to get rid of the dead skin cells. You walk out feeling rejuvenated and refreshed. 

Drupal 9 — Same as Drupal 8, But Without The Old Stuff

Drupal release timeline

In each new minor version of Drupal 8 there are some new features, and some old code is marked as “deprecated” (that just means that it’s time to stop using this, because it’s going to go away some day). After nine minor versions over almost five years, there’s now an accumulation of deprecated code. This deprecated code is like those dead skin cells that you go to the spa to get rid of.  So Drupal 9.0 will be the same as Drupal 8.9, just without the deprecated code. The two might even be released at the same time. 

Then, in Drupal 9.1, we see the cycle starting again: some new features, and some old code is marked as deprecated.

Don’t Rely on Deprecated Code

In the graphic above, you’ll notice that 8.9 does not have any more deprecated code than 8.8. That means that once a website is upgraded to 8.8, we can then start the process of ensuring that the site isn’t using any deprecated code. 

If you are an Advomatic client, we’ll create a ticket in your queue to clean out all uses of deprecated code. In fact, if you’ve done a project with us recently, we’ve already started doing this as part of the Q/A process in our two-week sprints. 

A Window of Almost Two Years for This Cleanup

Drupal timeline by quarters

This is the timeline for the next several versions of Drupal.  We’ve got about 2 years to make this change — more than enough time. 

Alternating Minor Versions

We handle all the technical stuff for you. But the purpose of the website is not for us to have a technical toy to play with, it’s to advance the mission of your non-profit. So we want to devote most of our time and effort towards your web strategy. While we could upgrade your website to the newest version every six months, it’s not the best use of your money or time. So we alternate versions. That means that your Drupal 8 website is either always on an even minor version, or an odd minor version. 

We’ll likely continue that pattern as we cross the threshold into Drupal 9. That means that this process could be delayed by 6 months from what you see here.

Flipping the Switch

Once we’ve cleaned up all the deprecated code, then we’re ready to upgrade the site to Drupal 9.  Remember: this is nothing like past major upgrades in Drupal. Instead it’s just like the minor upgrades from Drupal 8.6 → 8.7 → 8.8 etc.

Conclusion

The key takeaway is that this whole process should be almost seamless. We’ll create a few tickets in the queue to prep for the upgrade, and then for the upgrade itself.  But the majority of our time will still be spent on advancing your mission. Over the years to come the website content and its presentation will be able to continually evolve, all without a costly major upgrade. 

Thanks to Amanda Luker for the charts!

Dec 15 2015
Dec 15

Tom Friedhof

Senior Software Engineer

Tom has been designing and developing for the web since 2002 and got involved with Drupal in 2006. Previously he worked as a systems administrator for a large mortgage bank, managing servers and workstations, which is where he discovered his passion for automation and scripting. On his free time he enjoys camping with his wife and three kids.

Dec 02 2015
Dec 02

Now that the release of Drupal 8 is finally here, it is time to adapt our Drupal 7 build process to Drupal 8, while utilizing Docker. This post will take you through how we construct sites on Drupal 8 using dependency managers on top of Docker with Vagrant.

Keep a clean upstream repo

Over the past 3 or 4 years developing websites has changed dramatically with the increasing popularity of dependency management such as Composer, Bundler, npm, Bower, etc... amongst other tools. Drupal even has it's own system that can handle dependencies called Drush, albiet it is more than just a dependency manager for Drupal.

With all of these tools at our disposal, it makes it very easy to include code from other projects in our application while not storing any of that code in the application code repository. This concept dramatically changes how you would typically maintain a Drupal site, since the typical way to manage a Drupal codebase is to have the entire Drupal Docroot, including all dependencies, in the application code repository. Having everything in the docroot is fine, but you gain so much more power using dependency managers. You also lighten up the actual application codebase when you utilize dependency managers, because your repo only contains code that you wrote. There are tons of advantages to building applications this way, but I have digressed, this post is about how we utilize these tools to build Drupal sites, not an exhaustive list of why this is a good idea. Leave a comment if you want to discuss the advantages / disadvantages of this approach.

Application Code Repository

We've got a lot going on in this repository. We won't dive too deep into the weeds looking at every single file, but I will give a high level overview of how things are put together.

Installation Automation (begin with the end in mind)

The simplicity in this process is that when a new developer needs to get a local development environment for this project, they only have to execute two commands:

$ vagrant up --no-parallel
$ make install

Within minutes a new development environment is constructed with Virtualbox and Docker on the developers machine, so that they can immediately start contributing to the project. The first command boots up 3 Docker containers -- a webserver, mysql server, and jenkins server. The second command invokes Drush to build the document root within the webserver container and then installs Drupal.

We also utilize one more command to keep running within a seperate terminal window, to keep files synced from our host machine to the Drupal 8 container.

$ vagrant rsync-auto drupal8

Breaking down the two installation commands

vagrant up --no-parallel

If you've read any of [my]({% post_url 2015-06-04-hashing-out-docker-workflow %}) [previous]({% post_url 2015-07-19-docker-with-vagrant %}) [posts]({% post_url 2015-09-22-local-docker-development-with-vagrant %}), I'm a fan of using Vagrant with Docker. I won't go into detail about how the environment is getting set up. You can read my previous posts on how we used Docker with Vagrant. For completeness, here is the Vagrantfile and Dockerfile that vagrant up reads to setup the environment.

Vagrantfile






require 'fileutils'

MYSQL_ROOT_PASSWORD="root"

unless File.exists?("keys")
Dir.mkdir("keys")
ssh_pub_key = File.readlines("#{Dir.home}/.ssh/id_rsa.pub").first.strip
File.open("keys/id_rsa.pub", 'w') { |file| file.write(ssh_pub_key) }
end

unless File.exists?("Dockerfiles/jenkins/keys")
Dir.mkdir("Dockerfiles/jenkins/keys")
FileUtils.copy("#{Dir.home}/.ssh/id_rsa", "Dockerfiles/jenkins/keys/id_rsa")
end

Vagrant.configure("2") do |config|

config.vm.define "mysql" do |v|

    v.vm.provider "docker" do |d|
      d.vagrant_machine = "apa-dockerhost"
      d.vagrant_vagrantfile = "./host/Vagrantfile"
      d.image = "mysql:5.7.9"
      d.env = { :MYSQL_ROOT_PASSWORD => MYSQL_ROOT_PASSWORD }
      d.name = "mysql-container"
      d.remains_running = true
      d.ports = [
        "3306:3306"
      ]
    end

end

config.vm.define "jenkins" do |v|

    v.vm.synced_folder ".", "/srv", type: "rsync",
        rsync__exclude: get_ignored_files(),
        rsync__args: ["--verbose", "--archive", "--delete", "--copy-links"]

    v.vm.provider "docker" do |d|
      d.vagrant_machine = "apa-dockerhost"
      d.vagrant_vagrantfile = "./host/Vagrantfile"
      d.build_dir = "./Dockerfiles/jenkins"
      d.name = "jenkins-container"
      
      d.volumes = [
          "/home/rancher/.composer:/root/.composer",
          "/home/rancher/.drush:/root/.drush"
      ]
      d.remains_running = true
      d.ports = [
          "8080:8080"
      ]
    end

end

config.vm.define "drupal8" do |v|

    v.vm.synced_folder ".", "/srv/app", type: "rsync",
      rsync__exclude: get_ignored_files(),
      rsync__args: ["--verbose", "--archive", "--delete", "--copy-links"],
      rsync__chown: false

    v.vm.provider "docker" do |d|
      d.vagrant_machine = "apa-dockerhost"
      d.vagrant_vagrantfile = "./host/Vagrantfile"
      d.build_dir = "."
      d.name = "drupal8-container"
      d.remains_running = true
      
      d.volumes = [
        "/home/rancher/.composer:/root/.composer",
        "/home/rancher/.drush:/root/.drush"
      ]
      d.ports = [
        "80:80",
        "2222:22"
      ]
      d.link("mysql-container:mysql")
    end

end

end

def get_ignored_files()
ignore_file = ".rsyncignore"
ignore_array = []

if File.exists? ignore_file and File.readable? ignore_file
File.read(ignore_file).each_line do |line|
ignore_array << line.chomp
end
end

ignore_array
end

One of the cool things to point out that we are doing in this Vagrantfile is setting up a VOLUME for the composer and drush cache that should persist beyond the life of the container. When our application container is rebuilt we don't want to download 100MB of composer dependencies every time. By utilizing a Docker VOLUME, that folder is mounted to the actual Docker host.

Dockerfile (drupal8-container)

FROM ubuntu:trusty


ENV PROJECT_ROOT /srv/app
ENV DOCUMENT_ROOT /var/www/html
ENV DRUPAL_PROFILE=apa_profile


RUN apt-get update
RUN apt-get install -y \
	vim \
	git \
	apache2 \
	php-apc \
	php5-fpm \
	php5-cli \
	php5-mysql \
	php5-gd \
	php5-curl \
	libapache2-mod-php5 \
	curl \
	mysql-client \
	openssh-server \
	phpmyadmin \
	wget \
	unzip \
	supervisor
RUN apt-get clean


RUN curl -sS https://getcomposer.org/installer | php
RUN mv composer.phar /usr/local/bin/composer


RUN mkdir /root/.ssh && chmod 700 /root/.ssh && touch /root/.ssh/authorized_keys && chmod 600 /root/.ssh/authorized_keys
RUN echo 'root:root' | chpasswd
RUN sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/' /etc/ssh/sshd_config
RUN mkdir /var/run/sshd && chmod 0755 /var/run/sshd
RUN mkdir -p /root/.ssh
COPY keys/id_rsa.pub /root/.ssh/authorized_keys
RUN chmod 600 /root/.ssh/authorized_keys
RUN sed 's@session\s*required\s*pam_loginuid.so@session optional pam_loginuid.so@g' -i /etc/pam.d/sshd


RUN composer global require drush/drush:8.0.0-rc3
RUN ln -nsf /root/.composer/vendor/bin/drush /usr/local/bin/drush


RUN mv /root/.composer /tmp/


RUN sed -i 's/display_errors = Off/display_errors = On/' /etc/php5/apache2/php.ini
RUN sed -i 's/display_errors = Off/display_errors = On/' /etc/php5/cli/php.ini


RUN sed -i 's/AllowOverride None/AllowOverride All/' /etc/apache2/apache2.conf
RUN a2enmod rewrite


RUN echo '[program:apache2]\ncommand=/bin/bash -c "source /etc/apache2/envvars && exec /usr/sbin/apache2 -DFOREGROUND"\nautorestart=true\n\n' >> /etc/supervisor/supervisord.conf
RUN echo '[program:sshd]\ncommand=/usr/sbin/sshd -D\n\n' >> /etc/supervisor/supervisord.conf





WORKDIR $PROJECT_ROOT
EXPOSE 80 22
CMD exec supervisord -n

We have xdebug commented out in the Dockerfile, but it can easily be uncommented if you need to step through code. Simply uncomment the two RUN commands and run vagrant reload drupal8

make install

We utilize a Makefile in all of our projects whether it be Drupal, nodejs, or Laravel. This is so that we have a similar way to install applications, regardless of the underlying technology that is being executed. In this case make install is executing a drush command. Below is the contents of our Makefile for this project:

all: init install

init:
vagrant up --no-parallel

install:
bin/drush @dev install.sh

rebuild:
bin/drush @dev rebuild.sh

clean:
vagrant destroy drupal8
vagrant destroy mysql

mnt:
sshfs -C -p 2222 [email protected]:/var/www/html docroot

What this commmand does is ssh into the drupal8-container, utilizing drush aliases and drush shell aliases.

install.sh

The make install command executes a file, within the drupal8-container, that looks like this:

#!/usr/bin/env bash

echo "Moving the contents of composer cache into place..."
mv /tmp/.composer/* /root/.composer/

PROJECT_ROOT=$PROJECT_ROOT DOCUMENT_ROOT=$DOCUMENT_ROOT $PROJECT_ROOT/bin/rebuild.sh

echo "Installing Drupal..."
cd $DOCUMENT_ROOT && drush si $DRUPAL_PROFILE --account-pass=admin -y
chgrp -R www-data sites/default/files
rm -rf ~/.drush/files && cp -R sites/default/files ~/.drush/

echo "Importing config from sync directory"
drush cim -y

You can see on line 6 of install.sh file that it executes a rebuild.sh file to actually build the Drupal document root utilizing Drush Make. The reason for separating the build from the install is so that you can run make rebuild without completely reinstalling the Drupal database. After the document root is built, the drush site-install apa_profile command is run to actually install the site. Notice that we are utilizing Installation Profiles for Drupal.

We utilize installation profiles so that we can define modules available for the site, as well as specify default configuration to be installed with the site.

We work hard to achieve the ability to have Drupal install with all the necessary configuration in place out of the gate. We don't want to be passing around a database to get up and running with a new site.

We utilize the Devel Generate module to create the initial content for sites while developing.

rebuild.sh

The rebuild.sh file is responsible for building the Drupal docroot:

#!/usr/bin/env bash

if [ -d "$DOCUMENT_ROOT/sites/default/files" ]
then
echo "Moving files to ~/.drush/..."
mv \$DOCUMENT_ROOT/sites/default/files /root/.drush/
fi

echo "Deleting Drupal and rebuilding..."
rm -rf \$DOCUMENT_ROOT

echo "Downloading contributed modules..."
drush make -y $PROJECT_ROOT/drupal/make/dev.make $DOCUMENT_ROOT

echo "Symlink profile..."
ln -nsf $PROJECT_ROOT/drupal/profiles/apa_profile $DOCUMENT_ROOT/profiles/apa_profile

echo "Downloading Composer Dependencies..."
cd $DOCUMENT_ROOT && php $DOCUMENT_ROOT/modules/contrib/composer_manager/scripts/init.php && composer drupal-update

echo "Moving settings.php file to $DOCUMENT_ROOT/sites/default/..."
rm -f $DOCUMENT_ROOT/sites/default/settings\*
cp $PROJECT_ROOT/drupal/config/settings.php $DOCUMENT_ROOT/sites/default/
cp $PROJECT_ROOT/drupal/config/settings.local.php $DOCUMENT_ROOT/sites/default/
ln -nsf $PROJECT_ROOT/drupal/config/sync $DOCUMENT_ROOT/sites/default/config
chown -R www-data \$PROJECT_ROOT/drupal/config/sync

if [ -d "/root/.drush/files" ]
then
cp -Rf /root/.drush/files $DOCUMENT_ROOT/sites/default/
    chmod -R g+w $DOCUMENT_ROOT/sites/default/files
chgrp -R www-data sites/default/files
fi

This file essentially downloads Drupal using the dev.make drush make file. It then runs composer drupal-update to download any composer dependencies in any of the modules. We use the composer manager module to help with composer dependencies within the Drupal application.

Running the drush make dev.make includes two other Drush Make files, apa-cms.make (the application make file) and drupal-core.make. Only dev dependencies should go in dev.make. Application dependencies go into apa-cms.make. Any core patches that need to be applied go into drupal-core.make.

Our Jenkins server builds the prod.make file, instead of dev.make. Any production specific modules would go in prod.make file.

Our make files for this project look like this so far:

dev.make

core: "8.x"

api: 2

defaults:
  projects:
    subdir: "contrib"

includes:
  - "apa-cms.make"

projects:
  devel:
    version: "1.x-dev"

apa-cms.make

core: "8.x"

api: 2

defaults:
projects:
subdir: "contrib"

includes:

- drupal-core.make

projects:
address:
version: "1.0-beta2"

composer_manager:
version: "1.0-rc1"

config_update:
version: "1.x-dev"

ctools:
version: "3.0-alpha17"

draggableviews:
version: "1.x-dev"

ds:
version: "2.0"

features:
version: "3.0-alpha4"

field_collection:
version: "1.x-dev"

field_group:
version: "1.0-rc3"

juicebox:
version: "2.0-beta1"

layout_plugin:
version: "1.0-alpha19"

libraries:
version: "3.x-dev"

menu_link_attributes:
version: "1.0-beta1"

page_manager:
version: "1.0-alpha19"

pathauto:
type: "module"
download:
branch: "8.x-1.x"
type: "git"
url: "http://github.com/md-systems/pathauto.git"

panels:
version: "3.0-alpha19"

token:
version: "1.x-dev"

zurb_foundation:
version: "5.0-beta1"
type: "theme"

libraries:
juicebox:
download:
type: "file"
url: "https://www.dropbox.com/s/hrthl8t1r9cei5k/juicebox.zip?dl=1"

(once this project goes live, we will pin the version numbers)

drupal-core.make

core: "8.x"

api: 2

projects:
  drupal:
    version: 8.0.0
    patch:
      - https://www.drupal.org/files/issues/2611758-2.patch

prod.make

core: "8.x"

api: 2

includes:

- "apa-cms.make"

projects:
apa_profile:
type: "profile"
subdir: "."
download:
type: "copy"
url: "file://./drupal/profiles/apa_profile"

At the root of our project we also have a Gemfile, specifically to install the compass compiler along with various sass libraries. We install these tools on the host machine, and "watch" those directories from the host. vagrant rsync-auto watches any changed files and rsyncs them to the drupal8-container.

bundler

From the project root, installing these dependencies and running a compass watch is simple:

$ bundle
$ bundle exec compass watch path/to/theme

bower

We pull in any 3rd party front-end libraries such as Foundation, Font Awesome, etc... using Bower. From within the theme directory:

\$ bower install

There are a few things we do not commit to the application repo, as a result of the above commands.

  • The CSS directory
  • Bower Components directory

Deploy process

As I stated earlier, we utilize Jenkins CI to build an artifact that we can deploy. Within the jenkins job that handles deploys, each of the above steps is executed, to create a document root that can be deployed. Projects that we build to work on Acquia or Pantheon actually have a build step to also push the updated artifact to their respected repositories at the host, to take advantage of the automation that Pantheon and Acquia provide.

Conclusion

Although this wasn't an exhaustive walk thru of how we structure and build sites using Drupal, it should give you a general idea of how we do it. If you have specific questions as to why we go through this entire build process just to setup Drupal, please leave a comment. I would love to continue the conversation.

Look out for a video on this topic in the next coming weeks. I covered a lot in this post, without going into much detail. The intent of this post was to give a 10,000 foot view of the process. The upcoming video on this process will get much closer to the Tarmac!

As an aside, one caveat that we did run into with setting up default configuration in our Installation Profile was with Configuration Management UUID's. You can only sync configuration between sites that are clones. We have overcome this limitation with a workaround in our installation profile. I'll leave that topic for my next blog post in a few weeks.

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web