Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Nov 10 2020
Nov 10

You Might Also Like

The success of a CMS project ultimately depends upon the content it contains. No matter how “on-time” or “under-budget” a project might be, editors have to be able to create and manage content, and they need to be able to do it without feeling the need to pull their own hair out.

If editors on your team do not like the editorial experience, disillusionment will begin to set in, and that disillusionment can cascade throughout your organization. Remember, your editors are likely the most prolific users of your website. If they don’t like it, your CMS implementation is headed for the scrapheap.

How do you balance ease-of-use with the constraints of your design system? Are there common things that can be done that ensure editors have a good experience? And how do we define “good?”

What is “good?”

For this article, we’ll use the following standard: A good editorial experience is one that allows an editor to add content in accordance with the organization’s goals with minimal frustration. If there are strict standards and safeguards in place, an editor still feels empowered and does not feel the desire to circumvent these safeguards.

We are not shooting for a perfect editorial experience. That will never exist. This is still software we are talking about. It has to conform to a lot of requirements, and many of those requirements will conflict. But we want to be careful to ensure that we don’t make the perfect the enemy of the good, and the standard above allows for some compromises and tradeoffs.

You will also notice that this standard means that what is “good” can differ from organization to organization. Content goals vary, the preferences of editorial teams vary, and so we need to allow for some subjectivity.

But there are still some baseline commonalities we can recognize.

We want to be careful to ensure that we don’t make the perfect the enemy of the good.

Common best practices

What are some best practices that apply to any CMS and organization? These are good anchors to start with because, if you have these, it’s harder to get distracted by shiny new tools and approaches.

1. Consistency

Jumping from editing one content type to another should carry some familiarity. As much as possible should be the same from form to form, down to colors and font sizes. This gives editors less they have to think about, which results in less confusion.

Different fields and options might exist, but give editors some anchor points, so they don’t get lost. For example, have common fields, like Title and Author, appear in the same place on all forms that have them.

Do not have multiple ways to include images or videos. Do not have one text field that allows direct embedding of an img tag but another that only allows a special embed code. This is a recipe for frustration. Choose one way to do it, and stick with it, and if it needs to change, apply the change consistently.

2. Clear, relevant labeling

Fields should be labeled, and additional helper text should provide relevant context. This seems obvious, but sometimes it is so obvious that you can forget to pay attention to it. Often this text is written and placed by developers, and so they bring their own bias and context, and what they put in is, at best, unhelpful and sometimes downright confusing.

Helper text should make sense to editors and use the language they are familiar with. It should explain things in a way so they can predict the results of their actions.

Bad helper text: “This is an image field.”

Good helper text: “An image that will be displayed at the top of the article and be shown when the article is in a list. The minimum size is 1000x600.”

Bad helper text: “The categories this article belongs to.”

Good helper text: “Add links to one or more category pages. If you input 'apples, oranges,’ links to 'mysite/apples' and 'mysite/oranges' will automatically be displayed on the article, and this article will be included in the lists on those pages.

Keep things contextual, not just factual. Editors usually don’t care about technical details, but they do care about the hierarchy of the page and the effects on the site that they can see.

This is a place where usability tests can really help. Run them to find places where people are getting confused and craft text that addresses their questions and uncertainty. Make an intentional effort.

3. Make the right things easy and the wrong things hard

Your experience should be designed to encourage your editors to do the right things.

One common example: should all images embedded within content have “alt” text? (The answer to that is yes). Make “alt” text required for every image so the content cannot be saved unless it is filled in. For background images, they should have empty alt text so screen readers will skip over them, so just don’t show an alt text field for those and let your CMS handle putting in the empty attribute.

Should each update to content create a new revision so changes can be rolled back? If so, that should be easy to do. Make it hard for someone to mess this up: don’t even show them the option.

Don’t make onerous formatting requirements that editors have to ensure themselves. Make proper formatting easy.

Have you ever been frustrated by an online form that requires you to enter a phone number with exact spacing and parentheses and offers no help? It's like trying to put a puzzle together, but all the pieces are flipped to the side with no picture, and that’s an extra cognitive load your editors don’t need.

4. Clear, obvious error handling

If an editor does something wrong, don’t be coy. Don’t leave them guessing. State the error clearly and be specific, making the message hard to miss. Ideally, the error will show up next to the problem in question, though this isn’t always possible to do. Clarity is the most important.

Bad error message: “An error has occurred.”

Better error message: “The ‘summary’ field allows a maximum of 160 characters.”

Error messages should make it obvious that something went wrong, and some colors communicate “wrong” more than others. Therefore, don’t highlight errors in green. Also, take into consideration editors who might be color blind or have other visual impairments, so do not rely on color alone to signify an error.

5. Simplify to necessity

Plan and design for the 80%. Do not have lots of fields and options that most editors will never use. Do your best to slim down the experience to only what is necessary. Do not, in other words, present your editors with a form that includes everything, even the kitchen sink.

What about the other 20%?

Once you have figured out what that 20% actually is, a process driven by your business goals and not by what people think they want, accommodate the need.

For example, only a small number of your articles might have a Featured Video, so most editors will not use it. That field should not clutter their experience. But video is still important, so certain editors should be able to find it and use it with no trouble.

This can be accomplished via permission management. If only some editors need the field, you can hide the field from others by denying them access. 

If all editors need access, just sporadically, the Featured Video field could be moved under a tab labeled “Advanced” or to a sidebar labeled “Add Video.” The fields are there, but they do not distract from the main editorial path.

Long, confusing forms sometimes result from a single content type being responsible for lots of different contexts and page designs. An “article” might be used for a blog post, a news release, a video post with a transcript, and a podcast. This requires lots of fields to take into account the different uses. 

But this heightens the risk of confusion and burnout.

Sometimes, simplifying will mean revisiting your content model and having distinct content types for each audience and/or presentation. 

Having sensible defaults also helps simplify things. Default to what the 80% needs, but make it easy for the 20% to change that default.

6. Easy wayfinding

Good navigation and architecture aren’t helpful only for your site visitors. Your editors would also appreciate not feeling lost when they need to edit a specific page or content block.

Pay attention to the organization on the administration side. Are your editors forced to use your normal site search, the one for visitors, to find the content they need? Maybe that’s fine. If it isn’t, though, have you provided a good alternative?

Some of this depends on how big your editorial team is and how they work. If a team is responsible for a few categories, make it easy to filter by their categories. If you have a system of revision approval, make sure the process is clear and that the next steps are easy to figure out.

Inline editing can be helpful here, but it can also be overrated and difficult to integrate with certain workflows. As long as an editor has a clear path to get where they need to go, that is often enough.

7. Avoid busywork

Don’t enforce things “just because.” Don’t require editors to fill out fields that are not really required for any business logic.

For example, some systems have a revision log field that is required whenever content is updated. This can be helpful for large teams, but sometimes, the only person who ever reads those revision logs is the ghost who lives in the server. AKA “no one.” 

Don’t make your editors fill out stuff that nobody cares about. There might be legal requirements for you to store information, and in that case, someone obviously needs to care about it. The question then becomes: how do I make the process better?

Likewise, do not make every piece of content go through a five-step approval process if only one person needs to give approval. Do what is necessary to ensure standards are met, but having faith and trust in your team is better than arbitrary safeguards.

Drupal-specific tools to help

Drupal has a lot of functionality, out of the box, that will help you deliver a good editorial experience, like places for form element helper text, requiring alt text on images, good form validation, and the ability to set default values. 

Drupal core also gives you the ability to have different “form displays” for each content type, though taking advantage of them requires a bit more work.

Text formats and WYSIWYG

Drupal comes with a robust text format system, and editors can have access to different formats based on their permission level. Take advantage of these. Do not leave extra buttons on the WYSIWYG editor that no one uses. Keep it simple.

Contextual links

Drupal includes drop-down links for privileged users at various places on a site, allowing editors to easily jump to the edit form of a block or content type. Additional links can be defined with custom modules.

If you are creating listings with Views, Drupal provides a way to add contextual links to each result that is shown. This can be an easy win.

Shortcuts

Drupal comes with a shortcut menu that each editor can easily access. You can set up permissions in a way for editors to choose their own shortcut set, giving them some say in their own experience. Set up some sensible defaults for the most common tasks, and let them tweak. 

This is simple in concept but can go a long way in improving the overall experience.

Inline Form Errors

This module is included in Core but is disabled by default. When enabled, error messages are placed next to the relevant form elements themselves instead of just as a summary at the top of the form.

Contrib extras to help you craft a good editorial experience

There are other tools and groups of modules you should pay attention to when building an editorial experience in Drupal. Using them does not guarantee a good experience, but they can make it easier to achieve one.

  • Field Group - Perfect for simplifying your forms and hiding less-used fields. A great tool for catering toward the 80%. If you end up grouping fields according to a specific pattern(field type, whether a field is required, frequency of use,) stay consistent from content type to content type.
  • Field Permissions - Another way to limit and simplify your forms, but with more explicitness. Useful for larger teams with a hierarchy of editors where access to some fields need to be enforced beyond just moving them around visually.
  • Entity Browser - A widget for entity reference fields, this allows more user-friendly selection beyond simple autocomplete. If set up correctly, it also allows inline creation of other entities before they are referenced. This can drastically improve the usability for editors. However, it depends on good practices and discipline for field naming and descriptions, so you cannot ignore the basics.
  • Embed - You can also set up easy embedding for custom elements and structured data, tying them to a button on the WYSIWYG.
  • Entity Embed - Integrates with Embed to trigger Entity Browsers to enable embedding in text fields.  It’s easy to go overboard with this module and cause more confusion, so be intentional about what you use it for.
  • Inline Entity Form - Another widget for entity reference fields. Good for creating content to reference. If you don’t need to re-use content and don’t need to search for content to reference, this can work better than Entity Browser in some situations.
  • Allowed Formats - Drupal core allows you to limit text formats by user role, but Allowed Formats will enable you to do it per field. It is very common for a text field to allow only links, for example, enabling you to have the appropriate format selected and all others hidden. This is a great way to make the wrong things harder.
  • Entityqueue - Sometimes, editors need to be able to explicitly define the order of content on a page and not depend upon things like a published date or other metadata. This makes it easy for them to reference content and order it how they wish.
  • Automatic Entity Label - Used for content that is only referenced by other content, so editors do not have to worry about creating labels. This can help avoid some busywork.
  • Override Node Options - Limit access to default fields on nodes. This fills in some gaps from Field Permissions.
  • Chosen and Select2 - If you have long select lists, either of these can make them more usable.
  • Linkit - If your editors link to lots of internal content and files, this module makes it easier to perform those tasks.
  • Require on Publish - Sometimes there are fields that editors do not want to fill out until content is actually published, and they just want to save their work or view a preview. This has provided a huge boost in usability for some editorial teams.

When editors are happy…

There is no such thing as a perfect editorial experience, but you can provide a good one that will keep your editors happy and help them avoid frustration. Don’t ignore the most prolific users of your website.

If your editors are happy, it is much more likely that your business objectives will be met, there will be less turnover and burnout, and your CMS has a better chance of sticking around for a while.

At Lullabot, we have seen the good, the bad, and the ugly of CMS experiences. Part of our work in design and strategy is to figure out how to empower your editors and not leave them behind. And we would love to help you. Reach out if you have any questions

Nov 05 2020
Nov 05

Matt and Mike talk with Sascha Eggenberger about the Gin admin theme, including its editorial interface changes, relationship to the Claro theme, future and more!

Gin admin theme showing the node edit page
Oct 27 2020
Oct 27

An Accessibility Checklist for Your Website

This article is part 2 of a series that outlines key points to consider when designing and building more accessible websites.

It takes a coordinated team effort in making sure a site is accessible. Whether it's reviewing your design for color contrast, conducting usability testing to assure you address any usability issues, or developing for readability and proper text spacing, great teamwork makes for great accessibility.

In our previous article, we discussed things like color, links, and forms. In this article, we'll be discussing best practices around spacing, imagery, and typography.

Accessibility Checklist

Images-vs-Text - HTML text and using SVGs are useful for people who need to enlarge text or zoom in and out without dithering.

Font Choice - Readability is key.

Heading Order - Heading tags, when used in order, allows screen reader users to jump to sections based on the heading.

Text Spacing & Vertical Rhythm Unite Size (VRU) - Think about text and spacing before designing a site.

Text on Images - Provide color contrast between images and text.

Whitespace & positioning - Add balance of whitespace and large touch targets for images and text.

Images -vs- Text 

When in doubt, always use HTML text. When images of text are needed, ensure that the format allows for visual clarity when zooming in and out. Scalable Vector Graphics (SVGs) are suggested in this case.

The ability to zoom in and out is useful for people with visual impairment. SVGs scale without dithering and keeps things legible. SVGs are text-based and can be searched, indexed, scripted, and compressed. They can be created and edited with any text editor or drawing software.

Font Choice

When choosing typography for a site, readability should be the top concern. With standard fonts (i.e., Arial, Georgia, Serif, Sans-serif, etc.) being available on most user’s devices, it’s best to either use standard fonts for your site or to set a standard font as one or more of the backup fonts for the site. Check out WebAIM Fonts for more information.

Generally, long body copy is more legible if set in a serif face. The serifs help connect the letters and words, making it easier to read. Sans-serif fonts work well for shorter copy and headlines. That said, highly decorative serif fonts can be more challenging to read than san-serif fonts. Whichever you choose, make sure your fonts are clean and crisp and have nice letter spacing.

When choosing font-weights and styles, try and limit the number of font families & styles you use to improve site performance. Limiting yourself to two typefaces (a pair) usually works well to create a clear hierarchy within a type system. A great font pairing website to check out is Fontjoy.

Further resources when choosing fonts:

Heading Order

It’s important to remember that headings must always be in a logical order of H1, H2, H3, H4, H5, and H6. Also, there should only be one H1 heading per page. Although it’s a good idea to have the font sizing match the logical order of the headings, it’s not required, and the headings can have various font sizes.

Try and limit the number of heading sizes to four. This will help with consistency in how they’re used and rules around using them. Make sure to avoid any fonts smaller than 14px. Small font sizes can make legibility difficult, especially for those who are visually impaired. Think about how these font sizes translate to all the various breakpoints across your designs. 

Many designers begin by choosing a base font size (used for body copy) and determine the appropriate line-height. A line height that is 1.5x font size is a good starting point. This base font size and line-height can then used to determine an underlying baseline grid. All other font sizes used are multiples or are divisible by this base font size. 

Further typography resources

Text Spacing & Vertical Rhythm Unit Size (VRU)

When choosing fonts for a site, the ability for the fonts to adjust to the required text spacing per WCAG 1.4.12 Text Spacing must be taken into account. All text needs to be readable when the minimum text spacing settings are applied. Details for this standard can be found on W3’s Understanding Success Criterion 1.4.12: Text Spacing

There are two great bookmarklet tools, Text Spacing Bookmarklet tool (Dylan Barrell - @dylanb) and text spacing bookmarklet (Steve Faulkner - @stevef), that you can add to your bookmarks and use on any webpage to see how it would look with those spacings applied. Check out Knowbility Exploring WCAG 2.1 - 1.4.12 Text Spacing for more information.

A line-height of 1.5x font size is a good starting point. As mentioned earlier, designers usually start by defining their base font size and line-height to determine their underlying baseline grid. At Lullabot, we like to call the vertical spacing within a baseline grid a vertical rhythm unit, or VRU. 

Further reading on setting up your baseline grid

Text on images

Color contrast of text on images should always be tested when creating designs. Different techniques can be added to images to alleviate this issue, including darkening the image, adding a transparent background behind the text, or adding a solid background behind the text with a gradient to the image. Check out CSS Tricks: Design Considerations: Text on Images for more information. 

When testing color contrast, use a tool such as ColorZilla to check the background color closest to the text. Then check that color against the text color using a color contrast tool like WebAIM’s Contrast Checker. Text needs to meet a color contrast of at least 3:1 when it’s above 18.66px and bold and 4.5:1 when it’s below 18.66px.

Make sure the text is readable without sacrificing the background image quality. Maintaining both will help provide more context to what you're trying to communicate. Good design is a balance of visual treatments, usability, and communicating your message clearly.

Further reading and resources

Whitespace and positioning

Spacing between images should always be a consideration when designing a site in order to avoid confusion, especially for users with cognitive disabilities. Clickable images that are too close together become difficult to use for mobile device users as well. In tablet and mobile view, icon images should become larger for the user, making it easier for them to identify and click. Space should also be available for the user to easily scroll through the page without accidentally clicking on an active element on the page.

Negative or white space not only helps focus the user on the content at hand, but it helps increase its readability. Finding the right balance of space between elements helps to clarify relationships, adds emphasis, and ultimately makes for a better user experience.

The lack of white space results in a jumble of content and ultimately overwhelms users. It's hard to scan, hard to navigate, and eventually, your users may feel overwhelmed and leave your site.

There are two types of white space you should pay attention to; macro-space and micro-space. Macro-space examples are things like the white space above and below the header and footer of your site, as well as the outside gutters and between sections. Micro-space examples are those areas between titles and paragraphs and the leading space within paragraphs.

Further reading on spacing & placement

Wrap up

Let's be more thoughtful as teams by thinking about ways to make our sites more accessible. When we do, we will be enabling more inclusive experiences for everyone.

And finally, here's a great article to help you advocate for greater accessibility in your workplace, and for more tools to persuade, check out our article on the positive business impacts of improving accessibility

Oct 21 2020
Oct 21

You Might Also Like

The Drupal 8 to Drupal 9 upgrade path represents a big change for the Drupal world. The big change is that…your organization’s website won’t require a big change. The first iteration of Drupal 9 is just Drupal 8 with all of the deprecated code removed, mimicking Symfony’s model of major release upgrades.

This is good news. Keeping your platform up-to-date with the next version is no longer an “all hands on deck” situation.

As with all changes, however, this new model comes with its own challenges and problems. You will need to shift your own thinking and habits. When it comes to your Drupal website, your organization will need to begin running a marathon that goes on for years. Your relationship with outside vendors will take on a new cadence.

In this brave new world where upgrading to the next major Drupal release isn’t a big re-platforming effort, what does web development strategy look like?

Establish a cadence for web development work

One good thing about major re-platforming efforts is that they have to be planned for in advance. Budget and time have to be allocated. It acts as a huge celestial star, with everything else gradually falling into orbit around the big initiative. It draws out intention, direction, and sometimes enthusiasm, and none of these are bad things to have. 

Having this large, common goal that everyone sees with clarity can make a lot of this stuff come more easily, but now, you need to figure out how to create and harness these things while not depending on the existence of a monolithic target that dominates the landscape. And you need to maintain what you have at a sensible pace.

Planning out the release cycle

 Like Drupal 8, Drupal 9 requires frequent updates. To get the latest security updates, you need to stay on the latest minor point release (9.1, 9.2, etc.). With this, you also have the possibility of getting new features that have been included, and you should be aware of them. Our article on Drupal 8 release planning is still relevant for Drupal 9. In summary:

  1.  Build a schedule of releases and support windows for your software. Not only for Drupal 9 but also for any contributed modules and other software that is part of your hosting stack.
  2.  Schedule updates ahead of time, and do not let the desire for new features cannibalize these dates. These should be scheduled from the top by project managers. This might mean a sprint every month or so 100% dedicated to updates. These should be just as visible as other initiatives that are being developed and be treated as equally important.
  3. Promote any new features that were rolled out by these updates.

Planning new initiatives and features

Ideally, with a more iterative approach to development, stakeholders stay more involved and informed, and therefore better discussions can be had around the website. With the old Drupal upgrade model, there was a risk of the dreaded stakeholder swoop: someone swoops in, lists a bunch of requirements without regard to overall goals and priorities, and swoops back out. Sometimes they aren’t seen again until close to launch.

That risk still exists but is mitigated by the extra number of touchpoints required from a more long-term, iterative approach. There are more starts and mini-launches. If a typical swooping stakeholder wants to get something done, they will have to do a lot more swooping, which might start to look more like informed involvement.

Regardless, you need to be more intentional with planning out and prioritizing new features. You can’t let every neat idea, frustration, or new design collect in a bucket over the course of two years, only to implement them on the new platform. With iterative development, your organization will need to communicate more, not less.

Set up regular touchpoints with stakeholders and domain experts. This will look different for every organization. The Marketing department might be the main driver. In that case, you’ll need to regularly meet with the person who has the authority to make requests and set priorities, in addition to domain experts that can answer questions and provide deeper knowledge. In smaller organizations, this could all be invested in one person.

If your website represents the needs of many different silos, like a company selling multiple products, then you will need regular meetings for each different product team. The same standards apply. You need someone with authority and domain knowledge.

These meetings should match up with your project management philosophy. For example, are you running sprints via agile? Invite these folks to sprint planning. Requirements with large uncertainties can trigger the creation of a targeted discovery phase with stakeholders, which becomes part of a sprint, just like all of the other tickets.

Regular usability tests are another way to find potential improvements. These don’t have to take a lot of time and money. The book Don’t Make Me Think outlines a simple framework that anyone can implement, and running them once per month is usually enough to fill any gaps in your pipeline.

Re-factoring and technical debt

Codebases tend to gather junk over time. This comes in the form of disorganized code, “temporary” fixes that have become permanent when no one was paying attention, things that work but could use performance tuning, and modules included in the codebase that aren’t used anymore.

Previous upgrade cycles allowed messes to be stacked up into a closet somewhere. When the migration or re-platforming came along, everything from that closet could be safely dragged out and lit on fire. Easy clean-up.

With an iterative model, you can’t afford to keep pushing things off. Eventually, that closet will need to be so big that it takes up the entire house.

Start making an inventory of things that need to be cleaned up, and start adding these as tasks for your team. Maybe your goal is to complete two technical debt tickets per month. Maybe you start smelling something really foul and need an entire sprint dedicated to a refactor. Maybe a new feature will be easier to implement if some other code is reorganized, so you add that task as a pre-requisite. 

However you do it, do it with intention and planning. That closet is not going to clean itself.

With an iterative model, you can’t afford to keep pushing things off.

Allocating development resources

You have planned out the release cycle. You have a list of new feature requests that is constantly growing thanks to your increased communication with stakeholders. And you have identified good targets for refactoring. 

Now what?

Prioritize and allocate. This, of course, depends on your team's size and the number of stakeholders involved in the work. Your project managers have their work cut out for them because they also have to worry about allocating themselves properly within the changing tides of shifting priorities.

You might give each stakeholder a team they work with exclusively. This helps people grow comfortable working together and builds rapport. You are less likely to need project kickoffs each time something starts. 

You might also rotate developers and teams so everyone has experience doing a bit of everything. That way, there is some overlap in case of emergencies or turnover. It also keeps things fresh and can aid in preventing burnout. Let developers speak up and tell you what they’re thinking, and, if possible, indulge their preferences.

It can be helpful to have the same person in charge of the release schedule day after day, month after month. But does someone actually want that job day-to-day? Maybe they do. But you should be sure.

For smaller teams, you may have to split things up by month or quarter. Maybe November and December are the months everyone focuses on technical debt. Maybe the first quarter of the year is reserved for new design initiatives and higher priority feature rollouts.

But do not let security and software updates get lost in the shuffle. Do not get lost in the weeds of zealous re-factoring. Do not ignore the needs of your stakeholders. This can feel like a juggling act, but it is one your organization must master to keep your website secure, relevant, and successful.

Celebrate

There is no more “big reveal” and launch of a new website. The highs and lows, the victories and stress, are hopefully flattened out to more manageable peaks and valleys. That doesn’t mean you can’t celebrate the completion of smaller initiatives, however. You absolutely should. Announce them, treat them as huge milestones, make your hand sore from giving so many high fives and pats on the back.

This can be done in many ways.

  • Public recognition in meetings and internal newsletters.
  • Demos and learning days where people responsible for the work show off what they have done and learned.
  • Small parties that happen immediately after lunch.

Whatever fits within your culture, do it.

Since each initiative and feature stands more on its own, it doesn’t get drowned out in the excitement of a “big reveal” where the gloss of so many new features can blind people from seeing certain parts of the work that have been done.

Iterative releases allow for more focus. And they give more opportunities for kudos. Take advantage of them.

Revisiting site architecture and content

Historically, many organizations have used major Drupal version migrations as a cadence to review information architecture. In part, this has been because content migrations across major versions haven't always been 1:1 migrations, so organizations have had to undertake information architecture (IA) work to decide what to migrate and where it should go. 

The other aspect of this is that in more complex migrations, it might have been faster to remove old content and deprecated content types than to spend the time migrating them, especially in the event of a custom migration.

The upside of migrating from Drupal 8 to Drupal 9 and beyond is that there is no content migration. The downside is that organizations now need another cadence for undertaking information architecture and strategy projects. This is similar to creating a new pattern of web development work, and a lot of the tips for development also apply here.

In fact, architecture and strategy should drive most new development work. Get into the habit of doing periodic IA and content audits. Every year or every six months. Whatever makes sense for your organization. It’s part of having a well-kept house. 

Some example questions you could start asking:

  • Is historic content still meeting your needs? Can it be updated, moved to another content type, or archived?
  • Does the site contain one-time-use fields that should be deprecated?
  • Do content types need to be trimmed?
  • Have the site's navigation needs changed?
  • Does our taxonomy structure still meet our needs and goals?
  • Have we added new departments or consolidated older ones? Are they represented properly?
  • Has our audience shifted? Have we started targeting new audiences?
  • How has our business changed? Have new competitors required us to think of different approaches and goals?

Sometimes your IA might require a major shift, and that shift needs to happen to a website that needs to remain up and running. For example, consolidating two content types into one new content type.

The good news is that the robust, well-tested migration tools that would have aided you during a full upgrade are there to help you accomplish a smaller shift. Drupal migrate tools are great at Importing content and pouring that content into different structures. Take advantage of them, even if it might feel like overkill at first.

Architecture and strategy should drive most new development work.

Bringing in outside help

Large upgrades and migration efforts demanded expertise and more developer hours, but without this traditional demand, is there still room to hire outside help?

Yes, there is. It can make a lot of sense, depending on your situation. In many circumstances, establishing a longer-term relationship with a vendor can yield even more gains, as the external team isn’t just around for six months to a year, then off to the next thing. Trust has time to grow. Communication settles into familiar rhythms. Everyone involved becomes more comfortable with each other.

Long-term engagements were still possible before the Drupal 9 paradigm, of course. At Lullabot, we have worked with some of our clients for 5+ years, which has enabled us to contribute in unique ways.  But without the inevitable, looming migration in the distance, it allows for additional possibilities, and we are excited about the potential.

There are many ways outside expertise can be utilized beyond a big project crunch. Keep in mind the lines between the following categories are fuzzy, but they are good places to start when trying to determine if you want to hire an external vendor.

Support and maintenance

If you want your own team to focus on new features or work you consider higher-value, using external help for support and maintenance of your existing infrastructure can provide a level of consistency that allows you to forge ahead.

Put a specialized team in charge of managing your release cycle. They manage the software updates and work within your schedule. 

A skilled support team can also help manage bug fixes. This frees up your developers’ time, so they aren’t bouncing around between tasks, losing a little bit of productivity each time. Help them avoid the concentration whiplash.

How do you know you might want help with support?

  • Assigning responsibility for the release cycle is like a game of hot potato. No developer really wants it.
  • The backlog of “urgent” bugs is growing month-over-month instead of shrinking. Complaints from stakeholders begin to grow.
  • Your software is continually months out of date, and it requires everyone’s attention to rectify the situation.

Fill gaps of expertise

Even if you have a large, diverse development team, you probably have some experience gaps. Technology is complicated. It’s hard for a team to stay up to date with everything that’s going on all of the time, and it can help to bring in some specialists.

Security audits. Accessibility audits. Performance audits. These are good opportunities for someone to come in, give you some actionable items, and ride off into the sunset. Each round of these helps educate your staff, as well. Depending on the scale of your website, it might make sense to augment your team with this type of experience for long-term engagement instead of periodic audits.

DevOps and continuous integration is also something that can benefit from a dedicated resource. A good expert in this area can help make your entire team more productive. They can set up and maintain the automatic deployment and testing of code, manage local development setups, and help enforce best practices.

Content strategy and design are areas that can benefit from outside perspectives. Good talent in these areas can help make your internal projects more successful by forcing you to clarify priorities.

Increase development velocity

Sometimes, you just don’t have the resources required to meet your goals. Too many initiatives with too many people demanding their pound of flesh. These requests and requirements could all be funneled through the Marketing department, or maybe your company has multiple departments that each own a section of the website.

Either way, you need more help, and you need that help to hit the ground running. An expert team can be integrated in several ways.

  • They can augment the current team with no change in structure. With this model, they become additional team members for you to utilize, whether they are project managers, content strategists, designers, or developers. The aim is to increase the general velocity of your development work.
  • They can come in with more focused intent as a differentiated team. They are assigned to a stakeholder or a specific initiative, so important work can get pushed forward without interrupting your normal development workflow. Close collaboration can still happen, but the external team has different priorities they will focus on.
  • A mix of both paradigms. There are no hard and fast lines to draw, and being flexible has its advantages. For example, after a team has completed a specific initiative, they move on to another one, or the team is split up and dispersed throughout other internal teams so domain knowledge can be spread. Or maybe they move to a support role.

Brave new world

Despite not having a big re-platforming effort on the horizon, web development in a Drupal 9 world still requires planning, thought, and intention. Release cycles need to be managed. New work needs to be planned and developed. Stakeholders need to be kept happy.

Drupal 9 makes it easier to take advantage of new features while keeping your site secure, successful, and relevant, but you can’t push things off anymore. No more waiting for the big migration to get rid of technical debt or re-work your information architecture. No more sweeping things under the rug until spring cleaning.

New habits need to be formed. New cadences need to be implemented. Exciting times are ahead as we juggle with this new reality.

Oct 15 2020
Oct 15

Perspectives of Women in Technology

To highlight and share stories of women in technology, we've rounded up a group of Lullabots to talk about their journeys into technology, which tools they use on a regular basis, the skills they're learning, and what advice they'd share with their younger selves. 

Here are the stories of Katherine "Kat" Shaw, Senior Front-end DeveloperApril Sides, Senior Developer, and Jaden Johnston, Head of Business Development at Tugboat.qa.

Tell us about your journey into technology.

Kat: I started working in web development 20+ years ago to raise my now 22-year-old son at home. I'm self-taught, and I've had the pleasure of working for several great companies and organizations, including Spencer Reed Group, Autodesk, Douglas County, Kansas, and now Lullabot. I've built wonderful friendships over those years, and I've developed a lot of useful skills that I carry with me now. While working in the local government, I honed my accessibility skills and became an ardent advocate in making the web equal for all. 

April: I initially wanted to learn 3D animation in college but realized I didn't quite have the patience for it. Along the way, I discovered web design and development. However, as I reached graduation, I was more interested in print design. 

I had an internship that turned into a part-time job out of college and kicked-off my career doing print and web design/development. I went on to work in higher education, doing both print and web work, but then decided I wanted to focus on back-end development. 

I moved from Communications to the IT department, but I ended up spending a lot of time in Sharepoint. It was then that I decided I wanted to be a Drupal developer. I responded to an obscure Drupal Developer ad on Monster.com, which ended up being a government contractor job working on websites for the United States Global Change Research Program.

After a few years in the federal government, I decided to try remote agency work. I spent a few years at Mediacurrent learning Drupal 8 and trying my hand at a little project management. Looking for a change in environment, I took a chance and applied to Lullabot. I feel fortunate every day to be right where I'm supposed to be.

Jaden: My professional journey through tech started with an interesting, albeit necessary, stint in tech support. My first real tech job was in a network support center supporting back-end processing for banks and financial institutions. I learned so much in this position and quickly found that this was the road I was meant to be on and grew much of my seemingly-natural technical ability. After a few years of moving up the career ladder of that company, I knew that I wanted to focus more on the web development industry. I'd been dabbling in building small websites, graphic design, and photography, so I decided to jump ship from tech support and take any role I could in a creative agency setting.

Landing my first agency job as a Production Coordinator helped me develop and hone in project management skills, and I was grateful to have had technical experience and be well versed in early web-tech (like pre-CSS days!), so I could communicate effectively with dev teams and translate client needs into technical requirements for web and mobile projects.

I wanted to grow my ability to manage a lot of things at a time and had a desire to get involved in every aspect of the business that I could, so I learned best practices for resource allocation and agency traffic coordination. Eventually, I began to participate more regularly in business development activities and became the bridge between sales and production. I knew that I wanted to take on leadership roles at some point, so I challenged myself and took calculated risks to accelerate growth.

I crossed over from client services agency to product and doubled down on my selling skills. This role grew from inside sales to management to directing a cross-functional team and focusing on operational excellence, which allowed me to find even more of my calling in leadership, coaching, and helping others get better. Joining the Tugboat/Lullabot team is a dream come true. I am able to leverage nearly every skill in my repertoire AND gain new ones. There's always opportunity; you just gotta go get it! 

Which tools do you use on a regular basis in your work, and what types of skills are you currently learning?

Kat: I use several accessibility tools, including Color Contrast Checker, WhatFont, Axe, etc. I also use debugging tools in Drupal in order to troubleshoot coding issues during development. I'm currently focusing on CSS Grid, CSS Variables, BEM (Block Element Modifier), and React.

April: For communication, I use Slack, Google Meet, and Zoom. For planning, I typically use Google Suite, Dropbox, and Jira. For development, my current toolset includes a MacBook Pro, Drupal, Docker, Lando, PHPStorm, XDebug, Drush, Git, Bitbucket, and Tugboat.qa. A few other tools include Postman for authenticated API exploration, Monosnap for screenshots, and CopyClip to manage my clipboard.

Best tool of all: Google search.

Jaden: I have morphed from technical support to project management, business development, and now am using all of those strengths to lean into operations and leadership. I am driven by empowering and unblocking others to do their best work. My idea of success is seeing others grow.

I am currently learning how to unlearn and am realizing that a lot of the old things I was using in my previous role no longer apply. I need to adapt and grow and literally unlearn some of the methodologies and approaches I used in the past. It's challenging because you can come to trust yourself when you've got tried and true methods for certain things, but you cannot allow yourself to become inflexible and cemented in your ways.

What advice would you give to your younger self?

Kat: Keep up the hard work. You'll be fine, and it will be worth it in the end. Know your value!

April: Negotiate your salary. If you don't ask, the answer is always no. Figure out who you are, and don't be afraid to be yourself. Realize your personal values and use them to guide your life and career. Know that this quote is true: "Be who you are and say what you feel, because those who mind don't matter, and those who matter don't mind." - Dr. Seuss

Jaden: Trust your gut. Lead with empathy. Never stop learning. Never assume. You got this.

What's important to you when you build, create, and use technology?

Kat: To build things with accessibility and mobile-first in mind, keep things clean and tidy, to indent properly, document properly, and always focus on learning new things so that I'm always growing. Also, always share what you know with others. That's what it's all about. 

April: I like to solve complex problems and build solutions that solve problems for people.

Jaden: That it works the way I want it to. Just kidding, sort of. It needs to work for all of us.

What's a fun fact, favorite hobby, secret talent, or big goal you'd like to share?

Kat: I know a lot of unimportant things about movies, tv shows, and music, although I find it fascinating. My brother and I speak in a language of quotes from movies, and I can recognize a lot of songs after hearing just a few notes. 

I love to learn! I constantly look up people on Wikipedia to learn more about them, regardless of who they are. A big goal for me is to KonMari my life because I'm a big believer in "less is more," and I want that for my family and me. Another goal was achieved by becoming a member of the Lullabot team!

April: I'm a huge LEGO and Harry Potter fan. I collected all 22 of the 2018 LEGO Harry Potter/Fantastic Beasts minifigures with only two duplicates. I also posted a photo series from the 2019 LEGO Harry Potter Advent Calendar, and I'm excited to do the same for 2020. 

Jaden: I love to sing. It's cathartic, and there's nothing like that feeling.

Do you have perspectives to share? We'd love to hear them as we're all learning, growing, and getting better at our craft. Leave us a comment or tweet us @Lullabot.

Oct 07 2020
Oct 07

About host Matt Kleve

Portrait of Matt Kleve

Matt Kleve has been a Drupal developer since 2007. His previous work in the media sparks a desire to create lean, easy to use workflow processes.

About host Mike Herchel

Thumbnail

A senior front-end developer, Mike is also a lead of the Drupal 9 core "Olivero" theme initiative, organizer for Florida DrupalCamp, maintainer for the Drupal Quicklink module, and an expert hammocker

Sep 16 2020
Sep 16

You Might Also Like

Olivero is a new front-end theme for Drupal 9, meant to be included in Drupal core as the default. Several Lullabots took an initial idea and, on a tight timeline, designed and developed an MVP of the new theme so it could receive feedback from the wider Drupal community.

We are familiar with typical design projects, where we help design something specific for a single client for a specific purpose. But what happens when this specificity goes out the window?

When designing a theme for an open-source CMS, one that anyone can use for any kind of site, that familiar specificity disappears. Defining the audience and the stakeholders for a project like this is tricky, and some solutions can create new problems.

In other words, here be dragons. And picture that phrase in bold, elaborate type.

In many ways, designing a new core theme for Drupal represents the worst-case scenario for a design process. This does not mean the experience was negative or bad. It simply means there were many unknown unknowns. Many intersections where the project could fail. No clear stakeholders to provide things like clear goals and direct feedback. These challenges can introduce chaos into a design project.

But it is at the margins where the most learning and growth take place. The more difficult the problem, the more creative the problem-solvers need to be. New experiences forged in these circumstances can help bring new insight to more typical, day-to-day scenarios.

So how did we navigate this scenario? And what lessons did we learn along the way? 

The first stakeholders

You have to start somewhere, and in this case, we started with ourselves. Members of our team saw a need for a new theme for Drupal. In his keynote at DrupalCon Seattle, Dries Buytaert shared how experts still love Drupal. Beginners…not so much. 

We know through research that at least some of that negative sentiment is due to a dated out-of-the-box experience. The default theme, Bartik, is now ten years old. This is quite an accomplishment. Creating something that lasts for ten years, especially in technology, is commendable. Nevertheless, there are needs not being met.

To get started, we needed to approximate some stakeholders, and so we, as users and practitioners of Drupal, became the first stakeholders. We also needed to establish some high-level goals to define what we were hoping to accomplish.

  • An updated modern design. The new theme needed to feel current. Our hope was that it could last for five years.
  • Functionality that supports new features. The new theme needed to support new Drupal functionality like secondary navigation, embedded media, and Layout Builder.
  • WCAG AA Conformance. The new theme needed to pass Drupal’s stringent accessibility standards, with things like proper contrast, large enough typography, proper focus states, and more.

We could not remain the only stakeholders forever, of course, but you have to start somewhere.

What are we designing?

We also needed to establish a scope of the project and define what would be required for an MVP and what were just nice-to-haves. 

We used Bartik as a base to prime the process. Working closely with our development team, we compiled a list of all theme components in Bartik and how they could appear, complete with examples and screenshots. This helped bring an ephemeral idea down to the ground and give it some weight. Something we could point to and look at.

Once we had this list in a good spot, we created quick wireframes to review with the developers to ensure we had captured all the important bits.

Defining the audience

Who is the audience for this product? This is one of the fundamental questions for any design project and it is usually not that difficult to answer. When designing an open-source CMS theme, or something that allows lots of flexibility, defining your audience becomes a challenge.

There is a narrow path we tried to walk in answering this question, and on either side, we saw potential pitfalls.

Don’t try to include everything for everyone

In these kinds of projects, there can be a strong temptation to include everything. There are so many potential users, and Drupal can do so much. We want to please everyone!

It’s not something we could accomplish, even if we wanted to. The needs are too varied. Trying to include everything for anyone would result in something that’s unwieldy or unusable for everyone.

We needed focus and intent. 

Don’t design for the average

This is sometimes known as the “average man problem.” In the late 1940s, the US military realized a large number of pilots kept losing control of their planes, and this problem showed up across different types of aircraft. Too many planes were crashing.

For three decades, the design of cockpits conformed to the size of the average man. They first tried to update the averages, just in case the average man had gotten bigger, but they discovered something more important. There was no such thing as an “average man.” 

Not a single pilot fit within the most important dimensions of a cockpit, and so most cockpits had been designed for someone who literally did not exist.

This eventually led to adjustable yokes, seats, pedals, etc. The book, The End of Average, describes the details, and an episode of the 99% Invisible podcast also covered this topic.

If we couldn’t design for the average user, who could we design for? What would we need to make adjustable?

Establishing design principles to measure against

First, we defined some design principles that helped describe what success would look like in a memorable way. These principles also needed to hold true as we learned more about our audience and stakeholders, so they needed to be flexible and high-level.

The principles we settled on:

  • Simple - Avoid unnecessary visual elements, colors, effects, and complexity.
  • Modern - Take advantage of modern browser capabilities and interaction patterns, like the Intersection Observer API.
  • Focused - Embrace high contrast and color saturation for drawing attention to what’s important. Design defaults for the 80% audience.
  • Flexible - Provide options and overrides for varied needs and preferences.

What can we say about our audience and their sites?

There is an ocean of things that you just cannot know about people who will use this theme, and yet another ocean that could hold all of the different sites they could try to build. But there were a few things that we felt were safe to assume about our audience and what they would build.

  • Includes beginners - People new to Drupal. The default theme gets used by people kicking the tires. 
  • Limited resources - Both time and expertise. Probably no programming or design skills. If you do have these skills, you are probably making a custom theme. 
  • More limited content models - The base content types would be doing a lot of heavy lifting, so the theme needed to serve the base content types really well.
  • Varied content - Despite a simple content model, things like title and label lengths, as well as the number of menu items, can vary widely.

Defining our stakeholders

We couldn’t remain our only stakeholders forever. That was good enough for the initial push and motivation, but we couldn’t assume to speak for the whole community. We at least needed a better approximation to start building initial consensus, but out of the thousands of potential stakeholders, many of whom are passionate about Drupal and its direction, how did we narrow the choice?

We started asking these questions:

  • Who will provide feedback, when, and how?
  • What kind of feedback are we looking for?
  • What does approval look like? Who will approve which designs, and how?

Establishing a proxy group

To reign in the numbers and get some focus, we established a proxy group. This group included various Drupal core maintainers as well as our team of designers and developers.  We wanted to do as much due diligence to weed out potential problems and refine our assumptions before putting it in front of the larger community so that eventual community feedback would be more focused.

This group led to a lot of changes in the designs, even after some of the development work had begun.

Some of the things that changed:

  • Initially, we had a style for uppercase labeling, used for things like RSS feed labels and share labeling. Because of localization and other varied uses, it was not possible to guarantee conciseness. 
  • Color palettes as we tested for accessibility.
  • Form states were refined and refined for visual clarity and accessibility while maintaining the language of the design.

Expanding our stakeholders

Eventually, we needed to engage the community for feedback, so we did some work to define what type of feedback we were looking for, along with a timeline. Making it completely open-ended could open the floodgates and drown us in too much noise. 

We published an overview of our journey and our guidelines for feedback and then continued the refinement process.

The community feedback did not feel like an overwhelming onslaught, which felt like a success and validation of our process up to this point. They provided great insights and helped catch a few things that we missed, but the work we had done with the proxy group had been effective. 

Developing the visual approach

What should the new theme look like? This was the first question to answer. To do that, we facilitated discussion and aggregated the results of those discussions.

Finding the voice and tone

The proxy group consisted of stakeholders located in multiple timezones who already had full days. In a normal design project, our stakeholders typically prioritize their involvement because, at some level, it falls within their day-to-day responsibilities. This project, however, was outside of all of our stakeholders' immediate responsibilities. 

We needed a less time-consuming, but high-impact exercise for our proxy group to do on their own time without a specific due date.

A spectrum analysis, which involves a list of sliding scales, going from one adjective to the other, worked well. Though each adjective had a positive connotation, each pair of adjectives conflicted (formal vs. casual, warm vs. cool), and forcing people to choose helped initiate some really good discussions. The “why” is more important than the actual selection, and everyone reserved the right to change their mind.

We usually do this exercise in person, but given the unique scenario and stakeholder makeup, that was impossible. Instead, we used Invision’s comment functionality to create the actual points on the scales, which made it easy to ask follow-up questions.

Here are the results of the exercise:

This exercise delivered the voice and tone we should aim for:

  • Formal
  • Light & Bright
  • Contemporary
  • Professional
  • Approachable
  • Novel
  • Cool
  • Familiar
  • High Contrast

Visual exploration

Our team uses Zoom Mocks for our projects, an alternative to style tiles, to explore different visual approaches. To use this paradigm, you choose a wireframe, then zoom in on a section of that wireframe to get into specifics. When you settle on a direction, you move to a different section of the wireframe and gradually build out the context. Styled elements become part of that wireframe.

We were not looking for an official sign off of “yes” for a particular style, as that would have required a larger feedback loop. Our timeline did not allow for that. It also would have required more involvement from our stakeholder proxy group. Instead, we depended on more directional feedback (“looks good so far”), understanding that the design would evolve and change as the system extended to other pages and components.

Typography challenges

Finding suitable type for the theme presented a challenge of its own. We wanted both a serif and a sans-serif font face, and the chosen fonts had to meet the following requirements:

  • Be widely available.
  • Utilize the SIL Open Font License to be compatible with Drupal’s packaging requirements.
  • Have a wide variety of font weights to fit various contexts (while being careful not to go overboard for performance reasons.)

In the end, we chose Metropolis for our san-serif font and Lora for our serif font.

 Creating the design system

This is where the rubber started to hit the road. To begin creating the design system, we started with the out-of-the-box landing page and the article page, which we assumed would be the worst-case scenarios from the least amount of content to the most content.

For the article page, we created a hypothetical blog article announcing the new theme, and this allowed us to work out the more complex patterns of the design system. We refined typography usage: headers, quotation copy, body copy, and captions. Other patterns received attention as well, like tables, lists, and comments. This allowed us to see how everything worked together in a common context.

After that, we moved on to secondary navigation, the search bar, and the branding area. The navigation, in particular, had to be flexible, as it was one of those elements that epitomized potential chaos. There are no inherent limits to the number of items you can add to a Drupal menu, nor inherent limits to the length of the text of a single menu item. We could make no assumptions on content in this area.

To solve this problem and make it easier to bring the navigation down to tablet and mobile screen sizes, we wrapped the entire element in a hamburger element after it hit a max-width. Doing this kept things clean and orderly regardless of the number of items crammed into the main navigation.

To accommodate Layout Builder and the various ways content is placed on a page, we made sure to establish good spacing and vertical rhythm., which dovetailed nicely with the responsive grid system. Altogether, they helped create a balanced and predictable flow on the page, one that scales well based on the content available.

To the future

Olivero is still a work in progress. As more people use it and feedback comes in, the theme continues to evolve, which is great. Olivero is a theme for the community, and it has a broad target audience that is itself shifting and evolving.

Designing for this large and varied audience proved a challenge, but a fun one. We are proud of the solutions we created, and we feel like we hit all of our initial goals while remaining true to our design principles. The theme is simple and modern, provides good flexibility, and meets our high standards for accessibility.

Finding some order in the chaos allowed us to use the design expertise and services we bring to more typical projects and to use them in creative ways. We hope our contribution serves the Drupal community well for years to come, and we look forward to the future of Olivero.

Sep 02 2020
Sep 02

You Might Also Like

With the announcement of an extension on official community support for Drupal 7, now through November of 2022, many organizations were able to let out a sigh of relief. Their investments in Drupal 7 will continue to serve their needs. 

In addition, Drupal 7’s Extended Support(D7ES) program has announced its vetted vendors for paid support of Drupal 7 through November of 2025. Drupal 7 is here to stay for a long time, and this is good news, especially for organizations with limited time, limited budget, and limited support staff. If you are one of those organizations, you can feel safe and secure. Settle in for the long haul. Continue to improve your existing site, knowing you have plenty of time to see the ROI you need.

But eventually, like all good things, Drupal 7 will come to an end. The destination is now further down the road, and you can enjoy the journey at a more leisurely pace than expected, but eventually, you will arrive at the end of the path. You will need to transition away from Drupal 7.

You must eventually let go and say goodbye.

Since this sunset period is now longer, it is a good idea to take advantage of it. Without the threat of impending deadlines, you can use this extra time to start preparing your website for its inevitable evolution and do it in a calm, controlled, and orderly fashion. Some effort now can reduce risk and pay dividends later.

What are some things you can start doing to prepare your Drupal 7 site for migration?

We’ll cover the following topics:

Begin to Shift Your Coding and Site Building Culture

There is a vast difference between the architecture of Drupal 7 and the architecture of Drupal 8+ (currently Drupal 8 and Drupal 9), and these differences will only grow with subsequent versions of Drupal. This is reflected in how custom code is written. 

For example, object-oriented code and paradigms in PHP are here to stay, and your team should begin to adjust to this new reality, if they haven’t already. 

Because of how Drupal 7 works, it resists a lot of OOP best-practices. But there are still things you can do to help your efforts down the road. These apply for both new features and refactoring of old code.

Move Business Logic to Classes

Instead of having long walls of code crammed into hooks or preprocess functions, start to encapsulate your business logic in well-named and organized classes. This helps improve readability and maintainability. It will also be less of a lift to move this logic to a newer version of Drupal.

Form alter hooks are a common way to customize Drupal. You can’t escape hooks altogether, but you can improve their clarity and make them less intimidating.

Instead of something like this:

function example_form_alter(&$form, &$form_state, $form_id) {
  if ($form_id == 'example_node') {
    $form['example_field'] = [
      '#type' => 'textfield',
      '#title' => t('Example field'),
      '#weight' => -10,
      '#maxlength' => 60,
    ];
  
    $form['submit'] = 'example_submit_handler';
  }

  if ($form_id == 'example_user') {
    // different stuff is added or hidden
  }

}

You can start using something like this:

function example_form_alter(&$form, &$form_state, $form_id) {
  $exampleFormManager = new ExampleFormManager();
  $exampleFormManager->alterForm($form, $form_state);
}

The ExampleFormManager class should then have all the methods necessary to determine when and how to change the form. If everything is well-named and documented, this will make it easier to port over to Drupal 8+. 

Write these classes for your future selves. In a year or two, you don’t want to be looking at this code, scratching your head, silently cursing the person who coded it.

Another opportunity to wrap business logic in classes is with entity wrappers. Let’s take the example of related articles for a node. In a theme preprocess function, you might grab a node’s taxonomy term, pass that to a function located in some other module, load the related nodes based on some criteria, and then format them back in the preprocess function.

Instead of this, you could create an Article entity wrapper:

class Article {
  private node;

  public function getRelatedArticles() {
    $related_nids = $this->getRelatedArticleNids();
    return $this->getRelatedArticlesRenderArray($related_nids);   
  }

  private function getRelatedArticleNids()  {
    // run the queries or pass to other internal function
  }

  private function getRelatedArticlesRenderArray(array $related_nids) {
    // construct a render array
  }
}

When the time comes to eventually migrate, you won’t be combing through a multitude of module and theme folders to figure out how in the world your site populates related articles. It will be there, packed up, and ready to move.

This process not only helps your code be more organized and portable, but it also helps your team start stretching beyond procedural modes of thinking, if they haven’t already. Shifting to Drupal 8+ won’t be as big of a shock.

Gravitate Toward Solutions that Rely on Structured Content 

At its most basic, this means avoiding Blocks and Panels.

Blocks underwent a fundamental change in how they were implemented for Drupal 8 and attempting to migrate them can introduce extra complexity. It is often easier to recreate them by hand or with a custom script.

If you have only a handful of blocks, this should not be a problem. But don’t let them get out of hand. Lean more on content entities to solve problems, as the migration landscape for them is documented, tried, and tested.

The easiest way for Block usage to get out of hand is with Panels, where almost anything can go. Their very nature tends toward the wild and unstructured. Panels can create additional risk when approaching a migration, and Panels that have not had strong content governance can end up being a garden made up of nothing but weeds. Better to just pull everything up, lay new soil, and start over.

This might be fine. As long as your eyes are open and you expect this to be the case, the risk can be planned for and mitigated.

Instead of a Panel, think about using an Entity with set Entity Reference fields, which then map to pre-defined templates. This type of thinking might require another paradigm shift, however, and for that, you might want some help.

Engage a Content Strategist

Preparing your Drupal 7 site for the future, no matter how far away that future might be, requires some thinking about how you envision that future. 

  • What goals are you trying to achieve? 
  • What message are you trying to broadcast? 
  • What tools make for happy editors? 
  • How does your content and CMS best serve these needs?

A content strategist can help you answer these questions and more. 

This expertise can come in many forms. You could hire one. You could contract with an agency for limited engagements. You could nurture the curiosity of someone on your team, so they grow to have the skills required.

What can a good content strategist help you do?

Audit Your Content

The more content you have, the more varied the content types, the more complex an eventual migration will be. You don’t want to spend time and brainpower migrating old content that isn't valuable.

A content audit can help you determine which content is stale and which content needs to be either archived or updated. It can also help surface duplication.

One of the most important benefits of a content audit, however, is finding content with problematic structure, or content that lacks proper structure altogether. Is everything being crammed into the body field? As a result, is the final look of a page different based on the editor who built it? Even pages of the same content type?

What are some things that raise red flags when seen in the body field?

  • Metadata about the content itself, like author information or reading time.
  • Iframes from various sources, with inconsistent sizes.
  • Different methods of image embedding from one article to the next.
  • Related links, some of which are outdated.
  • Presentational HTML markup and CSS.

The SQueaLer Drush tool can be helpful for this.

Each of these will require different special cases when migrating, making the process more complex. Cleaning them up now will save time and money later while making your site more maintainable in the present.

Create a Plan for Content Governance

Auditing and cleaning up your content is great, but if you don’t change your old habits, weeds will keep popping up. Re-assess who is allowed to create content and what they are allowed to do within that purview.

Has your audit determined three different ways images are being embedded? Work toward settling on a single way, with input from your editors. Enforce it as best you can.

Have you spent some time identifying problematic content that doesn’t fit with your strategy? Work toward processes that help prevent the creation of more problematic content.

How much time should you spend per month cleaning up or archiving old content? Will it be one person’s job or a shared responsibility? Who makes the final decision on what content needs to be updated? The answers to these questions will be different for each organization.

If your editors keep cramming things that don’t belong in the body field, you should see it as a sign, one covered with blinking red lights and loud alarms, telling you that the system is not meeting their needs. So, find out those needs. 

A good content strategist can help facilitate these conversations, help you find ways to temporarily fill these gaps while you are still on Drupal 7, and help you determine what your future should eventually look like. Address these potential problems as soon as you are able. Start thinking about them early. 

If you don’t, they will continue to haunt your future site, even if you end up leaving Drupal. These specters are platform-neutral.

Take Time to Model Your Content

Many websites grow in an ad-hoc manner, with features pasted on whenever a stakeholder has a flight of fancy. Even if the site launched after a careful period of requirements gathering and needs analysis, its evolution afterward can descend into chaos. 

New content types are added for that big seasonal push, and then never get used again. Editors need to make a landing page look just right, and inject things in the body field as a quick fix. Questions and new requirements are always based on whether the site can do something, rather than whether the site should do something. “No” becomes the forgotten vocabulary word.

Now is a good opportunity to take stock. This is related to our “Gravitate Toward Solutions that Rely on Structured Content” recommendation above.

Your content audit has surfaced issues you need to fix at the structural level. Your content governance plan needs some guard rails to help with enforcement. Your Drupal 7 site needs to align better with the goals and message of your organization.

This can mean new content types, but it can also mean a modification of existing content types. 

Some common examples of structure changes:

  • Moving related article links out of the body field and into their own Entity Reference field.
  • Additional (or fewer) taxonomy fields.
  • Summary and Lede fields.
  • An “Authors” Entity Reference field, separate from the default author field.

For media management, think about using the Media module to help ensure proper metadata surrounding media, and to help enforce a single solution for including media in content. There is also a migration path waiting for you when you are ready.

Properly modeling your content, and beginning to implement that model, will net many benefits. It will be easier to change the presentation of your site consistently. Opportunities for content re-use will become clearer. Your next content audit will be less thorny. 

While you may give up some flexibility in the day-to-day creation of content, it pays off with more holistic flexibility. For example, if you desire to go down a decoupled route, with your content served via an API, structured content gives you that option.

But equally important, modeled and structured content will make your migration much, much easier. Well-structured content is always easier to migrate than unstructured content. 

And if your content strategist has helped you answer the proper questions, a lot of the big questions surrounding migration will be answered as well. This means a chunk of the hard work required will already have been done.

Keep Everything Secure and Up to Date

Your Drupal 7 site should be kept up to date. That means Drupal core and all the modules you use, but it also means your hosting environment. Don’t remain stuck on past versions of PHP. The latest versions of Drupal 7 support the latest versions of PHP 7, which has a nice symmetry to it.

All of this will not only keep your current site secure and performant, but it will also make a future migration easier. The latest versions of contrib modules are usually closer to their Drupal 8+ counterparts in terms of organization and how they store data. They also can take advantage of PHP 7 improvements and syntax.

This ties into our first recommendation of shifting your coding culture. Staying up to date with PHP will allow your team to start using methods and styles that do not exist for PHP 5. Being familiar with PHP 7 will ease the eventual transition.

Conclusion

Drupal 7 has been given a new lease on life, and if you are currently dependent on Drupal 7, you can breathe a little easier. Your CMS will be supported for another year, and even longer if you reach out to a qualified D7ES vendor.

Use this time wisely. Prepare for the future. Now that you know it will be around for a while, you can feel safe making smart investments in the platform and in your team. Make your Drupal 7 site work better for you in the present, while at the same time making your eventual migration go smoother.

You and your team can be seen as agents of foresight, clarity, and poise, recognized for preparing the organization to make the leap.

If you would like to start the conversation around some of these items, Lullabot’s support and maintenance team is ready to help. Through developer mentorship and close collaboration, we can help you squeeze every last drop out of your Drupal 7 investment.

Aug 19 2020
Aug 19

Continuous Deployment, Infrastructure as Code, and Drupal

Following up with the first three parts of the Continuous Deployment, Infrastructure as Code, and Drupal series, where we used DigitalOcean as the cloud provider, in this article we will see how to set up a continuous deployment workflow that uses Google Cloud. Based on the promise of the Cloud Native Foundation, it should be very similar.

We have set up a demo repository where you can discover the following resources that we will cover in the following sections:

Let’s begin with an overview of the cluster setup.

The Kubernetes cluster setup

We begin by creating a Kubernetes cluster at Google Cloud’s Kubernetes Engine. The free tier is enough to accomplish what we want for this article so there is no need to spend money if you just want to tinker with Kubernetes at Google Cloud. Follow the steps at Create Cluster and leave the default options as they are.

Here is a screenshot of the Kubernetes Engine dashboard after creating the cluster:

In order to interact with the cluster via the command line, we need to install the Google Cloud SDK. In particular, the gcloud command is the one we need to perform actions such as downloading cluster details or performing deployments. Here is a link to the gcloud installation instructions for all platforms.

With the SDK installed, we can download the cluster details to start issuing commands against it. Here is the command for doing so:

gcloud container clusters get-credentials my-first-cluster-1 --zone us-central1-a --project heroic-bliss-279608

The above command downloads the cluster details and sets the cluster as the default context for kubectl, the command line interface to manage Kubernetes clusters. Here is how we can see the available clusters:

[email protected]:~:$ kubectl config get-contexts
CURRENT   NAME                                                    
          do-sfo2-drupster                                        
*         gke_heroic-bliss-279608_us-central1-a_my-first-cluster-1

The star above indicates the active context, meaning that kubectl is set to use the Google Cloud cluster that we just created. As for do-sfo2-drupster, this is the DigitalOcean cluster that we used at the Continuous Deployment, Infrastructure as Code, and Drupal series.

That’s it for the cluster setup. In the next section we will set up GitHub actions to perform deployments using these credentials but it is still useful to have access to the cluster from our local environment in order to analyze its state and manage secrets.

Continuous Deployment via GitHub Actions

In this section we will analyze a GitHub Actions workflow. When code is pushed to master branch, it will perform the following tasks:

  1. Builds a Docker image containing the code and the operative system. Then pushes it to GitHub Packages.
  2. Installs gcloud: Google Cloud’s command line interface.
  3. Downloads credentials and details of the Kubernetes cluster at Google Cloud.
  4. Performs a deployment and updates the database.

Here is a diagram that illustrates the above:

Here are the contents of the GitHub Actions workflow:

https://github.com/juampynr/drupal8-gcloud/blob/master/.github/workflows/ci.yml

on:
  push:
    branches:
      - master
name: Build and deploy
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/[email protected]
        with:
          fetch-depth: 1

      - name: Build, push, and verify image
        run: |
          echo ${{ secrets.PACKAGES_TOKEN }} | docker login docker.pkg.github.com -u juampynr --password-stdin
          docker build --tag docker.pkg.github.com/juampynr/drupal8-gcloud/drupal8-gcloud:${GITHUB_SHA} .
          docker push docker.pkg.github.com/juampynr/drupal8-gcloud/drupal8-gcloud:${GITHUB_SHA}
          docker pull docker.pkg.github.com/juampynr/drupal8-gcloud/drupal8-gcloud:${GITHUB_SHA}

      - name: Install gcloud
        uses: GoogleCloudPlatform/github-actions/[email protected]
        with:
          version: '295.0.0'
          project_id: ${{ secrets.GCP_PROJECT_ID }}
          service_account_key: ${{ secrets.GCP_SA_KEY }}
          export_default_credentials: true

      - name: Check gcloud credentials
        run: |
          gcloud container clusters get-credentials ${{ secrets.GCP_CLUSTER_ID }} --zone us-central1-a --project ${{ secrets.GCP_PROJECT_ID }}
          kubectl config get-contexts

      - name: Deploy to Google Cloud
        run: |
          sed -i 's||docker.pkg.github.com/juampynr/drupal8-gcloud/drupal8-gcloud:'${GITHUB_SHA}'|' $GITHUB_WORKSPACE/kubernetes/drupal-deployment.yaml
          sed -i 's||${{ secrets.DB_PASSWORD }}|' $GITHUB_WORKSPACE/kubernetes/drupal-deployment.yaml
          sed -i 's||${{ secrets.DB_PASSWORD }}|' $GITHUB_WORKSPACE/kubernetes/mysql-deployment.yaml
          sed -i 's||${{ secrets.DB_PASSWORD }}|' $GITHUB_WORKSPACE/kubernetes/mysql-deployment.yaml
          kubectl apply -k kubernetes
          kubectl rollout status deployment/drupal

      - name: Update database
        run: |
          POD_NAME=$(kubectl get pods -l tier=frontend -o=jsonpath='{.items[0].metadata.name}')
          kubectl exec $POD_NAME -c drupal -- vendor/bin/robo files:configure
          kubectl exec $POD_NAME -c drupal -- vendor/bin/robo database:update

If you have read Continuous Deployment, Infrastructure as Code, and Drupal: Part 2, then the above will look familiar. The step to build and push a Docker image containing the application is identical to the step to perform a deployment and update the database. What changes? How to authenticate against Google Cloud. Let’s see that in further detail.

Building and pushing the Docker image

Once GitHub Actions has checked out the code, it performs the following step in which it builds a Docker image, then pushes it to the Docker registry within the GitHub project (aka GitHub Packages), and finally pulls the image. Here is the step:

https://github.com/juampynr/drupal8-gcloud/blob/master/.github/workflows/ci.yml

      - name: Build, push, and verify image
        run: |
          echo ${{ secrets.PACKAGES_TOKEN }} | docker login docker.pkg.github.com -u juampynr --password-stdin
          docker build --tag docker.pkg.github.com/juampynr/drupal8-gcloud/drupal8-gcloud:${GITHUB_SHA} .
          docker push docker.pkg.github.com/juampynr/drupal8-gcloud/drupal8-gcloud:${GITHUB_SHA}
          docker pull docker.pkg.github.com/juampynr/drupal8-gcloud/drupal8-gcloud:${GITHUB_SHA}

Each time code is pushed to master branch, GitHub Actions will build a Docker image and store it along with the repository as a Package. Here you can see the section within the repository where you can access to all available Packages. Each package is a Docker image containing all the required files to run the application such as Composer dependencies or configuration files:

The reason why the step ends with a docker pull is because docker push may fail due to a network error, in which case we don’t want to proceed with the workflow. docker pull will attempt to fetch the image from GitHub Packages and return an error if it wasn’t able to do so.

Installing Google Cloud

We are using the official Google Cloud action to authenticate against Google Cloud. Here is the step within the GitHub Actions workflow:

https://github.com/juampynr/drupal8-gcloud/blob/master/.github/workflows/ci.yml

      - name: Install gcloud
        uses: GoogleCloudPlatform/github-actions/[email protected]
        with:
          version: '295.0.0'
          project_id: ${{ secrets.GCP_PROJECT_ID }}
          service_account_key: ${{ secrets.GCP_SA_KEY }}
          export_default_credentials: true

The setup-gcloud command requires the following parameters:

  • The project id, which we have stored as a GitHub Secret.
  • A Service Account, which is a secret encoded as a Base64 string.

A service account, according to the official docs at Google’s Security and Identity, “is a special kind of account used by an application or a virtual machine (VM) instance, not a person. Applications use service accounts to make authorized API calls.”. The main benefit of service accounts is that they save you from having to enter your personal account details (in this case, your Google username and password) on remote services.

You can find the steps to create a service account in the Google Cloud documentation. Make sure that you select the role Kubernetes Engine Developer role. The result will be a json file that you can encode via cat key.json | base64 and save it as a GitHub secret.

Downloading the cluster details

Once we have authenticated against Google Cloud via the GitHub action in the previous section, downloading the cluster details is a trivial task since gcloud has a command to accomplish just that. Here is the step within the GitHub Actions workflow:

https://github.com/juampynr/drupal8-gcloud/blob/master/.github/workflows/ci.yml

      - name: Check gcloud credentials
        run: |
          gcloud container clusters get-credentials ${{ secrets.GCP_CLUSTER_ID }} --zone us-central1-a --project ${{ secrets.GCP_PROJECT_ID }}
          kubectl config get-contexts

The first command, gcloud container clusters get-credentials, is what downloads the credentials and stores them so future kubectl commands can use them. The second command, kubectl config get-contexts, is just a verification to print out the available Kubernetes clusters. Here is the output of this and the previous step for a GitHub Actions run:

Running cron jobs via Kubernetes

The set of Kubernetes objects that the demo repository uses is located at https://github.com/juampynr/drupal8-gcloud/tree/master/kubernetes. These are are almost identical to the ones used for a Kubernetes cluster hosted at DigitalOcean, which we covered in detail at Continuous Deployment, Infrastructure as Code, and Drupal: part 3. What’s different for Google Cloud is how we make the CronJob object to authenticate and run Drupal cron. Here are its contents:

https://raw.githubusercontent.com/juampynr/drupal8-gcloud/master/kubernetes/drupal-deployment.yaml

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: drupal-cron
spec:
  schedule: "*/2 * * * *"
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: drupal-cron
              image: juampynr/gcloud-cronjob:latest
              volumeMounts:
                - name: google-cloud-key
                  mountPath: /var/secrets/google
              command: ["/bin/sh","-c"]
              args:
                - gcloud auth activate-service-account --key-file=/var/secrets/google/key.json;
                  gcloud container clusters get-credentials `cat /var/secrets/google/cluster_id` --zone=`cat /var/secrets/google/cluster_zone` --project=`cat /var/secrets/google/project_id`;
                  POD_NAME=$(kubectl get pods -l tier=frontend -o=jsonpath='{.items[0].metadata.name}');
                  kubectl exec $POD_NAME -c drupal -- vendor/bin/drush core:cron;
          volumes:
            - name: google-cloud-key
              secret:
                secretName: creds
          restartPolicy: OnFailure

There are a few things worth pointing out from the above object:

  • schedule: "*/2 * * * *" means that Kubernetes will execute the contents of the command section every two minutes.
  • concurrencyPolicy: Forbid avoids running cron if it is already running.
  • image: juampynr/gcloud-cronjob:latest is a Docker image that contains the gcloud and kubectl commands.
  • There are a few secrets that gcloud requires in order to authenticate against Google Cloud and then identify and download the cluster details. We have created a Kubernetes secret with them and mounted such secret as a volume at /var/secrets/google.
  • Finally, the args section sends a command to the container running Drupal via kubectl exec $POD_NAME -c drupal -- vendor/bin/drush core:cron, which runs Drupal cron.

Conclusion

So far the promise of the Cloud Native Foundation is proven right: running a web application using Kubernetes is very similar using different cloud providers.

The main difference between hosting it at DigitalOcean versus Google Cloud is authentication, but once you have downloaded the Kubernetes cluster configuration, everything else is identical. This is a great advantage in terms of portability as it allows large projects to use multiple cloud providers and smaller ones to move from one cloud provider to another.

If you want to try this setup with any of your Drupal projects, feel free to copy the GitHub Actions workflow and the Kubernetes objects from the demo repository.

Jul 29 2020
Jul 29

Continuous Deployment, Infrastructure as Code, and Drupal

Welcome to the last article of this series, which will take you through a set of Kubernetes objects that define the infrastructure of the sample Drupal site that we have been using. The previous article covered the GitHub Actions workflow that orchestrated the Docker image build, push, and deployment to a Kubernetes cluster. This article will demonstrate the Kubernetes setup. 

Note: If you have never heard of Kubernetes before, the official site has a wonderful tutorial with graphics explaining its main concepts.

An overview of the Kubernetes setup

From the official documentation: "To work with Kubernetes, you use Kubernetes API objects to describe your cluster's desired state.

Kubernetes API objects are YAML files that define the state of the cluster. You can find these files in the directory called definitions within the demo repository. Here is a screenshot with its contents:

 There are two resources, one for Drupal and one for MySQL, and a Kustomization file that reference them. Here are the contents of definitions/kustomization.yaml:

resources:
- mysql-deployment.yaml
- drupal-deployment.yaml

In the next section, we will explore the MySQL resource that defines how MySQL gets deployed and another one for Drupal.

The MySQL resource

The file definition/mysql-deployment.yaml defines how MySQL gets deployed and runs within the cluster. It is composed of three Kubernetes objects:

Taking a look at them one by one, first, here is the service:

apiVersion: v1
kind: Service
metadata:
  name: drupal-mysql
  labels:
    app: drupal
spec:
  ports:
    - port: 3306
  selector:
    app: drupal
    tier: mysql
  clusterIP: None

Notice the following settings listed above:

And now, here's how to claim storage for the database:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
  labels:
    app: drupal
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi

In the object above, 20GB of storage is claimed to host the database files, and this storage space is connected with MySQL via a volume in the below object where how to deploy MySQL into the cluster is defined:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: drupal-mysql
  labels:
    app: drupal
spec:
  selector:
    matchLabels:
      app: drupal
      tier: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: drupal
        tier: mysql
    spec:
      containers:
      - image: mysql:5.6
        name: mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql
              key: admin-password
        - name: MYSQL_DATABASE
          value: drupal
        - name: MYSQL_USER
          value: drupal
        - name: MYSQL_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql
              key: nonadmin-password
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pv-claim

The deployment object is longer than the other two, mainly because it requires further configuration to define environment variables that Drupal will use to connect with MySQL. Two things are worth highlighting:

  • Passwords were obtained from secret objects, which are managed by the Kubernetes cluster. A secret was created to host the root and non-root passwords via the command line by following these instructions from the official documentation.
  • We mounted the persistent volume claim that we defined in the previous object at /var/lib/mysql, which is where MySQL stores its data. This will persist the database even while pods are recreated during deployments.

That's it for MySQL! Now it's time to check out the Drupal application.

The Drupal resource

The Drupal resource has the same Kubernetes objects as the MySQL one with a slightly different configuration. Additionally, it defines a CronJob object to run Drupal's background tasks periodically.

You can find the resource with all of the following objects in the repository under definitions/drupal-deployment.yaml. In the service below, port 80 is exposed and handed to the load balancer, which allows incoming requests to the web application.

apiVersion: v1
kind: Service
metadata:
  name: drupal
  labels:
    app: drupal
spec:
  ports:
    - port: 80
  selector:
    app: drupal
    tier: frontend
  type: LoadBalancer

The next object is the storage, which is similar to the MySQL one that we claimed for 20GB. This is attached to the container running the web application via a volume:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: dr-pv-claim
  labels:
    app: drupal
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi

The heart of the Drupal resource is the Deployment object, where the configuration of the web application is defined. Here it is:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: drupal
  labels:
    app: drupal
spec:
  selector:
    matchLabels:
      app: drupal
      tier: frontend
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: drupal
        tier: frontend
    spec:
      volumes:
        - name: drupal-persistent-storage
          persistentVolumeClaim:
            claimName: dr-pv-claim
      containers:
        - image: 
          name: drupal
          imagePullPolicy: Always
          env:
            - name: DB_HOST
              value: drupal-mysql
            - name: DB_NAME
              value: drupal
            - name: DB_USER
              value: drupal
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysql
                  key: nonadmin-password
          ports:
          - containerPort: 80
            name: drupal
          volumeMounts:
          - name: drupal-persistent-storage
            mountPath: /var/www/html/web/sites/default/files
      imagePullSecrets:
        - name: regcred

There are a few things to point out in the above object:

  • The image is defined as a placeholder: - image: . The reason is that a Docker image is built and pushed every time changes are pushed to the GitHub repository. Therefore, the GitHub workflow, at run time, will turn into something like docker.pkg.github.com/juampynr/drupal8-do/app:foo, where foo is the hash of the last commit pushed to the repository.
  • The password is obtained via a Secret object. It's the same object that we saw for MySQL in the previous section.
  • The volume was mounted at /var/www/html/web/sites/default/files, the default location to host public files in a Drupal application. This will persist files while Kubernetes recreates pods, which happens during deployments.
  • GitHub Packages requires authentication to pull Docker images, even if they belong to public repositories. Kubernetes lets you define a secret with the credentials to authenticate against a Docker image registry. Following these instructions, the secret was created.

The last object for the Drupal resource is a CronJob that runs Drupal cron every five minutes:

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: drupal-cron
spec:
  schedule: "*/5 * * * *"
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: drupal-cron
              image: juampynr/digital-ocean-cronjob:latest
              env:
                - name: DIGITALOCEAN_ACCESS_TOKEN
                  valueFrom:
                    secretKeyRef:
                      name: api
                      key: key
              command: ["/bin/bash","-c"]
              args:
                - doctl kubernetes cluster kubeconfig save drupster;
                  POD_NAME=$(kubectl get pods -l tier=frontend -o=jsonpath='{.items[0].metadata.name}');
                  kubectl exec $POD_NAME -c drupal -- vendor/bin/drush core:cron;
          restartPolicy: OnFailure

The CronJob object uses a custom Docker image that has doctl (DigitalOcean's command-line interface) and kubectl (the Kubernetes command-line interface) preinstalled. Then, it uses these two commands to download the cluster configuration and execute a Drush command against the container running Drupal.

Next steps

Getting the Drupal application up and running in Kubernetes has been a fabulous experience and is a world worthy of continued exploration. Here are some things to figure out:

  • Performing rolling updates instead of recreating pods, so there is no downtime during deployments
  • Performing load tests against the current infrastructure by doing common tasks in Drupal like saving a node, uploading an image, or requesting a page with cold caches
  • Setting up automatic backups for the database and file system
  • Triggering automatic or manually rollbacks when a deployment fails
  • Setting up secure traffic via an SSL certificate
  • Adjusting the architecture to enable scaling the replica set up and down. Here are several pages of the official documentation that explain how to do this:
  • Checking out the existing helm charts for Drupal
  • Test running the Drupal 8 application at AWS, Google Cloud, and Linode
  • Test hosting platforms that support Drupal over Kubernetes, such as DDEV-Live and Lagoon

Acknowledgments

As a final note, this is a learning process. If you have any tips, feedback, or want to share anything about this topic, please post it as a comment here or via social networks. Thanks in advance!

Jul 24 2020
Jul 24

People who build websites sometimes choose Drupal because of its reputation for accessibility. Technically, features added to Drupal core must conform with the World Wide Web Consortium (W3C) WCAG 2.0 and ATAG 2.0 guidelines. That said, only a small subset of web developers could provide any specifics about those guidelines. People who participate in the Drupal project often trust that Drupal websites, at least "out of the box," will be among the most accessible by people with disabilities. Lullabot, like many other Drupal agencies, regularly shares information about Drupal's accessibility features. We have a generous group of people in Lullabot who call themselves the "Accessibots" who speak at conferences and regularly impart advice about how to prevent inaccessible websites.

While some developers rightly focus on how to make websites accessible, this article focuses on the why. Why does the Drupal community hold accessibility in such high esteem? The Drupal community strives to provide a diverse and inclusive space, so building accessible websites clearly supports those goals. This article explores the stated reasons for Drupal's continued commitment to accessibility beyond the obvious truth that making websites accessible is the right thing to do.

Accessibility is hard work

Much of the Internet is an "inaccessible Internet" that excludes millions of disabled people because making websites accessible requires considerable effort. In the current form of Drupal's Values and Principles — a document intended to "evolve over time" — the last two sentences of the second principle acknowledge this difficulty:

Making security, privacy, accessibility, multilingual capability, and usability top priorities is hard work, but it's worth it. We want to build 'good software.'

Appropriately enough, the second principle offers a value judgment that includes accessibility among the features of "good software." Presumably, nobody wants to build "bad software," but professional web developers know that accessibility, like a robust code test suite, usually falls rather low on the list of priorities. Non-disabled stakeholders commonly, and understandably, prioritize new features.

Building accessible websites requires effort on multiple levels. For instance, those involved in the building process must keep informed about the accessibility guidelines, colors, images, links, buttons, forms, tools for testing the accessibility of a website, and more. Next, they might need to use their knowledge to convince their stakeholders about the value of building accessible websites. Finally, they must know how to make websites accessible using their chosen tools in a way that conforms to the guidelines.

The virtue of accessibility

To counteract the natural human desire to see fancy new features, teams building websites can sometimes provide statistics to make their case for building accessible websites (e.g., "1 in 4 US adults live with a disability"), but frequently they must use value statements like the ones we find on Drupal's Values and Principles page to make a case for prioritizing accessibility. Further, "accessibility" is just one item on a long list of requirements (along with security, privacy, usability, etc.) that we do because of their "inherent worth."

How can we back up the claim, "it's worth it"? What does "worth it" even mean? Grammar books and coaches instruct writers to avoid the word "it" (for instance, in The Writer's Diet, Helen Sword calls "it" a "waste word"). However, the objective here is not to critique word choice, but rather to highlight the intentionally vague nature of this logic. For instance, understand "it's worth it" to indicate that the time or money required to make an accessible website increases the monetary value. More likely, though, we should understand that the idealized version of the Drupal community — the one we want to inhabit, or believe we are inhabiting — values the inherent dignity and worth of all people, with no person excluded.

Put another way; our idealized Drupal community does not build accessible software because of Title III of the Americans with Disabilities Act (in the US), the Accessibility for Ontarians with Disabilities Act (AODA), or some other legal requirement. We build accessible software because we value people. We have opinions about right and wrong, good and bad, etc., and we — ideally — value all people.

We cannot afford to be careless

Further, we understand accessibility requires effort and diligence. The second principle states,

Drupal has such a large impact on the digital landscape that our community cannot afford to be careless.

To make accessible websites requires non-carelessness (or carefulness or conscientiousness). Such a statement indicates a Drupal community that makes careful, calculated decisions about all code considered for Drupal core.

Accessibility is one of five "core gates" — essentially a "checklist" used for evaluating proposed additions to the Drupal codebase. While not every individual patch needs to pass all of the gates, all changes — small and large, front-end and back-end, from routing to rendering, media to menus, forms to fields, and everything else — could inadvertently have negative consequences and therefore must be considered with reference to these "gates" in order to ensure non-carelessness. These core gates do not indicate whether a proposed change is "essential" or not, for example, but they help core maintainers agree on minimum requirements.

To claim that attributes, such as accessibility, are "worth it," also suggests long-term thinking. New features must be accessible both now and for future users. We strive for a continuous flow of inclusive features, accessible to all people. As the web changes, this imagined Drupal community will change with it.

Drupal as a safe haven

The people who create the Drupal software aspire to provide a safe haven for all people. The second principle makes frequent references to safety:

[The Drupal community's] responsibility to ensure that the web is inclusive of every person and accounts for everyone's safety [and that] when software is unsafe, countless people are put at risk of financial, social, and physical harm ... Our community needs to ensure that everyone has access to Drupal and that everyone can use Drupal easily and safely.

Drupal provides safety to organizations such as libraries, governments, universities, and other groups that seek to provide inclusive "digital experiences." Such groups aspire to offer a "safe space" for all people. Drupal does not need to claim that Drupal is safe, and all other software is unsafe. We might compare the consistently safe environment offered by the Drupal software to the "safe space" envisioned by the compassionate people working on Drupal Diversity and Inclusion. Drupal can offer safety to users, safe files, safe HTTP methods, and all other varieties of safety. Further, Drupal offers safety to all people who use the software, disabled or not.

Access by everyone

In 1997, the inventor of the World Wide Web, Tim Berners-Lee, said, "The power of the web is in its universality. Access by everyone, regardless of disability, is an essential aspect." Unfortunately, most people building websites today are not building accessible websites. By some estimates, up to 98 percent of the web does not comply with accessibility standards. People building non-accessible websites might possess the required technical competency, but they get asked to build websites in settings with an ethos that does not prioritize accessibility. The people behind initiatives such as Global Accessibility Awareness Day (GAAD) seek to change the ethos.

Accessibility, we must conclude, is a feature that sets Drupal apart from the majority of the web. Not all Drupal websites are accessible, but Drupal offers a lot to people trying to build accessible websites. Although accessibility requires hard work and diligence, the Drupal community has concluded that proposed technical changes to the Drupal project are necessarily ethical choices and that we must carefully consider their impact on all people.

Jul 22 2020
Jul 22

New to Drupal 8.9 and 9.0 is the ability to create the HTML element within a native Drupal menu that can be used to toggle secondary menus (such as drop-downs or mega-menus) in a usable and accessible way.

Common inaccessible menu patterns

It's common to see links (instead of buttons) used to toggle submenus. The result of this pattern is typically inaccessible for keyboard navigation and assistive devices such as screen readers.


Hyperlinks ( tags) should only be used when navigating to a route where the URL will change. If clicking the element shows or hides something or performs some other action, a

element should be used. For a more nuanced explanation, see Marcy Sutton's article, Links vs. Buttons in Modern Web Applications.

Creating buttons using the Drupal admin interface

As stated earlier, Drupal 8.9 and later include support for the button element when creating menu items within Drupal's menu interface. To do so, simply enter route:

into the link field when creating the menu entry.

Once entered, the menu item will be output as a

element instead of an tag. However, your work is not yet done. Assuming that you're going to create a submenu, you need to make the menu respond to click and hover events in an accessible manner.

Attaching appropriate aria labels to the menu items

To indicate the current state of the

element, you need to attach the aria-expanded attribute. In its default closed state, it should be set to false, and then toggle to true when the submenu is open.
Services

In addition to indicating the state, you need to create a relationship between the parent element and its child

    submenu element. To do this, you first need to create a unique id attribute on the submenu's
      element and then create an aria-controls attribute on the
element that corresponds to the submenu's id attribute. 
Services

All of this can be done within the menu's Twig template. However, the aria-attributes should not be created by Twig because you can't be sure that JavaScript is enabled, and the functionality will work properly. These attributes will be added later in the JS.

The menu.html.twig template below is adapted from the new Olivero theme. This template creates these relationships and adds some useful CSS class names for styling the links and buttons.

{#
/**
 * @file
 * Theme implementation for a menu.
 *
 * Available variables:
 * - menu_name: The machine name of the menu.
 * - items: A nested list of menu items. Each menu item contains:
 *   - attributes: HTML attributes for the menu item.
 *   - below: The menu item child items.
 *   - title: The menu link title.
 *   - url: The menu link url, instance of \Drupal\Core\Url
 *   - localized_options: Menu link localized options.
 *   - is_expanded: TRUE if the link has visible children within the current
 *     menu tree.
 *   - is_collapsed: TRUE if the link has children within the current menu tree
 *     that are not currently visible.
 *   - in_active_trail: TRUE if the link is in the active trail.
 *
 * @ingroup themeable
 */
#}
{% import _self as menus %}
{#
  We call a macro which calls itself to render the full tree.
  @see https://twig.symfony.com/doc/1.x/tags/macro.html
#}
{% set attributes = attributes.addClass('menu') %}
{{ menus.menu_links(items, attributes, 0) }}
{% macro menu_links(items, attributes, menu_level) %}
  {% set primary_nav_level = 'menu--level-' ~ (menu_level + 1) %}
  {% import _self as menus %}
  {% if items %}
    
    {% set attributes = attributes.removeClass(primary_nav_level) %} {% for item in items %} {% if item.url.isrouted and item.url.routeName == '' %} {% set menu_item_type = 'nolink' %} {% elseif item.url.isrouted and item.url.routeName == '' %} {% set menu_item_type = 'button' %} {% else %} {% set menu_item_type = 'link' %} {% endif %} {% set item_classes = [ 'menu__item', 'menu__item--' ~ menu_item_type, 'menu__item--level-' ~ (menu_level + 1), item.in_active_trail ? 'menu__item--active-trail', item.below ? 'menu__item--has-children', ] %} {% set link_classes = [ 'menu__link', 'menu__link--' ~ menu_item_type, 'menu__link--level-' ~ (menu_level + 1), item.in_active_trail ? 'menu__link--active-trail', item.below ? 'menu__link--has-children', ] %}
  • {# A unique HTML ID should be used, but that isn't available through Twig yet, so the |clean_id filter is used for now. @see https://www.drupal.org/project/drupal/issues/3115445 #} {% set aria_id = (item.title ~ '-submenu-' ~ loop.index )|clean_id %} {% if menu_item_type == 'link' or menu_item_type == 'nolink' %} {{ link(item.title, item.url, { 'class': link_classes }) }} {% if item.below %} {{ menus.menu_links(item.below, attributes, menu_level + 1) }} {% endif %} {% elseif menu_item_type == 'button' %} {{ link(link_title, item.url, { 'class': link_classes, 'data-ariacontrols': item.below ? aria_id : false, }) }} {% set attributes = attributes.setAttribute('id', aria_id) %} {{ menus.menu_links(item.below, attributes, menu_level + 1) }} {% endif %} </li> {% endfor %}
  • {{>
{% endif %} {% endmacro %}

Show and hide the submenu with CSS

Because the submenu should not be shown until activated, you need to hide it both visually and from the accessibility tree (so assistive devices cannot see it). To do this, you can either use display: none or visibility: hidden within our CSS.

.menu--level-2 {
  visibility: hidden;
}

To show the menu when the element’s aria-expanded attribute is toggled to true, use the following CSS selector.

[aria-expanded="true"] + .menu--level-2 {
  visibility: visible;
}

JavaScript to initialize and toggle the submenu

The CSS above will show the submenu when the

's aria-expanded attribute is true. To set this attribute, you need JavaScript. 

Note: The JavaScript examples below use modern syntax and methods that are not compatible with Internet Explorer 11. If you support this browser, make sure that you transpile the JS and include appropriate polyfills.

Create the aria attributes

Because you don't want to have aria attributes that will indicate non-existent functionality if JS is disabled, wait for JavaScript to add the aria-attributes.

The code snippet below integrates with Drupal Behaviors to create the aria attributes for each button.

(Drupal => {
  function initSubmenu(el) {
    el.setAttribute('aria-controls', el.dataset.ariacontrols);
    el.setAttribute('aria-expanded', 'false');
  }

  Drupal.behaviors.submenu = {
    attach(context) {
      context.querySelectorAll('.menu__link--button').forEach(el => initSubmenu(el));
    },
  };
}) (Drupal);

Toggle the submenu on button click

Below, a click event listener is attached to each button, which activates a toggleSubmenu function, which flips the aria-expanded attribute.

(Drupal => {
  function initSubmenu(el) {
    el.addEventListener('click', toggleSubmenu);
    el.setAttribute('aria-controls', el.dataset.ariacontrols);
    el.setAttribute('aria-expanded', 'false');
  }
  function toggleSubmenu(e) {
    const button = e.currentTarget;
    const currentState = button.getAttribute('aria-expanded') === 'true';
    button.setAttribute('aria-expanded', !currentState);
  }
  Drupal.behaviors.submenu = {
    attach(context) {
      context.querySelectorAll('.menu__link--button').forEach(el => initSubmenu(el));
    },
  };
}) (Drupal);

Toggling the submenu on hover

It's a common requirement to show the submenu when hovering over the button with a mouse pointer. To show the menu on hover, toggle the button's aria-expanded attribute on mouseover and mouseout events attached to the button's

  • parent. Attaching the events to the parent ensures the menu stays open when the end-user moves the mouse pointer off of the button and onto its submenu.

    The refactored JavaScript below expands the click event to add mouseover and mouseout events on the parent

  • item.
    (Drupal => {
      /**
       * Add necessary event listeners and create aria attributes
       * @param {element} el - List item element that has a submenu.
       */
      function initSubmenu(el) {
        const button = el.querySelector('.menu-link--button');
        button.setAttribute('aria-controls', button.dataset.ariacontrols);
        button.setAttribute('aria-expanded', 'false');
        button.addEventListener('click', e => toggleSubmenu(e.currentTarget, !getButtonState(e.currentTarget)));
        el.addEventListener('mouseover', toggleSubmenu(button, true));
        el.addEventListener('mouseout', toggleSubmenu(button, false));
      }
    
      /**
       * Toggles the aria-expanded attribute of a given button to a desired state.
       * @param {element} button - Button element that should be toggled.
       * @param {boolean} toState - State indicating the end result toggle operation.
       */
      function toggleSubmenu(button, toState) {
        button.setAttribute('aria-expanded', toState);
      }
    
      /**
       * Get the current aria-expanded state of a given button.
       * @param {element} button - Button element to return state of.
       */
      function getButtonState(button) {
        return button.getAttribute('aria-expanded') === 'true';
      }
    
      Drupal.behaviors.submenu = {
        attach(context) {
          context.querySelectorAll('.menu-item--has-children').forEach(el => initSubmenu(el));
        },
      };
    }) (Drupal);

    Non-JavaScript fallback

    The functionality to show and hide the submenus only works if JavaScript is enabled. Fortunately, you can provide a non-JS alternative with the CSS :focus-within and :hover pseudo-classes. If any element within the

  • receives focus or is hovered over, the submenu will appear.

    The selectors below take advantage of the fact that Drupal will attach a js CSS class to the HTML element when JavaScript is enabled. 

    Note: The :focus-within pseudo-class is not supported by Internet Explorer 11 or earlier versions of Edge.

    html:not(.js) .menu-item--has-children:focus-within .menu--level-2,
    html:not(.js) .menu-item--has-children:hover .menu--level-2,
    [aria-expanded="true"] + .menu--level-2 {
      visibility: visible;
    }

    Conclusion

    Drupal front-end developers used to jump through hoops to create

  • elements within Drupal's menu structure, but with the advent of this feature, it's easier than ever before. Using patterns similar to those above, you can create super accessible and WCAG compliant menus that are easy for all users to navigate.

    Special thanks to the Drupal core accessibility maintainers for helping me navigate and learn techniques such as these throughout the creation of the core Olivero theme.

    Jul 15 2020
    Jul 15

    Continuous Deployment, Infrastructure as Code, and Drupal

    The previous article of this series provided an overview of setting up a GitHub Actions workflow that would publish changes into a Kubernetes cluster. This article takes you through each of the steps of such a workflow.

    For reference, here is the GitHub Actions workflow in the sample repository at .github/workflows/ci.yml:

    on:
      push:
        branches:
          - master
    name: Build and deploy
    jobs:
      build:
        runs-on: ubuntu-latest
        steps:
          - uses: actions/[email protected]
            with:
              fetch-depth: 1
    
          - name: Build, push, and verify image
            run: |
              echo ${{ secrets.PACKAGES_TOKEN }} | docker login docker.pkg.github.com -u juampynr --password-stdin
              docker build --tag docker.pkg.github.com/juampynr/drupal8-do/app:${GITHUB_SHA} .
              docker push docker.pkg.github.com/juampynr/drupal8-do/app:${GITHUB_SHA}
              docker pull docker.pkg.github.com/juampynr/drupal8-do/app:${GITHUB_SHA}
          - name: Install doctl
            uses: digitalocean/[email protected]
            with:
              token: ${{ secrets.DIGITALOCEAN_ACCESS_TOKEN }}
    
          - name: Save cluster configuration
            run: doctl kubernetes cluster kubeconfig save drupster
    
          - name: Deploy to DigitalOcean
            run: |
              sed -i 's||docker.pkg.github.com/juampynr/drupal8-do/app:'${GITHUB_SHA}'|' $GITHUB_WORKSPACE/definitions/drupal-deployment.yaml
              kubectl apply -k definitions
              kubectl rollout status deployment/drupal
          - name: Update database
            run: |
              POD_NAME=$(kubectl get pods -l tier=frontend -o=jsonpath='{.items[0].metadata.name}')
              kubectl exec $POD_NAME -c drupal -- vendor/bin/robo project:files-configure
              kubectl exec $POD_NAME -c drupal -- vendor/bin/robo project:database-update

    There are several systems involved in the above workflow. Here is a diagram that illustrates what is happening:

    Building an image

    The first step at the GitHub Actions workflow builds a Docker image containing the operating system and the application code, along with its dependencies and libraries. Here is the GitHub Actions step:

     - name: Build, push, and verify image
      run: |
        echo ${{ secrets.PACKAGES_TOKEN }} | docker login docker.pkg.github.com -u juampynr --password-stdin
        docker build --tag docker.pkg.github.com/juampynr/drupal8-do/app:${GITHUB_SHA} .
        docker push docker.pkg.github.com/juampynr/drupal8-do/app:${GITHUB_SHA}
        docker pull docker.pkg.github.com/juampynr/drupal8-do/app:${GITHUB_SHA}

    The Docker images are named with the Git commit hash via the environment variable {GITHUB_SHA}. This way, you can match deployments with commits in case you need to review or roll back a failed deployment. We also created a personal access token to authenticate against GitHub Packages and saved it as a GitHub Secret as PACKAGES_TOKEN.

    The Dockerfile that docker build uses for building the Docker image is quite simple:

    FROM juampynr/drupal8ci:latest
    COPY . /var/www/html/
    RUN robo project:build

    The base image is juampynr/drupal8ci. This image is an extension of the official Drupal Docker image, which extends from the official PHP Docker image with an Apache web server built in. It has a few additions like Composer and Robo. In a real project, you would use Ubuntu or Alpine as your base image and define all the libraries that your application needs.

    The last command at the above Dockerfile, robo project:build, is a Robo task. Robo is a PHP task runner used by Drush among others. Here is the output of running this task in a GitHub Actions run:

    Step 3/3 : RUN robo project:build
    199 ---> Running in 2b550db4d25e
    200 [Filesystem\FilesystemStack] _copy [".github/config/settings.local.php","web/sites/default/settings.local.php",true]
    201 [Composer\Validate] Validating composer.json: /usr/local/bin/composer validate --no-check-publish
    202 [Composer\Validate] Running /usr/local/bin/composer validate --no-check-publish
    203./composer.json is valid
    204 [Composer\Validate] Done in 0.225s
    205 [Composer\Install] Installing Packages: /usr/local/bin/composer install --optimize-autoloader --no-interaction
    206 [Composer\Install] Running /usr/local/bin/composer install --optimize-autoloader --no-interaction
    207Loading composer repositories with package information
    208Installing dependencies (including require-dev) from lock file

    In essence, robo project:build copies a configuration file and downloads dependencies via Composer.

    The result of the docker build  command is a Docker image that needs to be published somewhere the Kubernetes cluster can pull it during deployments. In this case, we are using GitHub Packages to host Docker images. 

    Connecting GitHub Actions with the Kubernetes cluster

    At this point, we have built a Docker image with our application containing the latest changes. To deploy such an image to our Kubernetes cluster, we need to authenticate against it. DigitalOcean has a Kubernetes action that makes this easy via doctl, its command-line interface.

    - name: Install doctl
      uses: digitalocean/[email protected]
      with:
        token: ${{ secrets.DIGITALOCEAN_ACCESS_TOKEN }}
    
    - name: Save cluster configuration
      run: doctl kubernetes cluster kubeconfig save drupster

    The first step uses DigitalOcean’s GitHub action to install doctl while the second one downloads and saves the cluster configuration, named drupster.

    As for the DigitalOcean setup, there is nothing fancy about it. You can just sign up for a Kubernetes cluster which uses three nodes (aka Droplets in DigitalOcean’s lingo):

    The Kubernetes Dashboard link at the top right corner takes you to a page that shows an overview of the cluster:

     The above dashboard is a great place to monitor deployments and look for errors. Speaking of deployments, this is what the next section covers.

    Deploying the image into the cluster

    Here is the step that deploys the Docker image to DigitalOcean:

    - name: Deploy to DigitalOcean
      run: |
        sed -i 's||docker.pkg.github.com/juampynr/drupal8-do/app:'${GITHUB_SHA}'|' $GITHUB_WORKSPACE/definitions/drupal-deployment.yaml
        kubectl apply -k definitions
        kubectl rollout status deployment/drupal

    The first line fills a placeholder with the image being deployed, which came from the commit hash. The next two lines perform the deployment and verify its status.

    Note: We will inspect each of the Kubernetes objects in the next article of this series, but if you are curious, you can find them here.

    After the deployment is triggered, the Kubernetes master node will destroy the pods and create new ones containing the new configuration and code. This process is called rollout, and it may take a variable time depending on the scale of the deployment. In this case, it took around 20 seconds.

    If something goes wrong while pods are being recreated, there is the chance to undo the changes by rolling back to a previous deployment configuration via kubectl rollout undo deployment/drupal.

    Updating the database

    If the deployment succeeded, then the next step is to import configuration changes and run database updates. An additional command was added to ensure that file permissions are correct due to issues found while setting up the workflow). Here is the step:

    - name: Update database
      run: |
        POD_NAME=$(kubectl get pods -l tier=frontend -o=jsonpath='{.items[0].metadata.name}')
        kubectl exec $POD_NAME -c drupal -- vendor/bin/robo project:files-configure
        kubectl exec $POD_NAME -c drupal -- vendor/bin/robo project:database-update

    The above step completes the workflow. 

    Next in this series…

    The next and last article in this series will cover each of the Kubernetes objects in detail.

    Acknowledgments

    If you have any tips, feedback, or want to share anything about this topic, please post it as a comment here or via social networks. Thanks in advance!

    Jul 08 2020
    Jul 08

    Drupal 7 to 9 Upgrade

    Drupal 9 was recently released, Drupal 8 will reach end-of-life November 2021, and Drupal 7’s life will expire in November 2022 (the date was extended due to COVID-19). With a window of a little under a year and a half to upgrade from 7 to 9, it's time to make a plan, or pursue other options if that timeline isn't viable. But what other options are there?

    Migrate Drupal 7 to another CMS

    Drupal 8 and Drupal 9 aren't the only migration options to get sites off a no-longer-supported Drupal 7. Another option is to move to a different CMS before Drupal 7 reaches end-of-life. 

    One possibility is to switch to Backdrop CMS. Backdrop CMS is a fork of Drupal 7, which offers a built-in upgrade path for Drupal 7 sites. That means migrating to it from Drupal 7 should be relatively easy. While there are some changes in the way things work in Backdrop CMS, it should be a familiar option for longtime Drupal 7 developers.

    While Drupal has been making strides in the enterprise space and working to deliver content through multiple channels(e.g. mobile apps, marketing platforms, decoupled backends), Backdrop targets smaller budget projects that are primarily concerned with websites that deliver content through traditional HTML pages.

    For users looking to buy time before upgrading to Drupal 9, Backdrop may not be the right solution because it's intended to be a permanent alternative to Drupal. While Drupal 7 and Backdrop are very similar, there currently no migration path available to move from Backdrop to Drupal 9.

    There are other CMS options out there, but few provide an automatic or easy migration path from Drupal 7. WordPress offers migration plugins to expedite the move from Drupal 7, but the mental paradigms don't map directly, and developing for WordPress involves a learning curve for longtime Drupal developers. The difficulty of a transition to any other CMS will vary depending on the complexity of your Drupal site and the solution you choose, but most will probably require some time and effort to migrate existing Drupal 7 data into a new destination.

    Build a custom CMS

    Another option is to build and migrate to a completely custom CMS. We’ve seen industries, for instance, the media industry, decide to develop proprietary CMS solutions that address their unique needs and demands.

    Building and maintaining a custom CMS is a huge commitment of time and money: the cost is higher, development is slower, and security is more likely to suffer. The result may be weird legacy paradigms and mounting technical debt as the number of years in a custom platform increases. There is no direct path to migrate from Drupal 7 to a custom CMS, so migrations will be custom. In some cases, the move will entail disposing of legacy content and data entirely and starting over because the complexity of migrating content is substantial.

    Time will tell whether businesses will continue to invest in maintaining and developing these custom CMS platforms, or whether they'll return to mainstream platforms like Drupal 9. Generally, though, the cost of building and maintaining a secure, scalable, robust custom solution will likely dwarf the cost of building with Drupal, even a highly customized Drupal site.

    Retire a Drupal 7 website

    Another solution is to convert some or all of the existing Drupal 7 website to a static site to eliminate the need to maintain it. For a deeper dive into the technical details of transforming a dynamic Drupal site into a static site, check out Sending a Drupal Site Into Retirement.

    Event sites for events that have passed may be good candidates for retirement. Organizations that want to retain historical event information and session summaries, but no longer want to actively support or make updates to these sites, may find that converting them to static sites is a good option.

    Another case for converting a site to a static site includes retiring a site that is infrequently or never updated, where the site maintenance costs more than the value of the site. This would be a good fit for an old sub-section of a site that won't be updated anymore or a department that has been closed or doesn't need to provide new information. 

    Start with a clean slate

    For organizations with a lot of custom code, organizations that are doing a big redesign, or organizations that are dramatically changing their information architecture, starting an entirely new version of the site with a clean slate is a good option. Starting fresh is particularly helpful for organizations who want to clean up, rewrite, or remove old content anyway.

    If there is a desire to maintain some of the old site content as an archive or for historical purposes, some old material could be preserved as a static site as described above, while creating new content in a fresh install.

    Stay on Drupal 7 with an Extended Support agreement

    Organizations that are unable to upgrade to Drupal 9 before Drupal 7 reaches the end-of-life deadline should consider an Extended Support (ES) agreement. The Drupal 7 ES program involves partnering with a Drupal Association-vetted vendor, assures that the vendor coordinates responsible disclosure of security issues and fixes, and ensures that the work toward fixes is shared publicly. While the bug and security fixes that Drupal 7 ES vendors create are released to the public so non-customers can apply them, there are advantages to being a customer of a D7ES vendor. Those benefits will vary by vendor but include, for example: prioritizing the fixes a customer needs, applying the public fixes, performing usual site maintenance, providing custom development, helping with critical site outages, etc.

    While this solution is an excellent short-term option — in fact, it's probably the best option for organizations that don't have the budget to complete a large migration project at this time — it’s not a long-term solution. The Drupal 7 ES program is projected to run for three years, so enterprises must migrate or move to another platform by November 2025.

    The ongoing cost associated with Drupal 7 life support is a maintenance cost, unlike development work that might get amortized over the years for a replatforming. However, it's a necessary cost in today's world where organizations can't afford to be without security updates, and using a platform without security support is actually against many enterprise security policies.

    Use Drupal 7 without support

    Finally, it is theoretically possible for organizations to continue to use Drupal 7 after end-of-life without an ES agreement. Drupal 7 won't instantaneously break when that fateful day in November 2022 arrives. However, the fact that Drupal 7 is already getting less community support as attention shifts to Drupal 8 and Drupal 9 means it's already falling behind as a platform. As a CMS designed in 2011, it's not optimized to deal with the changes in site usage and the internet in general that have taken place in the last decade. In short, it's not an ideal CMS platform to tackle the next decade.

    More critically, the lack of official community security support is a dealbreaker for many organizations. Many simply can't use a platform that doesn't get security updates; it's against security policies.

    Security is an increasing concern for websites that store and use personal information; GDPR, CalOPPA, CCPA, and PIPEDA all stipulate that enterprises must protect personal data. Sites that don't stay up-to-date with security patches and fixes face major liability risks. 

    These risks are a concern for any site that contains sensitive personal information but might be a particular concern for .edu and .gov sites. Millions are paid out due to security breaches under violations of the federal Drivers Privacy Protection Act (DPPA), the Federal Education Privacy Rights Act (FERPA), and the Health Insurance Portability and Accountability Act (HIPAA). Therefore, both .gov and .edu sites must maintain high-security standards to mitigate legal liability in the event of a data breach.

    Beyond liability considerations and security policies, it's also bad business to maintain websites that aren’t actively getting security fixes. Reputation damage when personal information is compromised costs organizations substantial revenue opportunities. Security exploits can also compromise the websites themselves, giving outsiders the ability to alter or take down a website. These security considerations are risks that organizations can't afford to take. 

    In short, while you can use Drupal 7 without support after end-of-life, you shouldn't. So now is the time to come up with a plan for whether to upgrade to Drupal 9 or to pursue some other option for your Drupal 7 site.

    Prepare a path forward

    Regardless of the direction chosen, organizations that aren't ready to upgrade to Drupal 9 must make a decision and devote development resources to preparing a path forward. For most, that means planning for a Drupal upgrade; for others, that means retiring an existing Drupal site. With an end to official security support, there just isn’t a third option. (This article was updated 7/10/2020)

    Jun 30 2020
    Jun 30

    Continuous Deployment, Infrastructure as Code, and Drupal

    Having devoured DevOps books for the past year and a half, it was interesting to find that some of them contained the terms Infrastructure as Code and Continuous Deployment. It was amazing to learn from the DevOps Handbook that companies were already doing these things in the early 2000s. They achieved it via virtualization while nowadays many projects are doing it via containerization, yet the goals of these two concepts are still the same:

    • Infrastructure as code: define all your infrastructure in files, down to the network interface level.
    • Continuous deployment: make production deployments a chore. Small, frequent, and, most important: automatic.

    To experiment with these two concepts, here is a Drupal demo application. You can check out the repository at https://github.com/juampynr/drupal8-do.

     The below screen recording shows a change being made in the configuration (the website's title), committed, and pushed. The GitHub Actions workflow, Kubernetes dashboard, and the Drupal application are then monitored:

    It works! It is far from perfect, but it is a giant step as it is an entirely different way of hosting and deploying a site. Here is the homepage running:

    In the next section, you will see an overview of the setup.

    The setup

    Here is the root of the GitHub repository with some annotations:

    • .github contains the workflow explained in the previous section.
    • definitions contain the Kubernetes objects.
    • Dockerfile defines how to build the project's image.

    Here is a diagram that illustrates what happens behind the screens when pushing code changes to the GitHub repository:

    In the above diagram, GitHub Actions takes care of building the project and creating a release. At the same time, Kubernetes is in charge of the deployment, detaching the database volume from the MySQL pod and re-attaching it to the pod containing the new configuration. The same happens for the Drupal pod, which has a volume to host public files. If you are new to Kubernetes, there is a great introduction with animated diagrams on the official site.

    Below is the GitHub Actions workflow, located in the repository at .github/workflows/ci.yml:

    on:
      push:
        branches:
          - master
    name: Build and deploy
    jobs:
      build:
        runs-on: ubuntu-latest
        steps:
          - uses: actions/[email protected]
            with:
              fetch-depth: 1
    
          - name: Build, push, and verify image
            run: |
              echo ${{ secrets.PACKAGES_TOKEN }} | docker login docker.pkg.github.com -u juampynr --password-stdin
              docker build --tag docker.pkg.github.com/juampynr/drupal8-do/app:${GITHUB_SHA} .
              docker push docker.pkg.github.com/juampynr/drupal8-do/app:${GITHUB_SHA}
              docker pull docker.pkg.github.com/juampynr/drupal8-do/app:${GITHUB_SHA}
    
          - name: Install doctl
            uses: digitalocean/[email protected]
            with:
              token: ${{ secrets.DIGITALOCEAN_ACCESS_TOKEN }}
    
          - name: Save cluster configuration
            run: doctl kubernetes cluster kubeconfig save drupster
    
          - name: Deploy to DigitalOcean
            run: |
              sed -i 's||docker.pkg.github.com/juampynr/drupal8-do/app:'${GITHUB_SHA}'|' $GITHUB_WORKSPACE/definitions/drupal-deployment.yaml
              sed -i 's||${{ secrets.DRUPAL_CRON }}|' $GITHUB_WORKSPACE/definitions/drupal-deployment.yaml
              kubectl apply -k definitions
              kubectl rollout status deployment/drupal
    
          - name: Update database
            run: |
              POD_NAME=$(kubectl get pods -l tier=frontend -o=jsonpath='{.items[0].metadata.name}')
              kubectl exec $POD_NAME -c drupal -- vendor/bin/robo files:configure
              kubectl exec $POD_NAME -c drupal -- vendor/bin/robo database:update

    In essence, it builds a Docker image, pushes it, and then deploys it along with any other configuration changes into a Kubernetes cluster hosted on DigitalOcean.

    While setting up the workflow, there was a time when a check seemed necessary to find out why the site was not working. A search for how to open an SSH connection with the cluster turned up nothing. Further research revealed that you don't use SSH to connect to the cluster, but instead, you use kubectl to open a shell against a container.

    Kubernetes contexts (aka cluster management)

    Kubernetes uses the concept of context to manage cluster configurations. For example, here is the output of listing contexts locally:

    [email protected]:~:$ kubectl config get-contexts
    CURRENT  NAME              CLUSTER           AUTHINFO               
    *        do-sfo2-drupster  do-sfo2-drupster  do-sfo2-drupster-admin
             minikube          minikube          minikube      

    do-sfo2-drupster is the current context, referencing the DigitalOcean cluster used for this article. The second one in the list refers to a cluster created by minikube, a development tool to create and run a Kubernetes cluster locally.

    When the GitHub Actions workflow in the above section runs the command doctl kubernetes cluster kubeconfig save drupster, it is downloading the cluster configuration, saving it at ~/.kube/config, and setting it as the default context. From that point, all kubectl commands will run against such cluster.

    doctl is the command-line interface to interact with DigitalOcean.

    Opening an interactive session against a container

    With the cluster configuration in place for kubectl, here is how to run a command against the container running Drupal. First, you need to identify the pod that hosts the container that runs Drupal, which you can figure out via kubectl get pods:

    [email protected]:~:$ kubectl get pods
    NAME                            READY   STATUS      RESTARTS   AGE
    drupal-5b4f9996cf-dlvtw         1/1     Running     0          29h
    drupal-cron-1590506940-zxglx    0/1     Completed   0          3m10s
    drupal-cron-1590507000-g25g2    0/1     Completed   0          2m10s
    drupal-cron-1590507060-7vmns    0/1     Completed   0          70s
    drupal-cron-1590507120-rnn6t    0/1     Completed   0          10s
    drupal-mysql-6594bfd66b-9j6zd   1/1     Running     0          29h

    The pod to look for is drupal-5b4f9996cf-dlvtw since the other pods are for cron runs and MySQL. With the pod identifier you can run commands against the drupal container running within it:

    [email protected]:~:$ kubectl exec drupal-5b4f9996cf-dlvtw -c drupal -- ls
    Dockerfile Makefile README.md RoboFile.php composer.json composer.lock config definitions docker-compose.yml docs scripts traefik.yml vendor web

    We just sent an ls command to the container which returned the list of files and directories of the application. If run several commands against the Drupal container is required, you can open an interactive session:

    [email protected]:~:$ kubectl exec drupal-5b4f9996cf-dlvtw -c drupal -i -t -- bash
    [email protected]:/var/www/html# ls
    Dockerfile  Makefile  README.md  RoboFile.php  composer.json  composer.lock  config  definitions  docker-compose.yml  docs  scripts  traefik.yml  vendor  web

    As a final note, this is a learning process. If you have any tips, feedback, or want to share anything about this topic, please post it here as a comment or tweet us @lullabot or @juampynr. Thanks in advance!

    The next article in this series will cover each of the GitHub Actions steps in detail.

    Acknowledgments

    Jun 19 2020
    Jun 19

    Is technology by itself good, bad, or neutral? Does the application of technology as its own process carry any inherent viewpoint or judgment? Are technologists — the engineers, scientists, builders, makers, and creators who build, utilize, and share technology — cognizant of their role in the design and functioning of their own tools? If technology advances internal beliefs in a more efficient, widespread manner, is it important to examine those internal beliefs first? 

    These types of questions, as well as questions adjacent to technology — such as around privacy, security,  and surveillance — have been part of an ongoing discussion inside Lullabot. As strategists, designers, and developers, we not only do our work, but we also help our clients reach their own audiences, shape their own business models, and spread their own values into the world.

    Where there is great power there is great responsibility.

    Spider-Man's "Uncle Ben" and Winston Churchill

    Discussions about values, morals, and ethics happen daily in workplaces across the country, with tech-specific examples, including, body cameras and police accountability nationwide, GitHub employees protesting the sale of the software they build to U.S. Immigration and Customs Enforcement, Salesforce employees protesting over continuing contracts with the U.S. Customs and Border Protection, the Moral Machine, which gathers a human perspective on moral decisions made by machine intelligence (such as self-driving cars), and the National Institute of Standards and Technology(NIST) report on how face-recognition algorithms fare at identifying people from different demographics.

    Values are implicit in those of us architecting, producing, and deploying technology, and we have been grappling for the past year with the creation of a living Engineering Values Statement.

    During our 2019 team retreat, Matt Westgate mentioned "engineering values" in his presentation, and our Design & Strategy team led a company exercise inferring values from a hypothetical build of a podcast page. From this work, some wanted to clarify and publicly state the type of work that we engage in to help identify what would be a good fit versus what would not be a good fit for our work from a sales perspective, as well as to share this information with clients, staff, and new hires. 

    The evolution of our engineering values came out of continued focus and clarification of our core values.  As the retreat drew to a close, some volunteers decided to start meeting on a regular basis to come up with the parameters of the project with the ultimate goal of encapsulating a list of succinct engineering values for circulation.

    The idea was to create a reference list to help with various situations, such as:

    • The sales team outlining our default values when we begin an engagement
    • Account managers evaluating values with clients during ongoing engagements
    • Evaluating technology for ourselves or our clients
    • Arising conflicts from engineering problems with ourselves and our clients

    The committee's call for volunteers was: 

    • "We’re looking for 4 to 7 members who can attend a 30-minute meeting each week, along with an hour or two of help with whatever tasks we come up with. Of course, client work will ebb and flow, so we don’t expect this to be 100% every week. And while we’ve called this 'engineering values,' that doesn’t mean this group is only for 'engineers.' We’re sure anyone who's interested will have something valuable to contribute."

    Our volunteer group organized around a series of Paper docs, and met via Zoom every week for a half hour with the first task of brainstorming questions and ideas for what we'd like to ask ourselves and the team. Through the initial creation of a charter document, the group identified a list of "must-have" deliverables as well as some "nice-to-haves:"

    Deliverables

    • A summary of interviews and feedback from the team, with any notable commonalities or conflicts in values.
    • An initial draft of an engineering values document, presented to leadership and the team.
    • A summary of research on what Drupal agencies and the broader industry communicate as their engineering values.
    • A final document based on a round of feedback from the team.

    Nice to haves / to be determined

    • Whatever resources, if any, the sales team would like
    • A page on our company website
    • A marketing announcement (e.g., blog post, podcast, etc)

    In terms of logistics, a weekly meeting for thirty minutes worked well, as not all participants could attend all meetings. Any action items identified were in manageable increments and all could asynchronously connect through a shared folder that included meeting notes and the actual list of values. The recurring calendar invite was held by one person and periodically circulated to the greater team as new hires joined, with a specific focus on making sure the group represented a cross-section of business roles, backgrounds, and geographical basis. Discussions centered around a shared wiki-style document. The facilitator role was held by different participants as availability and capacity changed over time. The working group also opened a Slack channel to post information as well as align around comments, next steps, and action items.

    In terms of process, the group came up with a first round through assessing personal, individual engineering values, and documenting those, discussing in small groups, and aggregating into an initial group-wide list. We sent a Google Form to the entire staff asking, "What are your engineering values?" This triggered a follow-up activity to do a roundtable on whether or not team members consider themselves engineers. We discovered that not everyone (even those who are developers) considered themselves "engineers." Through continuous feedback and sending reminders to participate approximately every six weeks, the group continued to identify who was not in the room and which voices needed to be added to the conversation. 

    Lessons learned

    1. Proposing an initiative

    The project, like any other project, demanded that the working group align around timelines, success criteria, deliverables and scoping, and getting leadership approval and feedback.

    2. Including those not present

    Leaning on other groups, such as our Diversity, Equity, and Inclusion and Accessibility groups, helped unblock us throughout the year. Also, inviting others to participate in ways like a one-off interview provided a nice balance of respecting time and getting unique feedback. 

    3. Space for different perspectives

    Including multiple voices helped form a more comprehensive discussion. Including different perspectives didn't end at the invitation: we followed up. We found we needed to be prepared for surprising disagreements and leaned on the idea that working group leaders should facilitate instead of participate.

    4. It’s OK to opt out (or opt in)

    Everyone in the working group also had client responsibilities and sometimes project scheduling changed, which meant participation by one or many in the group became difficult or impossible. Other times, a project ended so time was available to participate. We asked participants to commit to blocks of availability at a time (say 2-3 months) so the group could rely on individuals.

    5. Delegate!

    Sometimes, initiative leaders had to be direct to not allow their own absences to become blockers. We used dice to assign a facilitator out of available participants.

    In terms of feedback, the team prepared for a presentation at the 2020 Lullabot Team Retreat, to share the first draft with the greater team.  This list had 13 values on it. By soliciting feedback in the form of comments on the document (comments are visibly tied to a specific username), as well as index cards (snapshots of cards were anonymous), the committee gathered a wide variety of responses. Incorporating retreat feedback, the group continued to narrow down the list to six core engineering values. Along the way the working group continued to ask for feedback from operations, marketing, sales, and other departments. We published the values to an internal blog and released them to a private GitHub repository.

    Our team is happy to have phased out the working group and successfully handed over the document, moving to a living document. While we’ve closed the loop on this first phase of the work, we now look forward to discussing these values with the community and engaging around these values with those working in technology who have similar conversations happening within their own organizations.

    With many thanks to Andrew Berry, who facilitated the Engineering Values Committee, for his feedback and review.

    Members of the Engineering Values Committee included: Andrew Berry, Brian Skowron, Darren Petersen, Hawkeye Tenderwolf, Helena McCabe, James Sansbury, Marcos Cano, Mateu Aguiló Bosch, Matt Oliveira, Matt Robison, and Monica Flores.

    Jun 18 2020
    Jun 18

    Drupal 7 to 9 Upgrade

    If you're one of the 70% of Drupal sites that are still on Drupal 7 at the time of this writing, you may be wondering what the upgrade path looks like to go from Drupal 7 to Drupal 9. What does the major lift look like to jump ahead two Drupal versions? How is this different than if you'd upgraded to Drupal 8 sometime in the last few years? And how long will it be before you have to do it again?

    Upgrading via Drupal 8

    Before the release of Drupal 9, the best path for Drupal 7 sites to upgrade to Drupal 9 was to upgrade to Drupal 8. The big selling point in Drupal 9's evolution is that updating from a late version of Drupal 8 to Drupal 9.0 is more like an incremental upgrade than the massive replatforming effort that the older Drupal migrations used to entail. Sites that jumped on the Drupal 8 bandwagon before Drupal 9.0 was released could benefit from the simple upgrade path from Drupal 8 to Drupal 9.0 instead of another big migration project.

    Migrating to Drupal 8 is still a good option for Drupal 7 sites, even though Drupal 9 is now out.

    You might find that essential modules or themes you need are ready for Drupal 8 but not yet available for Drupal 9. The Drupal 8 to Drupal 9 upgrade path for many modules and themes should be relatively trivial, so many of them should be ready soon. But, there could be some outliers that will take more time. In the meantime, you can do the heavy lift of the Drupal 7 to Drupal 8 migration now, and the simpler Drupal 8 to Drupal 9 upgrade later, when everything you need is ready.

    The Drupal 7 to Drupal 8 migration

    The Drupal 7 to Drupal 8 upgrade involves some pretty significant changes. Some of the things you previously needed to do via contributed modules in Drupal 7 are now included in Drupal 8 core. However, the way you implement them may not be the same as some refactoring might be required to get feature parity when you migrate to Drupal 8. 

    The migration itself isn't a straight database upgrade like it was in Drupal 6 to Drupal 7; instead, you can migrate your site configuration and site content to Drupal 8. You have a choice of doing it two ways: 

    1. Migrate everything, including content and configuration, into an empty Drupal 8 installation (the default method).
    2. Manually build a new Drupal 8 site, setting the content types and fields up as you want them, and then migrate your Drupal 7 content into it. 

    For a deeper dive into what these migrations look like, check out An Overview for Migrating Drupal Sites to 8

    Planning migrations

    The Migration Planner is a helpful tool you may want to consider in your migration planning process. This tool queries a database to generate an Excel file that project managers or technical architects can use to help plan migrations. Developers who are performing the migrations can then use the spreadsheets.

    Performing migrations

    Core comes with some capability to migrate content automatically. If your site sticks to core and common contributed content types and fields, you may be able to use these automatic migrations. However, if your site relies heavily on contributed modules or custom code, an automatic migration might not be possible; you may need a custom migration approach.

    The Drupal Migrate UpgradeMigrate Plus and Migrate Tools modules are good starting points for performing a custom migration. They add things like Drush support for the migration tasks and migration support for some non-core field types. You can access several custom migration processors that make it easy to do some fairly complex migrations. This can be done just by adding a couple of lines to a YAML file, like an entity_lookup processor that will take text from Drupal 7 content and do a lookup to determine what Drupal 8 entity the text refers to.

    Drupal 7 works on older versions of PHP but recommends a minimum of 7.2. If you're migrating from an older Drupal 7 site, there may be several other platform requirements to investigate and implement. 

    Tooling and paradigm shifts

    With the change to Drupal 8, developers are also expected to use new tools. You now use Composer to add modules and their dependencies, rather than Drush. Twig has replaced PHPTemplate as the default templating engine. Some core paradigms have shifted; for instance, developers need to learn to think in terms of events, or extending objects, instead of the old system of hooks. Many hooks still work, but they will probably be deprecated over time, and the new methods are safer ways to write code. The changes aren't insurmountable, but your organization must invest in learning the new way of doing things. You'll need to account for this education overhead when coming from Drupal 7; development teams may need more time to complete tasks as they learn new tools and paradigms.

    Drupal 8's deprecation model

    In addition to big changes in Drupal 8 core and implementation details, Drupal 8 also features a deprecation model that's familiar in the software world, but new in Drupal version upgrades. Instead of deprecating a bunch of code when there's a major version upgrade, Drupal 8 has introduced a gradual deprecation model. 

    As features and improvements are made in Drupal 8's codebase, old methods and functions are marked as deprecated within the code. Then, a few versions later - or in Drupal 9 - that code is removed. This gives development teams a grace period of backward compatibility, during which they can see alerts that code is deprecated, giving organizations time to implement the new code before it's completely removed. 

    The deprecated code also provides an easy hint about how to rework your code using new services and methods. Just look at what the hook does, and do that directly in your code.

    This gradual deprecation model is one of the core reasons that the Drupal 9 upgrade is more like a minor version release for Drupal 8 than a major replatforming effort.

    Jumping straight to Drupal 9

    With that said, can you jump ahead from Drupal 7 to Drupal 9? If you want to skip over Drupal 8 entirely, you can jump directly to Drupal 9. The Drupal 7 migration ecosystem is still available in Drupal 9. Drupal 9 contains the same migrate_drupal module you need to migrate to Drupal 8. There has been discussion around possibly moving this module to a contributed module by Drupal 10, although no decision has been made at the time of this writing.

    If you intend to go this route, keep in mind that all of the considerations when upgrading from Drupal 7 to Drupal 8 apply if you jump straight to Drupal 9, as well. You'll still have to manage the migration planning, deal with tooling and paradigm shifts, and consider platform requirements.

    Ultimately, however, jumping directly from Drupal 7 to Drupal 9 is a valid option for sites that haven't migrated to Drupal 8 now that Drupal 9 is released. 

    When to migrate to Drupal 9

    Whichever route you choose, whether you're going to migrate via Drupal 8 or straight to Drupal 9, you should start the migration from Drupal 7 to Drupal 9 as soon as possible. Both Drupal 7 and Drupal 8 will reach end-of-life in November 2021, so you've got less than a year and a half to plan and execute a major platform migration before you'll face security implications related to the end of official Drupal security support. We'll cover that in more detail later in this series. 

    For any site that's upgrading from Drupal 7, you'll need to do some information architecture work to prepare for the migration to Drupal 8 or Drupal 9. Once you're on Drupal 8, though, the lift to upgrade to Drupal 9 is minimal; you'll need to look at code deprecations, but there isn't a major content migration to worry about. Check out our Preparing for Drupal 9 guide for more details around what that planning process might look like.

    But what about waiting for a later, more stable version of Drupal 9, you ask? This is a common strategy in the software world, but it doesn't apply to the Drupal 9 upgrade. Because Drupal 9 is being handled more like an incremental point-release upgrade to Drupal 8, there aren't any big surprises or massive swaths of new code in Drupal 9. The core code that powers Drupal 9 is already out in the world in Drupal 8. There are no new features in the Drupal 9.0 release; just the removal of code that has already been deprecated in minor versions of Drupal 8.

    Going forward, the plan for Drupal 9 is to release new features every six months in minor releases. The intent is for these features to be backward compatible, and to bring Drupal into the current era of iterative development versus the major replatforming projects of olde. There aren't any big surprises or major reliability fixes on the horizon for Drupal 9; just continued iteration on a solid platform. So there's no need or benefit to waiting for a later version of Drupal 9!

    How to migrate to Drupal 9

    Plan for migration

    Planning for a Drupal 7 to Drupal 8 or Drupal 7 to Drupal 9 migration becomes a question of scope. Do you just want to migrate your existing site's content into a modern, secure platform? Or are you prepared to make a bigger investment to update your site by looking at information architecture, features, and design? Three factors that will likely shape this decision-making process include:

    • Time and budget
    • Developer skillset
    • Release window

    Time and budget for a migration

    How much time are you able to allocate for what is likely to be a major replatforming effort? What's your budget for the project? Do you need to launch before an important date for your organization, such as college registration or an important government deadline? Can your budget support additional work, such as a design refresh? 

    For many organizations, getting the budget for a large project is easier as a one-time ask, so doing the design refresh as part of the migration project may be easier than migrating, and then planning a separate design project in six months. In other organizations, it may be difficult to get enough budget for all the work in one project, so it may be necessary to spread the project across multiple phases; one phase for the migration, and a separate phase for design.

    When factoring in the time and budget for additional work, keep in mind that things like revisiting a site's information architecture could save you time and money in the migration process. Budgeting the time to do the work up-front can dramatically save in time and cost later in the migration process, by reducing unnecessary complexity before migrating instead of having to work with custom migrations to bring over content and entity types that you don't use anymore. This also improves maintainability and saves time for developers and editors doing everyday work on the new site.

    Consider developer skills when planning your migration

    10 years is a long time for developers to be working with a specific framework. If you've been on Drupal 7 since 2011, your developers are likely very experienced with "the Drupal 7 way" of doing things. Many of those things change in Drupal 8. This is a big factor in developer resistance around upgrading to Drupal 8 and Drupal 9.

    Composer, for example, is a huge change for the better when it comes to managing dependencies. However, developers who don't know how to use it will have to learn it. Another big difference is that a lot of Drupal 8 and Drupal 9's core code is built on top of Symfony, which has changed many mental paradigms that experienced Drupal developers are accustomed to using. While some things may seem unchanged - a Block is still a Block, for example - the way they're implemented is different. Some things don't look the same anymore; developers will encounter things like needing to use YAML files instead of hooks to create menu items. Even debugging has changed; things like simple debugging via print() statement doesn't always cut it in the new world, so many developers are using IDEs like PHPStorm, or a host of plugins with other editors, just to code effectively in newer versions of Drupal.

    All of this change comes with overhead. Developers must learn new tools and new ways of doing things when switching from Drupal 7 to Drupal 9. That learning curve must be factored into time and budget not only for the migration itself but for ongoing development work and maintenance after the upgrade. Progress during sprints will likely slow, and developers may initially feel resistant or frustrated while they learn the new ways of doing things.

    Bringing in outside help during the migration process can mitigate some of this learning overhead. Partnering with an experienced Drupal development firm means your migration can be planned and implemented more quickly. One thing to consider when selecting an outside partner is how closely they collaborate with your internal team. When choosing a Drupal development firm to collaborate with your internal team, consider the value of partnering with experienced developers who can "teach" your internal teams how to do things. This reduces the learning curve for a team that's heavily experienced with older Drupal versions and can help your team get up to speed more quickly - saving money during the first year of your new site.

    Plan a release window

    The other aspect of planning for the Drupal 7 to Drupal 9 upgrade is planning a release window. Plan to have your migration project complete before Drupal 7 is scheduled to reach end-of-life in November 2021. If you can't make that deadline, then start planning now for an Extended Support engagement to keep your site secure until you're able to complete the migration.

    You'll want to plan the release window around key dates for your organization, and around other support windows in your stack. For example, if you're a retailer, you may want to have the migration completed before the end of Q3 so you're not upgrading during holiday initiatives. Education organizations may plan their release during slow periods in the school's calendar, or government websites may need to be ready for key legislation. 

    When it comes to your stack, you'll want to plan around other important release windows, such as end-of-support for PHP versions, or upgrading to Symfony 4.4. This is particularly important if you need to upgrade dependencies to support your Drupal 7 to Drupal 9 migration. Check out Drupal 8 Release Planning in the Enterprise for more insights about release planning.

    Revisit information architecture, features, and design

    Because the jump from Drupal 7 to Drupal 9 is so substantial, this is a good time to revisit the information architecture of the site, do a feature audit, and consider whether you want to make design changes. 

    Is it time to update your site's information architecture?

    Before you jump into a Drupal 9 upgrade project, you should perform an audit of your existing Drupal 7 site to see what you want to carry forward and what you can lose along the way. Did you set up a content type that you only used once or twice, and never touched again? Maybe you can delete that instead of migrating it. Are you using a taxonomy that was set up years ago, but no longer makes sense? Now is a good time to refine that for the new version of your site.

    Content migration is also a relatively easy time to manipulate your data. You can migrate Drupal 7 nodes or files into Drupal 9 media entities, for instance. Or migrate text fields into address fields or list fields into taxonomy terms. Or merge multiple Drupal 7 content types into a single Drupal 9 content type. Or migrate content from a deprecated Drupal 7 field type into a different, but supported, Drupal 9 field type. These kinds of things take a bit more work in the migration, but are completely possible with the Migration toolset, and are not difficult for developers with migration experience. The internet is full of articles about how to do these kinds of things.

    In addition to the fine details, it's also a good time to take a look at some big-picture questions, like who is the site serving? How has this changed since the Drupal 7 version of the site was established, and should you make changes to the information architecture to better serve today's audience in the upcoming Drupal 9 site? 

    Have your feature needs changed?

    Drupal 7 was released in 2011. Nearly a decade later, in 2020, the features that seemed important at Drupal 7's inception have changed. How have the feature needs of your content editors changed? Has your site become media-heavy, and do your content editors need large searchable image archives? Do you want to deliver a dynamic front-end experience via a bespoke React app, while giving content editors a decoupled Drupal framework to work in? 

    Many editors love the new Layout Builder experience for creating customized site pages. It's something that doesn't exist in Drupal 7 core and is arguably better than what you get even when you extend Drupal 7 with contributed modules. Drupal 8 and 9 have built-in media handling and a WYSIWYG editor, eliminating the need for dozens of Drupal 7 contributed modules that do not always cooperate with each other, and focusing developer attention on the editorial UX for a single canonical solution.

    Revisit the needs of your content editors and site users to determine whether any existing features of the current site are no longer important and whether new feature needs warrant attention in the upgrade process. This could be particularly helpful if you find that current features being provided by contributed modules are no longer needed; then you don't have to worry about whether a version of those modules is available in Drupal 8/9, and can deprecate those modules.

    Ready for a design update?

    If your Drupal 7 site hasn't had a design refresh in years, the upgrade process is a good time for a design refresh. Plan for a design refresh after the upgrade is complete. Drupal 9 will have a new default theme, Olivero, which features a modern, focused design that is flexible and conforms with WCAG AA accessibility guidelines. Olivero has not yet been added to Drupal core - it's targeted to be added in 9.1 - but it is available now as a contributed module any Drupal 8 or Drupal 9 site can use. Olivero is a great starting point for sites that want an updated design. 

    If you're planning a custom design project, keep accessibility and simplicity at the forefront of your design process. You may want to engage in the design discovery process with a design firm before you plan your Drupal 9 release; a good design partner may make recommendations that affect how you proceed with your migration.

    Perform the migration

    The process of migrating from Drupal 7 to Drupal 8 has improved since Drupal 8's initial release, but it can still be an intricate and time-consuming process for complex sites. We wrote An Overview for Migrating Drupal Sites to 8 to provide some insight around this process, but upgrading sites must:

    • Plan the migration
    • Generate or hand-write migration files
    • Set up a Drupal 8 site to actually run migrations
    • Run the migrations
    • Confirm migration success
    • Do some migration cleanup, if applicable

    Unlike prior Drupal upgrades, migrating to Drupal 8 isn't an automatic upgrade. A Drupal 7 site's configuration and content are migrated separately into a new Drupal 8 site. There are tools available to automate the creation of migration files, but if you've got a complex site that uses a lot of custom code or many contributed modules, you'll only go so far with automated tools. You'll need to revisit business logic and select new options to achieve similar results or deprecate the use of Drupal 7 contributed modules and custom code in your site to move forward to Drupal 8 and Drupal 9.

    Whether you're going upgrade to Drupal 8 and then Drupal 9, or migrating directly from Drupal 7 to Drupal 9, these migration considerations and the process itself will be the same. The only difference would be whether the new site you migrate content into is a Drupal 8 site or a Drupal 9 site.

    Upgrading from Drupal 8 to Drupal 9

    If you choose to go through Drupal 8, once you get to Drupal 8, finishing the migration to Drupal 9 is relatively easy. Upgrade to the latest version of Drupal 8; the upgrade to Drupal 9 requires Drupal 8.8.x or 8.9.x. Along the way, you'll be notified of any deprecated code or contributed modules you'll need to remove before upgrading to Drupal 9. Make sure any custom code is compatible with Drupal 9, and then update the core codebase to Drupal 9 and run update.php

    Voila! The long upgrade process is complete. 

    May 27 2020
    May 27

    Drupal 7 to 9 Upgrade

    Drupal 7, our much-loved CMS that was released in 2011, is nearing the end of its life. No, that's not hyperbole; Drupal 7 is scheduled to reach end-of-life in November 2021. Drupal 8 has been out for a few years, but at the time of this writing, Drupal core usage statistics indicate that only about 350,000 of the more than 1.1 million reporting Drupal core sites are using Drupal 8.x. Over 730,000 of those sites are still using Drupal 7. If your site is one of those 730,000 still on Drupal 7, should you upgrade to Drupal 9? 

    Drupal 7 is coming to an end

    Whether or not you choose to upgrade to Drupal 9, it's time to acknowledge one very important truth: Drupal 7 is coming to an end. After a decade in service, Drupal 7 will stop receiving official community support in November 2021, and the Drupal Association will stop supporting Drupal 7 on Drupal.org. Automated testing for Drupal 7 will stop being supported via Drupal.org, and Drupal 7 will no longer receive official security support.

    Beyond the loss of support for Drupal 7 core, there is less focus on the Drupal 7 version of many contributed modules. Some of them are quite stable and may work well into the future, but others are more neglected. The reality is that once module maintainers have moved their own sites to Drupal 8 or Drupal 9, they may lose interest in spending the time it takes to keep a Drupal 7 version of their code up to date.

    Upgrading from Drupal 7 is harder than from Drupal 8

    Drupal 8 was released in November 2015. When the Drupal Association announced Drupal 9, they discussed a big change coming to the Drupal ecosystem: Major Drupal version changes would no longer be a substantial replatforming effort, but would instead be a continuation of an iterative development process. In practice, this means that Drupal 9 is built in Drupal 8, using deprecations and optional updated dependencies. The result is that upgrading from Drupal 8 to Drupal 9 is just an iterative change from the final Drupal 8 version. Drupal 9.0 involves the removal of some deprecated code, but introduces no new features; it's a continuation of the fully-tested, stable codebase that is Drupal 8. Basically, Drupal 9.0 is just another release of Drupal 8. 

    On the other hand, Drupal 7 has significant differences from Drupal 8 and 9. The jump from Drupal 7 to Drupal 9 can be an enormous undertaking. Third-party libraries replaced huge swaths of custom Drupal code. The procedural code was reworked into object-oriented code. The code changes were massive. Upgrading a Drupal 7 site to Drupal 9 will bring it into the new upgrade paradigm, but there's quite a bit of work to do to get there.  So the question of whether, and how, to make the jump to Drupal 9 is more complicated.

    That leaves Drupal 7 sites with a handful of options:

    We’ll focus on the first option in this article, and the others later.

    Benefits of Drupal 8 and 9

    While Drupal 8 is a big change from Drupal 7, it features many developmental and editorial improvements that pay dividends for users who are willing to take the time to learn how to use them.

    Lots of contributed module functionality now in core

    One of the biggest upsides of Drupal 8 and Drupal 9 versus Drupal 7 is the fact that many of the things that require contributed modules in Drupal 7 are just baked into core now. This includes things like:

    • Layout Builder provides the ability to create customized page layouts that Panels or Display Suite provide in Drupal 7.
    • Blocks have been re-imagined to be fieldable and re-usable, things that require contributed modules like Bean in Drupal 7.
    • You don’t need to install a contributed module and third-party libraries to get a WYSIWYG editor; it’s built into core.
    • Views is in core, and most of the custom lists in core are now fully customizable views.
    • Media handling is not an add-on. It’s an integral feature. To get similar functionality in Drupal 7, you need a half dozen or more complicated contributed Media framework modules, each of which might require quite a bit of additional configuration. You can get a pretty decent media handling experience in Drupal 9 by doing nothing more than enabling the Media and Media Library modules and using the default configuration.
    • Web services are built in, like JSON:API.
    • Customized editorial workflows are now available in core, providing functionality that would have required contributed modules like Workbench Moderation or Workflow.

    That’s just to mention a few features; there are many things in core that would require contributed modules in Drupal 7.

    Maintaining this functionality is simplified by having more of it in core. Managing fewer contributed modules simplifies the process of keeping them in sync as you update versions and dependencies, and trying to decide what to do when you get a conflict or something breaks. As Drupal 7 development falls by the wayside, this is even more important, as it could take months - or longer - to get updates to Drupal 7 contributed modules, until they’re no longer supported at all after end-of-life.

    Having these solutions in core means everyone is using the same solution, instead of splintering developer focus in different directions. And having them in core means they’re well-tested and maintained.

    Composer gets us off the island

    One of the changes to Drupal since the Drupal 7 release is that Drupal 8 and 9 extensively use third party libraries like Symfony for important functionality, instead of relying on custom Drupal-specific code for everything. That move “off the island” has introduced a need to manage Drupal’s dependencies on those libraries. This is handled with yet another tool, a package called Composer.

    You need to manage the dependencies of these new top-level third-party libraries, but each of these libraries has dependencies on other libraries, which have dependencies on more libraries, creating a confusing spiderweb of dependencies and requirements and potential conflicts. Dependency management quickly becomes a maintenance nightmare. It’s a new tool to learn, but Composer is a great dependency manager. Taking the time to learn Composer gives developers a powerful new tool to deal with dependency management.

    Composer can do other things. If you add cweagans/composer-patches, it’s also a very useful tool for managing patches from Drupal.org. You can add a patches section to composer.json with links to the patches you want to watch. Composer will automatically apply the patches, and your composer.json file becomes a self-documenting record of the patches in use.

    You can read more about Composer in another Lullabot article: Drupal 8 Composer Best Practices.

    No more Features for configuration management

    In Drupal 7, many sites deploy configuration using the Features module. Depending on who you ask, using Features for configuration management could be regarded as a good thing or a bad thing. Many developers maintain that Drupal 8 (and therefore Drupal 9’s) Configuration Management system, which allows database configuration to be exported to YML files, is much easier than the Drupal 7 Features system. As with Composer, it takes time to learn, but it enables developers who understand the system to accomplish more with less effort. 

    Secure PHP support

    Drupal 7 sites could be running on deprecated versions of PHP, even as old as 5.3. Drupal 7 sites should already have moved to PHP 7, but could still be running on older, very outdated and insecure, versions of PHP. Drupal 7 currently works with PHP 7.3 but has problems with PHP 7.4. As PHP continues to progress and deprecate older versions, you may find that you can no longer keep your Drupal 7 site running on a secure version of PHP. Drupal 8 runs on PHP 7.0+, and Drupal 9 runs on and requires a minimum of PHP 7.3, so both provide a better window of compatibility with secure PHP versions.

    Resistance to migrating to Drupal 8 and 9

    There are some reasons why sites delay making this move:

    Lack of Drupal 8 versions of Drupal 7 contributed modules

    Early in Drupal 8’s release cycle, one of the big complaints about Drupal 8 was that many Drupal 7 contributed modules no longer worked in D8. It did take time for some contributed modules to be updated to Drupal 8. However, many Drupal 7 contributed modules were no longer needed in Drupal 8, because the functionality they provided is now a part of Drupal 8 core.

    If you haven’t checked the state of Drupal contributed modules in the last few years, take a look at what’s now available for Drupal 8. You can check the Drupal 8 Contrib Porting Tracker to find the status of popular Drupal 7 modules and see whether they’ve gotten a Drupal 8 stable release. You may find that modules that were missing early on are now available, or that you no longer need some contributed modules because that functionality is now managed in another way.

    More importantly, you don’t have to worry about lack of parity in Drupal 8 contributed modules when Drupal 9 is released; as long as the Drupal 8 module in question isn’t built on deprecated code, everything that works in 8.x should continue to work in Drupal 9. And if a D8 module is built on deprecated code, the maintainer should be aware of it. All the code that is being removed in Drupal 9 has already been deprecated in Drupal 8.8, so there won’t be any surprises for module or site maintainers.

    Maintenance overhead for small teams

    With the introduction of Drupal 8 and Drupal 9, the new paradigm in Drupal development is more frequent, smaller releases. This mirrors a larger trend in software development, where iterative development means frameworks make more frequent releases, and consequently, those releases aren’t supported as long. 

    This means you need to commit to keeping your site current with the latest releases. If you’re part of a small team managing a large Drupal site, you may simply not have the bandwidth or expertise to keep up with updates. 

    There are some tools to make it easier to keep a site current. There is an Automatic Updates module that might be helpful for small sites. That module is a work in progress, and it does not yet support contributed module updates or composer based site installs. These are planned for Phase 2. But this is a project to keep an eye on. 

    You can manage updates yourself using Composer and Drush. Sites of any size can also use  DependaBot, a service that creates automatic pull requests with updates. 

    And of course, some web hosts and most Drupal vendors will provide update services for a fee and just take care of this for you.

    The new way of doing things is harder

    The final complaint that has prevented many Drupal 7 sites from upgrading to Drupal 8 and Drupal 9 is that the new way of doing things is harder. Or, if not harder, different. There’s a lot to unpack here. In some cases, this reflects resistance to learning and using new tools. In other cases, it may be that long-time Drupal developers have a hard time learning new paradigms. Another option may be that developers are simply not interested in learning a new stack, and may no longer want to develop in new versions of Drupal. 

    Drupal 6 and 7 have a lot of “Drupalisms,” Drupal-specific, custom ways of doing things, so developers who have been deep into Drupal for a long time may feel the number of things to re-learn is overwhelming. Fortunately, the “new” things, such as Composer, Twig, and PHPUnit are used by other PHP projects, so there is a lot that Drupal 7 developers can learn that will be useful if they work on a Symfony or Laravel project, for example.

    Developing for Drupal 8 and Drupal 9 is certainly different compared to Drupal 7 and older versions. Some developers may choose this as a turning point to shift gears into other career paths, developing for a different stack, or making a more substantial change. But with the Drupal 7 end-of-life approaching, developers who don’t want to move to Drupal 8 and Drupal 9 must make some move, just as Drupal 7 sites must move to a modern platform.

    Security considerations

    In today's world, enterprises have a responsibility to protect their website users' personal data - and they face costly liability considerations if they don't. For many organizations, this means website security is a looming and ongoing concern. It's common for enterprise security policies to require that organizations only use services with ongoing security support. Relative to the Drupal 9 upgrade, this means that many enterprises can't continue to maintain Drupal 7 websites after they stop receiving security support.

    But what does “no more security support” actually mean?

    When Drupal 7 reaches end-of-life, the Drupal community at large will no longer provide “official” security updates or bug fixes. The Drupal Security Team will no longer provide support or Security Advisories for Drupal 7 sites. Automated or manual processes that you currently use to update your sites may no longer work.

    There is a bit of nuance to the lack of security support, however. The Drupal 7 ES program involves partnering with a Drupal Association-vetted vendor and assuring that the vendor is coordinating responsible disclosure of security issues and fixes while publicly sharing the work toward those fixes.

    Practically speaking, this means that even if you’re not partnered with an ES vendor, you can still get security patches for your site. However, websites using modules that aren’t actively supported by ES vendors won’t have the benefit of a partner to hunt down and fix issues with those modules, security, or otherwise. If you have modules or other dependencies that age out of security updates, such as the end-of-life of the PHP version you’re hosting, you may be left with a website with an increasing number of security holes.

    Additionally, after November 2021, Drupal 7 core and Drupal 7 releases on all project pages will be flagged as not supported. As a result, third-party scans may flag sites using Drupal 7 as insecure since they’ll no longer get official security support.

    No more bug fixes or active development

    Alongside security considerations, a lesser concern of the Drupal 7 end-of-life timeline is an official end to community-at-large bug fixes and active development. Drupal 7 development has already shifted to focus on Drupal 8 over the past several years, with Drupal 7 bugs lingering. For example, take a look at the Drupal.org issue queue for Drupal 7 core bugs; you’ll see issues that haven’t been updated for weeks or months, versus hours or days for Drupal 8/9 development issues.

    Questions to ask when migrating from Drupal 7

    So how do you decide which path is right for your organization? Here are some questions to ask.

    What are the skills and size of your development team?

    The shift from Drupal 7 to Drupal 8 and Drupal 9 involved a shift from Drupal-specific paradigms to incorporating more general object-oriented programming concepts. If your team consists of long-time Drupal developers who haven't done a lot of object-oriented programming, this paradigm shift involves a learning curve that does have an associated cost. For some budget-conscious organizations, this may mean it's more economical to remain on Drupal 7 while developers work on skilling up for Drupal 8/Drupal 9 paradigms.

    Another consideration is the size of your development team. If your team is small, you may need to engage an agency for help or explore the other alternatives mentioned above.

    What are the plans for the site?

    How much active development is being done on the site? Are you planning to add new features, or is the site in maintenance mode? What is your budget and plan to maintain the site; do you have developers devoted to ongoing maintenance, or is it one small priority among many competing priorities? 

    If you're planning to add new features, the best option is to migrate to Drupal 8 and Drupal 9. Drupal 9 is under active development, and these modern systems may already include the new features you want to add. If not, working in an ecosystem that's under active development generally reduces development overhead. 

    What is the life expectancy of the site?

    How many years do you expect the current iteration of the site to continue? Are you planning to use the site for three more years before a major redesign and upgrade? Eight more years? Sites with a shorter lifespan may be good candidates for Drupal 7 ES, while sites with longer life expectancies would benefit from upgrading to a modern platform with a longer lifespan.

    What code is the site using?

    Do an inventory of your site's code. What contributed modules are you using? What do you have that's custom? Drupal 8 upgrade evaluation is a good place to start. 

    Some Drupal 7 contributed modules have Drupal 8 and Drupal 9 versions available, while others no longer apply in a world with different programming paradigms. Still, others may now be a part of Drupal 9 core. 

    If you're using a lot of custom modules and code, migrating to Drupal 8 and Drupal 9 is a bigger project.  You might be able to mitigate some of that by altering the scope of your new site to take more advantage of the new capabilities of core and the available Drupal 8 contributed modules.

    What features do you want?

    Make a list of the features that are important to your organization. This should include features that your site currently has that you couldn't live without, and features you'd like to have but currently don't. Then, do a feature comparison between Drupal 8 and Drupal 9, and any other options you're considering. This may drive your decision to migrate, or you may decide that you can live without "must-have" features based on availability.

    Where to go from here

    Bottom line: with the Drupal 7 end-of-life date coming next year, now is the time to scope your site changes. But where do you go from here? The next few articles in this series explore how and when to upgrade from Drupal 7 to Drupal 9 and alternate solutions if upgrading isn’t a good choice for your organization. Stay tuned!

    May 21 2020
    May 21

    An Accessibility Checklist for Your Website

    Whether you're a developer or a designer, everyone has a role to play in making websites more accessible and usable for site visitors. Our design and development teams got together to create this checklist of high-priority tasks you can execute to make your site more accessible. These are some of the issues that affect users of assistive technologies the most, so they should be first on your list when you're forming a plan to improve website accessibility.

    Accessibility Checklist

    Color Choice - Choose a well-balanced set of complementary colors that meet color contrast standards.

    Color Contrast - When combining colors, verify that they have at least the minimum required color contrast.

    Link Style - Add at least two differentiators to links for users with visual disabilities

    Buttons - Remember color contrast requirements, states (i.e., hover, focus, etc.), and readability when designing and developing buttons for your site.

    Forms - Set up your forms for success by including accessible labels, fields, instructions, errors, and alerts.

    Color Choice

    • Create a visual design for your site using a balance of colors that isn't too distracting, while also not being too timid. This helps organize the site for all users, especially for those with disabilities. The effective use of color combinations in your website can establish a visual hierarchy, organize content, and draw distinctions between things such as foreground and background, or static areas and areas that are interactive. 
    • Use the primary color palette for things like calls to action (CTAs), icons, and any place where highlighting areas visually is important.
    • Save secondary colors to highlight less critical information, such as secondary CTAs or backgrounds.
    • Finally, apply neutral colors to things like text and backgrounds, or to tone things down when there are large areas of color.

    Color Contrast

    One of the designer's most important tasks is to check color contrasts. Once you learn how to do this, you can easily integrate this task into your design workflow. When you are checking color contrasts, you should:

    Web-based

    • Pull the hex value(s) and go to the WebAIM Contrast Checker tool
    • Enter your hex value(s) into the Foreground Color or Background Color field(s)
    • Using the sliders below the Foreground Color or Background Color, change the color values until the Contrast Ratio is at or above these minimum values:
      • For text that's at or over 18.66px and bold, look for a color contrast of at least 3:1
      • For text under 18.66px, look for a color contrast of at least 4.5:1
    • Pull the new hex value(s) and place them into your page

    Desktop-based

    • Download the Colour Contrast Analyser tool from The Paciello Group for Windows/macOS
    • Enter your hex value(s) into the Foreground Color or Background Color field(s)
    • Using the sliders below the Foreground Color or Background Color, change the color values until the Contrast Ratio is at or above these minimums:
      • For text that's at or over 18.66px and bold, look for a color contrast of at least 3:1
      • For text under 18.66px, look for a color contrast of at least 4.5:1
    • Pull the new hex value(s) and place them into your design

    Other useful tools

    • Colorblinding: This extension simulates what your website looks like to a visitor with a color vision impairment.
    • ColorZilla: Advanced Eyedropper, Color Picker, Gradient Generator and other colorful goodies

    Link Style

    Make sure links can be distinguished from regular text. When you use links in body content, they should be visually distinct in two different ways. One of these should be color, but the other differentiator should be independent of color to provide a distinction for colorblind users. When links are placed in the header, footer, or sidebar navigation, this differentiation is not required, although it is recommended. Having links underlined by default and removed on hover/focus is the best option, because most users expect that behavior, and it is also the default behavior of browsers. Other options include highlighting, dots, dashes, an outline, or bolded text.

    A focus state is required on all links, so be sure to include it when setting the hover state. This adds a solid border to a link when a user tabs to it using their keyboard, which helps keyboard-only users who use the keyboard to navigate instead of a mouse. 

    Make sure horizontal and vertical link groups have enough space to enable users to access them easily. Iconography can also help users, giving them another way to distinguish between links and plain text. Users understand content more quickly when paired with a visual cue, such as an icon. Finally, use descriptive text for links instead of general text like "More" or "Click here." Links should have some context to the content they're related to; however, make sure they are kept short and understandable.

    When designing links, think about the following states:

    • Default (unvisited)
    • Visited (already visited)
    • Hover (moused over)
    • Focus (focusable elements via the keyboard tab key, i.e., links, buttons, etc.)
    • Active (clicked on, i.e., tabs or navigation)
    • Disabled (not able to be activated by the user)

     Further reading on link style:

    Buttons

    When we talk about buttons, we're referring to regular form buttons and links that are styled as buttons. When developing for accessibility, form buttons should always be used in forms, and links should be used when you need to redirect the user to another page, site, or an anchor within the page.

    Buttons should have a clear and solid border color that meets color contrast against the background color and a background color that meets color contrast against the text. When you hover over a button, there should be a very noticeable differentiation in the background and text color. Inverting the colors is a good option; alternately, darken the background color, and invert the text color.

    When designing buttons, think about button sizing for both desktop and touch screen. Minimum touch targets should be comfortable for an adult finger to navigate successfully. The Web Content Accessibility Guidelines (WCAG) specify a minimum size of 44x44 pixels, or larger for users such as children or people with motor difficulties. 

    Create button labels that are easy to read. Choose sentence case or title case over uppercase, and make sure the font is big enough for easy readability. Make labels action-oriented, i.e., "Next step," or "Save recipe." Including iconography within your buttons can help users understand actions more quickly. Include button states in all designs. These states provide users with feedback that an action is about to happen. When designing buttons, think about the following states:

    • Default (what a button looks like without any interaction)
    • Hover (on desktop, when a user's cursor is hovering over a button)
    • Active (a user has clicked on, and it is selected)
    • Focus (when a user clicks on or uses a keyboard tab key)
    • Disabled (not active)

    Think about the overall button hierarchy within the system regarding primary, secondary, and tertiary buttons. This hierarchy lets users understand what the primary and secondary calls to action are on a page. 

     Further reading on buttons:

    Forms

    When designing forms, these tips can make them more readable and usable:

    Group related fields 

    • Group logical sections with clear headings, i.e., Personal information, Contact information, Billing information. 
    • Groups of fields (i.e. checkboxes, radio buttons, etc.) should be contained within a that includes a element. The element holds the title for the grouped fields, which will be displayed on the page.
    • Include ample white space between sections to provide a visual distinction across sections. 

    Single column forms

    • Forms are easiest to scan when form titles and fields are stacked in one column and aligned. This allows the eye to quickly scan down a single column instead of zig-zagging across multiple columns.

    Form labels

    • For text fields, it is best practice to place labels above corresponding form fields in a form. Place checkboxes and radio buttons to the right of each field.
    • Use a bolded font to help form labels stand out. A flag on whether the form field is required should be placed right after the label as well. This can be a red asterisk, red "REQUIRED" text, or something similar. Form labels can also contain brief instructions for the particular field; for example, Date (mm/dd/yyyy)
    • In addition to a label, each form field should have descriptive helper text and placeholder text. Left-aligning and stacking form labels over their respective fields improve scannability. Keep the labels short and sweet.
    • Don't use placeholder text as a label, as this text isn't available to screen readers. Placeholder text disappears when the user interacts with the field. This can be annoying for users who forget what the placeholder text said. For sighted users, placeholder text offers an excellent opportunity to give users brief instructions, or show them how to format their information.

    Form fields

    • Form fields should have a clear, solid border color that meets color contrast against the background color and a background color that meets color contrast against the text within the field.
    • The width of the form field should match the content it will contain. For instance, a date field would have a much shorter width than a name field that must accommodate long names.

    Form field states

    • When designing your form fields, include the various field states. These field states give the user visual cues about where they are within the form, and where they're going next.
    • Field states to include are default, focus, feedback, and disabled.

    Form instructions

    • Provide a brief list of form instructions directly above all input forms. A note about required fields is recommended, as well as required formats (i.e., dates).

    Form alerts and errors

    • Use form errors and alerts to concisely explain to users how to fix issues that prevent them from completing the form. Follow color contrast requirements with these alerts and errors.
    • Display alerts as a summary at the top of the form, including brief steps on how the user can fix the issues. You can also include links directly to the fields containing errors within the form. Display errors with each problematic form field to make it easier for a user to find specific error details. This may be inline, above, or under the field. 
    • When a field has an error, change the form field's border to another color. Red is recommended because it's universally understandable as an error in a form. In addition to a color change, another differentiator should be added to form fields when they receive errors. This could include an error icon within the field or to the left of the error message.
    • Try to keep the form lengths short. If fields aren't required, consider whether those form fields are truly necessary on the form. If you don't need them, leave them out altogether. The shorter the form, the fewer opportunities for errors.

    Further reading on forms:

    As designers and developers, we can help make the world more accessible to our users. Half the battle is knowing what needs you should be designing for, and the other half is applying the design during development with the best practices and requirements we've discussed. Let's start today by taking a closer look at our work and finding opportunities to make it more accessible, more usable, and more inclusive for everyone. 

    If you would like to ask us any questions about this article, please join us on Slack in the #accessibility channel. We look forward to seeing you there!

    May 18 2020
    May 18

    You Might Also Like

    This past Friday, the Olivero project reached a major milestone and tagged our first alpha release. You can download it on the Olivero project page. This release marks a point in time where the code is stable(ish!), looks good amazing, and is ready for additional testing!

    What is Olivero?

    Olivero is a new theme that is slated to make it into Drupal core in version 9.1 as the new default theme (replacing Bartik). It’s named after Rachel Olivero, who was a valued community member and accessibility advocate. 

    About this release

    This release has been a long time coming (read about the origin here) and has countless hours from dedicated volunteers poured into it with love. 

    This release is not perfect, in fact, we’ve stated that “perfect is the enemy of good” for our alpha releases! That being said, we’ve done extensive testing with various devices, and browsers (yes — this theme supports Internet Explorer 11), and have done lots of accessibility work, although more still needs to be done!

    You get a feature! You get a feature! Everyone gets a feature!

    Well… almost. Besides cross browser and accessibility work, we’ve included some common features in this initial release.

    • Dropdown menu support — this is self-explanatory, but until Drupal 9.1, core themes did not support multi-level menus. 
    • Option for “always-on” mobile navigation — This is intended for the use case where the page has more top-level menu items than can fit within the header.
    • Background color options for site branding — You can change the site branding (logo) background to suit your needs. The current options are white, gray, and blue (which is the default). 

    As of this release, we’re not including support for dark mode or changing the color of the theme (via color module, or anything else). However, this is on the horizon (likely after the initial core inclusion). 

    How you can help

    We’re at the point now where we can use some real-world testing. Please find things that break, we know they’re there!

    For example, I loaded up Lullabot.com with the new theme and discovered various default styling that can be tightened up (e.g., links within headers).


    We’re also at the point where we can use additional accessibility testing. Please fire up your screen readers, and decrease (or increase) your contrast and take a look!

    As with all free open source projects, please take a look at the issue queue to make sure that the issue isn’t already created.

    Tugboat helps save the day!

    We also want to give out a huge shoutout to the Tugboat team. If you aren’t familiar with Tugboat,  it’s a pull request builder that generates live, working websites from every PR, branch or tag.

    Through the Tugboat service, we worked to set up a Drupal multisite install that has deploy previews with content, without content, and using the minimal profile (note this is currently broken). 

    Check out the Tugboat preview here! Note that the “working” status of this Tugboat preview will be in flux, as we commit code, rebuild the preview, etc.

    Next steps

    We’re off to the races! The next step is incremental alpha releases (either weekly or biweekly). We’re also going to make a list of features that are necessary to tag our first beta and work toward that. 

    We hope to create our first Drupal core path in a month or so. This will give core committers the opportunity to put an eye on the codebase, so it can be included in version 9.1. We’re aiming to get this committed into core in late July-ish.

    Thank you, thank you, thank you!

    We have so many people helping out, and we wouldn’t be here without them. Here is the full list of committers (16 as of today), and this does not even count the people who are doing testing, etc! 
     

    May 15 2020
    May 15

    Matt and Mike talk with Putra Bonaccorsi and host Mike Herchel about Drupal 9's new front-end theme, and its past, present, and future. 

    May 13 2020
    May 13

    As the global pandemic continues to spread — causing widespread sickness and death, restricting in-person human contact, creating additional responsibilities at home or financial hardships, or any of the countless other changes to daily life that have resulted in feelings such as fear, anger, boredom, or uncertainty — this virus has forced some of us to reassess our values and our place in the world. While the majority of us who participate in the Drupal community remain focused squarely on technical issues, others might find now is an especially good time to take a closer look at Drupal's Values and Principles. For those of us privileged enough to have the time and space to consider more philosophical questions, we can ask if Drupal's stated values (still) align with our values, or even consider the role of Drupal in our lives when the pandemic subsides.

    This article — the first in a series of articles exploring Drupal's values and principles — considers Drupal's first principle, "impact gives purpose," which is one aspect of the first value, "prioritize impact." On one level, the first principle is merely practical. It concludes by prioritizing the "stakeholders" we should consider: "When faced with trade-offs, prioritize the needs of the people who create content (our largest user base) before the people who build sites (our second largest user base) before the people who develop Drupal (our smallest user base)." In its simplest form, this principle tells us that Drupal ranks the needs of content creators before the needs of the developers.

    However, the first principle offers considerably more depth. While acknowledging the practical nature of the Drupal software, it calls on us to aspire to a higher goal: "When contributing to Drupal, the goal is often to optimize the project for personal needs ('scratching our own itch'), but it has to be bigger than that." Thus, Drupal is presented as much more than simply a good product.

    The phrase "scratching our own itch" has become a platitude. It's everywhere. The Harvard Business Review called it "one of the most influential aphorisms in entrepreneurship." The phrase is well known among software developers in part because in his influential 1999 book, The Cathedral and the Bazaar, (the highly controversial) Eric S. Raymond wrote, "Every good work of software starts by scratching a developer's personal itch." In the Drupal community, however, we see ourselves as aspiring to much more. 

    As the first principle states, "Slowly, focus shifted from writing the perfect code to growing the project, and amplifying Drupal's impact through better marketing, user experience, and more." 

    Countless individuals and Drupal subgroups express their desire to impact people. For instance, the Drupal agency Palantir prioritizes impact that is "positive," "lasting," "thoughtful," and "deliberate." Over at ThinkShout, a Drupal agency that works "with courageous organizations that put people and the planet first," the "impact" they aspire to in their first core value "is driven by our sense of connectedness and desire to deliver meaningful, measurable results." Countless individuals and organizations in the Drupal community feel motivated by a sincere desire to positively "impact" other human beings.

    Drupal's first principle is especially ambitious in describing the impact of the Drupal community: "Prioritizing impact means that every community member acts in the best interest of the project." It seems unlikely that "every community member" can or should make the Drupal project their top priority. Though it may be idealized, it's a worthy goal. We also must reiterate that people will necessarily begin with their own needs.

    Contributions to the Drupal project should not come at personal expense. Imagine telling a single parent, who recently lost their job and wants to build a career with Drupal, to consistently act "in the best interest of the project." Change should come from individuals who have the capacity to help others. Part of why some of us contribute to Drupal is because we imagine another human being finding value in our work. We do not require those who benefit to give back. In this idealized form, we encourage people to participate, but we give with an open hand and no expectation of reciprocation. We contribute because we believe our actions have meaning. As the first principle states, "We derive meaning from our contributions when our work creates more value for others than it does for us."

    When we look inward to examine our value systems, we probably do not want to find a heap of clichés, and phrases like "prioritize impact" and "create value for others" might sound rather cliché to some ears. In fact, on various lists of "business buzzwords," the word "impact" takes the top slot. The noun "impact" comes from the Latin impactus, which means "collision" or "striking one thing against another." The cultural and historical context of "impact" doesn't negate its usefulness, but if the real goal is to "derive meaning," it might be helpful to reconsider this principle in more human terms.

    As previously noted, much of Drupal's first principle points toward bigger goals that extend beyond the conference room to a human-centered skill that good people work to cultivate: generosity. We seek to help others, both at home and in our careers. The business-friendly language in the first principle like, "maximize the impact the project can have on others," could, for at least some of us, be read as "practice generosity toward others." We seek to use Drupal for Good or even live in (with) Drutopia.

    Thanks to Drupal and its community, some of us possess the fortunate capacity to help others. If that describes you, then consider in what ways you have the opportunity to be generous. Toni Morrison — the iconic writer, activist, and college professor who became the first African-American woman to win the Nobel Prize in Literature — used to tell her students:

    "When you get these jobs that you have been so brilliantly trained for, remember that your real job is that if you are free, you need to free somebody else. If you have some power, then your job is to empower somebody else. This is not just a grab-bag candy game."

    In this case, Morrison's inspirational words apply not just to students, but to countless people in the Drupal community. Many in our community have freedom and power. We have the opportunity to help others. Help other Drupalers. Help kids. Help the homeless. Help anyone in need. Maybe even help Drupal and give to #DrupalCares. If your actions produce positive results, keep going!

    Ultimately, action matters more than language. Whether you feel motivated by the desire to make an impact, or you want to practice generosity, don't let up because the world has changed. Take another look at Drupal's Values & Principles and determine for yourself if they motivate you to action. This is not just a grab-bag candy game.

    May 06 2020
    May 06

    Drupal 8 to 9 Upgrade

    From a project planning perspective, what do organizations need to consider when planning for the Drupal 9 upgrade? What technical details should developers pay attention to? We’ve discussed who should upgrade to Drupal 9 and when to upgrade to Drupal 9. Here’s how to do it.

    Drupal 9 Upgrade Project Planning

    Plan a release window

    Drupal 9 has been released. Drupal 8.9.x is scheduled to reach end-of-life in November 2021, with older versions, such as 8.7.x, slated to stop receiving security support in June 2020. You first need to plan a release window to upgrade to Drupal 8.9.x during this timeframe to make sure your site is upgraded before the older Drupal versions are no longer supported.

    Once on Drupal 8.9, you can perform and release all the preparatory work described below. After that, you’ll be ready to plan a release window for the final upgrade to Drupal 9. 

    For more on planning a release window, check out Drupal 8 Release Planning in the Enterprise. Remember to factor in other development work, updates for the rest of your stack, and other ongoing development projects, and give yourself plenty of time to complete the work.

    Scope the upgrade project

    To scope the upgrade project, you'll need to consider a handful of factors:

    • Deprecated code that must be updated
    • Other development work that you'll do as part of the upgrade project

    We'll dive deeper into how to check for and correct deprecated code and APIs shortly, but first, let's take a look at other development work you might do as part of the upgrade project.

    Solicit feature requests from stakeholders

    Does your website deliver stakeholder-required features using contributed modules that haven't yet been updated for Drupal 9? Do your key stakeholders want new features to serve site users better or meet business objectives? 

    For many organizations, major Drupal re-platforming efforts have provided a cadence for other website development work, including new feature development. If it's been a while since your organization checked in with stakeholders, now might be a good time to do that. 

    Regardless of whether or not you plan to deliver new feature development in the Drupal 9 upgrade project, it's a good idea to make sure you won't lose Drupal 8 contributed modules that provide the functionality your stakeholders can't live without - unless you've got a new way to deliver that functionality in Drupal 9.

    Architecture, content, accessibility audits, and more

    For sites that are already on Drupal 8, the Drupal 9 upgrade is different than many previous major version updates; Drupal 8 to Drupal 9 does not require a content migration, so there's no real need to do a major information architecture audit and overhaul. In this new world, organizations should look at shifting the site redesign and content architecture cadence to an ongoing, iterative model.

    How to Prepare for Drupal 9

    Upgrade to Drupal 8.8 or Drupal 8.9

    If you haven't already updated your Drupal 8 site to the most recent version of Drupal 8.x, that's where you must start. Drupal 8.8 was a big milestone for API compatibility; it's the first release with an API that's fully compatible with Drupal 9. Practically speaking, this means that contributed modules released prior to 8.8 may not be compatible with Drupal 9.

    Beyond API compatibility, Drupal 8.8 and 8.9 introduce further bugfixes, as well as database optimizations, to prepare for Drupal 9. If you upgrade your website and all contributed modules and themes to versions that are compatible with 8.9, those parts of your site should be ready for Drupal 9. 

    Platform requirements 

    One change between Drupal 8 and Drupal 9 is that Drupal 9 requires a minimum of PHP 7.3. Drupal 8 recommends but does not require 7.3. There are new minimum requirements for MYSQL and MariaDB and other databases. And your Drush version, if you use Drush, must be Drush 10. If you need to update any of these, you should be able to do it while still on Drupal 8, if you like. There may be other changes to the Drupal 9 requirements in the future, so double-check the environment requirements.

    Audit for conflicting dependencies

    Composer manages third-party dependency updates and will update Drupal dependencies when you do Composer updates. However, if anything else in your stack, such as contributed modules or custom code, has conflicting dependencies, you could run into issues after you update. For this reason, you should check your code for any third-party dependency that conflicts with the core dependencies. 

    For example, Drupal 9 core requires Symfony 4.4, while Drupal 8 worked with Symfony 3.4. If you have contributed modules or custom code that depends on Symfony 3.4, you'll need to resolve those conflicts before you update to Drupal 9. If your code works with either version, you can update your composer.json to indicate that either version works. For instance, the following code in your module’s composer.json indicates that your code will work with either the 3.4 or 4.4 version of Symfony Console. This makes it compatible with both Drupal 8 and Drupal 9 and any other libraries that require either of these Symfony versions.

    {
      "require": {
        "symfony/console": "~3.4.0 || ^4.4"
      }
    }

    If you have code or contributed modules that require incompatible versions of third party libraries and won’t work with the ones used in Drupal 9, you’ll have to find some way to remove those dependencies. That may mean rewriting custom code, helping your contributed modules rewrite their code, or finding alternative solutions that don’t have these problems.

    Check for deprecated code

    Sites that are already on Drupal 8 can see deprecated code using a few different tools:

    • IDEs or code editors that understand `@deprecated` annotations;
    • Running Drupal Check,  PhpStan Drupal, or Drupal Quality Checker from the command line or as part of a continuous integration system to check for deprecations and bugs;
    • Installing the Drupal 8 branch of the Upgrade Status module to get Drupal Check functionality, plus additional scanning;
    • Configuring your test suite to fail when it tries to execute a method that calls a deprecated code path.

    See Hawkeye Tenderwolf’s article How to Enforce Drupal Coding Standards via Git for more ideas. That article explains how Lullabot uses GrumPHP and Drupal Quality Checker to monitor code on some of our client and internal sites.  

    Many organizations already have solutions to check for deprecated code built into their workflow. Some organizations do this as part of testing, while others do it as part of a CI workflow. In the modern software development world, these tools are key components of developing and maintaining complex codebases.

    While you can do this check in any version of Drupal 8, you’ll need to do a final pass once you upgrade any older Drupal 8 version to Drupal 8.8, because new deprecations have been identified in every release up to Drupal 8.8.

    Refactor, update and remove deprecated code

    If you find that your site contains deprecated code, there are a few avenues to fix it prior to upgrading to Drupal 9. Some of those tools include:

    Flag modules as Drupal 9 compatible

    Once you’ve removed deprecated code from your custom modules, flag them as being compatible with both Drupal 8 and Drupal 9, by adding the following line to your module’s info.yml file.

    core_version_requirement: ^8 || ^9

    What about contributed modules?

    If you're using contributed modules that are deprecated, work with module maintainers to offer help when possible to ensure that updates will happen. You can find Drupal 9 compatible modules, check reports in the drupal.org issue queue for Drupal 9 compatibility, or by checking Drupal 9 Deprecation Status.

    You should update all contributed modules to a Drupal 9-compatible version while your site is still on Drupal 8. Do this before attempting an upgrade to Drupal 9!

    Update to Drupal 9

    One interesting aspect of the Drupal 9 upgrade is that you should be able to do all the preparatory work while you’re still on Drupal 8.8+. Find and remove deprecated code, update all your contributed modules to D9-compatible versions, etc. Once that is done, updating to Drupal 9 is simple:

    1. Update the core codebase to Drupal 9.
    2. Run update.php.

    Drupal 9.x+

    The Drupal Association has announced its intent to provide minor release updates every six months. Assuming Drupal 9.0 releases successfully in June 2020, the Drupal 9.1 update is planned for December 2020, with 9.2 to follow in June 2021.

    To make Drupal 9.0 as stable as possible, no new features are planned for Drupal 9.0. The minor updates every six months may introduce new features and code deprecations, similar to the Drupal 8 release cycle. With this planned release cycle, there is no benefit to waiting for Drupal 9.x releases to upgrade to Drupal 9; Drupal 9.0 should be as stable and mature as Drupal 8.9.

    Other resources

    Apr 22 2020
    Apr 22

    Drupal 8 to 9 Upgrade

    Drupal 9 has been released. It waits, ready to take the baton from Drupal 8. But is your organization ready?

    Quick Drupal adoption isn't automatic. Historically, it's taken years for some significant Drupal versions to gain traction. With a relatively short window between the Drupal 9 release and Drupal 8's end-of-life, however, organizations must move more quickly to adopt Drupal 9 or make other arrangements.

    No penalty for early Drupal 9 adopters

    A common strategy for many technology release cycles is to avoid the initial version of a major software release. Some organizations wait until one or more point releases after a new version, while others prefer to wait months or even years after a major version release for things like bug fixes, additional features, and helpful resources created by early adopters. In the Drupal world, this delay is often exacerbated by waiting for contributed modules to be compatible with the new version.

    The nice thing about Drupal 9 is that there is no penalty for early adopters, so there's no reason to wait for a later version. The initial Drupal 9 version release introduces zero new features. Drupal 9.0 core code matches Drupal 8.9 core. The only differences between Drupal 9.0 and Drupal 8.8 or 8.9 are the removal of deprecated code and required upgrades to third-party dependencies.

    The primary consideration is whether or not your favorite contributed modules have declared that they are Drupal 9 compatible. With past upgrades, waiting for contributed modules to be ready for the new Drupal version caused months or even years of delays. But the Drupal 9 upgrade path for contributed modules is relatively easy, so they should be able to adapt quickly. Many modules are already compatible, and others will need minimal changes.

    When your code is ready

    One of the core components of the Drupal 9 upgrade is the removal of deprecated code in Drupal 8. However, this means that when you're planning your release window, you'll need to schedule some time for the pre-work of auditing and refactoring deprecated code. If you've already been doing this, you may not need to pad your schedule to compensate for this work. We'll dive deeper into how to get your code ready in a future article.

    You'll also need to give yourself time to address any third-party dependencies that require newer versions in Drupal 9. When you're looking at when to upgrade to Drupal 9, you should do it after you've had a chance to resolve any third-party dependency updates that conflict with other things in your stack. If you've got a contrib module or custom code that requires an older version of a third-party dependency, but Drupal 9 calls for a newer version of that dependency, you'll need to make a plan and address this conflict before you upgrade to Drupal 9.

    Consider other website work

    Many organizations have traditionally used major Drupal version migrations as a time to plan overall website redesign projects, information architecture work, and other web development projects. Because the upgrade to Drupal 9 is more like a minor release than a major one, there's no need to deep dive into information architecture - there's no migration! That means your organization needs to establish a new strategy for these projects; we're working on an upcoming article to cover web development strategy for Drupal 9 for more insights around this.

    If business logic dictates that your organization plan other web development projects for this year, make sure you give yourself time to complete the Drupal 9 upgrade before Drupal 8 reaches end-of-life in November 2021. 

    Take the availability of preferred partners and development teams into account

    If you're planning to work with vendor partners, make sure you factor their availability into your project plan. With an upgrade window of slightly over a year between the release of Drupal 9 and the end-of-life of Drupal 8, some vendor partners may have limited availability, especially if yours is a larger project. Planning ahead helps to ensure you can work with your preferred partners; otherwise, you might add the stress of working with a new partner into the mix.

    At the same time, don't forget about internal initiatives. You still have stakeholders you must serve. For example, doing new feature development for content editors while simultaneously maintaining an up-to-date platform consistent with your organization's security policies can mean a dance to prioritize development resources to meet everyone's priorities and deadlines. While this complicates the release planning process, it's essential to consider these factors when determining the timing of upgrading to Drupal 9.

    We dipped our toes into these considerations in Drupal 8 Release Planning in the Enterprise, and those recommendations remain relevant in these situations.

    Missing the Drupal 9 upgrade window

    To summarize, you should upgrade to Drupal 9 earlier rather than later. But what if your site can't upgrade to Drupal 9 before Drupal 8 reaches end-of-life? Unlike Drupal 7, Drupal 8 does not have an extended support program. The upgrade from Drupal 8 to Drupal 9 is such a minor replatforming effort compared to prior versions that the decision was made not to offer an extended support program for Drupal 8. 

    Support will continue through November 2021 for sites upgraded to 8.9.x, but support for that version ends when that Drupal 8 end-of-life date arrives. Older Drupal 8.x versions will cease getting support before that date; 8.7.x stops getting security support as of June 3, 2020, and security support ends for Drupal 8.8.x on December 2, 2020.

    Long-term, your organization needs a plan to upgrade to Drupal 9, or to consider other options. A future article in this series offers more information about what that plan might look like.

    Thanks to the Lullabot team for contributing to this article and to Dachary Carey for drafting it.

    Apr 15 2020
    Apr 15

    Drupal 8 to 9 Upgrade

    This article is the first in a series discussing Who, What, Why, and How Drupal 8 sites can upgrade to the upcoming Drupal 9 release. A future series will discuss upgrading Drupal 7 sites.

    Drupal 9 has been released, and with both Drupal 7 and Drupal 8 scheduled for end-of-life (EOL) in November 2021, it’s time to think about whether to upgrade Drupal sites to the new version. Upgrading to newer versions in the past was a significant replatforming effort that required a substantial investment and a non-trivial release window.

    The Drupal 8 to Drupal 9 upgrade is different, though; this is the first major version upgrade that’s reputed to be as simple as a minor point release. Can it really be that simple? Who should upgrade to Drupal 9?

    The Easy Path: Upgrading from Drupal 8

    Organizations that are already on Drupal 8 are several steps ahead in upgrading to Drupal 9. One of the biggest benefits of upgrading to Drupal 8 is that the platform and core code of Drupal 8 form the basis for Drupal 9. 

    Drupal 9.0 doesn’t introduce any new features or new code, so sites that are on the final Drupal 8 point release are essentially ready to upgrade to Drupal 9.0. No big lift; no major replatforming effort; no content migration; just a final audit to make sure the site doesn’t rely on any deprecated code or outdated Composer dependencies. 

    Sites that have kept up-to-date with Drupal 8’s incremental updates (see Andrew Berry’s article Drupal 8 Release Planning in the Enterprise) should be ready to go when it comes to core code. Many sites are already using automated tools or workflows to keep them up-to-date on code deprecations for contributed and custom modules. Ideally, you have been continuously replacing older versions of contributed modules with versions that have removed deprecated code, removing deprecated code in your custom code, and dealing with any Composer dependency conflicts. If so, the upgrade effort for your site should be relatively simple. The same is true if you rely on widely-used and well-supported contributed modules and have little custom code.

    If you have custom code and use less widely-used contributed modules, but you’ve been paying attention to code deprecations in your custom code and the readiness of your contributed modules, you’re probably in a good position to upgrade. If you have strong test coverage and aren’t relying on any deprecated third-party dependencies, you’re in even better shape. You shouldn’t see substantial changes from Drupal 8 to Drupal 9.0, so even custom code is likely to work without issue as long as it doesn’t rely on deprecated functions or methods that are removed. 

    The caveat is that if your custom code or contributed modules rely on older versions of Composer dependencies that are deprecated in Drupal 9 in favor of newer versions, you may need to do some refactoring to make sure that code works with the new third-party dependencies.

    Can you stay on Drupal 8 past its EOL?

    There should be no reason for anyone on Drupal 8 not to upgrade to Drupal 9.  There will be a small window of time until November 2021, during which the last Drupal 8 release will be supported with security updates. That allows time to make the necessary changes to move to Drupal 9. But after that, you’ll need to make the switch.

    When Drupal 6 reached its end of life, there was a Long Term Support(LTS) program, which made it possible to stay on Drupal 6 past its EOL. There is an LTS program for Drupal 7 and Lullabot is an approved vendor for this program; however, there will be no Long Term Support program for Drupal 8 because the upgrade path from Drupal 8 to Drupal 9 is much easier.

    If you don’t make the move, you’ll be on your own to deal with security updates and other maintenance and bug fixes for your Drupal 8 code. And that would likely be more expensive and time-consuming than just doing the upgrade.

    Prepare to upgrade or start considering alternatives.

    With the Drupal 9 upgrade being relatively uncomplicated for sites already on Drupal 8, it's easy to recommend that those sites upgrade. The main question is when, and what other options do you have? Later articles in this series will delve into more detail about how to prepare for the upgrade.

    Thanks to the Lullabot team for contributing to this article and to Dachary Carey for drafting it.

    About Drupal Sun

    Drupal Sun is an Evolving Web project. It allows you to:

    • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
    • Facet based on tags, author, or feed
    • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
    • View the entire article text inline, or in the context of the site where it was created

    See the blog post at Evolving Web

    Evolving Web