Mar 13 2019
Mar 13

This blog has been re-posted and edited with permission from Dries Buytaert's blog.

Three stars will align and the Open Web will win.

Today, the world wide web celebrates its 30th birthday. In 1989, Sir Tim Berners-Lee invented the world wide web and changed the lives of millions of people around the globe, including mine.

Tim Berners-Lee sitting in front of a computer showing the first website

Tim Berners-Lee, inventor of the World Wide Web, in front of the early web.

Milestones like this get me thinking about the positive impact a free and Open Web has had on society. Without the web, billions of people would not have been able to connect with one another, be entertained, start businesses, exchange ideas, or even save lives. Open source communities like Drupal would not exist.

As optimistic as I am about the web's impact on society, there have been many recent events that have caused me to question the Open Web's future. Too much power has fallen into the hands of relatively few platform companies, resulting in widespread misinformation, privacy beaches, bullying, and more.

However, I'm optimistic that the Open Web has a chance to win in the future. I believe we'll see three important events happen in the next five years.

First, the day will come when regulators will implement a set of laws that govern the ownership and exchange of data online. It's already starting to happen with GDPR in the EU and various state data privacy laws taking shape in the US. These regulations will require platforms like Facebook to give users more control over their data, and when that finally happens, it will be a lot easier for users to move their data between services and for the Open Web to innovate on top of these data platforms.

Second, at some point, governments globally will disempower large platform companies. We can't leave it up to a handful of companies to judge what is false and true, or have them act as our censors. While I'm not recommending governments split up these companies, my hope is that they will institute some level of algorithmic oversight. This will offer an advantage to the Open Web and Open Source.

Third, I think we're on the verge of having a new set of building blocks that enable us to build a better, next-generation web. Thirty years into the web, our data architectures still use a client-server model; data is stored centrally on one computer, so to speak. The blockchain is turning that into a more decentralized web that operates on top of a distributed data layer and offers users control of their own data. Similar to building a traditional website, distributed applications (dApps) require file storage, payment systems, user data stores, etc. All of these components are being rebuilt on top of the blockchain. While we have a long way to go, it is only a matter of time before a tipping point is reached.

In the past, I've publicly asked the question: Can we save the Open Web? I believe we can. We can't win today, but we can keep innovating and get ready for these three events to unfold. The day will come!

With that motivation in mind, I want to wish a special happy birthday to the world wide web!

Feb 26 2019
Feb 26

This post was created jointly by Michael Hess of the Security Working Group, and Tim Lehnen, Executive Director of the Drupal Association.

Last year, with the security release of SA-CORE-2018-002, the most significant security vulnerability since 2014, we heard the pain of site owners and development teams around the world staying up at all hours waiting for the complex security release process to complete and the patch to drop. We heard the pain of agencies and end-user organizations required to put teams on late shifts and overtime. We heard from some users who simply couldn't respond to patch their sites on the day of release, because of lack of resources or entrenched change management policies.

We've heard calls from the community for rotating the timezones for security advisories from release to release, or for having more on-call support from security team members across the globe, or simply for a longer horizon between the release of PSA and SA.

Yet at the same time, we're cognizant that these solutions would put increased burden on a security team composed of dedicated volunteers and contributors. There are a number of generous organizations who sponsor many of the members of the security team, but relying on their altruism alone is not a sustainable long-term solution—especially if we consider expanding the role of the security team to address the larger pain points above.

Last week, with the release of SA-CORE-2019-003, we heard these concerns for site owners and the sustainability of the security team echoed again.

The Security Team and the Drupal Association have been developing solutions for this issue for well over a year.

The goals are simple:

  • Provide a new service to the Drupal community, from small site owners to enterprise-scale end users, to protect their sites in the gap from security release to the time it takes them to patch.
  • Create a new model for sustainability for the Security Team, generating funding that 1) covers the operating costs of the program 2) can support security team operations and 3) can support additional Drupal Association programs.

Although the execution will take care and careful partnership, we are happy to announce that we've found a solution.

We're tentatively calling this: Drupal Steward. It is a service to be provided by the Drupal Association, the Security team, and carefully vetted hosting partners.

Drupal Steward will offer sites a form of mitigation through the implementation of web application firewall rules to prevent mass exploitation of some highly critical vulnerabilities (not all highly critical vulnerabilities can be protected in this fashion, but a good many can be - this method would have worked for SA-CORE-2018-002 for example).

It will come in three versions:

  • Community version - for small sites, low-budget organizations, and non-profits, we will offer a community tier, sold directly by the DA. This will be effectively at cost.
  • Self hosted version - for sites that are too large for the community tier but not hosted by our vendor partners.
  • Partner version - For sites that are hosted on vetted Drupal platform providers, who have demonstrated a commitment of contribution to the project in general and the security team in particular, protection will be available directly through these partners.

Next Steps

The Drupal Association and Security Team are excited to bring this opportunity to the Drupal Community.

We believe that the program outlined above will make this additional peace of mind accessible to the broadest base of our community possible, given the inherent costs, and are hopeful that success will only continue to strengthen Drupal's reputation both for one of the most robust security teams in open source, and for innovating to find new ways to fund the efforts of open source contributors.

We will announce more details of the program over the coming weeks and months as we get it up and running.

If you are a hosting company and are interested in providing this service to your customers, please reach out to us at [email protected].

Please also join us at DrupalCon for any questions about this program.

If you are a site owner and have questions you can join us in slack #drupalsteward.

For press inquiries, please contact us at: [email protected]

Feb 13 2019
Feb 13

This blog has been re-posted and edited with permission from Dries Buytaert's blog.

An abstract image of three boxes

The web used to be server-centric in that web content management systems managed data and turned it into HTML responses. With the rise of headless architectures a portion of the web is becoming server-centric for data but client-centric for its presentation; increasingly, data is rendered into HTML in the browser.

This shift of responsibility has given rise to JavaScript frameworks, while on the server side, it has resulted in the development of JSON:API and GraphQL to better serve these JavaScript applications with content and data.

In this blog post, we will compare REST, JSON:API and GraphQL. First, we'll look at an architectural, CMS-agnostic comparison, followed by evaluating some Drupal-specific implementation details.

It's worth noting that there are of course lots of intricacies and "it depends" when comparing these three approaches. When we discuss REST, we mean the "typical REST API" as opposed to one that is extremely well-designed or following a specification (not REST as a concept). When we discuss JSON:API, we're referring to implementations of the JSON:API specification. Finally, when we discuss GraphQL, we're referring to GraphQL as it used in practice. Formally, it is only a query language, not a standard for building APIs.

The architectural comparison should be useful for anyone building decoupled applications regardless of the foundation they use because the qualities we will evaluate apply to most web projects.

To frame our comparisons, let's establish that most developers working with web services care about the following qualities:

  1. Request efficiency: retrieving all necessary data in a single network round trip is essential for performance. The size of both requests and responses should make efficient use of the network.
  2. API exploration and schema documentation: the API should be quickly understandable and easily discoverable.
  3. Operational simplicity: the approach should be easy to install, configure, run, scale and secure.
  4. Writing data: not every application needs to store data in the content repository, but when it does, it should not be significantly more complex than reading.

We summarized our conclusions in the table below, but we discuss each of these four categories (or rows in the table) in more depth below. If you aggregate the colors in the table, you see that we rank JSON:API above GraphQL and GraphQL above REST for Drupal core's needs.

REST JSON:API GraphQL Request efficiency Poor; multiple requests are needed to satisfy common needs. Responses are bloated. Excellent; a single request is usually sufficient for most needs. Responses can be tailored to return only what is required. Excellent; a single request is usually sufficient for most needs. Responses only include exactly what was requested. Documentation, API explorability and schema Poor; no schema, not explorable. Acceptable; generic schema only; links and error messages are self-documenting. Excellent; precise schema; excellent tooling for exploration and documentation. Operational simplicity Acceptable; works out of the box with CDNs and reverse proxies; few to no client-side libraries required. Excellent; works out of the box with CDNs and reverse proxies, no client-side libraries needed, but many are available and useful. Poor; extra infrastructure is often necessary client side libraries are a practical necessity, specific patterns required to benefit from CDNs and browser caches. Writing data Acceptable; HTTP semantics give some guidance but how specifics left to each implementation, one write per request. Excellent; how writes are handled is clearly defined by the spec, one write per request, but multiple writes is being added to the specification. Poor; how writes are handled is left to each implementation and there are competing best practices, it's possible to execute multiple writes in a single request.

If you're not familiar with JSON:API or GraphQL, I recommend you watch the following two short videos. They will provide valuable context for the remainder of this blog post:

Request efficiency

Most REST APIs tend toward the simplest implementation possible: a resource can only be retrieved from one URI. If you want to retrieve article 42, you have to retrieve it from https://example.com/article/42. If you want to retrieve article 42 and article 72, you have to perform two requests; one to https://example.com/article/42 and one to https://example.com/article/72. If the article's author information is stored in a different content type, you have to do two additional requests, say to https://example.com/author/3 and https://example.com/author/7. Furthermore, you can't send these requests until you've requested, retrieved and parsed the article requests (you wouldn't know the author IDs otherwise).

Consequently, client-side applications built on top of basic REST APIs tend to need many successive requests to fetch their data. Often, these requests can't be sent until earlier requests have been fulfilled, resulting in a sluggish experience for the website visitor.

GraphQL and JSON:API were developed to address the typical inefficiency of REST APIs. Using JSON:API or GraphQL, you can use a single request to retrieve both article 42 and article 72, along with the author information for each. It simplifies the developer experience, but more importantly, it speeds up the application.

Finally, both JSON:API and GraphQL have a solution to limit response sizes. A common complaint against typical REST APIs is that their responses can be incredibly verbose; they often respond with far more data than the client needs. This is both annoying and inefficient.

GraphQL eliminates this by requiring the developer to explicitly add each desired resource field to every query. This makes it difficult to over-fetchdata but easily leads to very large GraphQL queries, making (cacheable) GET requests impossible.

JSON:API solves this with the concept of sparse fieldsets or lists of desired resource fields. These behave in much the same fashion as GraphQL does, however, when they're omitted JSON:API will typically return all fields. An advantage, though, is that when a JSON:API query gets too large, sparse fieldsets can be omitted so that the request remains cacheable.

REST JSON:API GraphQL Multiple data objects in a single response Usually; but every implementation is different (for Drupal: custom "REST Export" view or custom REST plugin needed). Yes Yes Embed related data (e.g. the author of each article) No Yes Yes Only needed fields of a data object No Yes; servers may choose sensible defaults, developers must be diligent to prevent over-fetching. Yes; strict, but eliminates over-fetching, at the extreme, it can lead to poor cacheability.

Documentation, API explorability and schema

As a developer working with web services, you want to be able to discover and understand the API quickly and easily: what kinds of resources are available, what fields does each of them have, how are they related, etc. But also, if this field is a date or time, what machine-readable format is the date or time specified in? Good documentation and API exploration can make all the difference.

REST JSON:API GraphQL Auto-generated documentation Depends; if using the OpenAPI standard. Depends; if using the OpenAPI standard (formerly, Swagger). Yes; various tools available. Interactivity Poor; navigable links rarely available. Acceptable; observing available fields and links in its responses enable exploration of the API. Excellent; autocomplete feature, instant results or compilation errors, complete and contextual documentation. Validatable and programmable schema. Depends; if using the OpenAPI standard. Depends; the JSON:API specification defines a generic schema, but a reliable field-level schema is not yet available. Yes; a complete and reliable schema is provided (with very few exceptions).

GraphQL has superior API exploration thanks to GraphiQL (demonstrated in the video above), an in-browser IDE of sorts, which lets developers iteratively construct a query. As the developer types the query out, likely suggestions are offered and can be auto-completed. At any time, the query can be run and GraphiQL will display real results alongside the query. This provides immediate, actionable feedback to the query builder. Did they make a typo? Does the response look like what was desired? Additionally, documentation can be summoned into a flyout, when additional context is needed.

On the other hand, JSON:API is more self-explanatory: APIs can be explored with nothing more than a web browser. From within the browser, you can browse from one resource to another, discover its fields, and more. So, if you just want to debug or try something out, JSON:API is usable with nothing more than cURL or your browser. Or, you can use Postman (demonstrated in the video above) — a standalone environment for developing on top of an anyHTTP-based API. Constructing complex queries requires some knowledge, however, and that is where GraphQL's GraphiQL shines compared to JSON:API.

Operational simplicity

We use the term operational simplicity to encompass how easy it is to install, configure, run, scale and secure each of the solutions.

The table should be self-explanatory, though it's important to make a remark about scalability. To scale a REST-based or JSON:API-based web service so that it can handle a large volume of traffic, you can use the same approach websites (and Drupal) already use, including reverse proxies like Varnish or a CDN. To scale GraphQL, you can't rely on HTTP caching as with REST or JSON:API without persisted queries. Persisted queries are not part of the official GraphQL specification but they are a widely-adopted conventionamongst GraphQL users. They essentially store a query on the server, assign it an ID and permit the client to get the result of the query using a GETrequest with only the ID. Persisted queries add more operational complexity, and it also means the architecture is no longer fully decoupled — if a client wants to retrieve different data, server-side changes are required.

REST JSON:API GraphQL Scalability: additional infrastructure requirements Excellent; same as a regular website (Varnish, CDN, etc). Excellent; same as a regular website (Varnish, CDN, etc). Usually poor; only the simplest queries can use GET requests; to reap the full benefit of GraphQL, servers needs their own tooling. Tooling ecosystem Acceptable; lots of developer tools available, but for the best experience they need to be customized for the implementation. Excellent; lots of developer tools available; tools don't need to be implementation-specific. Excellent; lots of developer tools available; tools don't need to be implementation-specific. Typical points of failure Fewer; server, client. Fewer; server, client. Many; server, client, client-side caching, client and build tooling.

Writing data

For most REST APIs and JSON:API, writing data is as easy as fetching it: if you can read information, you also know how to write it. Instead of using the GET HTTP request type you use POST and PATCH requests. JSON:API improves on typical REST APIs by eliminating differences between implementations. There is just one way to do things and that enabled better, generic tooling and less time spent on server-side details.

The nature of GraphQL's write operations (called mutations) means that you must write custom code for each write operation; unlike JSON:API the specification, GraphQL doesn't prescribe a single way of handling write operations to resources, so there are many competing best practices. In essence, the GraphQL specification is optimized for reads, not writes.

On the other hand, the GraphQL specification supports bulk/batch operations automatically for the mutations you've already implemented, whereas the JSON:API specification does not. The ability to perform batch write operations can be important. For example, in our running example, adding a new tag to an article would require two requests; one to create the tag and one to update the article. That said, support for bulk/batch writes in JSON:APIis on the specification's roadmap.

REST JSON:API GraphQL Writing data Acceptable; every implementation is different. No bulk support. Excellent; JSON:API prescribes a complete solution for handling writes. Bulk operations are coming soon. Poor; GraphQL supports bulk/batch operations, but writes can be tricky to design and implement. There are competing conventions.

Drupal-specific considerations

Up to this point we have provided an architectural and CMS-agnostic comparison; now we also want to highlight a few Drupal-specific implementation details. For this, we can look at the ease of installation, automatically generated documentation, integration with Drupal's entity and field-level access control systems and decoupled filtering.

Drupal 8's REST module is practically impossible to set up without the contributed REST UI module, and its configuration can be daunting. Drupal's JSON:API module is far superior to Drupal's REST module at this point. It is trivial to set up: install it and you're done; there's nothing to configure. The GraphQL module is also easy to install but does require some configuration.

Client-generated collection queries allow a consumer to filter an application's data down to just what they're interested in. This is a bit like a Drupal View except that the consumer can add, remove and control all the filters. This is almost always a requirement for public web services, but it can also make development more efficient because creating or changing a listing doesn't require server-side configuration changes.

Drupal's REST module does not support client-generated collection queries. It requires a "REST Views display" to be setup by a site administrator and since these need to be manually configured in Drupal; this means a client can't craft its own queries with the filters it needs.

JSON:API and GraphQL, clients are able to perform their own content queries without the need for server-side configuration. This means that they can be truly decoupled: changes to the front end don't always require a back-end configuration change.

These client-generated queries are a bit simpler to use with the JSON:API module than they are with the GraphQL module because of how each module handles Drupal's extensive access control mechanisms. By default JSON:API ensures that these are respected by altering the incoming query. GraphQL instead requires the consumer to have permission to simply bypass access restrictions.

Most projects using GraphQL that cannot grant this permission use persisted queries instead of client-generated queries. This means a return to a more traditional Views-like pattern because the consumer no longer has complete control of the query's filters. To regain some of the efficiencies of client-generated queries, the creation of these persisted queries can be automated using front-end build tooling.

REST JSON:API GraphQL Ease of installation and configuration Poor; requires contributed module REST UI, easy to break clients by changing configuration. Excellent; zero configuration! Poor; more complex to use, may require additional permissions, configuration or custom code. Automatically generated documentation Acceptable; requires contributed module OpenAPI. Acceptable; requires contributed module OpenAPI. Excellent; GraphQL Voyager included. Security: content-level access control (entity and field access) Excellent; content-level access control respected. Excellent; content-level access control respected, even in queries. Acceptable; some use cases require the consumer to have permission to bypass all entity and/or field access. Decoupled filtering (client can craft queries without server-side intervention) No Yes Depends; only in some setups and with additional tooling/infrastructure.

What does this mean for Drupal's roadmap?

Drupal grew up as a traditional web content management system but has since evolved for this API-first world and industry analysts are praising us for it.

As Drupal's project lead, I've been talking about adding out-of-the-box support for both JSON:API and GraphQL for a while now. In fact, I've been very bullish about GraphQL since 2015. My optimism was warranted; GraphQL is undergoing a meteoric rise in interest across the web development industry.

Based on this analysis, for Drupal core's needs, we rank JSON:API above GraphQL and GraphQL above REST. As such, I want to change my recommendation for Drupal 8 core. Instead of adding both JSON:API and GraphQL to Drupal 8 core, I believe only JSON:API should be added. That said, Drupal's GraphQL implementation is fantastic, especially when you have the developer capacity to build a bespoke API for your project.

On the four qualities by which we evaluated the REST, JSON:API and GraphQL modules, JSON:API has outperformed its contemporaries. Its web standards-based approach, its ability to handle reads and writes out of the box, its security model and its ease of operation make it the best choice for Drupal core. Additionally, where JSON:API underperformed, I believe that we have a real opportunity to contribute back to the specification. In fact, one of the JSON:API module's maintainers and co-authors of this blog post, Gabe Sullice (Acquia), recently became a JSON:API specification editor himself.

This decision does not mean that you can't or shouldn't use GraphQL with Drupal. While I believe JSON:API covers the majority of use cases, there are valid use cases where GraphQL is a great fit. I'm happy that Drupal is endowed with such a vibrant contributed module ecosystem that provides so many options to Drupal's users.

I'm excited to see where both the JSON:API specification and Drupal's implementation of it goes in the coming months and years. As a first next step, we're preparing the JSON:API to be added to Drupal 8.7.

Special thanks to Wim Leers (Acquia) and Gabe Sullice (Acquia) for co-authoring this blog post and to Preston So (Acquia) and Alex Bronstein(Acquia) for their feedback during the writing process.

 February 11, 2019

 11 min read time

 Permalink

Feb 07 2019
Feb 07

This blog has been re-posted and edited with permission from Dries Buytaert's blog.

I'm frequently sent examples of how Drupal has changed the lives of developers, business owners and end users. Recently, I received a very different story of how Drupal had helped in a rescue operation that saved a man's life.

The Snowdonia Ultra Marathon website

In early 2018, Race Director Mike Jones was looking to build a new website for the Ultra-Trail Snowdonia ultra marathon. He reached out to a good friend and developer, Rob Edwards, to lead the development of the website.

A photo of a runner at the Ultra-trail Snowdonia ultramarathon

© Ultra-trail Snowdonia and No Limits Photography

Rob chose Drupal for its flexibility and extensibility. As an organization supported heavily by volunteers, open source also fit the Snowdonia team's belief in community.

The resulting website, https://apexrunning.co/, included a custom-built timing module. This module allowed volunteers to register each runner and their time at every aid stop.

A runner goes missing

Rob attended the first day of Ultra-Trail Snowdonia to ensure the website ran smoothly. He also monitored the runners at the end of the race to certify they were all accounted for.

Monitoring the system into the early hours of the morning, Rob noticed one runner, after successfully completing checkpoints one and two, hadn't passed through the third checkpoint.

A photo of a runner at the Ultra-trail Snowdonia ultramarathon

© Ultra-trail Snowdonia and No Limits Photography

Each runner carried a mobile phone with them for emergencies. Mike attempted to make contact with the runner via phone to ensure he was safe. However, this specific area was known for its poor signal and the connection was too weak to get through.

After some more time eagerly watching the live updates, it was clear the runner hadn't reached checkpoint four and more likely hadn't ever made it past checkpoint three. The Ogwen Mountain Rescue were called to action.

Due to the terrain and temperature, searching for the lost runner on foot would be too slow. Instead, the mountain rescue volunteers used a helicopter to scan the area and locate the runner.

How Drupal came to the rescue

The area covered by runners in an ultra marathon like this one is vast. The custom-built timing module helped rescuers narrow down the search area; they knew the runner passed the second checkpoint but never made it to the third.

After following the fluorescent orange markers in the area pinpointed by the Drupal website, the team quickly found the individual. He had fallen and become too injured to carry on. A mild case of hypothermia had set in. The runner was airlifted to the hospital for appropriate care. The good news: the runner survived.

Without Drupal, it might have taken much longer to notify anyone that a runner had gone missing, and there would have been no way to tell when he had dropped off.

NFC and GPS devices are now being explored for these ultra marathon runners to carry with them to provide location data as an extra safety precaution. The Drupal system will be used alongside these devices for more accurate time readings, and Rob is looking into an API to pull this additional data into the Drupal website.

Stories about Drupal having an impact on organizations and individuals, or even helping out in emergencies, drive my sense of purpose. Feel free to keep sending them my way!

Special thanks to Rob EdwardsPoppy Heap (CTI Digital) and Paul Johnson (CTI Digital) for their help with this blog post.

Jan 29 2019
Jan 29

This blog has been re-posted and edited with permission from Dries Buytaert's blog.

The European Commission worked with the Drupal Security Team to set aside 89,000€ (or roughly $100,000 USD) for a Drupal bug bounty.

An image of a shield with the Drupal mascot

The European Commission made an exciting announcement; it will be awarding bug bounties to the security teams of Open Source software projects that the European Commission relies on.

If you are not familiar with the term, a bug bounty is a monetary prize awarded to people who discover and correctly report security issues.

Julia Reda — an internet activist, Member of the European Parliament (MEP) and co-founder of the Free and Open Source Software Audit (FOSSA) project — wrote the following on her blog:

Like many other organizations, institutions like the European Parliament, the Council and the Commission build upon Free Software to run their websites and many other things. But the Internet is not only crucial to our economy and our administration, it is the infrastructure that runs our everyday lives.

With more than 150 Drupal sites, the European Commission is a big Drupal user, and has a large internal Drupal community. The European Commission set aside 89,000€ (or roughly $100,000 USD) for a Drupal bug bounty. They worked closely with Drupal's Security Team to set this up. To participate in the Drupal bug bounty, read the guidelines provided by Drupal's Security Team.

Over the years I've had many meetings with the European Commission, presented keynotes at some of its events, and more. During that time, I've seen the European Commission evolve from being hesitant about Open Source, to recognizing the many benefits that Open Source provides for its key ICT services, to truly embracing Open Source.

In many ways, the European Commission followed classic Open Source adoption patterns; adoption went from being technology-led (bottom-up or grassroots) to policy-led (top-down and institutionalized), and now the EU is an active participant and contributor.

Today, the European Commission is a shining example and role model for how governments and other large organizations can contribute to Open Source (just like how the White House used to be).

The European Commission is actually investing in Drupal in a variety of ways — the bug bounty is just one example of that — but more about that in a future blog post.

 January 28, 2019

 1 min read time

 Permalink

Jan 25 2019
Jan 25

This blog has been re-posted and edited with permission from Dries Buytaert's blog.

Do I build my website with Drupal's built-in templating layer or do I use Drupal's decoupled or headless capabilities in combination with a JavaScript framework?

The pace of innovation in content management has been accelerating — driven by both the number of channels that content management systems need to support (web, mobile, social, chat) as well as the need to support JavaScript frameworks in the traditional web channel. As a result, we've seen headless or decoupled architectures emerge.

Decoupled Drupal has seen adoption from all corners of the Drupal community. In response to the trend towards decoupled architectures, I wrote blog posts in 2016 and 2018 for architects and developers about how and when to decouple Drupal. In the time since my last post, the surrounding landscape has evolved, Drupal's web services have only gotten better, and new paradigms such as static site generators and the JAMstack are emerging.

Time to update my recommendations for 2019! As we did a year ago, let's start with the 2019 version of the flowchart in full. (At the end of this post, there is also an accessible version of this flowchart described in words.)

A flowchart of how to decouple Drupal in 2019

Different ways to decouple Drupal

I want to revisit some of the established ways to decouple Drupal as well as discuss new paradigms that are seeing growing adoption. As I've written previously, the three most common approaches to Drupal architecture from a decoupled standpoint are traditional (or coupled), progressively decoupled, and fully decoupled. The different flavors of decoupling Drupal exist due to varying preferences and requirements.

In traditional Drupal, all of Drupal's usual responsibilities stay intact, as Drupal is a monolithic system and therefore maintains complete control over the presentation and data layers. Traditional Drupal remains an excellent choice for editors who need full control over the visual elements on the page, with access to features such as in-place editing and layout management. This is Drupal as we have known it all along. Because the benefits are real, this is still how most new content management projects are built.

Sometimes, JavaScript is required to deliver a highly interactive end-user experience. In this case, a decoupled approach becomes required. In progressively decoupled Drupal, a JavaScript framework is layered on top of the existing Drupal front end. This JavaScript might be responsible for nothing more than rendering a single block or component on a page, or it may render everything within the page body. The progressive decoupling paradigm lies on a spectrum; the less of the page dedicated to JavaScript, the more editors can control the page through Drupal's administrative capabilities.

Up until this year, fully decoupled Drupal was a single category of decoupled Drupal architecture that reflects a full separation of concerns between the presentation layer and all other aspects of the CMS. In this scenario, the CMS becomes a data provider, and a JavaScript application with server-side rendering becomes responsible for all rendering and markup, communicating with Drupal via web service APIs. Though key functionality like in-place editing and layout management are unavailable, fully decoupled Drupal is appealing for developers who want greater control over the front end and who are already experienced with building applications in frameworks like Angular, React, Vue.js, etc.

Over the last year, fully decoupled Drupal has branched into two separate paradigms due to the increasing complexity of JavaScript development. The so-called JAMstack (JavaScript, APIs, Markup) introduces a new approach: fully decoupled static sites. The primary reason for static sites is improved performance, security, and reduced complexity for developers. A static site generator like Gatsby will retrieve content from Drupal, generate a static website, and deploy that static site to a CDN, usually through a specialized cloud provider such as Netlify.

What do you intend to build?

The top section of the flowchart showing how to decouple Drupal in 2019

The essential question, as always, is what you're trying to build. Here is updated advice for architects exploring decoupled Drupal in 2019:

  1. If your intention is to build a single standalone website or web application, choosing decoupled Drupal may or may not be the right choice, depending on the features your developers and editors see as must-haves.
  2. If your intention is to build multiple web experiences (websites or web applications), you can use a decoupled Drupal instance either as a) a content repository without its own public-facing front end or b) a traditional website that acts simultaneously as a content repository. Depending on how dynamic your application needs to be, you can choose a JavaScript framework for highly interactive applications or a static site generator for mostly static websites.
  3. If your intention is to build multiple non-web experiences (native mobile or IoT applications), you can leverage decoupled Drupal to expose web service APIs and consume that Drupal site as a content repository without its own public-facing front end.

What makes Drupal so powerful is that it supports all of these use cases. Drupal makes it simple to build decoupled Drupal thanks to widely recognized standards such as JSON:APIGraphQLOpenAPI, and CouchDB. In the end, it is your technical requirements that will decide whether decoupled Drupal should be your next architecture.

In addition to technical requirements, organizational factors often come into play as well. For instance, if it is proving difficult to find talented front-end Drupal developers with Twig knowledge, it may make more sense to hire more affordable JavaScript developers instead and build a fully decoupled implementation.

Are there things you can't live without?

The middle section of the flowchart showing how to decouple Drupal in 2019

As I wrote last year, the most important aspect of any decision when it comes to decoupling Drupal is the list of features your project requires; the needs of editors and developers have to be carefully considered. It is a critical step in your evaluation process to weigh the different advantages and disadvantages. Every project should embark on a clear-eyed assessment of its organization-wide needs.

Many editorial and marketing teams select a particular CMS because of its layout capabilities and rich editing functionality. Drupal, for example, gives editors the ability to build layouts in the browser and drop-and-drag components into it, all without needing a developer to do it for them. Although it is possible to rebuild many of the features available in a CMS on a consumer application, this can be a time-consuming and expensive process.

In recent years, the developer experience has also become an important consideration, but not in the ways that we might expect. While the many changes in the JavaScript landscape are one of the motivations for developers to prefer decoupled Drupal, the fact that there are now multiple ways to write front ends for Drupal makes it easier to find people to work on decoupled Drupal projects. As an example, many organizations are finding it difficult to find affordable front-end Drupal developers experienced in Twig. Moving to a JavaScript-driven front end can resolve some of these resourcing challenges.

This balancing act between the requirements that developers prioritize and those that editors prioritize will guide you to the correct approach for your needs. If you are part of an organization that is mostly editorial, decoupled Drupal could be problematic, because it reduces the amount of control editors have over the presentation of their content. By the same token, if you are part of an organization with more developer resources, fully decoupled Drupal could potentially accelerate progress, with the warning that many mission-critical editorial features disappear.

Current and future trends to consider

A diagram showing a spectrum of site building solution; low-code solutions on the left and high-code solutions on the right

Over the past year, JavaScript frameworks have become more complex, while static site generators have become less complex.

One of the common complaints I have heard about the JavaScript landscape is that it shows fragmentation and a lack of cohesion due to increasing complexity. This has been a driving force for static site generators. Whereas two years ago, most JavaScript developers would have chosen a fully functional framework like Angular or Ember to create even simple websites, today they might choose a static site generator instead. A static site generator still allows them to use JavaScript, but it is simpler because performance considerations and build processes are offloaded to hosted services rather than the responsibility of developers.

I predict that static site generators will gain momentum in the coming year due to the positive developer experience they provide. Static site generators are also attracting a middle ground of both more experienced and less experienced developers.

Conclusion

Drupal continues to be an ideal choice for decoupled CMS architectures, and it is only getting better. The API-first initiative is making good progress on preparing the JSON:API module for inclusion in Drupal core, and the Admin UI and JavaScript Modernization initiative is working to dogfood Drupal's web services with a reinvented administrative interface. Drupal's support for GraphQL continues to improve, and now there is even a book on the subject of decoupled Drupal. It's clear that developers today have a wide range of ways to work with the rich features Drupal has to offer for decoupled architectures.

With the introduction of fully decoupled static sites as an another architectural paradigm that developers can select, there is an even wider variety of architectural possibilities than before. It means that the spectrum of decoupled Drupal approaches I defined last year has become even more extensive. This flexibility continues to define Drupal as an excellent CMS for both traditional and decoupled approaches, with features that go well beyond Drupal's competitors, including WordPress, Sitecore and Adobe. Regardless of the makeup of your team or the needs of your organization, Drupal has a solution for you.

Special thanks to Preston So for co-authoring this blog post and to Angie ByronChris HamperGabe SulliceLauri EskolaTed Bowman, and Wim Leers for their feedback during the writing process.

Accessible version of flowchart

This is an accessible and described version of the flowchart images earlier in this blog post. First, let us list the available architectural choices:

  • Coupled. Use Drupal as is without additional JavaScript (and as a content repository for other consumers).
  • Progressively decoupled. Use Drupal for initial rendering with JavaScript on top (and as a content repository for other consumers).
  • Fully decoupled static site. Use Drupal as a data source for a static site generator and, if needed, deploy to a JAMstack hosting platform.
  • Fully decoupled app. Use Drupal as a content repository accessed by other consumers (if JavaScript, use Node.js for server-side rendering).

Second, ask the question "What do you intend to build?" and choose among the answers "One experience" or "Multiple experiences".

If you are building one experience, ask the question "Is it a website or web application?" and choose among the answers "Yes, a single website or web application" or "No, Drupal as a repository for non-web applications only".

If you are building multiple experiences instead, ask the question "Is it a website or web application?" with the answers "Yes, Drupal as website and repository" or "No, Drupal as a repository for non-web applications only".

If your answer to the previous question was "No", then you should build a fully decoupled application, and your decision is complete. If your answer to the previous question was "Yes", then ask the question "Are there things the project cannot live without?"

Both editorial and developer needs are things that projects cannot live without, and here are the questions you need to ask about your project:

Editorial needs

  • Do editors need to manipulate page content and layout without a developer?
  • Do editors need in-context tools like in-place editing, contextual links, and toolbar?
  • Do editors need to preview unpublished content without custom development?
  • Do editors need content to be accessible by default like in Drupal's HTML?

Developer needs

  • Do developers need to have control over visual presentation instead of editors?
  • Do developers need server-side rendering or Node.js build features?
  • Do developers need JSON from APIs and to write JavaScript for the front end?
  • Do developers need data security driven by a publicly inaccessible CMS?

If, after asking all of these questions about things your project cannot live without, your answers show that your requirements reflect a mix of both editorial and developer needs, you should consider a progressively decoupled implementation, and your decision is complete.

If your answers to the questions about things your project cannot live without show that your requirements reflect purely developer needs, then ask the question "Is it a static website or a dynamic web application?" and choose among the answers "Static" or "Dynamic." If your answer to the previous question was "Static", you should build a fully decoupled static site, and your decision is complete. If your answer to the previous question was "Dynamic", you should build a fully decoupled app, and your decision is complete.

If your answers to the questions about things your project cannot live without show that your requirements reflect purely editorial needs, then ask two questions. Ask the first question, "Are there parts of the page that need JavaScript-driven interactions?" and choose among the answers "Yes" or "No." If your answer to the first question was "Yes", then you should consider a progressively decoupled implementation, and your decision is complete. If your answer to the first question was "No", then you should build a coupledDrupal site, and your decision is complete.

Then, ask the second question, "Do you need to access multiple data sources via API?" and choose among the answers "Yes" or "No." If your answer to the second question was "Yes", then you should consider a progressively decoupled implementation, and your decision is complete. If your answer to the second question was "No", then you should build a coupled Drupal site, and your decision is complete.

 Permalink

Jan 16 2019
Jan 16

This blog has been re-posted and edited with permission from Dries Buytaert's blog.

Eighteen years ago today, I released Drupal 1.0.0. What started from humble beginnings has grown into one of the largest Open Source communities in the world. Today, Drupal exists because of its people and the collective effort of thousands of community members. Thank you to everyone who has been and continues to contribute to Drupal.

Eighteen years is also the voting age in the US, and the legal drinking age in Europe. I'm not sure which one is better. :) Joking aside, welcome to adulthood, Drupal. May your day be bug free and filled with fresh patches!

Jan 09 2019
Jan 09

This blog has been re-posted and edited with permission from Dries Buytaert's blog.

Last year, I talked to nearly one hundred Drupal agency owners to understand what is preventing them from selling Drupal. One of the most common responses raised is that Drupal's administration UI looks outdated.

This critique is not wrong. Drupal's current administration UI was originally designed almost ten years ago when we were working on Drupal 7. In the last ten years, the world did not stand still; design trends changed, user interfaces became more dynamic and end-user expectations have changed with that.

To be fair, Drupal's administration UI has received numerous improvements in the past ten years; Drupal 8 shipped with a new toolbar, an updated content creation experience, more WYSIWYG functionality, and even some design updates.

A visual comparison of Drupal 7 and Drupal 8's administration UI

A comparison of the Drupal 7 and Drupal 8 content creation screen to highlight some of the improvements in Drupal 8.

While we made important improvements between Drupal 7 and Drupal 8, the feedback from the Drupal agency owners doesn't lie: we have not done enough to keep Drupal's administration UI modern and up-to-date.

This is something we need to address.

We are introducing a new design system that defines a complete set of principles, patterns, and tools for updating Drupal's administration UI.

In the short term, we plan on updating the existing administration UI with the new design system. Longer term, we are working on creating a completely new JavaScript-based administration UI.

A screenshot of the content administration using Drupal 8's Carlo theme

The content administration screen with the new design system.

As you can see on Drupal.org, community feedback on the proposal is overwhelmingly positive with comments like Wow! Such an improvement! and Well done! High contrast and modern look..

A screenshot of the spacing guidelines of Drupal 8's Carlo theme

Sample space sizing guidelines from the new design system.

I also ran the new design system by a few people who spend their days selling Drupal and they described it as "clean" with "good use of space" and a design they would be confident showing to prospective customers.

Whether you are a Drupal end-user, or in the business of selling Drupal, I recommend you check out the new design system and provide your feedback on Drupal.org.

Special thanks to Cristina ChumillasSascha EggenbergerRoy ScholtenArchita AroraDennis CohnRicardo MarcelinoBalazs KantorLewis Nyman,and Antonella Severo for all the work on the new design system so far!

We have started implementing the new design system as a contributed theme with the name Claro. We are aiming to release a beta version for testing in the spring of 2019 and to include it in Drupal core as an experimental theme by Drupal 8.8.0 in December 2019. With more help, we might be able to get it done faster.

Throughout the development of the refreshed administration theme, we will run usability studies to ensure that the new theme indeed is an improvement over the current experience, and we can iteratively improve it along the way.

Acquia has committed to being an early adopter of the theme through the Acquia Lightning distribution, broadening the potential base of projects that can test and provide feedback on the refresh. Hopefully other organizations and projects will do the same.

How can I help?

The team is looking for more designers and frontend developers to get involved. You can attend the weekly meetings on #javascript on Drupal Slack Mondays at 16:30 UTC and on #admin-ui on Drupal Slack Wednesdays at 14:30 UTC.

Thanks to Lauri EskolaGábor Hojtsy and Jeff Beeman for their help with this post.

File attachments:  drupal-7-vs-drupal-8-administration-ui-1280w.png carlo-content-administration-1280w.png carlo-spacing-1280w.png
Jan 05 2019
Jan 05

The Drupal Security Team is using funding from the EU-FOSS to pay for valid security issues found in Drupal 7 and 8 and top contributed modules. This program is open for participation by anyone.

The Drupal Security Team will be authorizing payment anywhere from €350 ($401) – €15.000 ($17,100) per issue. The more serious the issue, the more the Drupal Security Team will authorize for payment.

Who is running this program?

The Drupal Security Team with funds from the EU-FOSSA on the intigriti reporting platform.

What is covered?

Third-party libraries, even those bundled with Drupal core or contributed projects, are excluded from this program.

Can anyone participate?

You may not participate in this program if you fall into one of the following categories:

  • If you are a project maintainer (module, theme, etc), or you contribute a large amount to a project, you may not get paid for the project you maintain. This does not apply to Drupal Core.
  • You cannot report a bug you yourself created or committed. (If you find one, however, do report it via our normal processes (https://www.drupal.org/security-team/report-issue)

To get paid, you must have an account on Drupal.org.
Security Team members that are involved with the administration of this program and/or its funds are not eligible for payouts under the program.
This program is only valid for new issues submitted after 2019-01-29. (Duplicate reports of in-progress issues known to the Security Team may not eligible for payment.)
All issues submitted must be original research. Do not copy and paste results from a scanner without validating them first.

How can I get started?

Install a local copy of Drupal 8 or Drupal 7 from git (https://www.drupal.org/project/drupal/git-instructions). Find security issues such as XSS, SQL Injection, CSRF, Access Bypass etc.
Any submissions about a public Drupal website (including Drupal.org) will result in your account being blocked from any further payments. This bounty program applies to public Drupal open source code only. Not hosted website that run Drupal.

If you find a security issue you should:

  1. Write up the steps to reproduce the issue.
  2. Make sure you have a Drupal.org account created. Include this in your submission.
  3. Go to the intigriti platform and create a new report.
  4. WAIT, do not discuss or post anything anywhere yet. Members of the intigriti team and the Drupal Security Team must validate your report. This can take up to 3 weeks depending on the report and the complexity involved.
  5. If you have a valid report, we will issue payment. You still can not disclose this bug until we publish a release that fixes the bug.
  6. If you do not have a valid report, we will inform you as well.

What must be included in the report?

  • The reporter must provide a detailed explanation of the issue and steps to reproduce the issue.
  • The quality of the report will be taken into account when assigning a value to it.
  • We will also take into account the severity of the security issue.
  • Reporters will also need to confirm that the Drupal Security team will be the group to release the information.
  • Include your Drupal.org username.
  • Issues will be confirmed by the security team before payment is approved.

Do all security issues count?

Your testing should be done on a local environment, testing should NEVER be done on live Drupal sites. Any testing on ANY Drupal site that is not under your control (local or server you own) will automatically be rejected and your account blocked from further payment. We have a page on how to get started installing Drupal locally.

If a task requires the attacker to have advanced permissions as listed on our permission policy page (e.g. 'Access site reports', 'Administer users', 'Translate interface', etc.) will not be eligible for a payout. Other advanced permissions that are not white listed are at the discretion of the security team. Contributed modules (non-core code) must be eligible for a security advisory according to the security advisory policy. A list of projects currently eligible for the program is available.

Security issues excluded from the bounty program

The following are not being considered; while some may be legitimate security issues, they are out of scope for this bounty program.

  • Descriptive error messages (e.g. Stack Traces, application or server errors).
  • HTTP 404 codes/pages or other HTTP non-200 codes/pages.
  • Fingerprinting / banner disclosure on common/public services.
  • Disclosure of known public files or directories (e.g. robots.txt).
  • Clickjacking and issues only exploitable through clickjacking.
  • CSRF on forms that are available to anonymous users (e.g. the contact form).
  • Logout Cross-Site Request Forgery (logout CSRF).
  • Presence of application or web browser ‘autocomplete’ or ‘save password’ functionality.
  • Lack of Secure/HTTP-Only flags on non-sensitive Cookies.
  • Lack of Security Speedbump when leaving the site.
  • User enumeration.
  • Missing HTTP security headers
  • Any Denial of service attack.
  • Other exceptions not listed.
  • We would still like to know about issues in these categories, and you may still get credit for reporting them, but we will not be issuing payments for them.

Other questions?

Questions regarding additional specifics of this program should be emailed to [email protected].

Dec 21 2018
Dec 21

This blog has been re-posted and edited with permission from Dries Buytaert's blog.

Drupal 8 has been growing 40 to 50 percent year over year. It's a healthy growth rate. Regardless, it is always worth exploring how we can continue to accelerate that growth.

Earlier this week, I wrote about the power of removing obstacles to growth, and shared how Amazon approaches its own growth blockers. Amazon identified at least two blockers for long-term growth: (1) shipping costs and (2) shipping times. For more than a decade, Amazon has been focused on eliminating both. They have spent an unbelievable amount of creativity, effort, time, and money to eliminate them.

In that blog post, I promised to share my thoughts around Drupal's own growth barriers. What obstacles can we eliminate to fuel Drupal's long-term growth? Well, I believe the limitations to Drupal's growth can be summarized as:

  1. Make Drupal easy to evaluate and adopt
  2. Make Drupal easy for content creators and site builders
  3. Reduce the total cost of ownership for developers and site owners
  4. Keep Drupal relevant and impactful
  5. Promote Drupal and help Drupal agencies win

For those that have read my blog or watched my DrupalCon keynote presentations, none of these will come as a surprise. Just like Amazon's examples, fixing these obstacles have been, and will be, multi-year efforts.

A mountain images with 5 product strategy tracks leading to the top

Drupal's five product strategy tracks. A number of current initiatives is shown on each track.

1. Make Drupal easy to evaluate and adopt

We need to make it easy for more people to try Drupal. To help evaluators explore Drupal's possibilities, we improved the download and installation experience, and included a demonstration site with core. We made fantastic progress on this in 2018.

Now that we have improved the evaluator experience, I'd love to see us focus on the "new user" experience. When you put yourself in the shoes of a new Drupal user, you'd still find it hard to set up a local development environment. There are too many options, too little direction, and no one official way for how to get started with Drupal. The "new user" is not receiving enough attention, and that slows adoption so I'd love to see us focus no that in 2019.

2. Make Drupal easy for content creators and site builders

One of the most powerful trends I've noticed time and time again is that simplicity wins. People expect software to be functionally powerful and easy to use. This is especially true for content creators and site builders.

To make Drupal easier to use for content creators and site builders, we've introduced WYSIWYG and in-place editing in Drupal 8.0, and now we're working hard on media management, layout building, content workflows and a new administration and authoring UI.

A lot of these initiatives add tools to the UI that empower content creators and site builders to do more with less code. Long term, I believe that we need to more of these "no-code" or "low-code" capabilities in Drupal.

3. Reduce the total cost of ownership for developers and site owners

Developers want to be agile, fast and deliver high quality projects that add value for their organization. Developers don't want their tools to get in the way.

For Drupal this means that they want to build sites, including themes and modules, without being bogged down by complex upgrades, expensive migrations or cumbersome developer workflows.

For developers and site owners we have made upgrades easier, we adopted a 6-month innovation model, and we extended security coverage for minor releases. This removes the complexity from major upgrades, gives organizations more time to upgrade, and allows us to release new capabilities more frequently. This is a very big deal for developer and site owners!

In addition, we're working on improving Drupal's Composer support and configuration management capabilities. This will help developers automate and streamline their day-to-day work.

Longer term, improved Composer support could act as a stepping stone towards automated updates, which would be one of the most effective ways to free up a developer's time.

4. Keep Drupal relevant and impactful

The innovation in the Drupal ecosystem happens thanks to Drupal contributors. We need to attract new contributors to Drupal, and keep existing contributors excited. This means we have to keep Drupal relevant and impactful.

To keep Drupal relevant, we've been investing in making Drupal an API-first platform for many years now. Headless Drupal or decoupled Drupal is one of Drupal's competitive advantages. Drupal's web service APIs allow developers to use Drupal with their JavaScript framework of choice, push content to different channels, and better integrate Drupal with different technologies in the marketing stack.

Drupal developers can now do unprecedented things with Drupal that weren't available before. JavaScript and mobile application developers have been familiarizing themselves with Drupal due to its improved API-first capabilities. All of this keeps Drupal relevant, ensures that Drupal has high impact, and that we attract new developers to Drupal.

5. Promote Drupal and help Drupal agencies win

While Drupal is well-known as an Open Source project, there isn't a deep understanding of how Drupal is evolving or how Drupal compares to its competitors.

Drupal is improving rapidly every six months with each new minor version release, but I'm not sure we're getting that message out effectively. We need to promote our amazing progress, not only to everyone in the web development community, but also to marketers and content managers, who are now often weighing in heavily on CMS decisions.

We do an incredible job collaborating on code — thousands of us are helping to build Drupal — but we do a poor job collaborating on marketing, education and promotion. Imagine what could happen if these thousands of individuals and agencies would all collaborate on promoting Drupal!

That is why the Drupal Association started the Promote Drupal initiative, and why we're trying to rally people in the community to work together on creating pitch decks, case studies, and other collateral to promote and market Drupal.

Here are a few things already happening:

  • There is an updated Drupal Brand Book for organizations to follow as they design Drupal marketing and sales materials.
  • A team of volunteers is creating a comprehensive Drupal pitch deck that Drupal agencies can use as a starting point when working with new clients.
  • DrupalCon will have new Content & Digital Marketing Track for marketing teams responsible for content generation, demand generation, user journeys, and more; and a "Agency Leadership Track" for those running Drupal agencies.
  • We will begin work on a competitive comparison chart — contrasting Drupal with other CMS competitors like Adobe, Sitecore, Contentful, WordPress, Prismic, and more.
  • A number of local Drupal Associations are hiring marketing people to help promote Drupal in their region.

Just like all open source contribution, it takes many to move things forward. So far, 40 people have signed up to help with these marketing efforts. If your organization has a marketing team that would like to contribute to the marketing of Drupal, check out the Promote Drupal initiative page and please join the Promote Drupal team.

Educating the world about how Drupal is evolving, the amazing use cases we support, and how Drupal compares to old and new competitors will go a very long way towards raising awareness of the project and growing the businesses built on and around Drupal.

Final thoughts

After talking to hundreds of Drupal users and would-be users, as well as dozens of agency owners, I believe we're working on the right things. Overcoming these growth obstacles are multi-year efforts. While the various initiatives might change, I believe we'll keep working on these four tracks for the next decade. We've been making steady progress the last few years but need to remain both patient and committed to driving them home. Just like Amazon continues to work on their growth obstacles after more than a decade, I expect we'll be working on these four obstacles for many years to come.

File attachments:  product-strategy-mountain-december-2018.jpg
Dec 12 2018
Dec 12

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Unfortunately Dries' blog does not allow for comments at the moment, feel free to post them here.

At Drupal Europe, I announced that Drupal 9 will be released in 2020. Although I explained why we plan to release in 2020, I wasn't very specific about when we plan to release Drupal 9 in 2020. Given that 2020 is less than thirteen months away (gasp!), it's time to be more specific.

Shifting Drupal's six month release cycle

A timeline that shows how we shifted Drupal 8's release windows

We shifted Drupal 8's minor release windows so we can adopt Symfony's releases faster.

Before I talk about the Drupal 9 release date, I want to explain another change we made, which has a minor impact on the Drupal 9 release date.

As announced over two years ago, Drupal 8 adopted a 6-month release cycle (two releases a year). Symfony, a PHP framework which Drupal depends on, uses a similar release schedule. Unfortunately the timing of Drupal's releases has historically occurred 1-2 months before Symfony's releases, which forces us to wait six months to adopt the latest Symfony release. To be able to adopt the latest Symfony releases faster, we are moving Drupal's minor releases to June and December. This will allow us to adopt the latest Symfony releases within one month. For example, Drupal 8.8.0 is now scheduled for December 2019.

We hope to release Drupal 9 on June 3, 2020

Drupal 8's biggest dependency is Symfony 3, which has an end-of-life date in November 2021. This means that after November 2021, security bugs in Symfony 3 will not get fixed. Therefore, we have to end-of-life Drupal 8 no later than November 2021. Or put differently, by November 2021, everyone should be on Drupal 9.

Working backwards from November 2021, we'd like to give site owners at least one year to upgrade from Drupal 8 to Drupal 9. While we could release Drupal 9 in December 2020, we decided it was better to try to release Drupal 9 on June 3, 2020. This gives site owners 18 months to upgrade. Plus, it also gives the Drupal core contributors an extra buffer in case we can't finish Drupal 9 in time for a summer release.

A timeline that shows we hope to release Drupal 9 in June 2020

Planned Drupal 8 and 9 minor release dates.

We are building Drupal 9 in Drupal 8

Instead of working on Drupal 9 in a separate codebase, we are building Drupal 9 in Drupal 8. This means that we are adding new functionality as backwards-compatible code and experimental features. Once the code becomes stable, we deprecate any old functionality.

Let's look at an example. As mentioned, Drupal 8 currently depends on Symfony 3. Our plan is to release Drupal 9 with Symfony 4 or 5. Symfony 5's release is less than one year away, while Symfony 4 was released a year ago. Ideally Drupal 9 would ship with Symfony 5, both for the latest Symfony improvements and for longer support. However, Symfony 5 hasn't been released yet, so we don't know the scope of its changes, and we will have limited time to try to adopt it before Symfony 3's end-of-life.

We are currently working on making it possible to run Drupal 8 with Symfony 4 (without requiring it). Supporting Symfony 4 is a valuable stepping stone to Symfony 5 as it brings new capabilities for sites that choose to use it, and it eases the amount of Symfony 5 upgrade work to do for Drupal core developers. In the end, our goal is for Drupal 8 to work with Symfony 3, 4 or 5 so we can identify and fix any issues before we start requiring Symfony 4 or 5 in Drupal 9.

Another example is our support for reusable media. Drupal 8.0.0 launched without a media library. We are currently working on adding a media library to Drupal 8 so content authors can select pre-existing media from a library and easily embed them in their posts. Once the media library becomes stable, we can deprecate the use of the old file upload functionality and make the new media library the default experience.

The upgrade to Drupal 9 will be easy

Because we are building Drupal 9 in Drupal 8, the technology in Drupal 9 will have been battle-tested in Drupal 8.

For Drupal core contributors, this means that we have a limited set of tasks to do in Drupal 9 itself before we can release it. Releasing Drupal 9 will only depend on removing deprecated functionality and upgrading Drupal's dependencies, such as Symfony. This will make the release timing more predictable and the release quality more robust.

For contributed module authors, it means they already have the new technology at their service, so they can work on Drupal 9 compatibility earlier (e.g. they can start updating their media modules to use the new media library before Drupal 9 is released). Finally, their Drupal 8 know-how will remain highly relevant in Drupal 9, as there will not be a dramatic change in how Drupal is built.

But most importantly, for Drupal site owners, this means that it should be much easier to upgrade to Drupal 9 than it was to upgrade to Drupal 8. Drupal 9 will simply be the last version of Drupal 8, with its deprecations removed. This means we will not introduce new, backwards-compatibility breaking APIs or features in Drupal 9 except for our dependency updates. As long as modules and themes stay up-to-date with the latest Drupal 8 APIs, the upgrade to Drupal 9 should be easy. Therefore, we believe that a 12- to 18-month upgrade period should suffice.

So what is the big deal about Drupal 9, then?

The big deal about Drupal 9 is … that it should not be a big deal. The best way to be ready for Drupal 9 is to keep up with Drupal 8 updates. Make sure you are not using deprecated modules and APIs, and where possible, use the latest versions of dependencies. If you do that, your upgrade experience will be smooth, and that is a big deal for us.

Special thanks to Gábor Hojtsy (Acquia), Angie Byron (Acquia), xjm(Acquia), and catch for their input in this blog post.

Dec 10 2018
Dec 10

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

This week I was in New York for a day. At lunch, Sir Martin Sorrell pointed out that Microsoft overtook Apple as the most valuable software company as measured by market capitalization. It's a close call but Microsoft is now worth $805 billion while Apple is worth $800 billion.

What is interesting to me are the radical "ebbs and flows" of each organization.

In the '80s, Apple's market cap was twice that of Microsoft. Microsoft overtook Apple in the the early '90s, and by the late '90s, Microsoft's valuation was a whopping thirty-five times Apple's. With a 35x difference in valuation, no one would have guessed Apple to ever regain the number-one position. However, Apple did the unthinkable and regained its crown in market capitalization. By 2015, Apple was, once again, valued two times more than Microsoft.

And now, eighteen years after Apple took the lead, Microsoft has taken the lead again. Everything old is new again.

As you'd expect, the change in market capitalization corresponds with the evolution and commercial success of their product portfolios. In the '90s, Microsoft took the lead based on the success of the Windows operating system. Apple regained the crown in the 2000s based on the success of the iPhone. Today, Microsoft benefits from the rise of cloud computing, Software-as-a-Service and Open Source, while Apple is trying to navigate the saturation of the smartphone market.

It's unclear if Microsoft will maintain and extend its lead. On one hand, the market trends are certainly in Microsoft's favor. On the other hand, Apple still makes a lot more money than Microsoft. I believe Apple to be slightly undervalued, and Microsoft is to be overvalued. The current valuation difference is not justified.

At the end of the day, what I find to be most interesting is how both organizations have continued to reinvent themselves. This reinvention has happened roughly every ten years. During these periods of reinvention, organizations can fall out of favor for long stretches of time. However, as both organizations prove, it pays off to reinvent yourself, and to be patient product and market builders.

Dec 05 2018
Dec 05

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Last week, WordPress Tavern picked up my blog post about Drupal 8's upcoming Layout Builder.

While I'm grateful that WordPress Tavern covered Drupal's Layout Builder, it is not surprising that the majority of WordPress Tavern's blog post alludes to the potential challenges with accessibility. After all, Gutenberg's lack of accessibility has been a big topic of debate, and a point of frustration in the WordPress community.

I understand why organizations might be tempted to de-prioritize accessibility. Making a complex web application accessible can be a lot of work, and the pressure to ship early can be high.

In the past, I've been tempted to skip accessibility features myself. I believed that because accessibility features benefited a small group of people only, they could come in a follow-up release.

Today, I've come to believe that accessibility is not something you do for a small group of people. Accessibility is about promoting inclusion. When the product you use daily is accessible, it means that we all get to work with a greater number and a greater variety of colleagues. Accessibility benefits everyone.

As you can see in Drupal's Values and Principles, we are committed to building software that everyone can use. Accessibility should always be a priority. Making capabilities like the Layout Builder accessible is core to Drupal's DNA.

Drupal's Values and Principles translate into our development process, as what we call an accessibility gate, where we set a clearly defined "must-have bar." Prioritizing accessibility also means that we commit to trying to iteratively improve accessibility beyond that minimum over time.

Together with the accessibility maintainers, we jointly agreed that:

  1. Our first priority is WCAG 2.0 AA conformance. This means that in order to be released as a stable system, the Layout Builder must reach Level AA conformance with WCAG. Without WCAG 2.0 AA conformance, we won't release a stable version of Layout Builder.
  2. Our next priority is WCAG 2.1 AA conformance. We're thrilled at the greater inclusion provided by these new guidelines, and will strive to achieve as much of it as we can before release. Because these guidelines are still new (formally approved in June 2018), we won't hold up releasing the stable version of Layout Builder on them, but are committed to implementing them as quickly as we're able to, even if some of the items are after initial release.
  3. While WCAG AAA conformance is not something currently being pursued, there are aspects of AAA that we are discussing adopting in the future. For example, the new 2.1 AAA "Animations from Interactions", which can be framed as an achievable design constraint: anywhere an animation is used, we must ensure designs are understandable/operable for those who cannot or choose not to use animations.

Drupal's commitment to accessibility is one of the things that makes Drupal's upcoming Layout Builder special: it will not only bring tremendous and new capabilities to Drupal, it will also do so without excluding a large portion of current and potential users. We all benefit from that!

Dec 05 2018
Dec 05

The Drupal Association seeks volunteer organizations from Agency and Drupal site owners running production Drupal 8 sites for the creation of an official minor-release beta-testers program.

Since Drupal 8.0's release in November 2015, the Drupal community has successfully transitioned to a scheduled release process whereby two minor releases are made every year.

The most recent of these releases was Drupal 8.6, released in September 2018.

In a significant change from Drupal 7, these minor releases may contain new features and maintain backwards compatibility. This means that every six months there are new features in Drupal core, instead of waiting for the next major release.

This rapid acceleration in feature development has resulted in the need for greater real-world testing of upgrade paths and backwards compatibility. Drupal core has a vast automated test-suite comprising almost 25,000 tests—however, these can be greatly complemented by real-world testing of production sites. There are an infinite number of ways to put Drupal together that cannot always be handled in automated tests.

In order to improve the reliability of the minor-releases, the Drupal community—in conjunction with the Drupal Association—aims to develop a minor-release beta testers panel comprised of agencies and site-owners who maintain complex Drupal 8 production sites.

Many companies and Drupal users are looking to help with core development but aren't always sure where to start. Membership in this panel presents a new way to help the development of software that powers their website.

Who should apply?

Agencies and site owners who maintain large and complex Drupal 8 production sites. In particular, sites that use a wide range of contributed and custom modules or have large volumes of content.

What is involved?

When the beta release becomes available, the Drupal core committers will work in conjunction with the Drupal Association to contact the members of the beta-testing panel to advise that the next minor release is ready for testing.

Members of the panel will be asked to attempt updating to the beta using a staging version of their site (not straight-on production) and report back any issues found. New issues will be opened to track and resolve reported issues. The community will try to resolve the issue before the release deadline. If an issue cannot be resolved in time for the scheduled release, it will be documented in the release notes, or if it is severe enough, release managers may opt to revert the change that introduced the issue. Participants whose testing participation lapses may be removed from the program.

At the moment, testing of the new release occurs in a largely ad-hoc fashion, but once the program is established, this will become more structured and maintainers will have access to statistics regarding the breadth of testing. This will then inform release management decisions in regards to release preparedness.

What's in it for participants?

  • Updating early helps find issues beforehand, rather than after the release is out.

  • Reporting issues encountered lets you tap the wealth of experience of the Drupal core contributors that you'd not have the same level of access to if you update on your own after the release.

  • All organizations and individuals taking part in the testing will receive issue credits for both testing the update and fixing any issues that arise.

  • Satisfaction in the knowledge that you helped shape the next minor release of Drupal core.

  • Advanced preview of upcoming features in Drupal core.

Apply to participate in the program

Nov 14 2018
Nov 14

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

After months of hard work, the Drupal Governance Task Force made thirteen recommendations for how to evolve Drupal's governance.

If you want to go quickly, go alone. If you want to go far, go together.

Drupal exists because of its community. What started from humble beginnings has grown into one of the largest Open Source communities in the world. This is due to the collective effort of thousands of community members.

What distinguishes Drupal from other open source projects is both the size and diversity of our community, and the many ways in which thousands of contributors and organizations give back. It's a community I'm very proud to be a part of.

Without the Drupal community, the Drupal project wouldn't be where it is today and perhaps would even cease to exist. That is why we are always investing in our community and why we constantly evolve how we work with one another.

The last time we made significant changes to Drupal's governance was over five years ago when we launched a variety of working groups. Five years is a long time. The time had come to take a step back and to look at Drupal's governance with fresh eyes.

Throughout 2017, we did a lot of listening. We organized both in-person and virtual roundtables to gather feedback on how we can improve our community governance. This led me to invest a lot of time and effort in documenting Drupal's Values and Principles.

In 2018, we transitioned from listening to planning. Earlier this year, I chartered the Drupal Governance Task Force. The goal of the task force was to draft a set of recommendations for how to evolve and strengthen Drupal's governance based on all of the feedback we received. Last week, after months of work and community collaboration, the task force shared thirteen recommendations (PDF).

The proposal from the Drupal Governance Task Force

Me reviewing the Drupal Governance proposal on a recent trip.

Before any of us jump to action, the Drupal Governance Task Force recommended a thirty-day, open commentary period to give community members time to read the proposal and to provide more feedback. After the thirty-day commentary period, I will work with the community, various stakeholders, and the Drupal Association to see how we can move these recommendations forward. During the thirty-day open commentary period, you can then get involved by collaborating and responding to each of the individual recommendations below:

I'm impressed by the thought and care that went into writing the recommendations, and I'm excited to help move them forward.

Some of the recommendations are not new and are ideas that either the Drupal Association, myself or others have been working on, but that none of us have been able to move forward without a significant amount of funding or collaboration.

I hope that 2019 will be a year of organizing and finding resources that allow us to take action and implement a number of the recommendations. I'm convinced we can make valuable progress.

I want to thank everyone who has participated in this process. This includes community members who shared information and insight, facilitated conversations around governance, were interviewed by the task force, and supported the task force's efforts. Special thanks to all the members of the task force who worked on this with great care and determination for six straight months: Adam BergsteinLyndsey JacksonEla MeierStella PowerRachel LawsonDavid Hernandez and Hussain Abbas.

Nov 12 2018
Nov 12

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Layout Builder, which will be in the next release of Drupal 8, is unique in that it can work with structured and unstructured content, and with templated and free-form pages.

Content authors want an easy-to-use page building experience; they want to create and design pages using drag-and-drop and WYSIWYG tools. For over a year the Drupal community has been working on a new Layout Builder, which is designed to bring this page building capability into Drupal core.

Drupal's upcoming Layout Builder is unique in offering a single, powerful visual design tool for the following three use cases:

  1. Layouts for templated content. The creation of "layout templates" that will be used to layout all instances of a specific content type (e.g. blog posts, product pages).
  2. Customizations to templated layouts. The ability to override these layout templates on a case-by-case basis (e.g. the ability to override the layout of a standardized product page)
  3. Custom pages. The creation of custom, one-off landing pages not tied to a content type or structured content (e.g. a single "About us" page).

Let's look at all three use cases in more detail to explain why we think this is extremely useful!

Use case 1: Layouts for templated content

For large sites with significant amounts of content it is important that the same types of content have a similar appearance.

A commerce site selling hundreds of different gift baskets with flower arrangements should have a similar layout for all gift baskets. For customers, this provides a consistent experience when browsing the gift baskets, making them easier to compare. For content authors, the templated approach means they don't have to worry about the appearance and layout of each new gift basket they enter on the site. They can be sure that once they have entered the price, description, and uploaded an image of the item, it will look good to the end user and similar to all other gift baskets on the site.

The Drupal 8 Layout Builder showing a templated gift basket

Drupal 8's new Layout Builder allows a site creator to visually create a layout template that will be used for each item of the same content type (e.g. a "gift basket layout" for the "gift basket" content type). This is possible because the Layout Builder benefits from Drupal's powerful "structured content" capabilities.

Many of Drupal's competitors don't allow such a templated approach to be designed in the browser. Their browser-based page builders only allow you to create a design for an individual page. When you want to create a layout that applies to all pages of a specific content type, it is usually not possible without a developer.

Use case 2: Customizations to templated layouts

While having a uniform look for all products of a particular type has many advantages, sometimes you may want to display one or more products in a slightly (or dramatically) different way.

Perhaps a customer recorded a video of giving their loved one one of the gift baskets, and that video has recently gone viral (because somehow it involved a puppy). If you only want to update one of the gift baskets with a video, it may not make sense to add an optional "highlighted video" field to all gift baskets.

Drupal 8's Layout Builder offers the ability to customize templated layouts on a case per case basis. In the "viral, puppy, gift basket" video example, this would allow a content creator to rearrange the layout for just that one gift basket, and put the viral video directly below the product image. In addition, the Layout Builder would allow the site to revert the layout to match all other gift baskets once the world has moved on to the next puppy video.

The Drupal 8 Layout Builder showing a templated gift basket with a puppy video

Since most content management systems don't allow you to visually design a layout pattern for certain types of structured content, they of course can't allow for this type of customization.

Use case 3: Custom pages (with unstructured content)

Of course, not everything is templated, and content authors often need to create one-off pages like an "About us" page or the website's homepage.

In addition to visually designing layout templates for different types of content, Drupal 8's Layout Builder can also be used to create these dynamic one-off custom pages. A content author can start with a blank page, design a layout, and start adding blocks. These blocks can contain videos, maps, text, a hero image, or custom-built widgets (e.g. a Drupal View showing a list of the ten most popular gift baskets). Blocks can expose configuration options to the content author. For instance, a hero block with an image and text may offer a setting to align the text left, right, or center. These settings can be configured directly from a sidebar.

The Drupal 8 Layout Builder showing how to configure a block

In many other systems content authors are able to use drag-and-drop WYSIWYG tools to design these one-off pages. This type of tool is used in many projects and services such as Squarespace and the new Gutenberg Editor for WordPress (now available for Drupal, too!).

On large sites, the free-form page creation is almost certainly going to be a scalability, maintenance and governance challenge.

For smaller sites where there may not be many pages or content authors, these dynamic free-form page builders may work well, and the unrestricted creative freedom they provide might be very compelling. However, on larger sites, when you have hundreds of pages or dozens of content creators, a templated approach is going to be preferred.

When will Drupal's new Layout Builder be ready?

Drupal 8's Layout Builder is still a beta level experimental module, with 25 known open issues to be addressed prior to becoming stable. We're on track to complete this in time for Drupal 8.7's release in May 2019. If you are interested in increasing the likelihood of that, you can find out how to help on the Layout Initiative homepage.

An important note on accessibility

Accessibility is one of Drupal's core tenets, and building software that everyone can use is part of our core values and principles. A key part of bringing Layout Builder functionality to a "stable" state for production use will be ensuring that it passes our accessibility gate (Level AA conformance with WCAG and ATAG). This holds for both the authoring tool itself, as well as the markup that it generates. We take our commitment to accessibility seriously.

Impact on contributed modules and existing sites

Currently there a few methods in the Drupal module ecosystem for creating templated layouts and landing pages, including the Panels and Panelizer combination. We are currently working on a migration path for Panels/Panelizer to the Layout Builder.

The Paragraphs module currently can be used to solve several kinds of content authoring use-cases, including the creation of custom landing pages. It is still being determined how Paragraphs will work with the Layout Builder and/or if the Layout Builder will be used to control the layout of Paragraphs.

Conclusion

Drupal's upcoming Layout Builder is unique in that it supports multiple different use cases; from templated layouts that can be applied to dozens or hundreds of pieces of structured content, to designing custom one-off pages with unstructured content. The Layout Builder is even more powerful when used in conjunction with Drupal's other out-of-the-box features such as revisioning, content moderation, and translations, but that is a topic for a future blog post.

Special thanks to Ted Bowman (Acquia) for co-authoring this post. Also thanks to Wim Leers (Acquia), Angie Byron (Acquia), Alex Bronstein (Acquia), Jeff Beeman (Acquia) and Tim Plunkett (Acquia) for their feedback during the writing process.

Nov 09 2018
Nov 09

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

If you are still using PHP 5, now is the time to upgrade to a newer version of PHP.

PHP, the Open Source scripting language, is used by nearly 80 percent of the world's websites.

According to W3Techs, around 61 percent of all websites on the internet still use PHP 5, a version of PHP that was first released fourteen years ago.

Now is the time to give PHP 5 some attention. In less than two months, on December 31st, security support for PHP 5 will officially cease. (Note: Some Linux distributions, such as Debian Long Term Support distributions, will still try to backport security fixes.)

If you haven't already, now is the time to make sure your site is running an updated and supported version of PHP.

Beyond security considerations, sites that are running on older versions of PHP are missing out on the significant performance improvements that come with the newer versions.

Drupal and PHP 5

Drupal 8

Drupal 8 will drop support for PHP 5 on March 6, 2019. We recommend updating to at least PHP 7.1 if possible, and ideally PHP 7.2, which is supported as of Drupal 8.5 (which was released March, 2018). Drupal 8.7 (to be released in May, 2019) will support PHP 7.3, and we may backport PHP 7.3 support to Drupal 8.6 in the coming months as well.

Drupal 7

Drupal 7 will drop support for older versions of PHP 5 on December 31st, but will continue to support PHP 5.6 as long there are one or more third-party organizations providing reliable, extended security support for PHP 5.

Earlier today, we released Drupal 7.61 which now supports PHP 7.2. This should make upgrades from PHP 5 easier. Drupal 7's support for PHP 7.3 is being worked on but we don't know yet when it will be available.

Thank you!

It's a credit to the PHP community that they have maintained PHP 5 for fourteen years. But that can't go on forever. It's time to move on from PHP 5 and upgrade to a newer version so that we can all innovate faster.

I'd also like to thank the Drupal community — both those contributing to Drupal 7 and Drupal 8 — for keeping Drupal compatible with the newest versions of PHP. That certainly helps make PHP upgrades easier.

Nov 02 2018
Nov 02

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Configuration management is an important feature of any modern content management system. Those following modern development best-practices use a development workflow that involves some sort of development and staging environment that is separate from the production environment.

Configuration management example

Given such a development workflow, you need to push configuration changes from development to production (similar to how you need to push code or content between environments). Drupal's configuration management system helps you do that in a powerful yet elegant way.

Since I announced the original Configuration Management Initiative over seven years ago, we've developed and shipped a strong configuration management API in Drupal 8. Drupal 8's configuration management system is a huge step forward from where we were in Drupal 7, and a much more robust solution than what is offered by many of our competitors.

All configuration in a Drupal 8 site — from one-off settings such as site name to content types and field definitions — can be seamlessly moved between environments, allowing for quick and easy deployment between development, staging and production environments.

However, now that we have a couple of years of building Drupal 8 sites behind us, various limitations have surfaced. While these limitations usually have solutions via contributed modules, it has become clear that we would benefit from extending Drupal core's built-in configuration management APIs. This way, we can establish best practices and standard approaches that work for all.

Configuraton management initiative

The four different focus areas for Drupal 8. The configuration management initiative is part of the 'Improve Drupal for developers' track.

I first talked about this need in my DrupalCon Nashville keynote, where I announced the Configuration Management 2.0 initiative. The goal of this initiative is to extend Drupal's built-in configuration management so we can support more common workflows out-of-the-box without the need of contributed modules.

What is an example workflow that is not currently supported out-of-the-box? Support for different configurations by environment. This is a valuable use case because some settings are undesirable to have enabled in all environments. For example, you most likely don't want to enable debugging tools in production.

Configuration management example

The contributed module Config Filter extends Drupal core's built-in configuration management capabilities by providing an API to support different workflows which filter out or transform certain configuration changes as they are being pushed to production. Config Split, another contributed module, builds on top of Config Filter to allow for differences in configuration between various environments.

The Config Split module's use case is just one example of how we can improve Drupal's out-of-the-box configuration management capabilities. The community created a longer list of pain points and advanced use cases for the configuration management system.

While the initiative team is working on executing on these long-term improvements, they are also focused on delivering incremental improvements with each new version of Drupal 8, and have distilled the most high-priority items into a configuration management roadmap.

  • In Drupal 8.6, we added support for creating new sites from existing configuration. This enables developers to launch a development site that matches a production site's configuration with just a few clicks.
  • For Drupal 8.7, we're planning on shipping an experimental module for dealing with environment specific configuration, moving the capabilities of Config Filter and the basic capabilities of Config Split to Drupal core through the addition of a Configuration Transformer API.
  • For Drupal 8.8, the focus is on supporting configuration updates across different sites. We want to allow both sites and distributions to package configuration (similar to the well-known Features module) so they can easily be deployed across other sites.

How to get involved

There are many opportunities to contribute to this initiative and we'd love your help.

If you would like to get involved, check out the Configuration Management 2.0 project and various Drupal core issues tagged as "CMI 2.0 candidate".

Special thanks to Fabian Bircher (Nuvole), Jeff Beeman (Acquia), Angela Byron (Acquia), ASH (Acquia), and Alex Pott (Thunder) for contributions to this blog post.

Oct 29 2018
Oct 29

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

The cover of the Decoupled Drupal book

Drupal has evolved significantly over the course of its long history. When I first built the Drupal project eighteen years ago, it was a message board for my friends that I worked on in my spare time. Today, Drupal runs two percent of all websites on the internet with the support of an open-source community that includes hundreds of thousands of people from all over the world.

Today, Drupal is going through another transition as its capabilities and applicability continue to expand beyond traditional websites. Drupal now powers digital signage on university campuses, in-flight entertainment systems on commercial flights, interactive kiosks on cruise liners, and even pushes live updates to the countdown clocks in the New York subway system. It doesn't stop there. More and more, digital experiences are starting to encompass virtual reality, augmented reality, chatbots, voice-driven interfaces and Internet of Things applications. All of this is great for Drupal, as it expands its market opportunity and long-term relevance.

Several years ago, I began to emphasize the importance of an API-first approach for Drupal as part of the then-young phenomenon of decoupled Drupal. Now, Drupal developers can count on JSON API, GraphQL and CouchDB, in addition to a range of surrounding tools for developing the decoupled applications described above. These decoupled Drupal advancements represent a pivotal point in Drupal's history.

Decoupled Drupal sites

A few examples of organizations that use decoupled Drupal.

Speaking of important milestones in Drupal's history, I remember the first Drupal book ever published in 2005. At the time, good information on Drupal was hard to find. The first Drupal book helped make the project more accessible to new developers and provided both credibility and reach in the market. Similarly today, decoupled Drupal is still relatively new, and up-to-date literature on the topic can be hard to find. In fact, many people don't even know that Drupal supports decoupled architectures. This is why I'm so excited about the upcoming publication of a new book entitled Decoupled Drupal in Practice, written by Preston So. It will give decoupled Drupal more reach and credibility.

When Preston asked me to write the foreword for the book, I jumped at the chance because I believe his book will be an important next step in the advancement of decoupled Drupal. I've also been working with Preston So for a long time. Preston is currently Director of Research and Innovation at Acquia and a globally respected expert on decoupled Drupal. Preston has been involved in the Drupal community since 2007, and I first worked with him directly in 2012 on the Spark initiative to improve Drupal's editorial user experience. Preston has been researching, writing and speaking on the topic of decoupled Drupal since 2015, and had a big impact on my thinking on decoupled Drupal, on Drupal's adoption of React, and on decoupled Drupal architectures in the Drupal community overall.

To show the value that this book offers, you can read exclusive excerpts of three chapters from Decoupled Drupal in Practice on the Acquia blog and at the Acquia Developer Center. It is available for preorder today on Amazon, and I encourage my readers to pick up a copy!

Congratulations on your book, Preston!

Sep 17 2018
Sep 17

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Last week, nearly 1,000 Drupalists gathered in Darmstadt, Germany for Drupal Europe. In good tradition, I presented my State of Drupal keynote. You can watch a recording of my keynote (starting at 4:38) or download a copy of my slides (37 MB).

Drupal 8 continues to mature

I started my keynote by highlighting this month's Drupal 8.6.0 release. Drupal 8.6 marks the sixth consecutive Drupal 8 release that has been delivered on time. Compared to one year ago, we have 46 percent more stable Drupal 8 modules. We also have 10 percent more contributors are working on Drupal 8 Core in comparison to last year. All of these milestones indicate that the Drupal 8 is healthy and growing.

The number of stable modules for Drupal 8 is growing fast

Next, I gave an update on our strategic initiatives:

An overview of Drupal 8's strategic initaitives

Make Drupal better for content creators

A photo from Drupal Europe in Darmstadt

© Paul Johnson

The expectations of content creators are changing. For Drupal to be successful, we have to continue to deliver on their needs by providing more powerful content management tools, in addition to delivering simplicity though drag-and-drop functionality, WYSIWYG, and more.

With the release of Drupal 8.6, we have added new functionality for content creators by making improvements to the Media, Workflow, Layout and Out-of-the-Box initiatives. I showed a demo video to demonstrate how all of these new features not only make content authoring easier, but more powerful:

We also need to improve the content authoring experience through a modern administration user interface. We have been working on a new administration UI using React. I showed a video of our latest prototype:

Extended security coverage for Drupal 8 minor releases

I announced an update to Drupal 8's security policy. To date, site owners had one month after a new minor Drupal 8 release to upgrade their sites before losing their security updates. Going forward, Drupal 8 site owners have 6 months to upgrade between minor releases. This extra time should give site owners flexibility to plan, prepare and test minor security updates. For more information, check out my recent blog post.

Make Drupal better for evaluators

One of the most significant updates since DrupalCon Nashville is Drupal's improved evaluator experience. The time required to get a Drupal site up and running has decreased from more than 15 minutes to less than two minutes and from 20 clicks to 3. This is a big accomplishment. You can read more about it in my recent blog post.

Promote Drupal

After launching Promote Drupal at DrupalCon Nashville, we hit the ground running with this initiative and successfully published a community press release for the release of Drupal 8.6, which was also translated into multiple languages. Much more is underway, including building a brand book, marketing collaboration space on Drupal.org, and a Drupal pitch deck.

The Drupal 9 roadmap and a plan to end-of-life Drupal 7 and Drupal 8

To keep Drupal modern, maintainable, and performant, we need to stay on secure, supported versions of Drupal 8's third-party dependencies. This means we need to end-of-life Drupal 8 with Symfony 3's end-of-life. As a result, I announced that:

  1. Drupal 8 will be end-of-life by November 2021.
  2. Drupal 9 will be released in 2020, and it will be an easy upgrade.

Historically, our policy has been to only support two major versions of Drupal; Drupal 7 would ordinarily reach end of life when Drupal 9 is released. Because a large number of sites might still be using Drupal 7 by 2020, we have decided to extend support of Drupal 7 until November 2021.

For those interested, I published a blog post that further explains this.

Adopt GitLab on Drupal.org

Finally, the Drupal Association is working to integrate GitLab with Drupal.org. GitLab will provide support for "merge requests", which means contributing to Drupal will feel more familiar to the broader audience of open source contributors who learned their skills in the post-patch era. Some of GitLab's tools, such as inline editing and web-based code review, will also lower the barrier to contribution, and should help us grow both the number of contributions and contributors on Drupal.org.

To see an exciting preview of Drupal.org's gitlab integration, watch the video below:

Thank you

Our community has a lot to be proud of, and this progress is the result of thousands of people collaborating and working together. It's pretty amazing! The power of our community isn't just visible in minor releases or a number of stable modules. It was also felt at this very conference, as many volunteers gave their weekends and evenings to help organize Drupal Europe in the absence of a DrupalCon Europe organized by the Drupal Association. From code to community, the Drupal project is making an incredible impact. I look forward to celebrating our community's work and friendships at future Drupal conferences.

A summary of our recent progress

Sep 14 2018
Sep 14

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Seven months ago, Matthew Grasmick published an article describing how hard it is to install Drupal. His article included the following measurements for creating a new application on his local machine, across four different PHP frameworks:

Platform Clicks Time Drupal 20+ 15:00+ Symfony 3 1:55 WordPress 7 7:51 Laravel 3 17:28

The results from Matthew's blog were clear: Drupal is too hard to install. It required more than 15 minutes and 20 clicks to create a simple site.

Seeing these results prompted me to launch a number of initiatives to improve the evaluator experience at DrupalCon Nashville. Here is the slide from my DrupalCon Nashville presentation:

Improve technical evaluator experience

A lot has happened between then and now:

  • We improved the download page to improve the discovery experience on drupal.org
  • We added an Evaluator Guide to Drupal.org
  • We added a quick-start command to Drupal 8.6
  • We added the Umami demo profile to Drupal 8.6
  • We started working on a more modern administration experience (in progress)

You can see the result of that work in this video:

Thanks to this progress, here is the updated table:

Platform Clicks Time Drupal 3 1:27 Symfony 3 1:55 WordPress 7 7:51 Laravel 3 17:28

Drupal now requires the least time and is tied for least clicks! You can now install Drupal in less than two minutes. Moreover, the Drupal site that gets created isn't an "empty canvas" anymore; it's a beautifully designed and fully functional application with demo content.

Copy-paste the following commands in a terminal window if you want to try it yourself:

mkdir drupal && cd drupal && curl -sSL https://www.drupal.org/download-latest/tar.gz | tar -xz --strip-components=1
php core/scripts/drupal quick-start demo_umami

For more detailed information on how we achieved these improvements, read Matthew's latest blog post: The New Drupal Evaluator Experience, by the numbers.

A big thank you to Matthew Grasmick (Acquia) for spearheading this initiative!

Sep 13 2018
Sep 13

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Since the launch of Drupal 8.0, we have successfully launched a new minor release on schedule every six months. I'm very proud of the community for this achievement. Prior to Drupal 8, most significant new features were only added in major releases like Drupal 6 or Drupal 7. Thanks to our new release cadence we now consistently and predictably ship great new features twice a year in minor releases (e.g. Drupal 8.6 comes with many new features).

However, only the most recent minor release has been actively supported for both bug fixes and security coverage. With the release of each new minor version, we gave a one-month window to upgrade to the new minor. In order to give site owners time to upgrade, we would not disclose security issues with the previous minor release during that one-month window.

Old Drupal 8 security policy for minor releases

Illustration of the security policy since the launch of Drupal 8.0 for minor releases, demonstrating that previous minor releases receive one month of security coverage. Source: Drupal.org issue #2909665: Extend security support to cover the previous minor version of Drupal and Drupal Europe DriesNote.

Over the past three years, we have learned that users find it challenging to update to the latest minor in one month. Drupal's minor updates can include dependency updates, internal API changes, or features being transitioned from contributed modules to core. It takes time for site owners to prepare and test these types of changes, and a window of one month to upgrade isn't always enough.

At DrupalCon Nashville we declared that we wanted to extend security coverage for minor releases. Throughout 2018, Drupal 8 release managers quietly conducted a trial. You may have noticed that we had several security releases against previous minor releases this year. This trial helped us understand the impact to the release process and learn what additional work remained ahead. You can read about the results of the trial at #2909665: Extend security support to cover the previous minor version of Drupal.

I'm pleased to share that the trial was a success! As a result, we have extended the security coverage of minor releases to six months. Instead of one month, site owners now have six months to upgrade between minor releases. It gives teams time to plan, prepare and test updates. Releases will have six months of normal bug fix support followed by six months of security coverage, for a total lifetime of one year. This is a huge win for Drupal site owners.

New Drupal 8 security policy for minor releases

Illustration of the new security policy for minor releases, demonstrating that the security coverage for minor releases is extended to six months. Source: Drupal.org issue #2909665: Extend security support to cover the previous minor version of Drupal and the Drupal Europe DriesNote.

It's important to note that this new policy only applies to Drupal 8 core starting with Drupal 8.5, and only applies to security issues. Non-security bug fixes will still only be committed to the actively supported release.

While the new policy will provide extended security coverage for Drupal 8.5.x, site owners will need to update to an upcoming release of Drupal 8.5 to be correctly notified about their security coverage.

Next steps

We still have some user experience issues we'd like to address around how site owners are alerted of a security update. We have not yet handled all of the potential edge cases, and we want to be very clear about the potential actions to take when updating.

We also know module developers may need to declare that a release of their project only works against specific versions of Drupal core. Resolving outstanding issues around semantic versioning support for contrib and module version dependency definitions will help developers of contributed projects better support this policy. If you'd like to get involved in the remaining work, the policy and roadmap issue on Drupal.org is a great place to find related issues and see what work is remaining.

Special thanks to Jess and Jeff Beeman for co-authoring this post.

Sep 12 2018
Sep 12

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

We just released Drupal 8.6.0. With six minor releases behind us, it is time to talk about the long-term future of Drupal 8 (and therefore Drupal 7 and Drupal 9). I've written about when to release Drupal 9 in the past, but this time, I'm ready to provide further details.

The plan outlined in this blog has been discussed with the Drupal 7 Core Committers, the Drupal 8 Core Committers and the Drupal Security Team. While we feel good about this plan, we can't plan for every eventuality and we may continue to make adjustments.

Drupal 8 will be end-of-life by November 2021

Drupal 8's innovation model depends on introducing new functionality in minor versions while maintaining backwards compatibility. This approach is working so well that some people have suggested we institute minor releases forever, and never release Drupal 9 at all.

However that approach is not feasible. We need to periodically remove deprecated functionality to keep Drupal modern, maintainable, and performant, and we need to stay on secure, supported versions of Drupal 8's third-party dependencies. As Nathaniel Catchpole explained in his post "The Long Road to Drupal 9", our use of various third party libraries such as Symfony, Twig, and Guzzle means that we need to be in sync with their release timelines.

Our biggest dependency in Drupal 8 is Symfony 3, and according to Symfony's roadmap, Symfony 3 has an end-of-life date in November 2021. This means that after November 2021, security bugs in Symfony 3 will not get fixed. To keep your Drupal sites secure, Drupal must adopt Symfony 4 or Symfony 5 before Symfony 3 goes end-of-life. A major Symfony upgrade will require us to release Drupal 9 (we don't want to fork Symfony 3 and have to backport Symfony 4 or Symfony 5 bug fixes). This means we have to end-of-life Drupal 8 no later than November 2021.

Drupal 8 will be end-of-life by November 2021

Drupal 9 will be released in 2020, and it will be an easy upgrade

If Drupal 8 will be end-of-life on November 2021, we have to release Drupal 9 before that. Working backwards from November 2021, we'd like to give site owners one year to upgrade from Drupal 8 to Drupal 9.

If November 2020 is the latest we could release Drupal 9, what is the earliest we could release Drupal 9?

We certainly can't release Drupal 9 next week or even next month. Preparing for Drupal 9 takes a lot of work: we need to adopt Symfony 4 and/or Symfony 5, we need to remove deprecated code, we need to allow modules and themes to declare compatibility with more than one major version, and possibly more. The Drupal 8 Core Committers believe we need more than one year to prepare for Drupal 9.

Therefore, our current plan is to release Drupal 9 in 2020. Because we still need to figure out important details, we can't be more specific at this time.

Drupal 9 will be released in 2020

If we release Drupal 9 in 2020, it means we'll certainly have Drupal 8.7 and 8.8 releases.

Wait, I will only have one year to migrate from Drupal 8 to 9?

Yes, but fortunately moving from Drupal 8 to 9 will be far easier than previous major version upgrades. The first release of Drupal 9 will be very similar to the last minor release of Drupal 8, as the primary goal of the Drupal 9.0.0 release will be to remove deprecated code and update third-party dependencies. By keeping your Drupal 8 sites up to date, you should be well prepared for Drupal 9.

And what about contributed modules? The compatibility of contributed modules is historically one of the biggest blockers to upgrading, so we will also make it possible for contributed modules to be compatible with Drupal 8 and Drupal 9 at the same time. As long as contributed modules do not use deprecated APIs, they should work with Drupal 9 while still being compatible with Drupal 8.

Drupal 7 will be supported until November 2021

Historically, our policy has been to only support two major versions of Drupal; Drupal 7 would ordinarily reach end of life when Drupal 9 is released. Because a large number of sites might still be using Drupal 7 by 2020, we have decided to extend support of Drupal 7 until November 2021. Drupal 7 will be receive community support for three whole more years.

Drupal 7 will be supported until November 2021

We'll launch a Drupal 7 commercial Long Term Support program

In the past, commercial vendors have extended Drupal's security support. In 2015, a Drupal 6 commercial Long Term Support program was launched and continues to run to this day. We plan a similar paid program for Drupal 7 to extend support beyond November 2021. The Drupal Security Team will announce the Drupal 7 commercial LTS program information by mid-2019. Just like with the Drupal 6 LTS program, there will be an application for vendors.

We'll update Drupal 7 to support newer versions of PHP

The PHP team will stop supporting PHP 5.x on December 31st, 2018 (in 3 months), PHP 7.0 on December 3rd, 2018 (in 2 months), PHP 7.1 on December 1st, 2019 (in 1 year and 3 months) and PHP 7.2 on November 30th, 2020 (in 2 years and 2 months).

Drupal will drop official support for unsupported PHP versions along the way and Drupal 7 site owners may have to upgrade their PHP version. The details will be provided later.

We plan on updating Drupal 7 to support newer versions of PHP in line with their support schedule. Drupal 7 doesn't fully support PHP 7.2 yet as there have been some backwards-incompatible changes since PHP 7.1. We will release a version of Drupal 7 that supports PHP 7.2. Contributed modules and custom modules will have to be updated too, if not already.

Conclusion

If you are still using Drupal 7 and are wondering what to do, you currently have two options:

  1. Stay on Drupal 7 while also updating your PHP version. If you stay on Drupal 7 until after 2021, you can either engage a vendor for a long term support contract, or migrate to Drupal 9.
  2. Migrate to Drupal 8 by 2020, so that it's easier to update to Drupal 9 when it is released.

The announcements in this blog post made option (1) a lot more viable and/or hopefully helps you better evaluate option (2).

If you are on Drupal 8, you just have to keep your Drupal 8 site up-to-date and you'll be ready for Drupal 9.

We plan to have more specifics by April 2019 (DrupalCon Seattle).

Thanks for the Drupal 7 Core Committers, the Drupal 8 Core Committers and the Drupal Security Team for their contributions to this blog post.

Sep 12 2018
Sep 12

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

For the past two years, I've examined Drupal.org's commit data to understand who develops Drupal, how much of that work is sponsored, and where that sponsorship comes from.

I have now reported on this data for three years in a row, which means I can start to better compare year-over-year data. Understanding how an open-source project works is important because it establishes a benchmark for project health and scalability.

I would also recommend taking a look at the 2016 report or the 2017 report. Each report looks at data collected in the 12-month period between July 1st and June 30th.

This year's report affirms that Drupal has a large and diverse community of contributors. In the 12-month period between July 1, 2017 and June 30, 2018, 7,287 different individuals and 1,002 different organizations contributed code to Drupal.org. This include contributions to Drupal core and all contributed projects on Drupal.org.

In comparison to last year's report, both the number of contributors and contributions has increased. Our community of contributors (including both individuals and organizations) is also becoming more diverse. This is an important area of growth, but there is still work to do.

For this report, we looked at all of the issues marked "closed" or "fixed" in our ticketing system in the 12-month period from July 1, 2017 to June 30, 2018. This includes Drupal core and all of the contributed projects on Drupal.org, across all major versions of Drupal. This year, 24,447 issues were marked "closed" or "fixed", a 5% increase from the 23,238 issues in the 2016-2017 period. This averages out to 67 feature improvements or bug fixes a day.

In total, we captured 49,793 issue credits across all 24,447 issues. This marks a 17% increase from the 42,449 issue credits recorded in the previous year. Of the 49,793 issue credits reported this year, 18% (8,822 credits) were for Drupal core, while 82% (40,971 credits) went to contributed projects.

"Closed" or "fixed" issues are often the result of multiple people working on the issue. We try to capture who contributes through Drupal.org's unique credit system. We used the data from the credit system for this analysis. There are a few limitations with this approach, which we'll address at the end of this report.

What is the Drupal.org credit system?

In the spring of 2015, after proposing ideas for giving credit and discussing various approaches at length, Drupal.org added the ability for people to attribute their work to an organization or customer in the Drupal.org issue queues. Maintainers of Drupal modules, themes, and distributions can award issue credits to people who help resolve issues with code, translations, documentation, design and more.

Example issue credit on drupal org

A screenshot of an issue comment on Drupal.org. You can see that jamadar worked on this patch as a volunteer, but also as part of his day job working for TATA Consultancy Services on behalf of their customer, Pfizer.

Credits are a powerful motivator for both individuals and organizations. Accumulating credits provides individuals with a way to showcase their expertise. Organizations can utilize credits to help recruit developers, to increase their visibility within the Drupal.org marketplace, or to showcase their Drupal expertise.

Who is working on Drupal?

In the 12-month period between July 1, 2017 to June 30, 2018, 24,447, Drupal.org received code contributions from 7,287 different individuals and 1,002 different organizations.

Contributions by individuals vs organizations

While the number of individual contributors rose, a relatively small number of individuals still do the majority of the work. Approximately 48% of individual contributors received just one credit. Meanwhile, the top 30 contributors (the top 0.4%) account for more than 24% of the total credits. These individuals put an incredible amount of time and effort in developing Drupal and its contributed projects:

Rank Username Issues 1 RenatoG 851 2 RajabNatshah 745 3 jrockowitz 700 4 adriancid 529 5 bojanz 515 6 Berdir 432 7 alexpott 414 8 mglaman 414 9 Wim Leers 395 10 larowlan 360 11 DamienMcKenna 353 12 dawehner 340 13 catch 339 14 heddn 327 15 xjm 303 16 pifagor 284 17 quietone 261 18 borisson_ 255 19 adci_contributor 255 20 volkswagenchick 254 21 drunken monkey 231 22 amateescu 225 23 joachim 199 24 mkalkbrenner 195 25 chr.fritsch 185 26 gaurav.kapoor 178 27 phenaproxima 177 28 mikeytown2 173 29 joelpittet 170 30 timmillwood 169

Out of the top 30 contributors featured, 15 were also recognized as top contributors in our 2017 report. These Drupalists' dedication and continued contribution to the project has been crucial to Drupal's development. It's also exciting to see 15 new names on the list. This mobility is a testament to the community's evolution and growth. It's also important to recognize that a majority of the 15 repeat top contributors are at least partially sponsored by an organization. We value the organizations that sponsor these remarkable individuals, because without their support, it could be more challenging to be in the top 30 year over year.

How diverse is Drupal?

Next, we looked at both the gender and geographic diversity of Drupal.org code contributors. While these are only two examples of diversity, this is the only available data that contributors can currently choose to share on their Drupal.org profiles. The reported data shows that only 7% of the recorded contributions were made by contributors that do not identify as male, which continues to indicates a steep gender gap. This is a one percent increase compared to last year. The gender imbalance in Drupal is profound and underscores the need to continue fostering diversity and inclusion in our community.

To address this gender gap, in addition to advancing representation across various demographics, the Drupal community is supporting two important initiatives. The first is to adopt more inclusive user demographic forms on Drupal.org. Adopting Open Demographics on Drupal.org will also allow us to improve reporting on diversity and inclusion, which in turn will help us better support initiatives that advance diversity and inclusion. The second initiative is supporting the Drupal Diversity and Inclusion Contribution Team, which works to better include underrepresented groups to increase code and community contributions. The DDI Contribution Team recruits team members from diverse backgrounds and underrepresented groups, and provides support and mentorship to help them contribute to Drupal.

It's important to reiterate that supporting diversity and inclusion within Drupal is essential to the health and success of the project. The people who work on Drupal should reflect the diversity of people who use and work with the software. While there is still a lot of work to do, I'm excited about the impact these various initiatives will have on future reports.

Contributions by gender

When measuring geographic diversity, we saw individual contributors from 6 different continents and 123 different countries:

Contributions by continent

Contributions by country

The top 20 countries from which contributions originate. The data is compiled by aggregating the countries of all individual contributors behind each commit. Note that the geographical location of contributors doesn't always correspond with the origin of their sponsorship. Wim Leers, for example, works from Belgium, but his funding comes from Acquia, which has the majority of its customers in North America.

123 different countries is seven more compared to the 2017 report. The new countries include Rwanda, Namibia, Senegal, Sierra Leone, and Swaziland, Zambia. Seeing contributions from more African countries is certainly a highlight.

How much of the work is sponsored?

Issue credits can be marked as "volunteer" and "sponsored" simultaneously (shown in jamadar's screenshot near the top of this post). This could be the case when a contributor does the minimum required work to satisfy the customer's need, in addition to using their spare time to add extra functionality.

While Drupal started out as a 100% volunteer-driven project, today the majority of the code on Drupal.org is sponsored by organizations. Only 12% of the commit credits that we examined in 2017-2018 were "purely volunteer" credits (6,007 credits), in stark contrast to the 49% that were "purely sponsored". In other words, there were four times as many "purely sponsored" credits as "purely volunteer" credits.

Contributions by volunteer vs sponsored

A few comparisons between the 2017-2018 and the 2016-2017 data:

  • The credit system is being used more frequently. In total, we captured 49,793 issue credits across all 24,447 issues in the 2017-2018 period. This marks a 17% increase from the 42,449 issue credits recorded in the previous year. Between July 1, 2016 and June 30, 2017, 28% of all credits had no attribution while in the period between July 1, 2017 to June 30, 2018, only 25% of credits lacked attribution. More people have become aware of the credit system, the attribution options, and their benefits.
  • Sponsored credits are growing faster than volunteer credits. Both "purely volunteer" and "purely sponsored" credits grew, but "purely sponsored" credits grew faster. There are two reasons why this could be the case: (1) more contributions are sponsored and (2) organizations are more likely to use the credit system compared to volunteers.

No data is perfect, but it feels safe to conclude that most of the work on Drupal is sponsored. At the same time, the data shows that volunteer contribution remains very important to Drupal. Maybe most importantly, while the number of volunteers and sponsors has grown year over year in absolute terms, sponsored contributions appear to be growing faster than volunteer contributions. This is consistent with how open source projects grow and scale.

Who is sponsoring the work?

Now that we've established a majority of contributions to Drupal are sponsored, we want to study which organizations contribute to Drupal. While 1,002 different organizations contributed to Drupal, approximately 50% of them received four credits or less. The top 30 organizations (roughly the top 3%) account for approximately 48% of the total credits, which implies that the top 30 companies play a crucial role in the health of the Drupal project. The graph below shows the top 30 organizations and the number of credits they received between July 1, 2017 and June 30, 2018:

Top 30 organizations contributing to Drupal

The top 30 contributing organizations based on the number of Drupal.org commit credits.

While not immediately obvious from the graph above, a variety of different types of companies are active in Drupal's ecosystem:

Category Description Traditional Drupal businesses Small-to-medium-sized professional services companies that primarily make money using Drupal. They typically employ fewer than 100 employees, and because they specialize in Drupal, many of these professional services companies contribute frequently and are a huge part of our community. Examples are Chapter Three and Lullabot (both shown on graph). Digital marketing agencies Larger full-service agencies that have marketing-led practices using a variety of tools, typically including Drupal, Adobe Experience Manager, Sitecore, WordPress, etc. They tend to be larger, with the larger agencies employing thousands of people. Examples are Wunderman and Mirum. System integrators Larger companies that specialize in bringing together different technologies into one solution. Example system agencies are Accenture, TATA Consultancy Services, Capgemini and CI&T (shown on graph). Technology and infrastructure companies Examples are Acquia (shown on graph), Lingotek, BlackMesh, Rackspace, Pantheon and Platform.sh. End-users Examples are Pfizer (shown on graph) or NBCUniversal.

A few observations:

  • Almost all of the sponsors in the top 30 are traditional Drupal businesses. Companies like MD Systems (12 employees), Valuebound (58 employees), Chapter Three (33 employees), Commerce Guys (13 employees) and PreviousNext (22 employees) are, despite their size, critical to Drupal's success.
  • Compared to these traditional Drupal businesses, Acquia has nearly 800 employees and at least ten full-time Drupal contributors. Acquia works to resolve some of the most complex issues on Drupal.org, many of which are not recognized by the credit system (e.g. release management, communication, sprint organizing, and project coordination). Acquia added several full-time contributors compared to last year, however, I believe that Acquia should contribute even more due to its comparative size.
  • No digital marketing agencies show up in the top 30, though some of them are starting to contribute. It's exciting that an increasing number of digital marketing agencies are delivering beautiful experiences using Drupal. As a community, we need to work to ensure that each of these firms are contributing back to the project with the same commitment that we see from firms like Commerce Guys, CI&T or Acro Media. Compared to last year, we have not made meaningful progress on growing contributions from digital marketing agencies. It would be interesting to see what would happen if more large organizations mandated contributions from their partners. Pfizer, for example, only works with agencies and vendors that contribute back to Drupal, and requires that its agency partners contribute to open source. If more organizations took this stance, it could have a big impact on the number of digital agencies that contribute to Drupal
  • The only system integrator in the top 30 is CI&T, which ranked 3rd with 959 credits. As far as system integrators are concerned, CI&T is a smaller player with approximately 2,500 employees. However, we do see various system integrators outside of the top 30, including Globant, Capgemini, Sapient and TATA Consultancy Services. Each of these system integrators reported 30 to 85 credits in the past year. The top contributor is TATA with 85 credits.
  • Infrastructure and software companies also play an important role in our community, yet only Acquia appears in the top 30. While Acquia has a professional services division, more than 75% of the contributions come from the product organization. Other infrastructure companies include Pantheon and Platform.sh, which are both venture-backed, platform-as-a-service companies that were born from the Drupal community. Pantheon has 6 credits and Platform.sh has 47 credits. Amazee Labs, a company that is building an infrastructure business, reported 40 credits. Compared to last year, Acquia and Rackspace have slightly more credits, while Pantheon, Platform.sh and Amazee contributed less. Lingotek, a vendor that offers cloud-based translation management software has 84 credits.
  • We also saw three end-users in the top 30 as corporate sponsors: Pfizer (491 credits, up from 251 credits the year before), Thunder (432 credits), and the German company, bio.logis (319 credits, up from 212 credits the year before). Other notable customers outside of the top 30, include Workday, Wolters Kluwer, Burda Media, YMCA and OpenY, CARD.com and NBCUniversal. We also saw contributions from many universities, including University of Colorado Boulder, University of Waterloo, Princeton University, University of Adelaide, University of Sydney, University of Edinburgh, McGill University and more.

Contributions by technology companies

We can conclude that technology and infrastructure companies, digital marketing agencies, system integrators and end-users are not making significant code contributions to Drupal.org today. How can we explain this disparity in comparison to the traditional Drupal businesses that contribute the most? We believe the biggest reasons are:

  1. Drupal's strategic importance. A variety of the traditional Drupal agencies almost entirely depend on Drupal to support their businesses. Given both their expertise and dependence on Drupal, they are most likely to look after Drupal's development and well-being. Contrast this with most of the digital marketing agencies and system integrators who work with a diversified portfolio of content management platforms. Their well-being is less dependent on Drupal's success.
  2. The level of experience with Drupal and open source. Drupal aside, many organizations have little or no experience with open source, so it is important that we motivate and teach them to contribute.
  3. Legal reservations. We recognize that some organizations are not legally permitted to contribute, let alone attribute their customers. We hope that will change as open source continues to get adopted.
  4. Tools barriers. Drupal contribution still involves a patch-based workflow on Drupal.org's unique issue queue system. This presents a fairly steep learning curve to most developers, who primarily work with more modern and common tools such as GitHub. We hope to lower some of these barriers through our collaboration with GitLab.
  5. Process barriers. Getting code changes accepted into a Drupal project — especially Drupal core — is hard work. Peer reviews, gates such as automated testing and documentation, required sign-offs from maintainers and committers, knowledge of best practices and other community norms are a few of the challenges a contributor must face to get code accepted into Drupal. Collaborating with thousands of people on a project as large and widely-used as Drupal requires such processes, but new contributors often don't know that these processes exist, or don't understand why they exist.

We should do more to entice contribution

Drupal is used by more than one million websites. Everyone who uses Drupal benefits from work that thousands of other individuals and organizations have contributed. Drupal is great because it is continuously improved by a diverse community of contributors who are enthusiastic to give back.

However, the vast majority of the individuals and organizations behind these Drupal websites never participate in the development of the project. They might use the software as it is or don't feel the need to help drive its development. We have to provide more incentive for these individuals and organizations to contribute back to the project.

Consequently, this data shows that the Drupal community can do more to entice companies to contribute code to Drupal.org. The Drupal community has a long tradition of encouraging organizations to share code rather than keep it behind firewalls. While the spirit of the Drupal project cannot be reduced to any single ideology — not every organization can or will share their code — we would like to see organizations continue to prioritize collaboration over individual ownership.

We understand and respect that some can give more than others and that some might not be able to give back at all. Our goal is not to foster an environment that demands what and how others should give back. Our aim is not to criticize those who do not contribute, but rather to help foster an environment worthy of contribution. This is clearly laid out in Drupal's Values and Principles.

Given the vast amount of Drupal users, we believe continuing to encourage organizations and end-users to contribute is still a big opportunity. From my own conversations, it's clear that organizations still need need education, training and help. They ask questions like: "Where can we contribute?", "How can we convince our legal department?", and more.

There are substantial benefits and business drivers for organizations that contribute: (1) it improves their ability to sell and win deals and (2) it improves their ability to hire. Companies that contribute to Drupal tend to promote their contributions in RFPs and sales pitches. Contributing to Drupal also results in being recognized as a great place to work for Drupal experts.

What projects have sponsors?

To understand where the organizations sponsoring Drupal put their money, I've listed the top 20 most sponsored projects:

Rank Project name Issues 1 Drupal core 5919 2 Webform 905 3 Drupal Commerce 607 4 Varbase: The Ultimate Drupal 8 CMS Starter Kit (Bootstrap Ready) 551 5 Commerce Point of Sale (POS) 324 6 Views 318 7 Commerce Migrate 307 8 JSON API 304 9 Paragraphs 272 10 Open Social 222 11 Search API Solr Search 212 12 Drupal Connector for Janrain Identity Cloud 197 13 Drupal.org security advisory coverage applications 189 14 Facets 171 15 Open Y 162 16 Metatag 162 17 Web Page Archive 154 18 Drupal core - JavaScript Modernization Initiative 145 19 Thunder 144 20 XML sitemap 120

Who is sponsoring the top 30 contributors?

Rank Username Issues Volunteer Sponsored Not specified Sponsors 1 RenatoG 851 0% 100% 0% CI&T (850), Johnson & Johnson (23) 2 RajabNatshah 745 14% 100% 0% Vardot (653), Webship (90) 3 jrockowitz 700 94% 97% 1% The Big Blue House (680), Memorial Sloan Kettering Cancer Center (7), Rosewood Marketing (2), Kennesaw State University (1) 4 adriancid 529 99% 19% 0% Ville de Montréal (98) 5 bojanz 515 0% 98% 2% Commerce Guys (503), Torchbox (17), Adapt (6), Acro Media (4), Bluespark (1) 6 Berdir 432 0% 92% 8% MD Systems (396), Translations.com (10), Acquia (2) 7 alexpott 414 13% 84% 10% Chapter Three (123), Thunder (120), Acro Media (103) 8 mglaman 414 5% 96% 1% Commerce Guys (393), Impactiv (17), Circle Web Foundry (16), Rosewood Marketing (14), LivePerson (13), Bluespark (4), Acro Media (4), Gaggle.net (3), Thinkbean (2), Matsmart (2) 9 Wim Leers 395 8% 94% 0% Acquia (371) 10 larowlan 360 13% 97% 1% PreviousNext (350), University of Technology, Sydney (24), Charles Darwin University (10), Australian Competition and Consumer Commission (ACCC) (1), Department of Justice & Regulation, Victoria (1) 11 DamienMcKenna 353 1% 95% 5% Mediacurrent (334) 12 dawehner 340 48% 86% 4% Chapter Three (279), Torchbox (10), Drupal Association (5), Tag1 Consulting (3), Acquia (2), TES Global (1) 13 catch 339 1% 97% 3% Third and Grove (320), Tag1 Consulting (8) 14 heddn 327 2% 99% 1% MTech (325) 15 xjm 303 0% 97% 3% Acquia (293) 16 pifagor 284 32% 99% 1% GOLEMS GABB (423), Drupal Ukraine Community (73) 17 quietone 261 48% 55% 5% Acro Media (143) 18 borisson_ 255 93% 55% 3% Dazzle (136), Intracto digital agency (1), Acquia (1), DUG BE vzw (Drupal User Group Belgium) (1) 19 adci_contributor 255 0% 100% 0% ADCI Solutions (255) 20 volkswagenchick 254 1% 100% 0% Hook 42 (253) 21 drunken monkey 231 91% 22% 0% DBC (24), Vizala (20), Sunlime Web Innovations GmbH (4), Wunder Group (1), epiqo (1), Zebralog (1) 22 amateescu 225 3% 95% 3% Pfizer (211), Drupal Association (1), Chapter Three (1) 23 joachim 199 56% 44% 19% Torchbox (88) 24 mkalkbrenner 195 0% 99% 1% bio.logis (193), OSCE: Organization for Security and Co-operation in Europe (119) 25 chr.fritsch 185 0% 99% 1% Thunder (183) 26 gaurav.kapoor 178 0% 81% 19% OpenSense Labs (144), DrupalFit (55) 27 phenaproxima 177 0% 99% 1% Acquia (176) 28 mikeytown2 173 0% 0% 100% 29 joelpittet 170 28% 74% 16% The University of British Columbia (125) 30 timmillwood 169 1% 100% 0% Pfizer (169), Appnovation (163), Millwood Online (6)

We observe that the top 30 contributors are sponsored by 58 organizations. This kind of diversity is aligned with our desire to make sure that Drupal is not controlled by a single organization. These top contributors and organizations are from many different parts of the world, and work with customers large and small. Nonetheless, we will continue to benefit from an increased distribution of contribution.

Limitations of the credit system and the data

While the benefits are evident, it is important to note a few of the limitations in Drupal.org's current credit system:

  • Contributing to issues on Drupal.org is not the only way to contribute. Other activities, such as sponsoring events, promoting Drupal, and providing help and mentorship are also important to the long-term health of the Drupal project. Many of these activities are not currently captured by the credit system. For this post, we chose to only look at code contributions.
  • We acknowledge that parts of Drupal are developed on GitHub and therefore aren't fully credited on Drupal.org. The actual number of contributions and contributors could be significantly higher than what we report. The Drupal Association is working to integrate GitLab with Drupal.org. GitLab will provide support for "merge requests", which means contributing to Drupal will feel more familiar to the broader audience of open source contributors who learned their skills in the post-patch era. Some of GitLab's tools, such as inline editing and web-based code review, will also lower the barrier to contribution, and should help us grow both the number of contributions and contributors on Drupal.org.
  • Even when development is done on Drupal.org, the credit system is not used consistently. As using the credit system is optional, a lot of code committed on Drupal.org has no or incomplete contribution credits.
  • Not all code credits are the same. We currently don't have a way to account for the complexity and quality of contributions; one person might have worked several weeks for just one credit, while another person might receive a credit for ten minutes of work. In the future, we should consider issuing credit data in conjunction with issue priority, patch size, etc. This could help incentivize people to work on larger and more important problems and save coding standards improvements for new contributor sprints. Implementing a scoring system that ranks the complexity of an issue would also allow us to develop more accurate reports of contributed work.

Like Drupal itself, the Drupal.org credit system needs to continue to evolve. Ultimately, the credit system will only be useful when the community uses it, understands its shortcomings, and suggests constructive improvements.

Conclusion

Our data confirms that Drupal is a vibrant community full of contributors who are constantly evolving and improving the software. While we have amazing geographic diversity, we still need greater gender diversity, in addition to better representation across various demographic groups. Our analysis of the Drupal.org credit data concludes that most contributions to Drupal are sponsored. At the same time, the data shows that volunteer contribution remains very important to Drupal.

As a community, we need to understand that a healthy open source ecosystem includes more than the traditional Drupal businesses that contribute the most. We still don't see a lot of contribution from the larger digital marketing agencies, system integrators, technology companies, or end-users of Drupal — we believe that might come as these organizations build out their Drupal practices and Drupal becomes more strategic for them.

To grow and sustain Drupal, we should support those that contribute to Drupal and find ways to get those that are not contributing involved in our community. We invite you to help us continue to strengthen our ecosystem.

Sep 06 2018
Sep 06

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Last night, we shipped Drupal 8.6.0! I firmly believe this is the most significant Drupal 8 release to date. It is significant because we made a lot of progress on all twelve of Drupal 8 core's strategic initiatives. As a result, Drupal 8.6 delivers a large number of improvements for content authors, evaluators, site builders and developers.

What is new for content authors?

For content authors, Drupal 8.6 adds support for "remote media types". This means you can now easily embed YouTube or Vimeo videos in your content.

The Media Library in Drupal 8.6

The Media Library in Drupal 8.6

Content authors want Drupal to be easy to use. We made incredible progress on a variety of features that will help to achieve that: we've delivered an experimental media library, added the Workspaces module as experimental, providing sophisticated content staging capabilities, and made great strides on the upcoming Layout Builder. The Layout Builder is shaping up to be a very powerful tool that solves a lot of authoring challenges, and is something many are looking forward to.

The Workspaces module in Drupal 8.6

The Workspaces module in Drupal 8.6

Each initiative related to content authoring is making disciplined and steady progress. These features not only solve for the most requested authoring improvements, but provide a solid foundation on which we can continue to innovate. This means we can provide better compatibility and upgradability for contributed modules.

Top requests for content authors

The top 10 requested features for content creators according to the 2016 State of Drupal survey.

What is new for evaluators?

Evaluators want an out-of-the-box experience that allows them to install and test drive Drupal in minutes. With Drupal 8.6, we have finally delivered on this need.

Prior to Drupal 8.6, downloading and installing Drupal was a complex and lengthy process that ended with an underwhelming "blank slate".

Now, you can install Drupal with the new "Umami demo profile". The Umami demo profile showcases some of Drupal's most powerful capabilities by providing a beautiful website filled with content right out of the box. A demo profile will not only help to onboard new users, but it can also be used by Drupal professionals and digital agencies to showcase Drupal to potential customers.

The experimental layout builder in Drupal 8.6 with the Umami demo profile

The new Umami demo profile together with the Layout Builder.

In addition to a new installation profile, we added a "quick-start" command that allows you to launch a Drupal site in one command using only one dependency, PHP. If you want to try Drupal, you no longer have to setup a webserver, a database, containers, etc.

Last but not least, the download experience and evaluator documentation on Drupal.org has been vastly improved.

With Drupal 8.6, you can download and install a fully functional Drupal demo application in less than two minutes. That is something to be very excited about.

What is new for developers?

You can now upgrade a single-language Drupal 6 or Drupal 7 site to Drupal 8 using the built-in user interface. While we saw good progress on multilingual migrations, they will remain experimental as we work on the final gaps.

I recently wrote about our progress in making Drupal an API-first platform, including an overview of REST improvements in Drupal 8.6, an update on JSON API, and the reasons why JSON API didn't make it into this release. I'm looking forward to JSON API being added in Drupal 8.7. Other decoupled efforts, including a React-based administration application and GraphQL support are still under heavy development, but making rapid progress.

We also converted almost all of our tests from SimpleTest to PHPUnit; and we've added Nightwatch.js and Prettier for JavaScript developers. While Drupal 8 has extensive back-end test coverage, using PHPUnit and Nightwatch.js provides a more modern platform that will make Drupal more familiar to PHP and JavaScript developers.

Drupal 8 continues to hit its stride

These are just some of the highlights that I'm most excited about. If you'd like to read more about Drupal 8.6.0, check out the official release announcement and important update information from the release notes. The next couple of months, I will write up more detailed progress reports on initiatives that I didn't touch upon in this blog post.

In my Drupal 8.5.0 announcement, I talked about how Drupal is hitting its stride, consistently delivering improvements and new features:

In future releases, we plan to add a media library, support for remote media types like YouTube videos, support for content staging, a layout builder, JSON API support, GraphQL support, a React-based administration application and a better out-of-the-box experience for evaluators.

As you can see from this blog post, Drupal 8.6 delivered on a number of these plans and made meaningful progress on many others.

In future releases we plan to:

  • Stabilize more of the features targeting content authors
  • Add JSON API, allowing developers to more easily and rapidly create decoupled applications
  • Provide stable multilingual migrations
  • Make big improvements for developers with Composer and configuration management changes
  • Continually improve the evaluator experience
  • Iterate towards an entirely new decoupled administrative experience
  • ... and more

Releases like Drupal 8.6.0 only happen with the help of hundreds of contributors and organizations. Thank you to everyone that contributed to this release. Whether you filed issues, wrote code, tested patches, funded a contributor, tested pre-release versions, or cheered for the team from the sidelines, you made this release happen. Thank you!

Sep 05 2018
Sep 05

What's new in Drupal 8.6.0?

The most significant update to Drupal 8 in its history, this new release includes two new easy ways to install Drupal, a cooking magazine demo, oEmbed media support, stable upgrades for monolingual Drupal sites, a new media library and workspaces experimental modules, significant layout improvements, various REST fixes and testing improvements.

Download Drupal 8.6.0

oEmbed for media and a new experimental media library

New in this version is built-in stable oEmbed support for media. A new Remote video media type is shipped preconfigured to support embedding YouTube and Vimeo videos.

Media library with multiple images

In a new experimental module, you can now browse existing media and add new media using an integrated widget. Adding multiple media at once is also supported. The media library is based on Views and may be customized.

Umami food magazine demo included

Drupal 8.6.0 offers a new demo profile and theme in the installer. A beautiful, modern demonstration of Drupal's capabilities using an imaginary cooking site named Umami. Drupal's data modeling, listing, page composition and content moderation capabilities are showcased. Sample author and magazine editor users are created to experience different aspects of using Drupal's content management interface. People are invited to tinker with the demo and learn general Drupal concepts and practices.

Umami food magazine demo

The demo profile and theme should not be used on (or as a basis of) actual production or development sites since no backwards compatibility or upgrade paths are provided. Future versions of Umami will demonstrate multilingual capabilities, and as they become stable: media handling, layouts and so on.

New experimental workspaces module

The existing content moderation functionality is great when you need to move individual pieces of content through an editing and approval workflow. For example, use states like Draft, Archived, and Published, and specify which roles have the ability to move content between states.

Workspaces user interface in Drupal 8.6

When "packages" of content (maybe a few, a few hundred or even a few thousand items) need to be reviewed and deployed at once, you'll find the new experimental Workspaces module invaluable. Define multiple workspaces, make changes and deploy between them with an intuitive user interface.

Much improved experimental layout capabilities

The experimental Layout Builder module now supports per-display customizations (e.g. full mode vs. search result), so instead of defining the order of fields stacked on top of each other there, you can define layouts with dynamic sections. It is also possible to create one-off blocks now for use in a specific layout, which will not show up in the global block list. This is useful for things like a promotion only visible within a single landing page.

Stable upgrades for monolingual sites, multilingual improved

Migration support has been steadily improving. This release sees both Migrate Drupal (migrations from previous major Drupal versions) as well as Migrate Drupal UI (upgrade user interface) modules go stable. This means that, if you have a monolingual Drupal 6 or 7 site, you can now use a supported and built-in user interface to migrate your site to Drupal 8.

Multilingual migrations are still experimental and now wrapped in the Migrate Drupal Multilingual module. Significant improvements in this area include support for Drupal 7 Entity Translation migrations for nodes with Title module support. Further testing and implementation of the missing pieces is still required for this module to become stable.

We also saw lots of improvements in migrations for contributed modules in the past six months. Many of the most popular modules, Paragraphs, Field Collections, Multifield, Media, Workflow, and more all have some level of support.

Two new easy ways to install Drupal

Drupal depends on various external tools. To make it significantly easier to start a quick evaluator or development environment, a quick-start command is now included that only needs PHP on the system. Using the built-in webserver in PHP and the SQLite database, it sets up Drupal quickly and opens a browser ready to use:

$ curl -sSO https://www.drupal.org/download-latest/zip
$ unzip drupal-x.y.z.zip
$ php drupal-x.y.z/core/scripts/drupal quick-start
18/18 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓]
Congratulations, you installed Drupal!
Username: admin
Password: NlH5cmfEzsbS3DSs
Drupal development server started: <http://127.0.0.1:8888>
This server is not meant for production use.
One time login url: <http://127.0.0.1:8888/user/reset/1/1525772031/pyK4gRFkQSGGKJk0GhZRucybqROZ2zvV85JwQiD3bFY/login>
Press Ctrl-C to quit the Drupal development server.

The installer now also recognizes existing configuration and provides an option to install from that configuration. This allows to rebuild a site (without its content) locally for development.

MySQL 8 now supported

MySQL 8 includes several performance improvements and language/collation support changes. Drupal 8.6.0 supports MySQL 8. There are no plans to change database version requirements at this time.

Testing and REST improvements

The process of porting all tests from our own Simpletest implementation to PHPUnit is almost done. We have a total of 3,215 tests based on PHPUnit while 68 remain based on Simpletest in this release. The JavaScript testing system is also greatly improved by added support for Nightwatch.js, which supports writing automated tests in JavaScript itself. It is also now possible to upload files in REST requests among many other important bug fixes and improvements.

What does this mean for me?

Drupal 8 site owners

Update to 8.6.0 to continue receiving bug and security fixes. The next bugfix release (8.6.1) is scheduled for October 3 2018.

Updating your site from 8.5.6 to 8.6.0 with update.php is exactly the same as updating from 8.5.5 to 8.5.6. Drupal 8.6.0 also has updates to several dependencies. Modules, themes, and translations may need updates for these and other changes in this minor release, so test the update carefully before updating your production site.

Note that Drupal 8 will require PHP 7 starting in March 2019. If your site is hosted on PHP 5.5 or 5.6, you should begin planning to upgrade (and consider upgrading to PHP 7.2 for best results). See the Drupal core announcement about the PHP 5 end-of-life for more information.

Drupal 6 and 7 site owners

Drupal 7 is still fully supported and will continue to receive bug and security fixes throughout all minor releases of Drupal 8. Drupal 6 is no longer supported. You can now use the stable migration path for monolingual sites with the built-in upgrade user interface. For multilingual sites, please keep testing and reporting any issues you may find.

Translation, module, and theme contributors

Minor releases like Drupal 8.6.0 include backwards-compatible API additions for developers as well as new features. Read the 8.6.0 release notes for more details on the improvements for developers in this release.

Since minor releases are backwards-compatible, modules, themes, and translations that supported Drupal 8.5.x and earlier will be compatible with 8.6.x as well. However, the new version does include some changes to strings, user interfaces, internal APIs and API deprecations. This means that some small updates may be required for your translations, modules, and themes.

Thank you to everyone who contributed to Drupal 8.6!

Jul 31 2018
Jul 31
Posted by Dries on 31 July 2018

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Digital backpack

I recently heard a heart-warming story from the University of California, Davis. Last month, UC Davis used Drupal to launch Article 26 Backpack, a platform that helps Syrian Refugees document and share their educational credentials.

Over the course of the Syrian civil war, more than 12 million civilians have been displaced. Hundreds of thousands of these refugees are students, who now have to overcome the obstacle of re-entering the workforce or pursuing educational degrees away from home.

Article 26 Backpack addresses this challenge by offering refugees a secure way to share their educational credentials with admissions offices, scholarship agencies, and potentials employers. The program also includes face-to-face counseling to provide participants with academic advisory and career development.

The UC Davis team launched their Drupal 8 application for Article 26 Backpack in four months. On the site, students can securely store their educational data, such as diplomas, transcripts and resumes. The next phase of the project will be to leverage Drupal’s multilingual capabilities to offer the site in Arabic as well.

This is a great example of how organizations are using Drupal to prioritize impact. It’s always inspiring to hear stories of how Drupal is changing lives for the better. Thank you to the UC Davis team for sharing their story, and continue the good work!

Jul 19 2018
Jul 19

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

It's been 12 months since my last progress report on Drupal core's API-first initiative. Over the past year, we've made a lot of important progress, so I wanted to provide another update.

Two and a half years ago, we shipped Drupal 8.0 with a built-in REST API. It marked the start of Drupal's evolution to an API-first platform. Since then, each of the five new releases of Drupal 8 introduced significant web service API improvements.

While I was an early advocate for adding web services to Drupal 8 five years ago, I'm even more certain about it today. Important market trends endorse this strategy, including integration with other technology solutions, the proliferation of new devices and digital channels, the growing adoption of JavaScript frameworks, and more.

In fact, I believe that this functionality is so crucial to the success of Drupal, that for several years now, Acquia has sponsored one or more full-time software developers to contribute to Drupal's web service APIs, in addition to funding different community contributors. Today, two Acquia developers work on Drupal web service APIs full time.

Drupal core's REST API

While Drupal 8.0 shipped with a basic REST API, the community has worked hard to improve its capabilities, robustness and test coverage. Drupal 8.5 shipped 5 months ago and included new REST API features and significant improvements. Drupal 8.6 will ship in September with a new batch of improvements.

One Drupal 8.6 improvement is the move of the API-first code to the individual modules, instead of the REST module providing it on their behalf. This might not seem like a significant change, but it is. In the long term, all Drupal modules should ship with web service APIs rather than depending on a central API module to provide their APIs — that forces them to consider the impact on REST API clients when making changes.

Another improvement we've made to the REST API in Drupal 8.6 is support for file uploads. If you want to understand how much thought and care went into REST support for file uploads, check out Wim Leers' blog post: API-first Drupal: file uploads!. It's hard work to make file uploads secure, support large files, optimize for performance, and provide a good developer experience.

JSON API

Adopting the JSON API module into core is important because JSON API is increasingly common in the JavaScript community.

We had originally planned to add JSON API to Drupal 8.3, which didn't happen. When that plan was originally conceived, we were only beginning to discover the extent to which Drupal's Routing, Entity, Field and Typed Data subsystems were insufficiently prepared for an API-first world. It's taken until the end of 2017 to prepare and solidify those foundational subsystems.

The same shortcomings that prevented the REST API to mature also manifested themselves in JSON API, GraphQL and other API-first modules. Properly solving them at the root rather than adding workarounds takes time. However, this approach will make for a stronger API-first ecosystem and increasingly faster progress!

Despite the delay, the JSON API team has been making incredible strides. In just the last six months, they have released 15 versions of their module. They have delivered improvements at a breathtaking pace, including comprehensive test coverage, better compliance with the JSON API specification, and numerous stability improvements.

The Drupal community has been eager for these improvements, and the usage of the JSON API module has grown 50% in the first half of 2018. The fact that module usage has increased while the total number of open issues has gone down is proof that the JSON API module has become stable and mature.

As excited as I am about this growth in adoption, the rapid pace of development, and the maturity of the JSON API module, we have decided not to add JSON API as an experimental module to Drupal 8.6. Instead, we plan to commit it to Drupal core early in the Drupal 8.7 development cycle and ship it as stable in Drupal 8.7.

GraphQL

For more than two years I've advocated that we consider adding GraphQL to Drupal core.

While core committers and core contributors haven't made GraphQL a priority yet, a lot of great progress has been made on the contributed GraphQL module, which has been getting closer to its first stable release. Despite not having a stable release, its adoption has grown an impressive 200% in the first six months of 2018 (though its usage is still measured in the hundreds of sites rather than thousands).

I'm also excited that the GraphQL specification has finally seen a new edition that is no longer encumbered by licensing concerns. This is great news for the Open Source community, and can only benefit GraphQL's adoption.

Admittedly, I don't know yet if the GraphQL module maintainers are on board with my recommendation to add GraphQL to core. We purposely postponed these conversations until we stabilized the REST API and added JSON API support. I'd still love to see the GraphQL module added to a future release of Drupal 8. Regardless of what we decide, GraphQL is an important component to an API-first Drupal, and I'm excited about its progress.

OAuth 2.0

A web services API update would not be complete without touching on the topic of authentication. Last year, I explained how the OAuth 2.0 module would be another logical addition to Drupal core.

Since then, the OAuth 2.0 module was revised to exclude its own OAuth 2.0 implementation, and to adopt The PHP League's OAuth 2.0 Server instead. That implementation is widely used, with over 5 million installs. Instead of having a separate Drupal-specific implementation that we have to maintain, we can leverage a de facto standard implementation maintained by others.

API-first ecosystem

While I've personally been most focused on the REST API and JSON API work, with GraphQL a close second, it's also encouraging to see that many other API-first modules are being developed:

  • OpenAPI, for standards-based API documentation, now at beta 1
  • JSON API Extras, for shaping JSON API to your site's specific needs (aliasing fields, removing fields, etc)
  • JSON-RPC, for help with executing common Drupal site administration actions, for example clearing the cache
  • … and many more

Conclusion

Hopefully, you are as excited for the upcoming release of Drupal 8.6 as I am, and all of the web service improvements that it will bring. I am very thankful for all of the contributions that have been made in our continued efforts to make Drupal API-first, and for the incredible momentum these projects and initiatives have achieved.

Special thanks to Wim Leers (Acquia) and Gabe Sullice (Acquia) for contributions to this blog post and to Mark Winberry (Acquia) and Jeff Beeman (Acquia) for their feedback during the writing process.

Jul 13 2018
Jul 13

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

If you've ever watched a Drupal Camp video to learn a new Drupal skill, technique or hack, you most likely have Kevin Thull to thank. To date, Kevin has traveled to more than 30 Drupal Camps, recorded more than 1,000 presentations, and has shared them all on YouTube for thousands of people to watch. By recording and posting hundreds of Drupal Camp presentations online, Kevin has has spread knowledge, awareness and a broader understanding of the Drupal project.

I recently attended a conference in Chicago, Kevin's hometown. I had the chance to meet with him, and to learn more about the evolution of his Drupal contributions. I was struck by his story, and decided to write it up on my blog, as I believe it could inspire others around the world.

Kevin began recording sessions during the first community events he helped organize: DrupalCamp Fox Valley in 2013 and MidCamp in 2014. At first, recording and publishing Drupal Camp sessions was an arduous process; Kevin had to oversee dozens of laptops, converters, splitters, camcorders, and trips to Fedex.

After these initial attempts, Kevin sought a different approach for recording sessions. He ended up developing a recording kit, which is a bundle of the equipment and technology needed to record a presentation. After researching various options, he discovered a lightweight, low cost and foolproof solution. Kevin continued to improve this process after he tweeted that if you sponsored his travel, he would record Drupal Camp sessions. It's no surprise that numerous camps took Kevin up on his offer. With more road experience, Kevin has consolidated the recording kits to include just a screen recorder, audio recorder and corresponding cables. With this approach, the kit records a compressed mp4 file that can be uploaded directly to YouTube. In fact, Kevin often finishes uploading all presentation videos to YouTube before the camp is over!

Kevin Thull recording kit

This is one of Kevin Thull's recording kits used to record hundreds of Drupal presentations around the world. Each kit runs at about $450 on Amazon.

Most recently, Kevin has been buying and building more recording kits thanks to financial contributions from various Drupal Camps. He has started to send recording kits and documentation around the world for local camp organizers to use. Not only has Kevin recorded hundreds of sessions himself, he is now sharing his expertise and teaching others how to record and share sessions.

What is exciting about Kevin's contribution is that it reinforces what originally attracted him to Drupal. Kevin ultimately chose to work with Drupal after watching online video tutorials and listening to podcasts created by the community. Today, a majority of people prefer to learn development through video tutorials. I can only imagine how many people have joined and started to contribute to Drupal after they have watched one of the many videos that Kevin has helped to publish.

Kevin's story is a great example of how everyone in the Drupal community has something to contribute, and how contributing back to the Drupal project is not exclusive to code.

This year, the Drupal community celebrated Kevin by honoring him with the 2018 Aaron Winborn Award. The Aaron Winborn award is presented annually to an individual who demonstrates personal integrity, kindness, and above-and-beyond commitment to the Drupal community. It's named after a long-time Drupal contributor Aaron Winborn, who lost his battle with Amyotrophic lateral sclerosis (ALS) in early 2015. Congratulations Kevin, and thank you for your incredible contribution to the Drupal community!

Jul 09 2018
Jul 09

During my DrupalCon Nashville keynote, I shared a brief video of Mike Lamb, the Senior Director of Architecture, Engineering & Development at Pfizer. Today, I wanted to share an extended version of my interview with Mike, where he explains why the development team at Pfizer has ingrained Open Source contribution into the way they work.

[embedded content]

Mike had some really interesting and important things to share, including:

  1. Why Pfizer has chosen to standardize all of its sites on Drupal (from 0:00 to 03:19). Proprietary software isn't a match.
  2. Why Pfizer only works with agencies and vendors that contribute back to Drupal (from 03:19 to 06:25). Yes, you read that correctly; Pfizer requires that its agency partners contribute to Open Source!
  3. Why Pfizer doesn't fork Drupal modules (from 06:25 to 07:27). It's all about security.
  4. Why Pfizer decided to contribute to the Drupal 8's Workflow Initiative, and what they have learned from working with the Drupal community (from 07:27 to 10:06).
  5. How to convince a large organization (like Pfizer) to contribute back to Drupal (from 10:06 to 12:07).

Between Pfizer's direct contributions to Drupal (e.g. the Drupal 8 Workflow Initiative) and the mandate for its agency partners to contribute code back to Drupal, Pfizer's impact on the Drupal community is invaluable. It's measured in the millions of dollars per year. Just imagine what would happen to Drupal if ten other large organizations adopted Pfizer's contribution models?

Most organizations use Open Source, and don't think twice about it. However, we're starting to see more and more organizations not just use Open Source, but actively contribute to it. Open source offers organizations a completely different way of working, and fosters an innovation model that is not possible with proprietary solutions. Pfizer is a leading example of how organizations are starting to challenge the prevailing model and benefit from contributing to Open Source. Thanks for changing the status quo, Mike!

Jun 29 2018
Jun 29

Drupal is no longer the Drupal you used to know

Today, I gave a keynote presentation at the 10th annual Design 4 Drupal conference at MIT. I talked about the past, present and future of JavaScript, and how this evolution reinforces Drupal's commitment to be API-first, not API-only. I also included behind-the-scene insights into the Drupal community's administration UI and JavaScript modernization initiative, and why this approach presents an exciting future for JavaScript in Drupal.

If you are interested in viewing my keynote, you can download a copy of my slides (256 MB).

Thank you to Design 4 Drupal for having me and happy 10th anniversary!

Jun 27 2018
Jun 27

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

The Drupal community has done an amazing job organizing thousands of developers around the world. We've built collaboration tools and engineering processes to streamline how our community of developers work together to collectively build Drupal. This collaboration has led to amazing results. Today, more than 1 in 40 of the top one million websites use Drupal. It's inspiring to see how many organizations depend on Drupal to deliver their missions.

What is equally incredible is that historically, we haven't collaborated around the marketing of Drupal. Different organizations have marketed Drupal in their own way without central coordination or collaboration.

In my DrupalCon Nashville keynote, I shared that it's time to make a serious and focused effort to amplify Drupal success stories in the marketplace. Imagine what could happen if we enabled hundreds of marketers to collaborate on the promotion of Drupal, much like we have enabled thousands of developers to collaborate on the development of Drupal.

Accelerating Drupal adoption with business decision makers

To focus Drupal's marketing efforts, we launched the Promote Drupal Initiative. The goal of the Promote Drupal Initiative is to do what we do best: to work together to collectively grow Drupal. In this case, we want to collaborate to raise awareness with business and non-technical decision makers. We need to hone Drupal's strategic messaging, amplify success stories and public relation resources in the marketplace, provide agencies and community groups with sales and marketing tools, and improve the Drupal.org evaluator experience.

To make Promote Drupal sustainable, Rebecca Pilcher, Director of MarComm at the Drupal Association, will be leading the initiative. Rebecca will oversee volunteers with marketing and business skills that can help move these efforts forward.

Promote Drupal Fund: 75% to goal

At DrupalCon Nashville, we set a goal of fundraising $100,000 to support the Promote Drupal Initiative. These funds will help to secure staffing to backfill Rebecca's previous work (someone has to market DrupalCon!), produce critical marketing resources, and sponsor marketing sprints. The faster we reach this goal, the faster we can get to work.

I'm excited to announce that we have already reached 75% of our goal, thanks to many generous organizations and individuals around the world. I wanted to extend a big thank you to the following companies for contributing $1,000 or more to the Promote Drupal Initiative:

Thanks to many financial contributions, the Promote Drupal Initiative hit its $75k milestone!

If you can, please help us reach our total goal of $100,000! By raising a final $25,000, we can build a program that will introduce Drupal to an emerging audience of business decision makers. Together, we can make a big impact on Drupal.

Jun 20 2018
Jun 20

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

For the past two years, I've published the Who sponsors Drupal development report. The primary goal of the report is to share contribution data to encourage more individuals and organizations to contribute code to Drupal on Drupal.org. However, the report also highlights areas where our community can and should do better.

In 2017, the reported data showed that only 6 percent of recorded code contributions were made by contributors that identify as female. After a conversation in the Drupal Diversity & Inclusion Slack channel about the report, it became clear that many people were concerned about this discrepancy. Inspired by this conversation, Tara King started the Drupal Diversity and Inclusion Contribution Team to understand how the Drupal community could better include women and underrepresented groups to increase code and community contributions.

I recently spoke with Tara to learn more about the Drupal Diversity and Inclusion Contribution Team. I quickly discovered that Tara's leadership exemplifies various Drupal Values and Principles; especially Principle 3 (Foster a learning environment), Principle 5 (Everyone has something to contribute) and Principle 6 (Choose to lead). Inspired by Tara's work, I wanted to spotlight what the DDI Contribution Team has accomplished so far, in addition to how the team is looking to help grow diversity and inclusion in the future.

A mentorship program to help underrepresented groups

Supporting diversity and inclusion within Drupal is essential to the health and success of the project. The people who work on Drupal should reflect the diversity of people who use and work with the software. This includes building better representation across gender, race, sexuality, disability, economic status, nationality, faith, technical experience, and more. Unfortunately, underrepresented groups often lack community connections, time for contribution, resources or programs that foster inclusion, which introduce barriers to entry.

The mission of the Drupal Diversity & Inclusion Contribution Team is to increase contributions from underrepresented groups. To accomplish this goal, the DDI Contribution Team recruits team members from diverse backgrounds and underrepresented groups, and provides support and mentorship to help them contribute to Drupal. Each mentee is matched with a mentor in the Drupal community, who can provide expertise and advice on contribution goals and professional development. To date, the DDI Contribution Team supports over 20 active members.

What I loved most in my conversation with Tara is the various examples of growth she gave. For example, Angela McMahon is a full-time Drupal developer at Iowa State. Angela been working with her mentor, Caroline Boyden, on the External Link Module. Due to her participation with the DDI Contribution Team, Angela has now been credited on 4 fixed issues in the past year.

Improving the reporting around diversity and inclusion

In addition to mentoring, another primary area of focus of the DDI Contribution Team is to improve reporting surrounding diversity and inclusion. For example, in partnership with the Drupal Association and the Open Demographics Project, the DDI Contribution Team is working to implement best practices for data collection and privacy surrounding gender demographics. During the mentored code sprints at DrupalCon Nashville, the DDI Contribution Team built the Gender Field Module, which we hope to deploy on Drupal.org.

The development of the Gender Field Module is exciting, as it establishes a system to improve reporting on diversity demographics. I would love to use this data in future iterations of the 'Who sponsors Drupal development' report, because it would allow us to better measure progress on improving Drupal's diversity and inclusion against community goals.

One person can make a difference

What I love about the story of the DDI Contribution Team is that it demonstrates how one person can make a significant impact on the Drupal project. The DDI Contribution Team has grown from Tara's passion and curiosity to see what would happen if she challenged the status quo. Not only has Tara gotten to see one of her own community goals blossom, but she now also leads a team of mentors and mentees and is a co-maintainer of the Drupal 8 version of the Gender Field Module. Last but not least, she is building a great example for how other Open Source projects can increase contributions from underrepresented groups.

How you can get involved

If you are interested in getting involved with the DDI Contribution Team, there are a number of ways you can participate:

  • Support the DDI Contribution Team as a mentor, or consider recommending the program to prospective mentees. Join #ddi-contrib-team on Drupal Slack to meet the team and get started.
  • In an effort to deliberately recruit teams from spaces where people of diverse backgrounds collaborate, the DDI Contribution Team is looking to partner with Outreachy, an organization that provides paid internships for underrepresented groups to learn Free and Open Source Software and skills. If you would be interested in supporting a Drupal internship for an Outreachy candidate, reach out to Tara King to learn how you can make a financial contribution.
  • One of the long term goals of the DDI Contribution Team is to increase the number of underrepresented people in leadership positions, such as initiative lead, module maintainer, or core maintainer. If you know of open positions, consider understanding how you can work with the DDI Contribution Team to fulfill this goal.

I want to extend a special thanks to Tara King for sharing her story, and for making an important contribution to the Drupal project. Growing diversity and inclusion is something everyone in the Drupal community is responsible for, and I believe that everyone has something to contribute. Congratulations to the entire DDI Contribution Team.

Jun 15 2018
Jun 15

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Composer Initiative

At DrupalCon Nashville, we launched a strategic initiative to improve support for Composer in Drupal 8. To learn more, you can watch the recording of my DrupalCon Nashville keynote or read the Composer Initiative issue on Drupal.org.

While Composer isn't required when using Drupal core, many Drupal site builders use it as the preferred way of assembling websites (myself included). A growing number of contributed modules also require the use of Composer, which increases the need to make Composer easier to use with Drupal.

The first step of the Composer Initiative was to develop a plan to simplify Drupal's Composer experience. Since DrupalCon Nashville, Mixologic, Mile23, Bojanz, Webflo, and other Drupal community members have worked on this plan. I was excited to see that last week, they shared their proposal.

The first phase of the proposal is focused on a series of changes in the main Drupal core repository. The directory structure will remain the same, but it will include scripts, plugins, and embedded packages that enable the bundled Drupal product to be built from the core repository using Composer. This provides users who download Drupal from Drupal.org a clear path to manage their Drupal codebase with Composer if they choose.

I'm excited about this first step because it will establish a default, official approach for using Composer with Drupal. That makes using Composer more straightforward, less confusing, and could theoretically lower the bar for evaluators and newcomers who are familiar with other PHP frameworks. Making things easier for site builders is a very important goal; web development has become a difficult task, and removing complexity out of the process is crucial.

It's also worth noting that we are planning the Automatic Updates Initiative. We are exploring if an automated update system can be build on top of the Composer Initiative's work, and provide an abstraction layer for those that don't want to use Composer directly. I believe that could be truly game-changing for Drupal, as it would remove a great deal of complexity.

If you're interested in learning more about the Composer plan, or if you want to provide feedback on the proposal, I recommend you check out the Composer Initiative issue and comment 37 on that issue.

Implementing this plan will be a lot of work. How fast we execute these changes depends on how many people will help. There are a number of different third-party Composer related efforts, and my hope is to see many of them redirect their efforts to make Drupal's out-of-the-box Composer effort better. If you're interested in getting involved or sponsoring this work, let me know and I'd be happy to connect you with the right people!

Jun 06 2018
Jun 06

One of the most stressful experiences for students is the process of choosing the right university. Researching various colleges and universities can be overwhelming, especially when students don't have the luxury of visiting different campuses in person.

At Acquia Labs, we wanted to remove some of the complexity and stress from this process, by making campus tours more accessible through virtual reality. During my presentation at Acquia Engage Europe yesterday, I shared how organizations can use virtual reality to build cross-channel experiences. People that attended Acquia Engage Europe asked if they could have a copy of my video, so I decided to share it on my blog.

The demo video below features a high school student, Jordan, who is interested in learning more about Massachusetts State University (a fictional university). From the comfort of his couch, Jordan is able to take a virtual tour directly from the university's website. After placing his phone in a VR headset, Jordan can move around the university campus, explore buildings, and view program resources, videos, and pictures within the context of his tour.

[embedded content]

All of the content and media featured in the VR tour is stored in the Massachusetts State University's Drupal site. Site administrators can upload media and position hotspots directly from within Drupal backend. The React frontend pulls in information from Drupal using JSON API. In the video below, Chris Hamper (Acquia) further explains how the decoupled React VR application takes advantage of new functionality available in Drupal 8.

[embedded content]

It's exciting to see how Drupal's power and flexibility can be used beyond traditional web pages. If you are interesting in working with Acquia on virtual reality applications, don't hesitate to contact the Acquia Labs team.

Special thanks to Chris Hamper for building the virtual reality application, and thank you to Ash Heath, Preston So and Drew Robertson for producing the demo videos.

May 29 2018
May 29
Posted by Dries on 29 May 2018

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

In the beginning of the year I started doing some iOS development for my POSSE plan. As I was new to iOS development, I decided to teach myself by watching short, instructional videos. Different people learn in different ways, but for me, videos tutorials were the most effective way to learn.

Given that recent experience, I'm very excited to share that all of the task tutorials in the Drupal 8 User Guide are now accompanied by video tutorials. These videos are embedded directly into every user guide page on Drupal.org. You can see an example on the "Editing with the in-place editor" page.

These videos provide a great introduction to installing, administering, site building and maintaining the content of a Drupal-based website — all important skills for a new Drupalist to learn. Supplementing user guides with video tutorials is an important step towards improving our evaluator experience, as video can often convey a lot more than text.

Creating high-quality videos is hard and time-consuming work. Over the course of six months, the team at Drupalize.Me has generously contributed a total of 52 videos! I want to give a special shout-out to Joe Shindelar and the Drupalize.Me team for creating these videos and to Jennifer Hodgdon and Neil Drumm (Drupal Association) for helping to get each video posted on Drupal.org.

What a fantastic gift to the community!

May 25 2018
May 25
Posted by Dries on 25 May 2018

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Since the release of Drupal 8.0.0 in November 2015, the Drupal 8 core committers have been discussing when and how we'll release Drupal 9. Nat Catchpole, one of Drupal 8's core committers, shared some excellent thoughts about what goes into making that decision.

The driving factor in that discussion is security support for Drupal 8’s third party dependencies (e.g. Symfony, Twig, Guzzle, jQuery, etc). Our top priority is to ensure that all Drupal users are using supported versions of these components so that all Drupal sites remain secure.

In his blog, Nat uses Symfony as an example. The Symfony project announced that it will stop supporting Symfony 3 in November 2021, which means that Symfony 3 won't receive security updates after that date. Consequently, by November 2021, we need to prepare all Drupal sites to use Symfony 4 or later.

Nothing has been decided yet, but the current thinking is that we have to move Drupal to Symfony 4 or later, release that as Drupal 9, and allow enough time for everyone to upgrade to Drupal 9 by November 2021. Keep in mind that this is just looking at Symfony, and none of the other components.

This proposal builds on top of work we've already done on in the context of making Drupal upgrades easy, so upgrades from Drupal 8 to Drupal 9 should be smooth and much simpler than previous upgrades.

If you're interested in the topic, check out Nat's post. He goes in more detail about potential release timelines, including how this impacts our thinking about Drupal 7, Drupal 8 and even Drupal 10. It's a complicated topic, but the goal of Nat's post is to raise awareness and to solicit input from the broader community before we decide our official timeline and release dates on Drupal.org.

May 25 2018
May 25

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Last weekend, over 29 million people watched the Royal Wedding of Prince Harry and Meghan Markle. While there is a tremendous amount of excitement surrounding the newlyweds, I was personally excited to learn that the royal family's website is built with Drupal! Royal.uk is the official website of the British royal family, and is visited by an average of 12 million people each year. Check it out at https://www.royal.uk!

Royal uk

May 18 2018
May 18

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

As web applications have evolved from static pages to application-like experiences, end-users' expectations of websites have become increasingly demanding. JavaScript, partnered with effective user-experience design, enable the seamless, instantaneous interactions that users now expect.

The Drupal project anticipated this trend years ago and we have been investing heavily in making Drupal API-first ever since. As a result, more organizations are building decoupled applications served by Drupal. This approach allows organizations to use modern JavaScript frameworks, while still benefiting from Drupal's powerful content management capabilities, such as content modeling, content editing, content workflows, access rights and more.

While organizations use JavaScript frameworks to create visitor-facing experiences with Drupal as a backend, Drupal's own administration interface has not yet embraced a modern JavaScript framework. There is high demand for Drupal to provide a cutting-edge experience for its own users: the site's content creators and administrators.

At DrupalCon Vienna, we decided to start working on an alternative Drupal administrative UI using React. Sally Young, one of the initiative coordinators, recently posted a fantastic update on our progress since DrupalCon Vienna.

Next steps for Drupal's API-first and JavaScript work

While we made great progress improving Drupal's web services support and improving our JavaScript support, I wanted to use this blog post to compile an overview of some of our most important next steps:

1. Stabilize the JSON API module

JSON API is a widely-used specification for building web service APIs in JSON. We are working towards adding JSON API to Drupal core as it makes it easier for JavaScript developers to access the content and configuration managed in Drupal. There is a central plan issue that lists all of the blockers for getting JSON API into core (comprehensive test coverage, specification compliance, and more). We're working hard to get all of them out of the way!

2. Improve our JavaScript testing infrastructure

Drupal's testing infrastructure is excellent for testing PHP code, but until now, it was not optimized for testing JavaScript code. As we expect the amount of JavaScript code in Drupal's administrative interface to dramatically increase in the years to come, we have been working on improving our JavaScript testing infrastructure using Headless Chrome and Nightwatch.js. Nightwatch.js has already been committed for inclusion in Drupal 8.6, however some additional work remains to create a robust JavaScript-to-Drupal bridge. Completing this work is essential to ensure we do not introduce regressions, as we proceed with the other items in our roadmap.

3. Create designs for a React-based administration UI

Having a JavaScript-based UI also allows us to rethink how we can improve Drupal's administration experience. For example, Drupal's current content modeling UI requires a lot of clicking, saving and reloading. By using React, we can reimagine our user experience to be more application-like, intuitive and faster to use. We still need a lot of help to design and test different parts of the Drupal administration UI.

4. Allow contributed modules to use React or Twig

We want to enable modules to provide either a React-powered administration UI or a traditional Twig-based administration UI. We are working on an architecture that can support both at the same time. This will allow us to introduce JavaScript-based UIs incrementally instead of enforcing a massive paradigm shift all at once. It will also provide some level of optionality for modules that want to opt-out from supporting the new administration UI.

5. Implement missing web service APIs

While we have been working for years to add web service APIs to many parts of Drupal, not all of Drupal has web services support yet. For our React-based administration UI prototype we decided to implement a new permission screen (i.e. https://example.com/admin/people/permissions). We learned that Drupal lacked the necessary web service APIs to retrieve a list of all available permissions in the system. This led us to create a support module that provides such an API. This support module is a temporary solution that helped us make progress on our prototype; the goal is to integrate these APIs into core itself. If you want to contribute to Drupal, creating web service APIs for various Drupal subsystems might be a great way to get involved.

6. Make the React UI extensible and configurable

One of the benefits of Drupal's current administration UI is that it can be configured (e.g. you can modify the content listing because it has been built using the Views module) and extended by contributed modules (e.g. the Address module adds a UI that is optimized for editing address information). We want to make sure that in the new React UI we keep enough flexibility for site builders to customize the administrative UI.

All decoupled builds benefit

All decoupled applications will benefit from the six steps above; they're important for building a fully-decoupled administration UI, and for building visitor-facing decoupled applications.

Useful for decoupling of visitor-facing front-ends Useful for decoupling of the administration backend 1. Stabilize the JSON API module ✔ ✔ 2. Improve our JavaScript testing infrastructure ✔ ✔ 3. Create designs for a React-based administration UI ✔ 4. Allow contributed modules to use React or Twig ✔ ✔ 5. Implement missing web service APIs ✔ ✔ 6. Make the React UI extensible and configurable ✔

Conclusion

Over the past three years we've been making steady progress to move Drupal to a more API-first and JavaScript centric world. It's important work given a variety of market trends in our industry. While we have made excellent progress, there are more challenges to be solved. We hope you like our next steps, and we welcome you to get involved with them. Thank you to everyone who has contributed so far!

Special thanks to Matt Grill and Lauri Eskola for co-authoring this blog post and to Wim Leers, Gabe Sullice, Angela Byron, and Preston So for their feedback during the writing process.

May 09 2018
May 09

This blog has been re-posted and edited with permission from Dries Buytaert's blog. Please leave your comments on the original post.

Sample of Gender Identification Questions from the Open Demographics Initiative

© Open Demographics Initiative's gender identification questions

Last week, Nikki Stevens presented "Other, Please Specify" for TEDx at Arizona State University. In her TED Talk, Nikki shares the story behind the Open Demographics Initiative, which is developing a recommended set of questions that anyone can use to ask online community members about their demographics.

Nikki demonstrates how a majority of demographic surveys require users to conform to restrictive identity fields, which can alienate minority or underrepresented groups. The Open Demographics Initiative wants to develop forms that are more inclusive, in addition to giving people more control over the data and information they chose to disclose.

Inspired by Nikki's presentation, I reached out to the engineering team at the Drupal Association to see if there are plans to implement the Open Demographics Initiative's recommendations on Drupal.org. I was happy to learn that they are collaborating with the Open Demographics team to add the recommendations to the user registration process on Drupal.org.

Adopting Open Demographics on Drupal.org will also allow us to improve reporting on diversity and inclusion, which in turn will help us better support initiatives that advance diversity and inclusion. Plus, we can lead by example and inspire other organizations to do the same.

Thank you Nikki, for sharing the story behind the Open Demographics Initiative, and for helping to inspire change in the Drupal community.

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web