Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Apr 07 2021
Apr 07

Read our roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community. You can also review the Drupal project roadmap.

Drupal.org UpdatesDrupal contribution

Drupal's 20th Birthday Year

As we close out the first quarter of 2021, we continue the celebration of 20 years of Drupal with #DrupalFest and #DrupalCon!

#DrupalFest is a month-long celebration of all things Drupal, taking place online all around the world. DrupalFest lasts throughout the month of April. Most events are free, and we encourage everyone to attend, and even submit your own!

DrupalCon is right around the corner from April 12-16, happening online. This year's DrupalCon reflects a renewed focus on the strategic initiatives that drive innovation in Drupal. Each day has a half day of live programming for and then a half day of contribution, and all personas are welcome! Join us!

Increased focus on Strategic Initiatives

Speaking of strategic initiatives, the current primary initiatives being highlighted at DrupalCon and beyond are: 

  • Decoupled Menus - This initiative focuses on creating standardized tools and libraries for decoupled Drupal, starting with the menu system. This is the first step in making JavaScript front-ends a central part of the Drupal project. 
  • Easy out of the Box - This mega-initiative combines the efforts of Layout Builder, Media, and Claro to help empower content editors in Drupal to take advantage of the best that Drupal can offer.
  • Automatic Updates - This initiative is focused on the #1 most requested feature in Drupal: automatic updates. The initiative is building a robust and secure system for automatically updating Drupal, starting with security and patch releases.
  • Drupal 10 Readiness - The Drupal innovation train keeps rolling! The Drupal 10 Readiness initiative is rallying the community around what we need to reach our Drupal 10 release date, and helping site owners ensure they're ready for the upgrade when the time comes.

In addition to the content at DrupalCon, you can find ways to get involved in any of these initiatives by checking out the Drupal Strategic Initiative section on Drupal.org.

Decoupled Menu Initiative Support

General projects are a new content type on Drupal.org for code that does not fall into the neat categories of module, theme, or distribution. Instead, these can cover things like JavaScript Components, Drush Extensions, Install Profiles, Libraries, etc.

This is the first step in making Drupal a project greater than just PHP. This capability leans into Drupal's future in Decoupled applications, and in digital experiences beyond the web browser. 

Since the launch of general projects as a new content type on Drupal.org the Decoupled Menu Initiative has made great progress on creating standardized endpoints/libraries for decoupled Drupal solutions.

At DrupalCon North America the Decoupled Menus initiative leads invite you to a hackathon to begin to create applications for this work.

The rapid movement on this initiative shows how quickly the Drupal community can pivot into more robust and standardized Decoupled implementations, and furthers Drupal's lead in the marketplace.

Easy Out of the Box Support

For the Easy Out of the Box team, the Drupal Association has been focused on connecting the initiative leads to the Drupal Contribution Mentoring team, so that at DrupalCon there will be a variety of onramps to help new contributors support this work.

Easy Out of the Box is effectively three initiatives in one, focused on Layouts, Media, and the Claro administrative theme, so people with interest in any of those areas are more than welcome.

AutoUpdates Initiative Cross-Project Collaboration

The Drupal Association Engineering team continues its close collaboration with the AutoUpdates initiative team. Because AutoUpdates requires a server side component that will live on Drupal.org infrastructure, the engineering team needs to be closely involved.

This initiative has also had a heavy focus on cross-project collaboration - with three CMS partners in the PHP ecosystem collaborating together on the basic principles of supporting securely signed update packages.

Drupal    Typo3   Joomla

We're also collaborating with other partners, such as the Cloud Native Computing Foundations 'TUF'(The Update Framework) team, and the team behind Composer. 

Composer     Cloud Native Computing Foundation    The Update Framework

At DrupalCon North America the TUF team will be presenting about securing software package delivery - a topic that is sure to be interesting for all.

Drupal 10 Readiness Support

Drupal 10 is slated for release in June of 2022, which is only a little bit more than a year away. Fortunately, Drupal 10 follows the continuous innovation model of Drupal development that was so successful in the transition from Drupal 8 to Drupal 9. In essence, so long as site owners are up to date with the latest version of Drupal 9 they should be able to make the jump very easily. The only area of concern is deprecated code.

To that end, the Drupal Association engineering team collaborated with Gábor Hotjsy to set up automate code deprecation checking using the DrupalCI infrastructure. This allows the team to understand the most used instances of deprecated code, so that contributed module maintainers can be made aware of the need to update, and so that the Drupal Rector team(supported by Palantir.net) can begin creating automatic deprecation patches.

GitLab Merge Request Updates

Last year, Drupal.org migrated our community contribution tools to GitLab, by integrating the existing Drupal.org issue queues with GitLab's merge request functionality.

Thanks to these improvements, the complete contribution lifecycle can be completed entirely in the browser. As a contributor to Drupal you no longer need to use command line git, install a local development environment, or use a local IDE in order to make your contributions.

Since the initial launch, we've received feedback from many people in the community about improvements to usability with the Drupal.org issue queue integration. Looking at the child issues of this issue, we can see rapid usability improvements that have sped the pace of contribution.

More recently, we worked with our partners at Tugboat.qa to release live deployment previews of your code changes - first for Drupal Core, but now available for contributed projects on Drupal.org as well. This means that even reviewing visual changes or seeing your code deployed to a site can all be done without leaving your browser. This is a huge boon to all contributors, but especially to usability and accessibility experts who can much more easily view the impact of changes across issues.

Major improvements to Community events

In collaboration with the Events Organizers Working Group, the Drupal Association has updated the Drupal.org Community Events section. This new section represents a central repository for all of the events taking place across the Drupal Community, and will ultimately be the replacement for Groups.Drupal.org.

This section allows anyone in the community to submit their events, whether online or in-person, and provides a variety of views to help people find events they'd like to attend. Events can be filtered by type(con, camp, meetup, training, etc); proposed events can be submitted to help avoid scheduling conflicts; and calls for content/speakers can be promoted.

A feed of these events is made available for 3rd party tools built by the community, which is already being used to feed Drupical.com.

Local events are the heart of our community, so we hope that you'll help us by submitting your local events to this new tool!

Documentation updates

Led by community volunteer u/jhodgdon, Drupal.org's documentation tools have seen a variety of updates. In particular, the Drupal contributor guide is now much more complete, helping folks who are new to contribution in Drupal find a place to fit in and get started.

We've also deployed improvements that make it easier to understand whether the documentation you're reading is up-to-date, and how to report problems if you find them.

If you're looking for somewhere to contribute - helping to update documentation is a wonderful place to start!

Coming soon: Discover Drupal Portal

Coming up at DrupalCon is the announcement of a new program: Discover Drupal. This program is part of the Drupal Association's talent and education initiatives, and represents the Drupal Association's commitment to growing the Drupal talent pool and increasing diversity in our community.

With the official announcement just around the corner we won't spoil the details here, but very soon you'll be able to check out the new web portal for the Discover Drupal program and find out what it's all about.

Infrastructure Updates

Over the course of the last quarter the Drupal Association engineering team has provided a variety of feature updates for the Drupal project in terms of testing infrastructure: 

  • PHP8 Testing support - The Drupal Association provided PHP8 testing environments in DrupalCI, and so Drupal versions 9.1 and beyond are all fully PHP 8 compatible.Staying on the leading edge of compatibility gives Drupal the advantage of improved performance and security, and sets us up for success when it's time for Drupal 10.

  • Code Standards test for Drupal Core - Drupal Core tests now provide code standards testing results, saving a laborious manual step when reviewing core contributions.
  • GitLabCI/Pipelines - The Drupal Association has also enabled GitLabCI/Pipelines for these new general projects. This is a precursor to moving to GitLabCI for all Drupal CI uses. With direct maintainer control of the CI configuration for these projects, we can see automated workflows to support a wider variety of projects - allowing for more innovation. However, we need to be cognizant of cost controls as we open up this capability.

The year is off to a fast-paced, productive start and as always it is humbling and gratifying to see the great work that the community accomplishes with the tools the Drupal Association provides.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank: 

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.
Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Mar 22 2021
Mar 22

In November of 2020, shortly after launching Drupal.org's new issue forks and merge requests workflow, we announced another new feature: live deployment previews of Drupal Core issues, powered by Tugboat.qa - the same service that underlies SimplyTest.me.

Generating an automatic preview based on an open merge request is a huge boon to Drupal contributors, allowing them to preview the changes without having to set up a local dev server to deploy the changes to. It also makes it much easier to perform usability and accessibility reviews, as the visual impact of the changes is more readily visible.

Example:

This screenshot demonstrates where the previews appear in a contrib issue.

Example of live deploy previews

As we publish this, there are nearly 200 core issues that have preview ability to help accelerate the review process and increase the velocity of core contribution.

We're thrilled to announce that these same deployment previews are now available for contributed modules and themes on Drupal.org as well.

If you maintain a project on Drupal.org, enabling these previews is as simple as adding a tugboat.yml file to your project. Just follow the steps provided in our documentation. If you have feedback about the feature or would like to learn more, join the Tugboat Community Slack.

We hope the new feature makes contribution an even more rewarding experience.

Special request: Asking for your support for Ben Chavet

Ben Chavet is a long-time Drupal Community member, part of the extended Lullabot family, and one of the leads of the Tugboat.qa project. He worked very closely with the Drupal Association team in order to bring these live deployment features to the community. Now he needs your help. Ben is in intensive care with a very serious case of COVID-19, and his family needs your support. We appreciate your giving spirit to support one of our own.

Update

On Friday, March 26th, Ben passed away. His family, friends, and colleagues are grieving after this devastating loss. As a community we are reminded again of the seriousness and capriciousness of this deadly illness- its impact will last not only these 2 years of lockdown, but for the lifetimes of everyone who lost a loved one.

Ben's wife Suellen and children Samantha and Aiden will need support more than ever. If you have fond memories of Ben, please share your photos and comment with your stories here: https://photos.app.goo.gl/Tg2NRecKwxswNCvq5

If you are able to offer financial support, please donate here: https://www.gofundme.com/f/2av1u66cc0/

Jan 21 2021
Jan 21

Our community is in need of additional drupal.org site moderators (formerly "webmasters") to assist day-to-day site moderation tasks.

Over the past few years, the number of active site moderators in the Drupal.org site moderators queue has decreased significantly. Recognizing that this isn't a sustainable situation, the current active site moderators have embarked on an effort to recruit new volunteers.

Responsibilities of a site moderator

Site moderators are primarily tasked with responding to issues in the Drupal.org site moderators issue queue and the ongoing on-boarding of new community members. These tasks include:

  • Responding to issues in the Drupal.org site moderators issue queue.
  • Reviewing posts flagged as spam, deleting spam, and blocking users who post spam on Drupal.org web sites.
  • Confirming new Drupal.org users.
  • Responding to reports of projects posted on Drupal.org that do not meet community norms.
  • Other related tasks as described on the Moderation and maintenance documentation page.

Process to become a site moderator

In order to become a site moderator, there are a few easy steps:

  1. Familiarize yourself with site moderator responsibilities via the project https://www.drupal.org/project/site_moderators and the Site Moderator role description. https://www.drupal.org/community/contributor-guide/role/drupalorg-site_moderators
  2. Announce your intent to start participating in the #drupalorg-site-moderators channel in the Drupal Slack workspace.
  3. Begin participating in site moderator issues https://www.drupal.org/project/issues/site_moderators?categories=All. It is recommended that at first you ask questions and propose solutions to issues in a way that current site moderators can provide useful feedback.
  4. Once you have participated in the site moderator issue queue enough that you feel that you have a good handle on its various processes, create an issue in the queue asking for the Drupal.org "site moderator" role. It is highly recommended that you are active in the issue queue and Drupal Slack workspace channel, and have interacted with the other active site moderator before applying for this elevated role.

Why donate your time to Drupal?

If you've made it this far, then the obvious question is "Why should I become a Drupal.org site moderator?" Develop and exercise your people-skills while helping the Drupal community become a more sustainable, welcoming, friendly, and productive place. Regardless if you’re a developer, becoming a site moderator is a pretty straightforward way to build those skills.

In addition:

  • You'll work with other site moderators around the globe, building up your Drupal community network.
  • You'll have the opportunity to help nurture new community members in a meaningful way.
  • You'll have the opportunity to earn contribution credits.
  • Having the "site moderator" role is a position of leadership in our community. https://www.drupal.org/about/values-and-principles#choose-to-lead
  • If you or your organization uses Drupal and is looking for a way to give back, this is a small thing that will have a big impact.

First step

Interested? If you have any questions, pop into the Drupal Slack workspace #drupalorg channel and ask away. When you're ready, start with steps 1 and 2 above and start your journey!

Thanks to Donna Bungard (dbungard https://www.drupal.org/u/dbungard), Darren Oh (https://www.drupal.org/u/darren-oh), Jordana Fung ( jordana https://www.drupal.org/u/jordana), kiamlaluno (https://www.drupal.org/u/kiamlaluno), and Mike Anello (ultimike https://www.drupal.org/u/ultimike) for assisting with this blog post. 

Nov 11 2020
Nov 11

Tugboat.qa offers automated and on-demand deployment previews to accelerate development. In 2019, Tugboat was successfully integrated as the backbone of the community testbed Simplytest.me. Following that success, a community-lead initiative emerged to provide these previews for all of Drupal Core development, by integrating into the Drupal.org issue queue. Drupal Association CTO Tim Lehnen (hestenet on Drupal.org) opened an issue to explore this option in early August.

Why automated environments for Core issues?

Reviewing and testing Drupal.org issues has historically been the purview of experienced Drupal developers. Designers, end-users, and other stakeholders have been relegated to the role of watching over the shoulder of developers who drive demos from local development environments. This has been detrimental to fully involving those stakeholders in a development process that could improve Drupal for all users. With Drupal's ease-of-use, accessibility, and front-end features being central to the current strategic initiatives, it was time to ensure that these stakeholders are fully empowered. 

Automated deploy environments help bridge the technical divide and bring the ability to review changes to more users. Longtime Drupal contributor YesCT says:

The Tugboat integration with drupal.org issues makes reviewing and testing possible for people who have never been able to do it before. This project uses technology to lower the barriers of contributing to Drupal, and makes contributing more accessible to more people. It’s going to be a huge game-changer.
YesCT

In practice, visitors to the Drupal.org issue queue can click a link to tour a live, working version of a Drupal website wherever Tugboat deploy previews are enabled. As of November 10, 2020, this includes all Drupal Core issues.

Current UI:

Current UI for live previews in Drupal core issues

Upcoming UI revision:

Upcoming ui revision for live previews on Drupal Core merge requests

Drupal.org deploy previews - under the hood

The Drupal.org/Tugboat integration relied on a handful of contributors across a project that spanned months, but the core of this collaboration was a partnership between core contributor Juampy NR and Drupal.org architect Neil Drumm

As a senior developer at Lullabot, Juampy's continuous integration expertise, as well as his familiarity with Tugboat, made him an invaluable contributor to implementing the Tugboat side of the integration. Neil's expertise as architect, meanwhile, gave Juampy and Tugboat the guidance to find the right places in Drupal.org's custom logic to add code for this new integration. Simultaneous to the Tugboat integration, Juampy and Neil also worked to implement the Drupal Association GitLab integration, bringing issue forks and merge requests to the issue queue

Some of the challenges that this team faced included:

  • Drupal.org’s large database. Because of the size of the database, the team had to work remotely using a provisioned development environment. Remote debugging helped to catch and traverse the different events involved in the workflow.

  • Understanding where to write code. Drupal.org has a ton of custom logic, so finding the right place to add new code was not easy. The Drupal Association's guidance was key.

  • Integrating multiple systems. The team had to write interactions between Drupal.org, Tugboat, and GitLab, taking into account error responses, timeouts, and firewall restrictions.

Deployment preview workflow

Reducing friction by making changes visible and more accessible to all contributors

Before this integration, contributors had to take and upload screenshots of their changes, and reviewers had to replicate changes locally in order to see them in action. Tugboat reduces friction by making changes visible - and more accessible - to anyone who clicks the preview URL. This shortens review time considerably and invites participation from non-developers, working toward Drupal's value of respecting and valuing inclusivity.

Check out our documentation on Drupal.org to learn more!

The Drupal Association gives our thanks to TugboatQA/Lullabot, for contributing this feature to the Drupal project.

Nov 04 2020
Nov 04

Read our roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community. You can also review the Drupal project roadmap.

We're trying something new for our September update - a video! Let us know if you like this format in the comments.

Embedded below is a video we sent out to our Supporting Partners about the engineering initiatives that their support funded in September.

Here's what you'll learn about:

  • The issue fork and merge request beta - which makes Drupal contribution faster and easier, especially for new contributors.
  • The lazy-load initiative - which will make all Drupal sites on 9.1 and beyond faster.
  • Composer 2 support - Drupal now fully supports being built with Composer 2, which is much faster and less memory-hungry than Composer 1.
  • The Auto-Updates contribution week, which the DA held with community contributors from inside and outside of Drupal.
  • New efforts to match make organizations who want to contribute with initiatives that need help.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Sep 18 2020
Sep 18

Read our roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community. You can also review the Drupal project roadmap.

Drupal.org UpdatesGit Merge Request

GitLab Merge Request beta

Back in July we kicked off the GitLab merge request beta, and over the course of August more and more project maintainers opted-in. The merge request workflow is available on more than 150 projects, and in several core issues. This is a *dramatic* improvement in the contribution workflow, especially for newer contributors or contributors making 'drive-by' contribution - and makes the lives of our existing long-term contributors much easier.

We're getting ever closer to enabling this workflow on every project across Drupal.org - but we still need your help! You can try out the contribution workflow on any of the projects that have opted in, opt-in your own projects, or check out the workflow in the core issues trying the beta.

Check out these issues to opt-in your project or core issue.  We anticipate general availability across all projects within just a few more weeks as we gather the final feedback from our beta contributors.

Auto Updates contribution week(s)!

During the week of August 3rd, 2020, representatives from three of the most successful open source content management systems came together to collaborate on a mechanism for securing software updates. Each of these open source projects is based on PHP, with similar use-cases, users, and update delivery architecture. By teaming up across these three projects, and any others who choose to join, we hope to standardize on a secure update delivery and validation mechanism. With this mechanism in place, each project can then build on top of it for additional features for our respective communities, such as providing secure automatic updates in Drupal.

Drupal    Typo 3   Joomla

Drupal, Typo3, and Joomla spent the week evaluating The Update Framework (aka TUF), an initiative of the Cloud Native Computing Foundation to provide a standardized framework or managing secure updates, minimize the impact of any potential compromises, and to be flexible enough to be used across different software systems.

Over the course of the week we made significant progress by replicating the reference implementation and its test fixtures in php, hosting the work in a shared repository: https://github.com/php-tuf/php-tuf. As we left the contribution we agreed to regroup in September for an additional collaboration. (That second contribution week has now happened, so look for an update in our September blog post!)

How can you get involved?

If you're interested in contributing to the PHP-TUF effort, you can take a look at the GitHub repository for the project: https://github.com/php-tuf/php-tuf

If you're connected to the Drupal community and are interested in PHP-TUF, or the larger initiative to work towards automatic updates, you can join us in Drupal Slack in the #autoupdates channel.

Per-project issue summary templates

Contributing to Drupal can be intimidating for a newcomer. One of the first barriers is just knowing 'how do I file a bug or feature request?'

Drupal core has a project issue summary template that is predefined to ask the right questions to help build a useful, actionable issue report.

And now, thanks to community contributors, each project hosted on Drupal.org can now define its own issue summary template - extending this great contributor onboarding feature to all projects in the Drupal ecosystem.

Launched the Lazy Load initiative

An increasing goal of the Drupal Association is to foster and onboard more major contributors to the Drupal project. One of the organizations we've been talking to recently is Google. Google has a vested interest in the performance of the open web, and over the course of the past several years has begun hosting a CMS leadership summit, to bring representatives from many CMS projects together.

As part of this growing collaboration, the Drupal Association has helped to coordinate the Lazy-Load initiative, which seeks to load all images lazily in Drupal core by default. Google has generously sponsored this work, and we hope to see it included in Drupal 9.1.

DrupalCon Europe

Supporting DrupalCon Europe - now virtual!

DrupalCon Europe has gone virtual! Early in the year, many of us hoped that we might be able to come together in person in Barcelona this fall. Unfortunately, this was not to be. But the European community is not letting that stop them from having a great event - online!

The Drupal Association spent some time in August working with the DrupalCon Europe team to integrate their chosen virtual event platform with the events.drupal.org schedule!

Registration is open now: https://events.drupal.org/europe2020 

And you can browse the program here: https://events.drupal.org/europe2020/schedule/2020-12-08

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Sep 04 2020
Sep 04

As you’ve probably heard - merge requests via GitLab are finally here on Drupal.org.

On July 13th, the week of DrupalCon Global, we began the beta process allowing projects to opt in to using issue forks and merge requests. More than 100 projects have opted in so far. We’re getting close to enabling issue forks and merge requests for all projects, but we have a few more steps to go. In particular, we want to make sure that core contributors can test issue forks and merge requests for core.

As part of the beta program, we’re going to enable these features on specific core issues - nominated by core contributors as good issues to test these features.

Are you a core contributor who would like to try the new workflow on an issue?

Nominate your issue for the merge request beta

Contributors should opt-in issues which are:

  • Well-scoped.
  • Likely to be completable in days or weeks, rather than weeks or months.
  • Issues with enough complexity to exercise the forks and merge request system.
  • Issues that they themselves intend to work on.

Once these core contributors have signed off on the issue fork/merge request workflow - we’ll be enabling these features across all projects.

Keep your eye on this blog for further updates!

… one more thing

Tugboat.qa (a new venture by Drupal Association Supporting Partner Lullabot) are donating an enterprise license of their tool, which generates automated deployment previews of software. We’re going to be enabling this for core issues as part of our beta process as well, to see if these development previews can help accelerate core contribution, and what resources this tool might require in our infrastructure. Our thanks to Lullabot for the partnership!

Aug 11 2020
Aug 11

Read our roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community. You can also review the Drupal project roadmap.

DrupalCon Global

It's hard to believe that our last published update was back in April of 2020. Time seems to simultaneously crawl and disappear in the time of quarantine. All hands were on deck at the Drupal Association to facilitate the transition of DrupalCon from an in-person event to a virtual conference, and those efforts were rewarded:

The first DrupalCon Global event exceeded all expectations. Creating a 100% virtual event which captures the spirit of DrupalCon is no small task, but thanks to our sponsors and supporting partners we were able to set the benchmark for virtual open source conferences.

DrupalCon wasn't the only thing we worked for the past three months, of course, and we have a number of phenomenal updates that we'd love to share with you.

In fact, we spoke about this work during the Drupal.org Engineering Panel at DrupalCon Global, so what better way to update you than with that recording!

[embedded content]

Highlights

Looking for the TLDR? 

Don't worry - we know you're busy - here's the short version: 

  • We recapped the success of the #DrupalCares campaign, and reiterated our #DrupalThanks to all the contributors who made it possible to refocus on our mission work.
  • We shared new metrics available on https://www.drupal.org/metrics that help us understand community activity, project health, and community diversity.
  • We showed off the new community event listings - the beginning of a more modern replacement for the event-organizing features of groups.drupal.org.
  • We showed off the tools built for the new contributor guide - which are now in use by the community to help match contributors and their skills to what's needed for the DRupal project.
  • Drupal 9 released, not only on time, but in it's very first potential release window - on June 3rd, 2020. This is an incredible testament to the work of the community, and in particular the core maintainers.
  • We launched the beta for merge requests integrated into Drupal.org - you can opt in the projects you maintain.
  • We announced that Drupal Steward program was going live! And we're looking for 10-30 initial site owners ready to sign up, to help us refine our onboarding process.

But that's not all! 

Community contributed Drupal.org improvements

In addition to all the work that the Engineering team has done in recent months, the community has also swung into action to provide some great improvements to Drupal.org. 

Jun 17 2020
Jun 17

This is the sixth post in our series about integrating Drupal.org with a 3rd party developer tooling provider:

In March of last year the team here at the Drupal Association migrated all of the Git repositories at Drupal.org from bespoke Git hosting to a self-hosted GitLab instance. This was a major milestone in modernizing the developer workflow, immediately providing better search within project codebases, a better code viewer, and direct code editing in the browser for project maintainers.

However, we know that the primary feature the community has been waiting for is merge requests. We're very excited to report that we're now very close to opening up a beta test of merge requests integrated with Drupal.org issues.

Stepping back for a moment, let's remember the ideal contribution flow that we defined when evaluating our Developer tool options: 

Drupal Flow

  • The issue itself should remain a single-threaded conversation for discussing the particular problem to be solved.
  • Every issue will be associated with one canonical issue fork(with multiple branches, as needed) which can be collaborated on by any contributor, and merged by a project maintainer.
  • Contributors will be able to modify the code in these issue forks by:
    • Checking out the code and committing/pushing changes to the workspace.
    • Inline editing files using GitLab's web interface.
    • Legacy: uploading a patch.
  • All types of contribution—whether merge requests or the legacy patch workflow—will continue to trigger DrupalCI testing.
  • Issue forks can be rebased (manually or automatically) when needed to resolve conflicts with upstream commits.
  • Contributors and project maintainers will be able to comment on individual lines of code.

The foundation for this work is the ability to create these issue forks from an issue on Drupal.org. This involves building an interface from Drupal.org that creates an issue fork in GitLab associated with the Drupal.org issue, and then any Drupal.org user can push to that fork and branches within it. Maintainers may then merge this work from a branch on the issue fork the project.

That foundational work to create the necessary git hooks and access control management is now complete.

The next steps are:

How can you get involved?

If you are a contributed module maintainer, and would like to be a part of the early beta test program for Drupal.org-integrated GitLab merge requests, please indicate your interest by posting a comment to this issue with a list of projects you would like to opt-in.

When will merge requests be available to the community at large?

Creation of issue forks and branches will be available to a limited subset of projects this week, the week of June 14th, 2020. Over the course of the next several weeks we hope to implement the additional UI features that will display relevant information about the issue forks back in the Drupal.org issue, and ultimately allow merge requests themselves.

We hope to have the beta of merge request functionality available to some projects no later than DrupalCon Global, in Mid-July. From there, we'll work through feedback from our beta testers, and work with Drupal core maintainers, to enable issue forks for all projects in the coming months

We hope that you're as excited about this update as we are. It represents a lot of thought on the part of many community contributors, and a lot of work on the part of the Drupal Association staff.

Post-Script - What about DrupalSpoons?

As you read this update, many of you may be wondering how this work fits together with an emerging community initiative called DrupalSpoons. DrupalSpoons is an effort to not only use GitLab for merge requests, code viewing, inline editing, etc… but also to replace Drupal.org issues and DrupalCI with their GitLab equivalents, unifying the developer experience in a single UI.

To be clear, neither we at the Drupal Association, nor the DrupalSpoons initiative are recommending that projects at large begin migrating to the current experimental DrupalSpoons environment. Rather, DrupalSpoons is encouraging experienced maintainers who want to try the workflow as early adopters to try managing their projects on DrupalSpoons. Their goal is that initiative will prove new workflows and ideas that can be brought back into the official community tools on git.drupalcode.org.

The DrupalSpoons contributors have done some helpful and innovative work so far. We appreciate seeing contributors pushing the boundaries of what can be done with 'off-the-shelf' tools, and there's potentially quite a lot we might learn from this work that will inform future tooling for the Drupal project. For example, it's likely that we'll look to DrupalSpoons' use of GitLabCI as we look towards the next generation of DrupalCI.

We are always watching and listening to see what the community wants and needs for Developer Tools. Over time, new tools emerge that add significant value. As the Drupal Association, being responsible for maintaining these tools, we would like to approach each component independently (i.e: look at merge requests, issues, or GitLabCI, each independently) as the DrupalSpoons project continues.

In the meantime, we are moving forward with integrating GitLab merge requests with the Drupal.org issue queues. Once that's complete, we'd like to see a gap analysis to see what would be gained and lost when comparing Drupal.org issues with GitLab issues.

Jun 16 2020
Jun 16

The following case study was written collaboratively by hestenet and nnewton, explaining how we use rsync.net to manage backups for Drupal.org. The Drupal Association used rsync.net for many years prior to any partner relationship, and is now proud to count rsync.net among our Technology Supporters.

Drupal.org has been the home of the Drupal community for many years. Online since 2001, and fed by a global community of contributors, there is a tremendous amount of open source history recorded here.

It's critical that we safeguard that history for posterity, and of course all of our current activity so that we can maintain the momentum of the Drupal project.

Naturally, we've done a tremendous amount of work to make our infrastructure robust and fault tolerant from the top of the stack to the bottom. Individual servers use RAID storage, our infrastructure is built using highly-available pairs, and the Oregon State University Open Source Lab, our data center, has good data center hygiene and redundant power and cooling.

But disasters can and will happen, and this is why off-site backups are critically important.

Drupal.org uses rsync.net to manage off-site backups, and we highly recommend them as a solution. rsync.net is built on ZFS, a file system we have experience with and trust to be durable and offer cheap, immutable snapshotting. rsync.net gives you an empty filesystem to do anything you want with and works with any SSH or SFTP based tool. This standard approach allows us to easily use the service with existing tooling. We have used rsync.net for various purposes for almost ten years and have not had a single incident.

How exactly do we use rsync.net

rsync.net is actually configured as our primary backup location for all of the Drupal.org infrastructure. In addition to this, because we take advantage of rsync.net's geo-redundancy feature, rsync.net provides those backups in multiple, separate data centers. We also use rsync.net for a secondary backup layer for some select data pools that are already backed up in Amazon S3, or on the Open Source Lab's backup servers.

How do we have it configured?

For the Drupal.org infrastructure we use BorgBackup (https://www.borgbackup.org) to manage compression, encryption, and deduplication of our backup data. We then entrust rsync.net with ZFS snapshotting of the Borg data, providing us with points in time to easily roll back to. This gives us a sliding window of encrypted backups. It also gives us protection from malicious actors or ransomware as the rsync.net snapshots are immutable, or read-only.

The actual execution of borg and tracking of backups is done using a bash script that is placed on each server by our Puppet tree. Puppet also places a private key for encryption and the appropriate ssh private key for rsync.net access. We wrap Borg in a script due to our need to cleanly initialize new vaults when we spin up a new server, as well as our need to monitor Borg execution. One thing we have found is that it is difficult to detect silent failure for Borg specifically, so we have multiple points of feedback in the script. Our script functions as follows:

  1. Check if a vault on rsync.net exists for this host, if not create it.

  2. Backup the paths passed to this script to said vault for today’s date.

  3. Check the return of the last command, if it has failed email our monitoring endpoint to trigger an alert.

  4. Use Borg to pull statistics from the last backup, such as count of files backed up, chunks backed up, size of backup, etc.

  5. Massage those statistics into usable metrics and send them to statsd, where they will end up in our monitoring system.

With this process we both have an alert in the time of failure and also can create alerts based on the graphs we create from statsd. We do this to catch times when a backup may have succeeded but the amount of data backed up dramatically fell. That “success” may have been a failure in that case, just not an obvious one.

Example dashboard built from statsd information:

A major reason we value rsync.net is it presents a simple/standard ssh interface that allows us to use tooling we can customize to exactly what we need, as in the above example.

Why would we recommend this to others?

We share the Drupal community's love of simple, elegant, and technically excellent solutions to problems. Configuration and backup management with rsync.net ticks all of those boxes, and further the business-side is run in a very friendly way, with frequent increases in capacity available at very reasonable rates.

It's proven to be an effective and affordable way to use the funding we receive from the Drupal community to protect the project, and we believe you can trust it to protect your own projects as well.

May 20 2020
May 20

Read our roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community. You can also review the Drupal project roadmap.

Project NewsDrupal 9.0

Drupal 9 will be released on June 3rd, 2020

Despite the disruption of a global pandemic, the Drupal community has beaten the odds and Drupal 9.0 is on time for it's scheduled release window on June 3rd, 2020.

Drupal 9 represents the culmination of all of the features developed over the course of the Drupal 8 lifecycle, as well as the realization of the project's commitment to easy upgrades.

Drupal 9 beta 3 is available now, and the first release candidate will be available soon. We encourage you to participate in the Drupal beta test program, to help us ensure a smooth release.

Drupal 9 porting weekend from May 22-23

With the release of Drupal 9 only a couple weeks away, the community is coming together to support the effort to get modules ready for Drupal 9. After a successful Drupal 9 porting on April 28th, nearly 75% of the top 200 most used modules are already Drupal 9 compatible. Never before has so much of the contributed module ecosystem been ready even before the new release.

If you'd like to join in on the Drupal 9 module porting weekend, community member @kristen_pol has written a guide to the event.

New Drupal Brand assets available

The Drupal Association was very pleased to announce a new evergreen Drupal brand in time for the release of Drupal 9.

What does 'evergreen' mean?

The new branding is evergreen in the sense that it is no longer tied to a specific major version of Drupal. Whether for Drupal 9, Drupal 10, and beyond this new brand can remain the consistent identity for Drupal. This parallels the Drupal project's own development philosophy, where major version upgrades should no longer be a difficult process for end users.

With these new brand materials we hope to be able to unify the presentation of Drupal throughout the ecosystem, and help reintroduce Drupal to the world when the project inevitably gains more attention during Drupal 9's release.

We encourage you to begin using the new brand within your own materials as well - to support this effort.

Automated Deprecation Patches to port your module to Drupal 9

The Drupal Association is working together with the Drupal Rector team from Palantir and contributor Tedbow from Acquia to provide automatically generated Drupal Rector patches for all the projects on Drupal.org.

As of April, these patches are already available through DrupalCI. In May we hope to begin having a bot automatically post these fix patches to Drupal.org issues

More Drupal 9 Readiness tools

Drupal.org Updates

Events listing feature on Drupal.org

As we spoke about in last month's update, we've been working on a Event Listing Content Type on Drupal.org to help replace the aging Groups.Drupal.org subsite, and to support our global community by providing a central repository of events.

EDITED: The new event listings are being beta tested by the Event Organizers Working Group before a roll-out to initial community users.

Respectful Advertising

The COVID pandemic and its impact on DrupalCon has only emphasized the need for the Drupal Association to further diversify its revenue sources. We've made significant strides over the last several years, but as we've heard from many of you, that work must accelerate.

We've made a few changes over the course of April to start accelerating this revenue diversification:

In the most noticeable change, we've partnered with CarbonAds, a network that focuses on advertising to technical audiences, to create placements on Drupal.org.

These placements are hidden for Drupal Association members, so this program also helps promote DA membership as well, and does not put advertising in the workspace of our committed supporters.

Enhanced Membership Options

Speaking of membership - we also made some major overhauls to the Drupal Association membership program during the #DrupalCares campaign. Many members of the community reached out to us asking for more options for supporting the Drupal Association through membership, and we were happy to accommodate those requests.

There are now new membership levels for annual renewal, and we've also added a monthly membership option, which is now the default. We hope to continue to expand the membership program and its benefits to further support our fiscal stability in the future. 

#DrupalCares

A special thanks

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Apr 14 2020
Apr 14

Read our roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community. You can also review the Drupal project roadmap.

...what a changed world we live in compared to just a month ago. There are so many people across the world who make Drupal what it is, and Drupal.org is where we all come together.

Tim Lehnen (hestenet) | CTO - Drupal Association

Project News

Drupal 9 release window confirmed

The Drupal Core release managers and the Drupal Association team successfully released Drupal 9.0.0-beta in March, which means that we are on track for the June release window of Drupal 9. The target release date for Drupal 9 is June 3, 2020. 

There are several things you can do to help: 

  • Test the latest Drupal 9 beta. At the time of writing, Drupal 9.0.0-beta2 is the latest available version.
  • Test the Drupal 8.9.0-beta. This will be the last minor release of Drupal 8, to come out simultaneously with Drupal 9. 
  • If your organization would like to formally join the beta test program, you are welcome to do that as well. 

It's also time to get your own sites, contributed modules, and custom modules ready for Drupal 9. The community has built a variety of tools to help: 

#DrupalCares Fundraising Campaign

#DrupalCaresAt the end of March we also kicked off the #DrupalCares fundraising campaign. This campaign is specifically targeted at closing the budget gap caused by the impact of COVID-19. 

Drupal is used across the globe - but did you know Drupal is also in use by organizations on the front-line of the COVID-19 fight? The National Institutes of Health, the CDC National Prevention Information Network, Oxfam, Unicef and many others are using Drupal to spread key messages about safety and prevention. The Drupal Association supports these organizations with the infrastructure that hosts new releases and security updates. 

As of today, project founder Dries and Vanessa Buytaert have also announced a #DrupalCares matching campaign. They will match all individual donations, membership upgrades, and membership renewals through the end of April - up to $100,000. We appreciate their leadership and support. 

Drupal.org Updates

Semantic Versioning Available for All Projects

Semantic versioning is now available for all contributed projects on Drupal.org. All versions of Drupal greater than Drupal 8.8.3 are compatible with semantic versioning, and the old version format will continue to be supported until Drupal 10. 

Example of Semver

Using semantic versioning allows contributed project maintainers to follow the same pattern that has been so successful for the core release cycle--enabling core to release significant features every six months. The semver version pattern is: MAJOR.MINOR.PATCH. To summarize how this is used in the Drupal project: PATCH versions should include bug fixes and security releases; MINOR versions can include new features, but should remain backwareds compatible; MAJOR versions can break backwards compatibility and remove deprecated code.

Project maintainers who want to continue to support versions of Drupal at or below 8.8.3 should continue using the old version schema.

Major overhaul to Packaging Pipeline

Drupal PackagingAs we mentioned in last month's update, we've significantly overhauled how Drupal.org's packaging pipeline works, to provide better support for scaffolding Composer installations, to improve introspection, and to make it easier to enhance and maintain as Drupal continues to evolve as a software platform. 

There were several components to this packaging update: 

  • The Packaging Process is now a multi-step Jenkins pipeline, making it easier to observe packaging progress at each phase. 
  • The Packaging Process uses composer create project to ensure that the generated .zip and .tar.gz archives are ready to use Composer. 
  • The subtree splitter step of the packaging process uses a local path repository, so the packaging infrastructure does not depend on GitHub/Packagist
  • Many steps of the pipeline have had performance improvements and reductions in redundant work, to speed up the release process as much as possible. 
  • The use of local path repositories will also help reduce the risk of early embargo break for any security releases. 

Coming soon: Community event listings on Drupal.org

Groups.Drupal.org is well past its prime, but there's still a tremendous need for the community to organize around local events. Especially in light of shelter-in-place orders during COVID-19, we want to support the community in making their virtual events accessible to all.

In collaboration with the Event Organizers Working Group - we've been building a new event listings feature for drupal.org that will allow users to submit their DrupalCamps and other events, include a feed that can be aggregated by tools like Drupical.com, and provide a baseline for future features that will allow us to sunset Groups.Drupal.org

Drupal Association Updates

Contribution Recognition Committee

We know that the community at large is very interested in the future of Drupal.org's Contribution Credit system. The Contribution Recognition Committee has been holding regular meetings since its formation at DrupalCon Amsterdam in 2019, and has been making good progress gathering community input and creating early models for the next generation of the credit system.

The next phase of the committee's work is interviews with key stakeholders in the community, before assembling a list of final recommendations. Please understand:The CRC is on a brief pause as many committee members have found their time constrained for extra child care and other concerns during shelter-in-place orders for COVID-19, but we hope to resume progress as soon as possible. 

You can reach out to the committee here

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank: 

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Mar 13 2020
Mar 13

Read our roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community. You can also review the Drupal project roadmap.

Project News

Drupal 9 beta is closer than ever!

At the time of writing this post, there are fewer than three beta-blockers for Drupal 9. This hopefully means that we'll be seeing a beta release of Drupal 9 very soon. 

What does this mean for you?

Now's the time to get familiar with what's coming in Drupal 9, and to check your contributed or custom modules to see if you're ready to go. The community has put together a number of tools that you can use: the upgrade status module, the Drupal Check command line tool, and Drupal Rector.

We also need your help! We're looking for more individuals and organizations to participate in the Drupal Beta Test program. It's a great way to contribute to Drupal.

Call for Sponsors & Contributors: Automatic Updates

We're really proud of the work we accomplished in the first phase of the automatic updates initiative; in Drupal 7 and Drupal 8, sites that don't depend on Composer workflows now have complete support for securely and automatically updating Drupal Core. In the second phase of this work we want to extend that support to contributed projects, and to support Composer-based site installations. 

We need your help to make the second phase happen. Will you contribute?

Learn more on our call for sponsors & contributors post.

Drupal.org Updates

DrupalCon Minneapolis Program Update

In preparation for releasing the full DrupalCon Minneapolis speaker schedule, we've made some updates to the accepted sessions page. 

The newly redesigned page now highlights our excellent keynote speakers (to include Mitchell Baker from Mozilla!) as well as other featured speakers for this year's event. On top of that you can filter the list of sessions by track, to get a jumpstart on finding your favorite sessions, before the full schedule is released. 

Ready to enable Semantic Versioning for Contributed Projects

We've rearchitected the version management for contributed projects, so that they can begin using Semantic Versioning as we enter the Drupal 9 era. You can see an example of this in practice on this sample project: semver_example. 

We're coordinating with the Drupal core maintainers to select a window for enabling the new semver functionality across all projects. We want to ensure that Drupal end-users will still be able to find and easily understand which projects they can use once projects are able to be compatible with both D8 and D9, and are using semver version numbering. 

Not familiar with semantic versioning

The three digit numbering scheme (MAJOR.MINOR.PATCH) is designed to provide guide rails around API breaking changes. In Drupal core for example, patch releases are incremented whenever there are bug fixes or security releases. Minor releases indicate that new features have been introduced. And the Major version only changes when deprecated APIs are removed and fundamental architectural changes have been introduced.  Contributed project maintainers are encouraged to adopt the same pattern.

Updated display of releases

Speaking of releases - we've recently updated the display of releases to provide a cleaner view of release metadata. This should make it much easier to understand the history of recent releases, and to see at a glance which ones were bug fixes vs. feature releases vs. security releases. 

New Release Meta Data

You can see a detailed example by looking at the release history for Drupal core

Drupal usage stats by branch

Because of the six-month minor release cycle, it's become much more important to have more granular insight into what minor versions of Drupal are in use in the wild. 

Usage stats by branch

As you can see above, we've updated the usage stats for Drupal to display usage by branch. This is mostly useful for Drupal Core, but may be valuable for contrib maintainers as well as they look to understand which versions of their projects are in highest demand. 

Coming soon: An updated UX for project browsing

With the release of Drupal 9, it will be possible for contributed projects to be compatible with both major versions of Drupal. Perhaps more interestingly, because of the release of new features with minor versions, there are some projects that may only be compatible with a certain range of minor versions (e.g: 8.6.x - 9.2.x). 

This is a powerful improvement in ensuring that key modules are ready to use with Drupal 9 on day one, but it also has the potential to be confusing for Drupal site owners and evaluators who are trying to discover what projects they can use. We're looking to update the project browsing on Drupal.org to make sure discoverability doesn't suffer with this change. If you have good ideas about this user experience, please feel free to share them on the issue!

Drupal 9 Readiness

Packaging enhancements

Beginning with Drupal 8.8.0, Drupal needed to be packaged from the output of Composer create project, rather than as the simple contents of a git clone. These changes to packaging have additional ramifications for how we manage tagged releases for Drupal core, and in particular for how we manage security releases. We've been making a variety of updates to the Packaging pipeline since Drupal 8.8 to make the process more transparent, resilient, and performant, and that work continues. 

DrupalCI

DrupalCI: Support for new Postgres environments

Because minimum requirements are changing with Drupal 9, we've added new test environments for both Postgres 10 and Postgres 12

DrupalCI: Updated SQLite version

SQLite has also been updated within the DrupalCI test environment to version 3.26, to support testing on the correctly supported version. 

DrupalCI: Support for MariaDB environments

MariaDB forked from MySQL after the acquisition by Oracle, but at first had remained fairly consistent. However, with recent versions MariaDB has had to diverge, and so we are now providing explicit testing support for MariaDB, with test environments for versions 10.2.7 and 10.3.22. 

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank: 

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web