Feb 07 2020
Feb 07

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Project News

Get Ready for Drupal 9

Are you wondering what it will take to upgrade to Drupal 9? Good news - it's going to be easier than any other major version upgrade in Drupal's history.

The upgrade to Drupal 9 is just like any other Drupal upgrade, except that the new codebase has updated key dependencies Drupal relies on and removed deprecated code. As long as all the modules and custom code you use don't rely on deprecated code - you should be good to go.

As it turns out, many contributed or even custom modules only need a one-line change to be ready for Drupal 9. You can use these community created tools to check the status of your modules: the upgrade status module, or the Drupal Check command line tool. In many cases, you may just need to remove some deprecated code in favor of the more modern implementations. Drupal Rector can provide you with automated fixes for many of these deprecations. 

Still getting to grips with Composer?

Composer

If you're still getting to grips with using Composer after the changes in Drupal's 8.8.0 release, don't worry - there's help to be found. The community has extensively documented the different scenarios a site owner may find themselves in with this update.

If you've previously used one of the community created templates to manage your site to composer, there are instructions to migrating to the officially supported method.

If you've never used Composer at all - you're in luck - with 8.8.0 and beyond everything you need is already in place. 

Drupal.org Update

Drupal.org Packaging updates

As mentioned in our December update, we've been making major improvements to the Drupal.org packaging pipeline, to support packaging Drupal using Composer create project. We reached a major milestone at DrupalCamp New Jersey, allowing our packaging pipeline to properly support the Composer create project command when generating tar and zip files, and paving the way for enhancements to the core subtree splits.

Updating this pipeline is critical for ongoing releases of Drupal, and is part of paving the way for the Drupal 9 alpha release. We want to thank Acquia for donating time to help us get this work ready.

Preparing for contrib Semver

Per our roadmap for supporting Semver for contributed projects on Drupal.org, we have updated the way contrib version numbers are stored, making existing version numbers parseable when we convert to full semver. We also collaborated with core contributors at DrupalCamp New Jersey to identify and resolve a number of other related issues.

Drupal.org now has an example project which uses semantic versioning, which we are using as the testbed for this support, and to prove out any additional UI changes that we want to make before rolling this out to all other contributed projects.

Want to learn more about Semantic Versioning and how to use it properly within your projects? Semver.org can walk you through it.

More accessible formatting for the DrupalCon program schedule

It's almost time for the DrupalCon Minneapolis program to be published! To prepare for this launch, we've made updates to the program schedule to improve accessibility and readability for attendees.

In particular these updates have focused on line weights, spacing, and other formatting changes that should improve readability. With the accepted sessions being announced soon,  we're excited to see what you think!

Better social event submission tools for DrupalCon events

DrupalCon Minneapolis | May 18-22 2020Some of the best parts of DrupalCon are the social events that take place around it. They're a chance for the community to celebrate and build camaraderie, and an established tradition. We've made updates to the social event submission process to make getting your event listed easier than ever. 

Join the Drupal Community in person! 

By the way… have you registered for DrupalCon yet?

DrupalCon is the best place to come together with other members of the Drupal community in person. It's also the central meeting point for all of facets of the Drupal business ecosystem, so if you are end-user looking for training or a vendor to support your Drupal deployment - there's no better place to be than DrupalCon.

DrupalCon Minneapolis is going to be here any day now - so get your tickets before prices go up!

Can't make it to Minneapolis? Join us at DrupalCon Barcelona 2020 in September.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.
Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Jan 15 2020
Jan 15

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Project News

Get Ready for Drupal 9

The next minor release of Drupal will be 8.9 - to be released simultaneously with Drupal 9.0. The first target window for the release is this coming June, so now is the best time to get ready for the release.

As it turns out, many contributed or even custom modules only need a one-line change to be ready for Drupal 9. Check yours using: the upgrade status module, or the Drupal Check command line tool.

DrupalCon Minneapolis 2020

DrupalCon MinneapolisSpeaking of Drupal 9, DrupalCon Minneapolis is coming up from May 18-20. We expect their to be a large amount of programming and contribution focused on the upcoming release of Drupal 9. Minneapolis will be a great opportunity to get help with checking your module compatibility, or to find someone who can help you get your Drupal 7 or 8 site ready for the upgrade. Get your tickets now, before prices go up!

DrupalCon Europe 2020

Did you hear the news? DrupalCon Europe 2020 has been announced - and DrupalCon is coming back to Barcelona from Sep 14-17th.

Our partnership with Kuoni Congress continues, and we're excited to join you in beautiful Spain to celebrate Drupal 9 together.

Kuoni Congress

Mark your calendars, and bookmark the site - more info coming soon!

Drupal.org Update

Have you unwrapped automatic updates yet?

In November we finished the primary engineering work to support Phase 1 of the Automatic Updates initiative. In December we completed validation and testing, and launched the first stable release of the Automatic Updates module.

In its current form, Automatic Updates is available as a contributed module for both Drupal 7 and Drupal 8. After installing the module you'll be able to take advantage of three new features: 

  • When the Drupal Security Team identifies a critical release, they'll be able to publish a PSA that will be directly displayed in your admin interface. 
  • The module will run automated readiness checks to make sure your site is ready for the update, or to let you know if there are errors you need to fix.
  • The module will automatically apply Drupal core updates. 

What about automatic updates for contributed modules and composer dependencies?

The next phase of work on the Automatic Updates initiative is to support updates for contributed modules, and for sites managing Composer dependencies.

This is where we need your help! We're looking for organizational sponsors to help move this work forward. If you're interested in helping us move the initiative into the next phase, please contact the Association.

We want to thank: The European Commission, Acquia, Tag1, Mtech, and Pantheon for their support of Phase 1.

Expanding Drupal Solution content with Feature pages

About two years ago we decided to start featuring Drupal Solutions on Drupal.org. These Solutions are examples of Drupal being used in the real world in specific use cases. Our first series of this content was the Drupal Industry pages, highlighting the power for Drupal in specific industry verticals.

In December, we've just launched our next set of content, this time focusing on specific features of Drupal that set it apart from the competition. These Feature pages talk about the specific Drupal Solutions that are built around key features of the software.

Updating our packaging pipeline

Do you know what goes into a packaged release of Drupal? It's not just a git clone - and as of Drupal 8.8.0 the package you download from Drupal.org also includes Composer scaffold files.  As Drupal evolves, the way we deliver the software to users has to evolve along with it.

To support the increasingly sophisticated packaging process for Drupal, we started work on overhauling our packaging pipeline for Drupal releases. This work continues into January.

Preparing for contrib Semver

As part of Drupal 9's release we are working to migrate all of the projects on Drupal.org to properly use semantic versioning. Right now, contributed modules typically use a version format like: 7.x-1.6 or 8.x-1.7. The first part of this is just the platform version (D7 vs. D8), and the second part is the Major version and the Patch version.

We'll be migrating this version schema so that the current Major version remains the Major version, the current Patch version becomes the Minor version, and we'll add the ability to define new patch versions.

This enables several improvements. Firstly, contrib maintainers can now follow backwards compatibility policies similar to core, i.e: Major versions with backwards compatibility breaking changes, minor version with new features, and patch versions for bug fixes and security releases.  Secondly, because contributed modules can now be compatible with both Drupal 8 and Drupal 9, contrib semver will be an important part of keeping version management sane.

We've made some initial progress in that direction, and have a roadmap for completing this support.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Dec 19 2019
Dec 19

Holiday gift: Automatic Updates

As of today, the Automatic Updates contributed module for Drupal 7 and Drupal 8 has had its first stable release! This represents the completion of Phase 1 of the Automatic Updates Initiative, and the culmination of more than 18 months of work.

___________

Please note: this phase 1 work provides automatic update coverage for many scenarios, but not for every Drupal use case.

With this phase 1 release you can:

  • See Drupal PSAs that the security team determines are highly critical directly in your administrative interface.
  • Run readiness checks to determine if your site is eligible for automatic updates (is the filesystem writable, no custom edits to updated files, etc).
  • Automatically update Drupal 7 and Drupal 8 sites
    • Current limitations
      • Updates Drupal core, but not contributed projects
      • Right now, can update only if the sites are not built or maintained with Composer 
      • Database updates are supported in Drupal 8, but not supported in Drupal 7 (if there are DB updates, an automatic update in D7 will be rolled back)

The Automatic Updates module will not be an official solution until the second phase is complete and the feature becomes part of Drupal Core. However, we believe that even as a contributed module Automatic Updates provides a significant improvement in maintenance and cost of ownership for a broad base of existing Drupal 7 and 8 sites, particularly those managed by small, independent site owners.

___________

The second phase of this initiative will focus on solving the remaining requirements and getting automatic updates ready for inclusion in Drupal core. The outstanding requirements include:

  • Support for Composer-built sites
  • Support for contributed modules and themes
  • Support for a more robust roll-back process
  • More robust support for database updates

We are currently seeking sponsors for the second phase of this initiative, so please reach out to us at the Drupal Association if you'd like to support this work!

Many thanks!

Thank you to these individuals:

And to these organizations:

Dec 17 2019
Dec 17

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Writing this here in the United States, after having enjoyed the Thanksgiving holiday with friends and family, reminds me of how much we have to be thankful for in the Drupal community. We have a wonderful international community with incredible talent and thoughtfulness that goes into building software that touches millions of people. #DrupalThanks.

Project News

Reminder: Drupal 8.8.0 has been released! 

With the release of Drupal 8.8.0 comes some great improvements, including a stable Media Library and new experimental Administration UI called Claro, and a new Composer scaffold that means that even sites started from the tar.gz archive will be Composer ready.

Time to get ready for Drupal 9

The time is now to get ready for Drupal 9; it'll be here before you know it. To learn the latest about Drupal initiatives, follow @DropisMoving.

If you are a site owner, the best way you can get ready is to make sure you're up to date on the latest version of Drupal 8. From there, it'll be an easy upgrade to 9.

If you maintain a community module, or your own custom modules, you may have some work to do.  Many contributed or even custom modules only need a one-line change to be ready for Drupal 9. Check yours using: the upgrade status module, or the Drupal Check command line tool.

Drupal.org Update

Happy Holidays! We've wrapped a gift for you.

All engineering work on the Automatic Updates contrib module for Drupal is now complete. Pending any last feedback on the release candidate, the stable release of the module is scheduled for this Thursday December 19th.

If you're interested in supporting Phase 2 of automatic updates, with an A/B front-end controller, contrib updates, and support for updating Composer-based site installs, please contact the Association.

We want to thank: The European Commission, Tag1, Mtech, Acquia, and Pantheon for their support of Phase 1. When the stable release goes live we'll have a dedicated post talking about what's included in this first stable release and more completely thanking all of the individuals and organizations that helped us get here.

Updates to Update Status XML

The update status XML that Drupal.prg provides to inform all Drupal sites of available updates has been overhauled to include better support for Drupal 9, and to prepare for contributed modules to use semantic versioning. It now also includes support for more descriptive support and security information.

Session browsing improvements for Events.Drupal.org

November also saw us deploy some significant improvements to the session browser on Events.Drupal.org. With the call for papers ended and sessions under evaluation, we wanted to prepare a new experience.

Session Filtering and Search

Once accepted sessions are published for Minneapolis, you'll see new search facets and a title/description search that should make matching attendees with programming better than ever.

Our hard work behind the scenes

A lot of the work that goes into supporting the Drupal project and community happens behind the scenes. Visible improvements like the Automatic Updates feature, or even new releases of modules are just the tip of the iceberg. To make these things possible, the Drupal Association engineering team must keep the whole machine humming.

A large portion of the Drupal Association engineering team's work in November was spent on improving the services and infrastructure we provide to support Drupal core.

Drupal 8.8.0's release and its changes to Composer meant a rewrite of Drupal.org's packaging pipeline, updates to the Composer facade, and updates to DrupalCI to prepare for testing Drupal 8.9.x.

Supporting the automatic updates process required deploying a new Signing oracle server to sign and verify the integrity of update packages, as well as provisioning and deploying Hardware Security Modules (HSMs) to protect our root keys.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Nov 11 2019
Nov 11

    DrupalCon Amsterdam

Select members of the Drupal Association team have just returned from a wonderful DrupalCon Amsterdam. The revival of the European DrupalCon was a tremendous success, and we want to thank all of the volunteers and sponsors who made it possible. 

The content at the conference was great as well. This year the core initiative leads provided a comprehensive update, followed by the traditional #Driesnote the next day, which laid out the vision for Drupal 9's key initiatives moving forward. The DA also shared an update from the Drupal Association board, held a community town hall, and provided our bi-annual update from the engineering team. 

While DrupalCon was action-packed, we also moved forward a lot of other initiatives on the community's behalf in October. 

Project News

Reminder: Drupal 8.8.0 is coming soon! Drupal 8

Drupal 8.8.0-beta1 was released in early November, to be followed by the full release in early December. A variety of great features are landing in version 8.8.0, including improved Composer support in core, an updated Media Library, and updates to JSON:API. This is likely to be the last minor release before the simultaneous release of Drupal 8.9.0 and Drupal 9.0.0 next June.  

If you want to help ensure a smooth release, we invite you to join the Drupal Minor Release beta testing program.

Time to get ready for Drupal 9

The time is now to get ready for Drupal 9. At DrupalCon Amsterdam the core initiative team put out the call for the community at large to get ready. 

If you are a site owner, the best way you can get ready is to make sure you're up to date on the latest version of Drupal 8. From there, it'll be an easy upgrade to 9. 

If you maintain a community module, or your own custom modules, you may have some work to do.  Many contributed or even custom modules only need a one-line change to be ready for Drupal 9. Check yours using: the upgrade status module, or the Drupal Check command line tool.

Drupal.org Update

Automatic Updates Initiative needs your help!

For the last year we've been working together with the European Commission to bring automatic updates to Drupal. The first phase of this work covers updates for Drupal Core only, and only in non-Composer scenarios, but even so it should be able to protect many Drupal 8 and especially Drupal 7 site owners. 

The module is ready for testing now. Your feedback is welcome to help us make this first phase stable for production use by the community. 

You can also help by supporting Phase 2 of this initiative, which will include more advanced features like support for Composer-based sites, database updates, and a robust roll-back capability. We're looking for sponsors for this next round of work now. 

Forming a Contribution Recognition Committee

The Drupal Association's contribution credit system is an industry first in open source, and so we want to take great care on each step on this new journey. 

During the conference we also announced the formation of a Contribution Recognition Committee to govern the contribution credit algorithm which weights the order of the Drupal services marketplace on drupal.org. 

We are now seeking applications from community members who would like to sit on the committee. 

When will we enable GitLab merge requests?

Drupal + GitLabWhen we migrated Drupal.org's git repositories to GitLab, it was the first step on the road to modernizing and improving the project's collaboration tools. The most significant step in that journey will be enabling merge requests, and we know it's a feature that the community has been waiting for. 

So what's the hold up? 

There are a few factors that have held us back from enabling the feature sooner. First, we were waiting for the GitLab team to add support for Git object de-duplication. Beta support for this feature was added in GitLab version 12.0, and then enabled by default beginning with the release of GitLab version 12.1.

While waiting for these features, the Drupal Association engineering team focused on other major commitments: moving forward the Automatic Updates initiative, in partnership with the European commission, co-leading the Composer Initiative to improve support in Drupal core, and preparing Drupal.org to support the release of Drupal 9. 

While these other initiatives have overtaken much of our internal capacity, we're hoping to get back to the merge request feature very soon, and we're just as excited to release the feature as you are to begin using it! 

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank: 

  • Zyxware - Renewing Signature Supporting Partner
  • EPAM - Renewing Premium Supporting Partner
  • Datadog - Premium Technology Supporter
  • KWALL - Renewing Classic Supporting Partner
  • ANNAI - Renewing Classic Supporting Partner
  • SymSoft - Renewing Classic Supporting Partner
  • Forum One - Renewing Classic Supporting Partner
  • Catalyst IT - Renewing Classic Supporting Partner
  • Old Moon Digital - Renewing Classic Supporting Partner
  • Authorize.Net - *NEW* Classic Technology Supporter
  • SiteGround - Renewing Classic Hosting Supporter

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Oct 11 2019
Oct 11

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Note from the author: "I'm back in the hot seat! As many of you know, I took on the role of interim executive director in September 2018 while the Drupal Association underwent an executive search. This summer we found a fantastic new leader in Heather Rocker, and now that she's had a few months to settle in, I'm able to return to my regular duties, like bringing you these updates. Thanks for your patience!"

- Tim Lehnen(hestenet)

September was a flurry of activity here at the Drupal Association at large. Coming off a season of travel to a number of Drupal events, we headed straight into our semi-annual staff off-site here in Portland, and followed that up by attending Google's second-ever CMS Leadership summit.

Despite the whirlwind of events taking place in September, we've also landed some major milestones on our roadmap, and are hard at work getting some exciting things together to talk about with you all at DrupalCon Amsterdam at the end of October. As an added bonus, this month's report includes a short retrospective about the impact of the GitLab migration on our maintenance work. 

Project News

Composer Initiative work committed for release in Drupal 8.8.0

 Drupal Composer Facade

A major community initiative for Drupal 8.8.0 has been the push to make Drupal's internal scaffolding and filetree consistent, whether you start using Drupal by using the .zip download, or by using Composer from the get-go. Starting with Drupal 8.8.0 - no matter how you start your journey with Drupal, you'll be ready to use Composer's advanced dependency management when you need it.

Drupal Association engineering team member Ryan Aslett(mixologic) has been the initiative lead for this effort for more than a year. We're thrilled that his work and the work of many volunteers has been committed for release in Drupal 8.8.0!

We want to thank the following contributors for participating in this initiative in collaboration with us: 

The work is not over! There are still a number of clean ups and refinements being worked on in the Drupal core queue, and the Drupal Association team is working hard in October to ensure that Drupal.org itself will be ready to deliver these Composer-ready packages of Drupal 8.8.0 on release. 

Reminder: Drupal 8.8.0 is coming soon! Drupal 8

Speaking of Drupal 8.8.0 - it enters the alpha phase during the week of October the 14th, in preparation for release in December of this year.

Drupal 8.8.0 is the last minor release of Drupal before the simultaneous release of Drupal 8.9.0 and 9.0.0 next year. You can find more information about the Drupal release cycle here.

If you want to help ensure a smooth release, we invite you to join the Drupal Minor Release beta testing program.

Drupal.org Update

Preparing our infrastructure for Automatic Updates

In September we spent a good amount of time outlining the architectural requirements that will need to be met in order to support delivering the update packages that are part of the Automatic Updates initiative.

We are only in the first phase of this initiative, which focuses on: 1) Informing site owners of upcoming critical releases, 2) Providing readiness checks that site owners can use to validate they are ready to apply an update, and 3) offering in-place automatic updates for a small subset of use-cases (critical security releases).

As this initiative progresses, and begins to cover more and more use cases, it should greatly reduce TCO for site owners, and friction for new adopters. However, to make that forward progress we are seeking sponsors for the second phase of work.

Readying our secure signing infrastructure

Yubikey HSMWith the help of a number of community contributors (see below), a new architecture for a highly secure signing infrastructure has been laid out. As we roll into Q4 we'll get ready to stand this new infrastructure up and begin securing the first automatic updates packages.

Going into early October, a number of contributors came together at BadCamp to help advance this effort further. Without the collaboration between community members and Drupal Association staff, these initiatives would not be possible.

We'd like to thank the following contributors to the Automatic Updates/Secure Signing Infrastructure initiative: 

Supporting Drupal 9 readiness testing

In conjunction with the Drupal core team, the DA engineering team has been supporting the work to ensure that contributed projects are ready for the release of Drupal 9.

Early testing has shown that over 54% of projects compatible with Drupal 8 are *already* Drupal 9 ready, and we'll be continuing to work with the core team to get out the word about how to update the modules that are not yet compatible.

Infrastructure Update

A brief retrospective on the GitLab migration

Drupal + GitLabDrupal.org's partnership with GitLab to provide the tooling for Drupal and the ~40,000 contributed projects hosted on Drupal.org has been a significant step forward for our community. We're no longer relying on our own, home-brew git infrastructure for the project, and we're gradually rolling out more powerful collaboration tools to move the project forward. 

But what has that meant in terms of maintenance work for the Drupal Association engineering team?

There was some hope as we were evaluating tooling providers that making a switch would almost entirely eliminate the maintenance and support burden. While that was a hopeful outlook, the reality is that maintaining 'off-the-shelf' software can be at least as much work as maintaining mature existing tools.

GitLab in particular is still iterating at a tremendously rapid pace, releasing updates and new features every month. However, that speed of development has also meant frequent maintenance and security releases, meaning the DA team has had to update our GitLab installation almost once a week in some months.

Does that mean we're unhappy with the change? Absolutely not! We're still thrilled to be working with the GitLab team, and are excited about the new capabilities this partnership unlocks for the Drupal community (with more coming soon!).

However, it is a good lesson to anyone running a service for a large community that there's no free lunch - and a great reminder of why the support of Drupal Association members and supporting partners is so essential to our work.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank:

  • Tag1 - Renewing Signature Supporting Partner
  • Gitlab - *NEW* Premium Technology Supporter
  • Four Kitchens - Renewing Premium Supporting Partner
  • Phase2 - Renewing Premium Supporting Partner
  • WebEnertia - Renewing Premium Supporting Partner
  • Thunder - Renewing Premium Supporting Partner
  • Palantir -Renewing Premium Supporting Partner
  • Specbee - Renewing Premium Supporting Partner 
  • Pantheon - Renewing Premium Hosting Supporter
  • Cyber-Duck - *NEW* Classic Supporting Partner
  • Websolutions Agency - *NEW* Classic Supporting Partner
  • Unic - *NEW* Classic Supporting Partner
  • Kalamuna - Renewing Classic Supporting Partner 
  • ThinkShout - Renewing Classic Supporting Partner 
  • Amazee - Renewing Classic Supporting Partner 
  • Access - Renewing Classic Supporting Partner 
  • Studio Present - Renewing Classic Supporting Partner 
  • undpaul- Renewing Classic Supporting Partner 
  • Mediacurrent - Renewing Classic Supporting Partner 
  • Appnovation - Renewing Classic Supporting Partner 
  • Position2 - Renewing Classic Supporting Partner 
  • Kanopi Studios - Renewing Classic Supporting Partner 
  • Deeson - Renewing Classic Supporting Partner 
  • GeekHive - Renewing Classic Supporting Partner 
  • OpenSense Labs - Renewing Classic Supporting Partner 
  • Synetic - Renewing Classic Supporting Partner 
  • Axelerant - Renewing Classic Supporting Partner 
  • Centretek - Renewing Classic Supporting Partner 
  • PreviousNext - Renewing Classic Supporting Partner 
  • UniMity Solutions - Renewing Classic Supporting Partner 
  • Code Koalas - Renewing Classic Supporting Partner 
  • Vardot - Renewing Classic Supporting Partner 
  • Berger Schmidt - Renewing Classic Supporting Partner 
  • Authorize.Net - Renewing Classic Technology Supporter
  • JetBrains - Renewing Classic Technology Supporter
  • GlowHost - Renewing Classic Hosting Supporter
  • Sevaa - Renewing Classic Hosting Supporter
  • Green Geeks - Renewing Classic Hosting Supporter

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Oct 09 2019
Oct 09

There has been a lot of thoughtful debate about the Drupal project's contribution credit algorithm in recent weeks, and some great ideas have been proposed. In the meantime, however, we've also been monitoring concerns about gaming of the current algorithm. Making changes to the contribution credit algorithm must be done with care, because it has a significant impact on how we recognize contributions and also on the ecosystem of Drupal service providers. Today the Drupal Association engineering team deployed a quick tune-up to the contribution credit algorithm.

This change strengthens the existing multiplier which weighs issue credits based on the usage of the project. Issues on more highly used projects will now be weighted even higher, and those on lower usage projects will be weighted lower.

We're calling this a tune-up with good reason. This is not a fundamental update of the credit system; this is a small change to help resolve some recent issues with the current version of the algorithm.

Developing a true 'Contribution Credit 2.0' system is a much larger project, but one we're hoping to undertake soon.

In the meantime:

  • If you have feedback on the small change we made today, you can find the issue here.
  • If you'd like to propose new ideas about the next generation of the system, please do so in this issue.

As the community thinks about the contribution credit system, we encourage you to remember that there will always be a human element to recognizing contribution - and our goal should be a system that enables better interaction between project contributors.

Jun 27 2019
Jun 27

Drupal Association appoints Executive Director

We announced on April 30, 2019 that Heather Rocker has been selected as the next Drupal Association Executive Director.

Heather Rocker joins the Drupal Association with a strong history of leadership in technology and the nonprofit world. She was the first executive director of the Women in Technology Foundation, as well as the CEO of Girls Incorporated of Greater Atlanta. Most recently she acted as the Managing Principal of Systems Evolution, Inc. in Atlanta, Georgia. She also serves as Immediate Past-President of the Board of Directors of Georgia FIRST Robotics.

GitLab phase 1 Completed!

For the past 18 months, interoperability between the current Git infrastructure and migration to GitLab was a much larger effort than expected.  With several challenges: optimization of project syncing, disaster recovery, replication, high availability, and phasing out our old Git servers, to name a few. Each member of the Engineering Team has put a lot of time and effort in testing and addressing these challenges. We thank GitLab, The Drupal Association Engineering Team, and Partners for helping us achieve this launch date.

I hope you are excited as we are in achieving this milestone. 

The goals of phase 1 were to:

●      Transparently replace Drupal’s current underlying Git infrastructure (as part of this migration git.drupal.org’s SSH host key will change) for: repository hosting, maintainer permissions, code viewing with GitLab repositories, GitLab roles and permissions for maintainers, and the GitLab code viewing UI.

●      Enable inline code editing for project maintainers.

●      To unblock allowing pull requests, inline code editing for all contributors, and more in phase 2.

Read more about the Drupal Association partnership with GitLab at:  https://www.drupal.org/drupalorg/blog/developer-tools-initiative-part-5-gitlab-partnership

Please feel free to review our FAQs about GitLab at:  https://www.drupal.org/drupalorg/docs/gitlab-integration/gitlab-frequently-asked-questions

If you have questions, please join us in Drupal Slack on channel #gitlab.

Beginning of Gitlab Phase 2

This phase is focused on:  enabling merge requests, inline code editing, and web-based code review.  Phase 2 will enable benefits to developer velocity and collaboration:

●      By adding merge requests, contributing to Drupal will become much more familiar to the broad audience of open source contributors who've learned their skills in the post-patch era.

●      By adding inline editing and web-based code review, it will be much easier to make quick contributions. This not only lowers the barrier to contribution for people new to our community, it also saves significant effort for our existing community members, as they'll no longer need to clone work locally and generate patches.

●      Finally, by creating a tight integration between the Drupal.org issue queues and GitLab's development tools, we'll be able to transition to this new toolset without disrupting the community's existing way of collaborating.

The Engineering team and our partners are currently evaluating the timeline for Phase 2 and will update our issue tracker once it’s established.

Drupal Steward Program

The program will focus on protection that closes the gap between security releases and site updates. The Security Team and the Drupal Association have been developing this program for over a year.

The goals are:

●      Provide a new service to the Drupal community, from small site owners to enterprise-scale end users, to protect their sites in the gap from security release to the time it takes them to patch.

●      Create a new model for sustainability for the Security Team, generating funding that 1) covers the operating costs of the program 2) can support security team operations and 3) can support additional Drupal Association programs.

Drupal Steward will offer sites a form of mitigation through the implementation of web application firewall rules to prevent mass exploitation of some highly critical vulnerabilities (not all highly critical vulnerabilities can be protected in this fashion, but a good many can be—this method would have worked for SA-CORE-2018-002 for example).

It will come in three versions:

●      Community version - for small sites, low-budget organizations, and nonprofits, we will offer a community tier, sold directly by the DA. This will be effectively at cost.

●      Self-hosted version - for sites that are too large for the community tier but not hosted by our vendor partners.

●      Partner version - For sites that are hosted on vetted Drupal platform providers, who have demonstrated a commitment of contribution to the project in general and the Security Team in particular, protection will be available directly through these partners.

We will announce more details of the program over the coming weeks and months as we get it up and running.

If you are a hosting company and are interested in providing this service to your customers, please reach out to us at [email protected].


Updates to Drupal.org metrics and visualizations

Please visit our new graphical display of project usage information at:  https://www.drupal.org/project/usage/drupal. The usage graph now displays a stacked chart by version. In the long term, we have plans to revise how our usage data is tabulated to provide more accurate and useful data. 

Changes in credit attribution

We have made available by default giving commenters credit attribution as we believe comments are an important part of the development process. Although we deployed this successfully, we had to roll this back so we can refine our guidelines a bit. For now this issue is in backlog and expect to have our guidelines by the end of the year.

Introducing Community Group Sections in the Community Portal

/Community has been expanded to include more targeted sections upon landing at the /Community page. We’ve defined these sections in groups. We also have a framework that will enable these groups to tell us:

  • Who they are
  • What they do
  • How they work
  • Their latest updates
  • How you can get involved

Some of the early adopters that are now posted are:

Each section will have “standard” home page content, detailing the info above, as many content pages as the group can muster, and a blog that will go on to Drupal Planet.

Drupal 7 Extended Support - request for partners

With the upcoming release of Drupal 9 in June 2020, it's time to start thinking about migration. Drupal 7 will be supported through November 2021, but after that any end-users who haven't yet updated will want to partner with a Drupal 7 Extended Support vendor.

The Drupal Security Working Group is accepting applications for vendors who want to be part of the extended support program.

A renewed push on the Composer Initiative

At DrupalCon Seattle there was also renewed interest in the Composer Initiative—this initiative seeks to rationalize the way that Composer is used in Core, to make it easier to manage sites using Composer, and also to make it easier for sites set up using a non-Composer method to be converted to using Composer, something increasingly important as contributed modules add Composer dependencies.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank:

  • Srijan - Renewing Signature Supporting Partner
  • FFW - Renewing Signature Supporting Partner
  • Sixeleven - Renewing Signature Supporting Partner
  • Acquia - Renewing Signature Hosting Supporter
  • Nexcess.net - *NEW* Signature Hosting Supporter
  • Specbee - *NEW* Premium Supporting Partner
  • EPAM - Renewing Premium Supporting Partner
  • Elevated Third - Renewing Premium Supporting Partner
  • publicplan - Renewing Premium Supporting Partner
  • CI&T - Renewing Premium Supporting Partner
  • Capgemini - Renewing Premium Supporting Partner
  • Zoocha - Renewing Premium Supporting Partner
  • 1XInternet - Renewing Premium Supporting Partner
  • Message Agency - Renewing Premium Supporting Partner
  • Lullabot - Renewing Premium Supporting Partner
  • Aten - Renewing Premium Supporting Partner
  • Contegix - Renewing Premium Hosting Supporter
  • Platform.sh - Renewing Premium Hosting Supporter
  • Contegix - Renewing Premium Hosting Supporter
  • KWALL - Renewing Classic Supporting Partner
  • ANNAI - Renewing Classic Supporting Partner
  • Softescu - Renewing Classic Supporting Partner
  • SymSoft - Renewing Classic Supporting Partner
  • Vardot - Renewing Classic Supporting Partner
  • Forum One - Renewing Classic Supporting Partner
  • Berger Schmidt - Renewing Classic Supporting Partner
  • Catalyst IT - Renewing Classic Supporting Partner
  • Old Moon Digital - *NEW* Classic Supporting Partner
  • AddWeb Solution - *NEW* Classic Supporting Partner
  • Urban Insight - *NEW* Classic Supporting Partner
  • Asentech - *NEW* Classic Supporting Partner
  • One Shoe - Renewing Classic Supporting Partner
  • Hook 42 - *NEW* Classic Supporting Partner
  • Electric Citizen - Renewing Classic Supporting Partner
  • Beaconfire RED - Renewing Classic Supporting Partner
  • University Of Virginia - *NEW* Classic Supporting Partner
  • Bear Group - *NEW* Classic Supporting Partner
  • Dropsolid - *NEW* Classic Supporting Partner
  • ezCompany - *NEW* Classic Supporting Partner
  • Spry Digital - Renewing Classic Supporting Partner
  • Perpetuum - *NEW* Classic Supporting Partner
  • Factorial - Renewing Classic Supporting Partner
  • Acro Media - Renewing Classic Supporting Partner
  • Third & Grove - Renewing Classic Supporting Partner
  • Digital Circus - Renewing Classic Supporting Partner
  • Bounteous - Renewing Classic Supporting Partner
  • Cybage - Renewing Classic Supporting Partner
  • QED42 - Renewing Classic Supporting Partner
  • Digitalist Group - *NEW* Classic Supporting Partner
  • Red Crackle - *NEW* Classic Supporting Partner
  • SeeD - Renewing Classic Supporting Partner
  • Last Call - Renewing Classic Supporting Partner
  • CivicActions - Renewing Classic Supporting Partner
  • Morpht - Renewing Classic Supporting Partner
  • Tata Consultancy Services - Renewing Classic Supporting Partner
  • Wunder - Renewing Classic Supporting Partner
  • Promet Source - Renewing Classic Supporting Partner
  • Unleashed Technologies - Renewing Classic Supporting Partner
  • Digital Echidna - Renewing Classic Supporting Partner
  • Evolving Web - Renewing Classic Supporting Partner
  • Adapt - Renewing Classic Supporting Partner
  • Microserve - Renewing Classic Supporting Partner
  • ImageX - Renewing Classic Supporting Partner
  • Therefore - Renewing Classic Supporting Partner
  • Sevaa Group - Renewing Classic Hosting Supporter
  • DRUD - Renewing Classic Hosting Supporter
  • Smartling - Renewing Classic Technology Supporter

If you would like to support our work as an individual, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Dec 18 2018
Dec 18

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

DrupalCon

Give thanks with your Drupal Family at DrupalConDrupalCon Seattle | April 8-12 2019 - Register today!

What better way to connect with your Drupal family and give thanks for Drupal's impact than at DrupalCon? Still need to register? Coming on your own? Now's a great time to lock in a good price.

If your organization is sponsoring your trip, consider investing those end of year budgets in your registration... and invest in Drupal's success while you do it. Your team can also sign up for or renew your Supporting Partnership for steep discounts on ticket prices.

The schedule is available now; check out the specialty sessions and register before prices go up!

Kicking off planning for DrupalCon Amsterdam

Members of the Drupal Association team traveled to Europe to meet with Kuoni Congress and, the DrupalCon Europe advisory committee do a kick-off meeting and deep dive on the event planning for DrupalCon Amsterdam. This was our opportunity to dive deep into the event with the team, and it was a tremendously productive 2-day session.

More news about Amsterdam will be coming soon, so check back at https://events.drupal.org/amsterdam2019 soon!

Kuoni Meeting

(Image courtesy of Baddy Breidert)

Drupal.org Updates

New telemetry data about Drupal usage

In November we also re-architected the way we parse data from sites that call back to Drupal.org for updates. This allowed us to learn more about how Drupal is used in the wild. This graph shows the current distribution of PHP versions for Drupal 8 sites. Notably only about 20% of Drupal 8 sites are still using PHP 5, so the migration effort for the community may not be as big as some expected when PHP 5 reaches end of life.

Drupal 8 PHP versions

Finding a new Technical Program Manager

As Tim has stepped into the role of interim executive director, we've been looking to bring a new team member onboard to backfill some of his technical responsibilities. In November we interviewed candidates for our Technical Program Manager position. We're excited to have our new team member join in the new year!

Drupal.org/community changes live!

The changes outlined to the Drupal.org community home page that we outlined in our October update are now live. This new entry point to the Drupal community addresses the many different needs that a new member of our community might have, and the different personas that they might represent. The home of the community will continue to evolve over time, so expect to see more updates soon, and please offer your feedback here.

Drupal Association UpdatesLehman Associates

Executive search firm selected

As you know, we've begun the process of looking for our next Executive Director. In November we interviewed executive search firms to help us with this process, and in early December we announced that we've selected Lehman Associates to help us with our search. If you would like to read the candidate profile, or contact Lehman Associates to offer candidate suggestions or provide other feedback, please use the button below.

View the profile

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Dec 18 2018
Dec 18

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Lock in your DrupalCon tickets before the end of the yearDrupalCon Seattle | Apriul 8-12 2019 - Register today!

DrupalCon Seattle is shaping up to be an outstanding conference. If your organization is sponsoring your trip, now's a great time to use your 2018 budget to register to attend. Your team can sign up for or renew your Supporting Partnership for steep discounts on ticket prices. Coming to DrupalCon on your own? The schedule is available now, so peruse the offerings and register before prices go up!

Drupal.org Updates

A new taxonomy for DrupalCon sessions

As you've seen if you clicked the link to the schedule above, events.drupal.org was updated to support session submission by tag, rather than track, earlier this year. This provides more flexibility in finding the content you're interested in, and encourages sessions which cross the boundaries of traditional tracks.

Prototyping a new Try Drupal experience

In October we put together a visual prototype of our proposed revamp of the Try Drupal program. This includes a better, more targeted user experience for each persona, as well as the opportunity for more organizations to participate. More details will be shared soon as we get further along, but for a sneak preview you can review the operational update from our recent public board meeting.

Improving the experience of using Composer

In October significant progress was made on the initiative to Improve Drupal Core's use of Composer. In particular, kicking off the primary issue for building this better support into Core, as well as moving the issue for supporting Semantic Versioning for Contrib from a plan to the implementation phase. These changes will improve the user experience for Drupal users with composer based workflows, and especially for Drupal users who start sites without Composer, and then switch to Composer based workflows. This also lays the groundwork for necessary steps for supporting the Drupal 9 roadmap.

Promote Drupal

Releasing the first draft of the Drupal Brand Book

In October, with the feedback of the Promote Drupal volunteer team, we developed and released the initial draft of the Drupal Brand book. This is one of the materials created by the Promote Drupal initiative, in order to unify the brand presentation for Drupal across agencies, internal sales, and regions. This will be updated with a vision statement for Drupal's business strategy and market position.

Initial draft of Drupal brand book

A new Community Section

In October we also spent time creating a beta experience for a new Drupal.org/community landing page. This page focuses on the onboarding process, helping visitors identify their need and persona, so they can get to the segment of the community that is relevant to them. (Hint: this beta experience has since gone live!) If you have feedback about making the community portal better, you can leave your suggestions in the drupal_org_community issue queue.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Dec 07 2018
Dec 07

At Drupal Europe in September, we were very pleased that project founder Dries Buytaert highlighted a visual prototype of our upcoming integration with GitLab in his keynote.

This video outlines the migration phases that we discussed in the announcement of our partnership with GitLab. Our migration window for Phase 1 is targeted for the first weeks of January, and we hope Phase 2 to be completed shortly in the beginning of 2019.

So what has it taken to get this integration working between September and now?

We're now in the midst of serious migration testing. Testing and re-testing the process in our staging environment, putting load testing in place to stress test our integration, and doing user-validation testing to ensure that the workflows affected by this integration are working as expected.

All in all, we're thrilled with the progress, and very thankful for GitLab's close collaboration. We're excited to be moving the Drupal project to its next generation tooling soon. Once Phase 1 of our migration is complete, it'll be time for Phase 2 and our community will start seeing some tremendous improvements in efficiency and collaboration.

Cheers!

Tim Lehnen

Drupal Association - Executive Director

Oct 05 2018
Oct 05

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Drupal.org Updates

Drupal Europe 10-14 Sep 2018Thank you to the Drupal Europe team and attendees!

Members of the Drupal Association team joined the community to attend Drupal Europe in September. It was a fantastic event, and we had many great conversations with local community leaders, Supporting Partners, and others about the challenges and opportunities of the European market.

The Drupal.org Engineering Team also met with a number of contributors at Drupal Europe to move forward initiatives like improving Composer support for core, automatic updates, and more.

Reminder: DrupalCon Seattle Early Bird Registration is open nowDrupalCon Seattle April 8-12 2019

DrupalCon Seattle general registration is open now. The programming has been transformed to address the needs of Builders, Content and Digital Marketers, Agency Leaders, and Executives, while preserving that feeling of homecoming for the community that is central to every DrupalCon.

Have questions about the next evolution of DrupalCon? Head of Events Amanda Gonser was recently interviewed by Lullabot for their podcast and explains what's new and what's staying the same.

A video prototype of our integration with GitLab

Are you as excited as we are about the upcoming migration to GitLab? Watch this video for a visual prototype of the integration we're planning.

We should be announcing a window for our Phase 1 migration shortly.

Phase 1 of Improved Support for Composer begins

In September we moved forward with our multi-phase proposal for improving Drupal core's support for Composer workflows. There are still considerations under discussion, such as how to handle multi-site support, and the implementation details of the later phase. However, Phase 1 has now been broken into its own meta issue, with a goal of bringing these changes into Drupal as of release 8.7.

Seeking a Technical Program Manager

The Drupal Association seeks a Technical Program Manager (TPM) to join our Engineering team and shepherd key programs for Drupal.org that empower our global community to collaborate and build the Drupal project. A TPM is expected to be technically fluent, have excellent project management skills, and excel in internal and external communication. The Drupal Association serves one of the largest global open source communities — Drupal has pioneered open source for 17 years. Join our incredible, mission-driven team and make an important impact by building the tools that help our community build Drupal.

Join Promote Drupal

At Drupal Europe we kicked off the volunteer coordination for Promote Drupal. We've put together an introductory video that explains how to get your marketing teams involved.

Just go to the Promote Drupal landing page to sign up!

Further improvements for inclusivity on Drupal.org

Thanks to the community contributed work of @justafish, among others, Drupal.org user profiles now include fields for pronouns, primary language, and location, to help give people cultural context as they interact with each other online. We’ll be adding options to show this information with comments throughout Drupal.org.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Sep 06 2018
Sep 06

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Announcements

The Drupal Association announces our partnership with GitLab

In August we were pleased to announce a migration of the Drupal.org developer tools to GitLab.

This will be a three phase process, and we'll be announcing more soon at Drupal Europe and in future blog posts.

We want to say a special thank you to contributor Michael Hess, who volunteered hundreds of hours toward this initiative over the past year (and more).

Git remote URL changes for full projects and sandboxes

To support the migration to GitLab, Git authentication methods for Drupal.org hosted projects are changing.

In particular we have:

  • Deprecated password authentication for Git
  • Deprecated the full project Git remote format
    <username>@git.drupal.org/project/<yourproject>.git
    in favor of:
    [email protected]:project/<yourproject>.git
  • HTTPS clone urls for full projects will not be changing.
  • Sandbox URLs have changed from: <username>@git.drupal.org:sandbox/<author-username>/<project-node-id>.git
    to
    [email protected]:sandbox/<author-username>-<project-node-id>.git
  • HTTPS clones sandbox URLs have changed from:
    https://git.drupal.org/sandbox/<author-username>/<project-node-id>.git
    to
    https://git.drupal.org/sandbox/<author-username>-<project-node-id>.git

We have updated the version control instructions for Drupal.org projects, and put a message in our Git server for any user who makes a push using the deprecated format.

For more information, please review: https://drupal.org/gitauth

Drupal Europe 10-14 Sep 2018

Reminder: Drupal Europe is coming up soon

Drupal Europe is coming up in less than a week! Drupal Europe will be the largest gathering of the Drupal community in Europe and is a reimagining of this important community event as both technical conference and family reunion. The Drupal Association engineering team will be attending to connect with the community, provide updates on Drupal.org, and listen to some of the incredible speakers who will be in attendance.

Join the community in Darmstadt, Germany, from September 10-14, 2018. Make sure to register, book your travel, and secure accommodation: http://drupaleurope.org/

DrupalCon Seattle April 8-12 2019

Reminder: DrupalCon Seattle Early Bird Registration is open now

DrupalCon Seattle general registration is open now. The programming has been transformed to address the needs of Builders, Content and Digital Marketers, Agency Leaders, and Executives: https://events.drupal.org/seattle2019/registration

Please note that ticket prices have changed—and now Supporting Partners will receive a discounted rate compared to non-partner attendees. Joining the Supporting Partner program is a great way to support the Drupal project and the Drupal Association.

Drupal.org Updates

Drupal.org issues can be sorted by # of followers

To help Drupal.org project maintainers understand what issues are most important to people, the issue queues can now be sorted by the number of people following each issue. Here's an example of the Drupal Core issue queue sorted by follower count.

Documentation landing page updated

The Documentation Initiative was formed at DrupalCon Nashville to help improve the onboarding experience for users new to Drupal. A lot of work has been going on in the background, but one of the more visible changes is the new Documentation landing page.

Stable Download link and updated Try Drupal menu

Drupal 8.6 ships with an incredible new Quickstart feature—and to make the process of trying out Drupal as simple as possible, there is now a stable url to retrieve Drupal's latest release.

Similarly, we've updated the top navigation of Drupal.org. The Try Drupal button which appears at the top of every page now offers users the choice of trying an online demo or downloading the software directly.

Travel and Tourism industry page launched

Drupal has a powerful place in the Travel and Tourism space, in use everywhere from Princess Cruises to AAA. To promote Drupal's power in this industry we've launched our Travel and Tourism page.

Decoupled Solutions industry page launched

Perhaps the most powerful implementations of Drupal are Decoupled solutions, relying on Drupal as a stable back-end for content delivery to multiple endpoints with different front-end architectures. This approach is still new to many, our Decoupled industry page helps educate the market.

Staff Updates

Welcome Angie Sabin, Director of Operations

We're also very pleased to welcome Angie Sabin to the team. Angie comes from a non-profit background and is joining the Association to help us take finance and operations to the next level.

Welcome Kelly Delaney, Customer Success Coordinator

The Drupal Association is pleased to announce that Kelly Delaney has joined our team. As our new customer success coordinator, she'll work closely with supporting partners and sponsors to ensure that their needs are met and benefits are fulfilled.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.
Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Aug 16 2018
Aug 16

This is the fifth post in our series about integrating Drupal.org with a 3rd party developer tooling provider:

In this post we are announcing our migration to a new tooling provider, and outlining the phases of that migration process to take place over the next several months.

Announcing our partnership with GitLab

Wait, what?

Drupal Association + GitLab

Yes, in our four part series from December of last year it certainly looked like we were going in a different direction for the future of Drupal's developer stack.

So what changed?

Last year we laid out a model for integrating Drupal.org with a third party tooling provider, which we described as "Drupal Flow". This model was deliberately laid out to be agnostic to the provider we chose, so long as certain requirements were met. We worked with representatives from three potential providers at the time: GitHub, GitLab, and BitBucket, and each one had pros and cons. Once we had completed our evaluation, BitBucket was the only provider without hard blockers to the integration we wanted to build.

However, following our blog series, the GitLab team reached out directly to the Drupal Association team, and asked us to give them the chance to resolve the blockers and close the gaps in our integration.

At the same time, we saw an outpouring of feedback from our community asking us to see if we could find a way to make GitLab work.

And so we did.

The Agreement

For the past six months we've been working closely with Eliran Mesika, the Director of Partnerships at GitLab, in addition to CEO Sid Sijbrandij and members of GitLab's engineering team. They've escalated the internal priority of issues that blocked our adoption of GitLab, offered technical and financial support for the migration, and made a commitment to ongoing support for the Drupal project.

And so we're happy to announce that Drupal.org is going to be moving our code collaboration tools for our forty-five thousand projects to GitLab over the course of the coming months.

Three Phases to the Migration

Phase 1: Replacing Drupal.org's Git backend

The first phase of the Drupal.org migration

  • Transparently replace Drupal’s current underlying Git infrastructure (for repository hosting, maintainer permissions, code viewing) with GitLab repositories, GitLab roles and permissions for maintainers, and the GitLab code viewing UI.
  • Enable inline code editing (only for maintainers for this phase).
  • During this phase, Drupal.org will remain the primary source of information.  SSH keys, new projects, etc. will be created on Drupal.org.

This first phase, while modest, will bring some concrete benefits to the project:

  • Maintainers will be able to begin familiarizing themselves with GitLab's code collaboration tools.
  • Code viewing will receive a significant upgrade from CGIT to GitLab's built-in code viewer.
  • And Drupal.org's old Git server will be phased out.

Phase 2: Enabling Merge Requests, Inline Code Editing, and Web-based Code Review

  • The timeline for Phase 2 is dependent on GitLab’s resolution of a diskspace deduplication issue, which they have committed to on our behalf: https://gitlab.com/gitlab-org/gitlab-ce/issues/23029
  • Enable GitLab Merge Requests, GitLab inline code editing in the web UI, and GitLab web-based code review.
  • During this phase, Drupal.org will handle any 'create branch/merge request' integrations from the Drupal.org Issue queues, and related call-backs from GitLab into the Drupal.org issue comment stream.

Phase 2 is where we realize some tremendous benefits to developer velocity and collaboration:

  • By adding merge requests, contributing to Drupal will become much more familiar to the broad audience of open source contributors who learned their skills in the post-patch era.
  • By adding inline editing and web-based code review, it will be much easier to make quick contributions. This not only lowers the barrier to contribution for people new to our community, it also saves significant effort for our existing community members, as they'll no longer need to clone work locally and generate patches.
  • Finally, by creating a tight integration between the Drupal.org issue queues and GitLab's development tools, we'll be able to transition to this new toolset without disrupting the community's existing way of collaborating.

Phase 3: Evaluating Additional Features

Phase 3 has no strict timeline, but will be dependent on feedback from the community as they get up to speed on using the new GitLab-based contribution workflow for Drupal.

  • Evaluate additional features such as:

    • Integrating or replacing DrupalCI with GitLab CI
    • Enabling GitLab issues for a sub-set of projects
    • Enabling GitLab confidential issues for specific use-cases (security releases)
    • Possible MatterMost integration, etc.

These additional features may allow us to further improve the velocity of the Drupal project, or realize additional cost savings for the association. For example, we may be able to use GitLab's test runner integration to orchestrate tests across a wider variety of cloud platforms, helping us find the best pricing. We may be able to entirely replace security.drupal.org with a private issue tracker, eliminating an entire sub-site for the Drupal.org team to maintain. We may even be able to enhance existing community services like SimplyTest.me by integrating features like GitLab's AutoDevops tools to automatically create review environments for issues or branches.

We won't really know what's possible within the scope of our resources until the first two phases are completed, but this helps to show that by hitching our toolset to a partner that specializes in collaboration, we may be able to realize even more benefits for our community.

Changes to Git Remotes

  • Git remote urls for pushes to full projects have changed:
    • If you have an established Git remote in the format
      <username>@git.drupal.org:project/<yourproject>.git
      the format should be changed to:
      [email protected]:project/<yourproject>.git

  • HTTPS clone urls for full projects are unchanged.
  • HTTPS clone urls and Git remote urls for sandbox projects have changed:
    • For remotes of the format:
      <username>@git.drupal.org:sandbox/<username>/<node-id>.git
      the format should be changed to:
      [email protected]:sandbox/<username>-<nodeid>.git
    • Clone urls will be changing from:
      https://git.drupal.org/sandbox/<username>/<nodeid>.git
      to the format:
      https://git.drupal.org/sandbox/<username>-<nodeid>.git

Important: If you have any automated systems which authenticate to Git, such as CI pipelines or repo mirroring, ensure they are updated as well.

For more detailed information about these changes, as well as instructions for changing your Git remotes or setting ssh keys, please consult these instructions: https://drupal.org/gitauth

How to follow our progress

Issues for the Drupal.org migration to GitLab will be opened in the Drupal.org Infrastructure queue and tagged 'GitLab'.

For questions or concerns, please create an issue at https://www.drupal.org/node/add/project-issue/infrastructure

 

Learn more on the GitLab Blog

You can learn more about the decisions that brought us to this point on the GitLab blog.

Aug 07 2018
Aug 07

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

AnnouncementsDrupal Association Logo

Git remote URL changes for full projects and sandboxes

Git authentication methods for Drupal.org hosted projects are changing as we approach upgrading our developer tooling stack.

In particular we are:

We have updated the version control instructions for Drupal.org projects, and put a message in our Git server for any user who makes a push using the deprecated format.

For more information, please review: https://drupal.org/gitauth

Drupal Europe

Reminder: Drupal Europe is coming up soon

Drupal Europe is coming up in less than 40 days! Drupal Europe will be the largest gathering of the Drupal community in Europe and is a reimagining of this important community event as both technical conference and family reunion. The Drupal Association engineering team will be attending to connect with the community, provide updates on Drupal.org, and listen to some of the incredible speakers who will be in attendance.

Join the community in Darmstadt, Germany, from September 10-14, 2018. Make sure to register, book your travel, and secure accommodation: http://drupaleurope.org/

We want your feedback on ideas for Drupal Core

The Drupal Association has proposed several initiatives for Drupal Core - but before they can be officially adopted they need feedback from stakeholders in the community (even if it's just a "+1") and to reach community RTBC. Here are the proposals:

Drupal.org Updates

Staff retreat

In July the Drupal Association gathered together in Portland Oregon for our bi-annual staff retreat. At these retreats we discuss the progress made in the last six months, and our prioritization as an organization going into the next six month period.

Hightech users of Drupal

Hightech industry page launched

Drupal is the CMS of choice for a variety of companies in the high tech space, including organizations like Redhat, Cisco, and Tesla. Whether it is used in a front-facing application, as a decoupled back-end, or for an internal intranet experts in hightech defer to Drupal's example for their needs.

We launched a new industry page featuring these stories from high tech in July.

Drupal.org API updated for security advisories

To improve the automated toolchains built by organizations and individuals in the community to watch for new security advisories, we've updated the Security Advisory API. One of these changes ensures that the full canonical identifier for each advisory is included in the API data, which is a small but valuable change for anyone monitoring the API for advisory information.

Social Media Sharing for Events News

The DrupalCon news feed now includes social media sharing icons, so that you can better promote DrupalCon news and announcements to your networks. Word of mouth has always been a critical part of Drupal's success - so we hope that as featured speakers are announced, early bird registration begins, or the schedule is published, you will help us get the word out!

DrupalCon Seattle is coming up from April 8-12 2019, and we're featuring some bold new changes to support a variety of audiences from our traditional core of those people who build Drupal, to marketers and content editors, and to the agency sales forces that sell Drupal to the world.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Jul 13 2018
Jul 13

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Announcements

Last chance to vote in the Drupal Association board election

Elections for the Drupal Association board end on July 13th, 2018 at 5pm Pacific (in just over an hour at the time this is posted). There are nine candidates from 7 countries across six continents representing a wide variety of perspectives from the Drupal community. Anyone user who has been active in the last year and registered before the elections began is welcome to cast a ballot.

We encourage you to vote today and help guide the future of the Drupal Association.  

Reminder: Drupal Europe is coming up soon

Drupal Europe is coming up in less than 60 days! Drupal Europe will be the largest gathering of the Drupal community in Europe and is a reimagining of this important community event as both technical conference and family reunion. The Drupal Association engineering team will be attending to connect with the community, provide updates on Drupal.org, and listen to some of the incredible speakers who will be in attendance.

Join the community in Darmstadt, Germany from September 10-14, 2018. Make sure to register, book your travel, and secure accommodation: http://drupaleurope.org/

Project maintainers: Change your git remote configuration

Git authentication methods for Drupal.org hosted projects are changing as we approach upgrading our developer tooling stack. In particular we will be:

  • Deprecating password authentication for git

  • Deprecating the git remote format <username>@git.drupal.org/project/<yourproject>.git in favor of [email protected]:project/<yourproject>.git

We have updated the version control instructions for Drupal.org projects, and put a message in the git daemon for any user who makes a push using the deprecated format.

For more information, please review: https://drupal.org/gitauth

Drupal.org Updates

Ecommerce industry page launched

Since last year, one of our ongoing initiatives has been to develop more content on Drupal.org focused on specific industries. Drupal is an incredible powerful tool for building ambitious digital experiences, but it's flexibility can sometimes be overwhelming. These industry specific pages help Drupal evaluators discover how Drupal can be tailored for their specific needs, and highlight successful case studies of Drupal in the wild.

The Drupal Association has launched our sixth industry page promoting the power of Drupal for Ecommerce. We want to thank Commerce Guys for their contributions to getting this page off the ground.

Display project screenshots in a more user friendly way

For every new Drupal project that a developer or site-builder undertakes, time is spent evaluating distributions, modules, and themes to find integrations that will accelerate launching the project.

To improve the user experience for users evaluating modules on Drupal.org, we've implemented a new lightbox-style display for project screenshots.

Here's an example of a screenshot from the Token project:

Granted more maintainers the ability to give contribution credit

Since the introduction of contribution credits at the end of 2015, they've become an important part of the way the Drupal community recognizes individual and organizational contributions to the project. The Drupal Association Engineering team regularly reviews the contribution credit system to make small tweaks and adjustments to make the experience even better.

For our most recent update, Drupal.org now grants all project maintainers with the 'maintain issues' permission the ability to grant contribution credit, instead of just those users with 'Write to Version Control' permissions. This means that a much wider group of maintainers can now participate in granting credit.

Showing maintainer photos on top level Docs guides

Documentation is critically important to the Drupal project To make it easier for potential contributors to find out who they should reach out to for issues that affect the top levels of documentation, we've added maintainer information to the top level documentation guides.

Email confirmation when creating an organization node

To help more organizations that work with Drupal join our community, we now send an email confirmation to any user who creates an organization profile with information about becoming listed as a service provider, details about the contribution credit system, and information about becoming a Drupal Association member or supporting partner.

We encourage everyone in the Drupal community to ask your clients to create a Drupal.org organization profile. Bringing end-users into the contribution journey will be a key part of Drupal's long term health and success.

Contributing to the Open Demographics Initiative

One of our goals on the Drupal Association engineering team is to adopt the Open Demographics Initiative in our user registration process. As part of our effort to work towards that goal, we have contributed a machine readable version of the demographic questions and and answers to the ODI project.

We're hopeful that can be reviewed and committed soon, and be used as the basis for an ODI Drupal module.

Security Improvements

Added PSA and SAs to the /news feed

To increase the visibility of security notifications, Public Security Announcements and Security Advisories will now be included in the https://drupal.org/news feed.

Multi-value CVE field for Security Advisories

We've also updated the security advisory content type so that an advisory can be associated with multiple CVEs.

Infrastructure Updates

DrupalCI: Converted core javascript tests to use Chrome driver

The DA Engineering team has worked together with Core to convert the Core javascript tests from using PhantomJS to using Chrome Webdriver. This provides much more powerful and better supported tools for javascript development in Drupal.

DrupalCI: Reduced disk space usage of the DrupalCI dispatcher

One of the most important services the Drupal Association provides for the project is DrupalCI, the suite of tools used to test all of Drupal's code. These tools are very powerful, but also expensive to maintain, and something we have to monitor carefully. In June, we spent some time automating disk space management for the DrupalCI dispatcher, to help reduce the maintenance cost of keeping it running smoothly.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

  • OPIN - Renewing Signature Supporting Partner
  • Srijan - Renewing Signature Supporting Partner
  • Lullabot - Renewing Premium Supporting Partner
  • Aten - Renewing Premium Supporting Partner
  • Phase2 - Renewing Premium Supporting Partner
  • WebEnertia - *NEW* Premium Supporting Partner
  • Pantheon - Renewing Premium Hosting Supporter
  • Datadog - Renewing Premium Technology Supporter
  • Promet Source - Renewing Classic Supporting Partner
  • Evolving Web - Renewing Classic Supporting Partner
  • ImageX - Renewing Classic Supporting Partner
  • Adapt - Renewing Classic Supporting Partner
  • Green Geeks - Renewing Hosting Supporter
  • Microserve - Renewing Classic Supporting Partner
  • ThinkShout - Renewing Classic Supporting Partner
  • Amazee Labs - Renewing Classic Supporting Partner
  • Four Kitchens - Renewing Classic Supporting Partner
  • Access - Renewing Classic Supporting Partner
  • Appnovation - Renewing Classic Supporting Partner
  • Studio Present - Renewing Classic Supporting Partner
  • undpaul - Renewing Classic Supporting Partner
  • Position2 - Renewing Classic Supporting Partner
  • Blend Interactive - Renewing Classic Supporting Partner

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Jun 11 2018
Jun 11

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

AnnouncementsDrupal Association logo

Change your git remote configuration

We will be deprecating the git remote format <yourusername>@git.drupal.org/project/<yourproject>.git in favor of [email protected]:project/<yourproject>.git in preparation for changes to our developer tooling stack. If you used the <username>@ format for your git remotes, you should change your remote to the [email protected] format. You can use $ git remote set-url to make this change for existing repositories you have cloned.

We have updated the version control instructions on Drupal.org to reflect this change, and will be updating the git daemon to warn developers who are using the deprecated remote format.

Proposal: Improving Core's Relationship with Composer

In May, Mixologic from the Drupal Association engineering team worked with community members Mile23, Bojanz, Webflo, and others in the community to develop a proposal for improving Drupal Core's relationship with Composer.

In its simplest form, the proposal is to: Conceptually separate Drupal, the product, from Drupal's git repository, and provide a mechanism that creates a composer ready Drupal installation.

Going into early June, we've been circulating this proposal to the Core Committers, the Auto-Updates Initiative team, Contrib maintainers, Distribution maintainers etc.

Credit for non-code projects on Drupal.org

We're excited to announce that we've created a 'Community Projects' section in the Drupal.org issue queues. This section exists to record all the tremendous community labor exercised to promote the Drupal project in ways other than code. This format was pioneered by the Drupal Diversity and Inclusion group, who started recording their meeting minutes in the issue queues so they could provide contribution credit for attendees. This same model can be used by initiative coordinators, camp organizers, or any other Drupal community group that would like a place to recognize their work with the official contribution credit system.

The Contribution Credit system is one of the Drupal projects most successful innovations in the way that open source projects are managed, and it will continue to evolve and grow as time goes on.

Updates for GDPR

AEU GDPRre the words "We've updated our privacy policy" burned into your laptop screen yet? Well in May we did the same.  In particular, we've updated our Terms of Service, Privacy Policy, Digital Advertising Policy, and Git Contributor Agreement to clarify our compliance with the EU's GDPR. We also initiated a re-consent campaign for our marketing lists. If you have not re-consented to communications we strongly encourage you to do so

Launched the Customer Supporter program

Have you built great relationships with your Drupal customers? Help them contribute back to the project by becoming part of our Customer Supporter program.

Drupal.org Updates

Self-nominations for the Drupal Association board are live

Each year one of the two community-held seats on the Drupal Association board comes up for election. We opened the self-nomination process for this year, and some passionate and dedicated members of the community have already stepped forward with their candidacy.

To learn more, you can view our portal for the 2018 Elections Process. Key dates to remember:

  • Self nominations: 1-11 June, 2018
  • Meet the candidates: 12-29 June 2018
  • Voting: 2-13 July, 2018
  • Votes ratified, Winner announced: 25 July, 2018

Better landing pages for Drupal's strategic initiatives

Did you know there are 12 active Drupal strategic initiatives right now?

To help the initiative coordinators promote this work, and recruit more open source contributors to the cause, we've given initiative coordinators new landing page tools. Check out the first initiative to use this landing page: the Admin UI and Javascript Modernization Initiative.

These tools are the first step in improving the project management tools available to initiative coordinators to help move the Drupal project forward.

Historical user and organization contribution data is now available.

Drupal.org user profiles show the last year's worth of contributions by users. We chose the one year window deliberately, to promote the importance of a user's more recent activity. However, seeing a user's complete contribution history can be valuable as well. We've recently added a link to the bottom of this view to display that history.

Similarly, organization profiles have shown the last 90 days of contributions by organization. Again, we chose this very deliberately to emphasize the  importance of recent and ongoing contribution. However, as with user accounts, these profiles now also include a link to the organization's complete contribution history. You can see an example of where to find this link below:

Organizations - View all credit history

Expanded spam protections

After the sunsetting of Mollom in March of this year, we've been implementing a new set of tools to mitigate spam on Drupal.org. We expanded these protections in May, using a combination of bot detection, content analysis, rate limiting, and more to try and reduce the impact of spam attacks on Drupal.org. The less time the community spends wading through spam, the greater the velocity of the Drupal project.

And a Thank You

We'd also like to give a special shout out to contributor Wim Leers, for his incredibly kind 'Ode to the Drupal Association' about our work on the testing infrastructure. The nature of software engineering has a tendency to draw our attention to things that are broken, buggy, or unoptimized, and so when things are working well that success can sometimes feel invisible.

Fortunately, the Drupal community puts people first, and celebrates our collective success, and Wim's words are a tremendous example of that ethos.

Thank you, Wim - and thank you to everyone who takes the time to recognize the hard work and dedication of your fellow contributors.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

May 23 2018
May 23

Our global community includes many EU citizens and residents of the EEA, and we have taken steps to comply with the GDPR which takes effect on May 25, 2018.

Your rights under this law and how Drupal.org complies with GDPR

We've updated our Terms of Service, Privacy Policy, Git Contributor Agreement, and Digital Advertising Policy based on the requirements of the EU General Data Protection Regulation. We've also begun a campaign to reconfirm your consent to our marketing messages.

For easy and clear access to the changes: 

Human Readable Summary

Disclaimer: This summary is not itself a part of the Terms of Service, Privacy Policy, Git Contributor Agreement, or Digital Advertising Policy, and is not a legal document. It is simply a handy reference for understanding privacy rights and regulations. Think of it as the user-friendly interface to the legal language.

In plain language, regulations such as GDPR define the following roles, rights, and responsibilities:

  • Data Subject - this is you, the end user.
  • Data Controller - this is us, the Drupal Association as the owners and operators of Drupal.org and its sub-sites.
  • Data Processor - any other organization that processes personal data on behalf of the Data Controller.

Rights of the Data Subject

  • Right to be Informed - A data subject has the right to know whether personal information is being processed; where; and for what purpose.
     
  • Right to Access - A data subject has a right to access the information about them that is stored by the Data Controller.
     
  • Right to Rectification - A data subject has the right to correct any errors in the data about them. This can be done by editing your user account, or contacting the Drupal Association directly.
     
  • Right to Restrict Processing - A data subject has the right to request that data not be processed, and yet also not be deleted by the Data Controller.
     
  • Right to Object - A data subject has the right to opt out of marketing, processing based on legitimate interest, or processing for research or statistical purposes.
     
  • Right to be Forgotten - Also known as the right to revoke consent, the right to be forgotten states that a data subject has the right to request erasure of data, the cessation of processing by the controller, and halting processing of the data by third party processors.

    The conditions for this, as outlined in article 17, include the data no longer being relevant to original purposes for processing, or a data subjects withdrawing consent.

    It should also be noted that this right requires controllers to compare the subjects' rights to "the public interest in the availability of the data" when considering such requests.

  • Data Portability - A data subject has the right to receive a copy of their data in a 'commonly used and machine readable format.'

    This information is outlined in the sections below titled "Your Choices About Use and Disclosure of Your Information" and "Accessing and Correcting Your Information".

Responsibilities of the Data Controller and Data Processors

  • Privacy by Design - 'The controller shall..implement appropriate technical and organisational measures..in an effective way.. in order to meet the requirements of this Regulation and protect the rights of data subjects'. Article 23 of the GDPR calls for controllers to hold and process only the data absolutely necessary for the completion of its duties, as well as limit the access to personal data to those who need it to carry out these duties.
     
  • Breach Notification - The Data Controller must notify the appropriate data processing authority and any affected end user of any breach that might result in 'risk to the rights and freedoms of individuals' within 72 hours of becoming aware of the breach.

    A Data Processor must notify the Data Controller of any breach 'without undue delay.'

  • Data protection officer - A Data Controller or Processor must appoint a Data Protection Officer when: a Data Controller represents a public authority; or the core operations of the Controller require regular and systematic monitoring of Subjects on a large scale; or when the Controller's core operations depend on processing a large scale of special categories of data (including but not limited to health data, criminal conviction information, etc).
     

    The Drupal Association's core operations do not require the Association to establish a Data Protection Officer.

We take privacy and security very seriously, as all Drupal professionals do! We will continue analyzing the legal landscape and collecting feedback for future revisions.

If you have any questions or concerns about our GDPR compliance, or if you want to point out a mistake or provide a suggestion for the Terms of Service, Privacy Policy, Git Contributor Agreement, or Digital Advertising policy, you can send an email to [email protected].

Apr 30 2018
Apr 30

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Drupal.org UpdatesDrupal Association Logo

Drupal.org's new front page and persona pages launched

As you've probably seen by now, just before DrupalCon Nashville we launched a makeover of the Drupal.org front page. This was a research-based redesign focused on addressing the three key personas that come to Drupal.org: Developers, Marketers/Content Editors, and Agencies.

The new redesign simplifies the number of calls to action on the front page, and directs each of these personas into a more focused funnel, to ensure they are more likely to find the information they really need. To learn more about this redesign and the Promote Drupal initiative, read our recent blog post. We want to thank SixEleven for their help with this new design initiative.

Promote Drupal Initiative

Redesigning the front page was just the start, we kicked off DrupalCon by announcing a new 'Promote Drupal' initiative, asking the community to come together to help bring Drupal to new audiences, and to convince people who've used older versions in the past to give Drupal 8 another look.

We need your support to make the Promote Drupal initiative happen!

Updated top navigation and IA

Along with the front page changes, we've updated Drupal.org's top level IA, providing a more logical structure for navigating to the major areas of the site depending on a user's persona.

Drupal.org Top Nav Redesign

Promoting Nonprofit solutions built with Drupal

And last, but not least, in our efforts to #PromoteDrupal we've launched a new Nonprofit solution page, promoting the power of Drupal for Nonprofits and NGO's around the globe. Drupal has long been the choice for well-recognized, global nonprofit organizations to extend their reach and maximize their impact.

Drupal nonprofit

Simplify Drupal Initiatives

In project founder Dries Buytaert's keynote at DrupalCon Nashville he proposed a series of initiatives to simplify Drupal - lowering the barriers to adoption and improving the user experience of site administrators and content editors. Some of these initiatives are to improve features of Drupal core itself, whereas others are focused on the evaluator experience and will be managed in collaboration with the Drupal Association.

In particular, the Drupal Association will collaborate with the core initiatives teams on:

These initiatives are not going to be quick or easy. They rely on collaboration between the Drupal Association, Drupal's core committer team, and a variety of volunteers throughout the community. We'll need your help.

Drupal.org and GDPR

GDPR, the General Data Protection Regulation passed by the EU last year, begins enforcement on May 25th, 2018. We've been preparing for this new regulation for some time, and will be implementing a few changes in the coming weeks:

Drupal securitySecurity Release

SA-CORE-2018-003

Drupal Core coordinated a security release with the CKEditor team to ensure that the security fix for CKEditor was immediately available in Drupal 8. As Drupal becomes further integrated into a world of third party dependencies, this kind of coordination between open source projects becomes increasingly important. We want to thank the CKEditor team and the volunteer Drupal Security team for their hard work and careful collaboration.

SA-CORE-2018-004

After the release of SA-CORE-2018-002 in March, a related vulnerability was discovered and an additional security advisory for Drupal 7 and 8 released in April. If you have not yet updated your Drupal sites to address these vulnerabilities they may already be compromised. If that is the case, we encourage you to read this PSA, which provides some steps you can take.

Security releases tend to spark quite a bit of conversation in the community about the nature of software security, proprietary vs open source, and related issues. Community member @rickmanelius provided some much-needed context to keep these security focused efforts in perspective:

The recent SA-CORE-2018-004 and SA-CORE-2018-002 security advisories have sparked a lot of conversations in the Drupal community regarding all things security. IMHO, it's important to highlight several talking points to keep things in perspective.

— Rick Manelius, PhD (@rickmanelius) April 26, 2018

DrupalCI: Support for DrupalCI.yml

DrupalCI now supports the use of Drupalci.yml files in projects to customize and override elements of testing. This makes the testing capability of DrupalCI much more powerful and flexible for project maintainers. We're still working on documenting these new features, but you can read about the new features here.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Apr 27 2018
Apr 27

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

This month's update comes a bit later than usual, as we return from DrupalCon Nashville. Expect our April update to follow after shortly.

Representing Drupal at Google's CMS Leadership Summit

In mid-March, we attended the Google CMS Leadership Summit, as representatives of the Drupal project. The Summit was a one-day event hosted by Google to unite the 20 or so projects in the CMS space responsible for more than 50% of the content on the web.

The goal was to understand how to preserve an open web, by empowering better authoring experiences, content consumption, and performance in our CMS platforms.

This level of dialogue and engagement with an organization like Google is new and exciting for us, and we're looking forward to ongoing conversations, both with Google and with the other CMS project leaders they assembled at the event.

Drupal.org UpdatesDrupal Association Logo

Researching the anonymous traffic to Drupal.org

One of the focuses of the Drupal Association in 2018 has been to better understand our audience. When it comes to users who register on Drupal.org, and our DrupalCon attendees, we have quite a bit of information about who our users are.

However, when it comes to the wider ecosystem of Drupal users (evaluators and end-users who do not have Drupal.org accounts) we've been largely in the dark for most of the project's history. One way we want to improve this is by working with Drupal Core to add telemetry to Drupal, but that is an effort that will take some time.

In the meantime, we've implemented several Audience Insight tools to help us learn more about our anonymous users. Privacy is always a paramount concern, so we chose only insight tools which provide aggregate, anonymized data, and we wrapped those tools in our own implementation of Do-Not-Track so that we could ensure that user privacy preferences are respected.

The table below demonstrates the job functions held by the anonymous visitors to Drupal.org. (Please note: these job functions might be held within any kind of industry, this data is about the user's role, not their target market).

Job Function

D.O Front Page Visitors

All D.O Visitors

Diff

Engineering

26.20%

44.90%

-18.70%

Information Technology

14%

16.40%

-2%

Business Development

11.60%

9.60%

2.00%

Entrepreneurship

10.30%

10.70%

-0.40%

Arts and Design

7.70%

5.30%

2.40%

Media and Communication

6.60%

6%

0.60%

Marketing

6.30%

3.70%

2.60%

Education

6.10%

4.60%

1.50%

Operations

5.20%

3.50%

1.70%

Program and Project Management

4.30%

4.20%

0.10%

Sales

4.20%

2.60%

1.60%

Consulting

2.90%

3.30%

-0.40%

Research

2.40%

1.70%

0.70%

Community and Social Services

2.40%

1.80%

0.60%

Administrative

2.30%

1.30%

1.00%

We've used this data to inform the redesign of Drupal.org, as well as our new persona pages. Learn more about that process in our April update. The redesign work was carried out in collaboration with SixEleven, who also produced the DrupalCon brand and design for DrupalCon Nashville.

Nashville guitar

Preparing for DrupalCon Nashville

In the lead up to DrupalCon Nashville in April, the team was in high-gear preparing for the event. We participated in a panel about the future of pull requests on Drupal.org, the public board meeting, and handled the keynote livestream process.

DrupalCon is always an incredible opportunity for the team to connect with the community about upcoming initiatives, drupal.org support requests, and to plan for the future.

We were happy to see so many of you there, and we'll talk more about this in our April update.

Documentation enhancements

Our efforts to improve the quality of Drupal's documentation continued in March and April, as we added features:

  • New D7/D8 guides are now automatically approved, so new project contributors aren't blocked on documenting their projects.
  • Added Drupal version to page title for better searchability.
  • Follow/Unfollow links are available directly on the discuss page of any documentation.

In-context links to newer and older releases

To ensure that users are aware when there are newer releases than the one they may be looking at, we now provide newer and older release history on release nodes. In the sidebar of any release page you will see links and dates to related releases. Among other things, we hope this will prevent users from accidentally installing an older release when another new one has just come out!  Other releases

Email notifications for new maintainers

Encouraging succession planning in module maintership is an incredible challenge for any open source project. We want to encourage maintainers to invite contributors to their projects to help maintain those projects, but we recognize that we also have to make sure the appropriate tools are in place to make this a smooth process.

To make sure that new maintainers of projects are welcomed into the fold, we've added email notifications to let a user know when they've been added as a project co-maintainer. If you are invited by an existing maintainer of a project to help maintain it, you should now receive a warm welcome.

Drupal Security Team

Security Release

SA-CORE-2018-002

The Drupal Association Engineering Team collaborated with the Security Working Group and Security Team to coordinate 3 significant security releases in March and April.

The primary release was SA-CORE-2018-002, a highly critical security release for Drupal 7 and 8. For more information about all Drupal security releases and PSAs, please visit our security portal.

The volunteer Security Team has always been a tremendous asset to our community, and the Drupal Association is proud to support their work.

Infrastructure Updates

DrupalCI: Support for testing themes

DrupalCI has enabled support for testing Themes, so now Theme projects on Drupal.org can include tests. This has become more and more necessary as javascript becomes critical to modern web design, and we hope this will help accelerate the build out of themes for Drupal 8 and increase their quality.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Mar 07 2018
Mar 07

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Drupal.org UpdatesDrupal Association

Reimagining Drupal.org's front pages to serve distinct personas

Drupal serves a wide audience of users, from developers to marketers to content editors and beyond. Historically, Drupal.org has been focused on our community of contributors, whether those contributions are in code, documentation, volunteer support, camp organizing, etc. However, only 1 in 15 visitors to Drupal.org are an authenticated user, and the rest are primarily visiting Drupal.org as representatives of an end-user organization that is evaluating Drupal. We want to serve these visitors better.

In February we held an off-site in Portland to consolidate our research about the personas within end-user organizations who make the decision to adopt Drupal. We identified three key roles:

  • Technical evaluators - who are often developer evangelists within their organization

  • Marketing and business users -who are evaluating Drupal as a platform. They are interested in the editorial experience and time-to-market for building a solution that integrates with tools they already use

  • Agencies - who are already using Drupal for their clients, or are considering making it central to their business.

From there, we developed some initial concepts for a reimagining of the front page of Drupal.org to better serve these first three personas.

This work will carry us through DrupalCon Nashville and beyond, so expect additional updates over the coming months.

Contribution credit update

The Drupal project has an innovative system of crediting users and sponsoring organizations for the work they contribute. However, as a system that we've pioneered, there is always room for additional improvements. One area that needed improvement was the date used for the assignment of credit. In the past, the credit for a user or organization would be tracked to the timestamp of the latest activity on the issue. This was a good approximation, but additional comments after issue resolution would bump the date of the credit.

We've updated the way that contribution credits are calculated - so that it is now based on the date that the issue was closed(status last changed) instead of the date of the last change to the issue. This change affects both individual contribution credits and the marketplace ranking.

Documentation improvements

As our new team member Dhanya has come on board, she has helped make some great improvements to the documentation system, including: fixing the display of sidebar lists of guide contents, increasing the visibility of the current page indicators, and swapping the grid treatment for a more readable guide contents layout

Doc Guide Preview

Accessibility and readability

We've made two additional small fixes.

One for accessibility - improving the keyboard 'skip to…' links in the Drupal.org top navigation.

Skip to main

Skip to search

… And one for navigating issues, fixing a bug that prevented links to comments on multi-page threads. Now, any Drupal.org user who receives an email notification about a multi-page issue should be properly linked to the correct comment.

Preparing our live-streaming capability for DrupalCon Nashville

For DrupalCon Nashville, rather than relying on a vendor, we are going to be managing the live stream of the keynotes and closing session ourselves, together with the AV staff of the venue. In February we spent some time putting together our equipment and running some streaming tests.

Continued work to reduce our PCI scope

In February we finished migrating our donation process for both USD and Euro donations to new payment processors to reduce our PCI scope and thus maintenance costs. We've also launched the beta of the membership system, and will hopefully complete the migration of existing memberships soon.

If you are not yet a member of the Drupal Association, and would like to support us both by joining and helping us test the new membership system, you can sign up here. (If you are an existing member, please continue to process your renewals on the original system for now).

Infrastructure Updates

Git servers updated

We migrated our existing git infrastructure from bare metal servers to virtual machines, which will help to make our infrastructure more flexible and portable in the future. This has been an ongoing effort, and the git servers are among the last of the servers to be migrated.

Continued tuning of Perimeter XPerimeterX

PerimeterX helps to identify bad actor behavior and DDOS attempts and mitigate them at the edge of our network. We established our relationship in January, and throughout February have been monitoring and tuning our configuration to better protect Drupal.org. We've already managed to mitigate a persistent DDOS attack which has recurred every couple of months, and hopefully we can make more improvements to protect Drupal.org, and reduce the pager burden on our team.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Feb 26 2018
Feb 26

JetBrainsWe are happy to announce today that JetBrains is continuing their support of the Drupal Association and the Drupal project in a new way:

It has been part of JetBrains mission to give back to key contributors in the open source world, and we're grateful that they want to offer this opportunity as a benefit for Drupal.org contribution credits.

To receive the free PhpStorm Open Source license, please fill out this open source license request form. Be sure to include:

  •  Project Name: Drupal
  •  # of Developers: 1 - since you are requesting only a single license for yourself
  • Project role: Contributor
  • Your role in project: <-- This should include the link to your Drupal.org profile.
  • Fill in the rest of the fields as you are best able - the ones above are the key fields for this offer.

The Jetbrains team will be checking the profile to ensure you meet the 35 contributions threshold, they will be looking for the line which reads Credited on XXX issues fixed in the past 1 year. You can read more about PhpStorm and the Open Source support program by JetBrains.

Many thanks to JetBrains, we’re excited for their continued partnership.

Feb 20 2018
Feb 20

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

In January the Drupal Association kicked off the new year with our Winter Staff Retreat, where the whole organization came together to review the past 6 months and lay out our strategy for the new year. In addition to taking a big picture look at our upcoming priorities, we also made some great improvements this past month.

Before we dive into those, we'd like to welcome Dhanya Girish to the Drupal Association team. Her time has been generously sponsored by Zyxware and she joins us as a skilled engineer. Please welcome Dhanya!

Drupal.org Updates

Celebrating Drupal's 17th Birthday

To celebrate the 17th birthday of Drupal, we've embedded a wonderful video on the home page, celebrating some of the incredible things that have been built with Drupal. Drupal has grown from being a dorm-room experiment to being the driver for some of the most powerful digital experiences on the web. We can't wait to see what the next 17 years bring.

[embedded content]

Proposed new initiatives for collaboration with Core

The Drupal Association engineering team meets monthly with the Drupal Core committers to align our goals and ensure that we're on the same page about what the project needs moving forward. At the beginning of this year we outlined some new proposed initiatives that, in collaboration with Core, we believe could be a tremendous value to the project.

None of these initiatives can be accomplished quickly, nor can they be moved forward without the collaboration of key contributors and the core committers, but we strongly believe they will be important next steps for the future of Drupal.

Refresh of the Hosting Listings program

A major focus in January was the refresh of our hosting listings program. The new hosting listings now display all providers on the primary view, with a series of filters to help users find exactly the right hosting partner. This update also brings a much needed visual refresh to the listings. New filters and search facets will help end-users find the hosting partner that is right for their needs, industry, and budget, and will enable users to identify and support those platforms that support the project.

Promoting industry solutions in the marketplace

Over the last year, we've been focused on pushing a message about crafting the perfect solution for particular industries with Drupal. We link to this information off of the front page, but we've also added a contextual block in the sidebar that will appear whenever users filter the marketplace by the relevant industry.

These efforts are an ongoing part of better serving the various personas who come to Drupal.org. Look forward to hearing about even more changes on this front as we approach DrupalCon Nashville in April.

In-context issue tag explanations

One of the cornerstones of Drupal.org contribution and issue management is our issue tagging system. However, it's been difficult in the past to understand what each tag is used for, especially for tags that are carefully monitored and curated by project maintainers or the core committer team. We've enhanced the tagging system by providing in-context hover-states that describe what tags are used for.

Infrastructure Updates

Implemented PerimeterX for increased protection from DDOS, crawlers, badbots etc.

Would you believe that Drupal.org is the target of a bad crawler or outright DDOS attack at least once every two months? The team does an incredible job reducing the impact on end users of Drupal.org, but to make that job easier, we've partnered with PerimeterX for bad bot protection integrated with our CDN. PerimeterX integrates closely with Fastly, our CDN, and so can provide intelligent logic and protection at the edge.

We're starting off using this system simply for greater introspection into the nature of traffic patterns and the attacks of bad actors. We're gradually enabling the protection as we tune the system for Drupal.org's unique needs. We want to thank PerimeterX for supporting the project, and for helping to make Drupal.org a better place for our community.

DrupalCI: Support for yarn-based core tests

We've added a yarn build-step plugin to DrupalCI, allowing us to support new testing types, including nightwatch javascript tests in core and contrib. We want to provide a special thanks to community contributors who helped: justafish, dawhener, and mile23.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

  • PerimeterX - *NEW* Signature Technology Supporting Partner
  • publicplan - *NEW* Premium Supporting Partner
  • WebEnertia - *NEW* Premium Supporting Partner
  • Electric Citizen - *NEW* Classic Supporting Partner
  • Factorial - *NEW* Classic Supporting Partner
  • One Shoe - Renewing Classic Supporting Partner
  • DRUD - Renewing Classic Hosting Supporter

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Jan 05 2018
Jan 05

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Announcements

DrupalCon License program launched

DrupalCon In 2018, the Drupal Association made the difficult decision to pause DrupalCon Europe for the year, so that we could re-envision the program for greater scalability and sustainability. We're pleased to announce that we have launched a new program for licensing DrupalCon, so that local entities can bring this great event to their area. Read the announcement by Executive Director Megan Sanicki for more information.

Analysis of Developer Tooling options published

For more than a year, the Drupal Association has been evaluating options for improving the tooling used by developers on Drupal.org. We have recently concluded our study, and published a detailed analysis of the options, as well as our next steps. As expected, this blog series has sparked ongoing conversations about the future of our tools both among the community and with our potential partners - so look out for more updates.

Drupal.org Updates

Easier management of Drupal Association Membership

Drupalcon Individual Member Badge In December we worked on updates to membership management, so that Drupal Association members will be able to manage their membership information.

New DA Membership Directories launched

Last month we mentioned the launch of the new individual member directory on Drupal.org. In December we expanded on this work to update our directory of organization members as well. Drupalcon Organization Member Badge You can explore the directory of Drupal Association members here, as well as the new directory of organization members. If you have feedback on either of these directories, please let us know!

Akismet for spam protection

The content analysis tool Mollom is rapidly approaching its end of life. And so to continue to protect the Drupal community from spam, we have implemented Akismet on Drupal.org. We are currently running it in silent mode, side-by-side with Mollom and our other protection methods, to ensure a smooth transition.

DrupalCI: Chrome Webdriver available for JS testing

DrupalCI One of the major services provided by the Drupal Association is continuous integration testing for the Drupal project. An increasingly important component of this is our javascript testing stack. Previously we tested javascript for the project using PhantomJS. However, that library is now deprecated. We have since created a new testing environment running Chrome Webdriver, and are working with core and contrib developers to ensure it meets their needs.

Drupal.org Updates

Mitigating the risks of Spectre and Meltdown

By now everyone in the technology industry is likely aware of Meltdown and Spectre, the two major security vulnerabilities recently disclosed in major CPU architectures. Drupal Association staff are in close coordination with our infrastructure partners at Tag1Consulting, to ensure that any vulnerable machines in our infrastructure are protected as soon as possible, and our community's data is kept safe. ——— As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association. Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Dec 20 2017
Dec 20

This is the fourth post in our series about integrating Drupal.org with a 3rd party developer tooling provider:

With our plan to create modular integration points for our tooling options, we have a few clear steps for moving forward:

Phase 1: Prep work

  • Deprecation of password authentication for Git, since many external tooling services no longer support it.
  • Working with core to provide compatibility for semver versioning for contrib, both because this is needed for Composer, and because all of the third party developer toolsets we are considering have begun to standardize on semver.

Phase 2: Initial implementation, replacing current features

  • Replacement of custom, bespoke Twisted Git daemon with standard Bitbucket repositories.
  • Replacement of unmaintained CGit with supported Bitbucket code viewing.

Phase 3: New features

  • Integration of merge request 'hook' into issue queues, to allow contributors to use a pull request workflow instead of patches.
    • Modular - to be used with Bitbucket for now, but potentially another solution when more mature.
  • Integration of code review 'hook' into issue queues, to give us powerful inline code commenting tools.
    • Modular - to be used with Bitbucket for now, but potentially another solution when more mature.

Phase 4: Implement Hybrid Integrations for other toolsets

  • Updating project page integrations such that those projects which are already hosted on third party tools such as GitHub or GitLab (for example, Drush) can easily login with SSO, synchronize their repositories, and choose the canonical home of their issues.

On-going: Evaluation

  • Re-evaluate other tooling solutions as blocking issues are resolved and their feature-sets evolve.

So that's the update!

In short: after more than a year's evaluation of the current leaders in open source tooling solutions, including direct collaboration with several of those teams, we are going to focus on making Drupal.org modular to integrate with the best tooling solution as it develops. For now, we will be implementing key improvements for Drupal developers using Bitbucket for our repository hosting, code viewing/review, inline editing, and merge requests - integrated with the existing project pages and issue queues.

We'd like to thank the following people for their involvement in this initiative at various times through the process:

Drupal Association Staff

The Technical Advisory Committee (TAC)

Members of the community

The teams of our potential partner organizations

Each of these teams is committed to serving open source and we thank them for their collaboration during our evaluation process:

  • GitHub
  • GitLab
  • Atlassian/BitBucket
Dec 20 2017
Dec 20

This is the third post in our series about integrating Drupal.org with a 3rd party developer tooling provider:

Below are some rough mockups of what a modular integration between Drupal.org's issue queues and a third party toolset could look like. For the sake of this example, I've used Bitbucket as the tooling provider, as that is likely to be our interim toolset for the reasons outlined above. However, one can easily imagine these functions being substituted for their equivalents in other toolsets.

The following is only an early concept. These are by no means final designs for each of these integration points, but they will help to visualize how these tools will be integrated with Drupal.org.

Mockup: Workspaces in the issue summary

The biggest change for contributors would be the addition of a new place in the issue summary to contain these workspaces. For most issues we anticipate a single workspace, as most issues are resolved via collaboration on a single code path, however we would be able to generate multiple workspaces per issue for multiple solutions being proposed in parallel:

Code workspace in issue summary

The key functions of the Proposed code workspace are: the ability to make a new workspace, the ability to clone an existing workspace, the ability to create/view/edit a workspace merge request. The latest test result and ad-hoc testing options for each workspace would be provided here as well.

Mockup: Propose new Code

The "Propose new code" button is very simply a way to automatically generate a new workspace. This would only be used when a contributor wants to propose an alternate solution to whatever is currently under development. When pressed, we would generate a new workspace with the back-end tooling provider (whether as a fork, branch, or Git namespace) and add it to the table to be cloned, viewed for comments/inline editing, or ultimately to create a new merge request.

A second workspace in the issue summary

The new workspace will be generated with a name based on the issue id, and will present its own test result, clone, and merge request features.

If a user wants to collaborate on an existing solution instead, they can simply clone that workspace locally, or view it to make inline edits. Because some issues can languish for months, or even years, we also want to automate the process of rebasing these workspaces from their parents.

Mockup: Automatic comments as workspaces are updated

Whenever a change is made within the third party toolset, we would use the API to call back to Drupal.org and leave a system message describing the change, as well as linking to the relevant part of the third party toolset UI.

This is what an automated issue comment for code review might look like:

hestenet commented on 2893061-0 - "This is just a test comment" Read more...

This is what a comment for recently pushed changes would look like:

hestenet pushed code to: 2893061-0: View the diff This is a commit messag. Read more...

And this is what a comment for inline edits could look like:

hestenet used the inline editor to update 2893061-1: drupal/core/modules/update/update.api.php - View the diff

Mockup: Clone

The clone button would simply provide a basic modal pop-up with instructions for collaborators to clone the workspace locally with Git. This is not the final url pattern (one of our goals is to avoid changes in canonical git urls where possible) but it could look something like this:

$ git clone ssh://[email protected]/is/drupal.git

Mockup: Create Merge Request (in Bitbucket)

Merge requests allow contributors to propose changes to projects they don't maintain, and provide maintainers an easy way to view proposed changes.

The create merge request button would allow the user to view the current workspace and request that it be merged into the canonical parent. Only a project maintainer would have the permissions to complete the merge.

Create pull request

Code in a workspace with an open pull request can continue to be iterated on by the contributors. Code comments, diffs, and reviews are summarized on the request.

Mockup: View Merge Request (in Bitbucket)

View existing merge request

View merge request, like 'create merge request' would allow any collaborator to view the current workspace. This could also be used to initiate inline code edits, or leave code comments.

Mockup: Inline editing

Inline editing allows a quick and easy way for people to propose "standalone" changes to e.g. documentation, markup, etc. without having to have an entire development stack.

We would rely on the third party tooling provider's inline editing functionality. In the example below, we can see how an inline edit is made using Bitbucket:

Inline editor

Mockup: Code review

As with inline editing, we would rely on the third-party tooling provider's tools for code review. In the example below you can see the code review options provided by Bitbucket:

Code review tools and comments

As indicated above, these mockups are simply an illustration of what this integration could look like, not final designs. However, this has hopefully shown how we can introduce a hybrid model to integrate the best features of a third party tooling provider (pull requests, code review, inline editing), while retaining the essential nature of the drupal-flow.

Dec 20 2017
Dec 20

This is the second post in our series about integrating Drupal.org with a 3rd party developer tooling provider:

In this update we'll talk about the specific options we've been evaluating to improve the tools for developers on Drupal.org. For each of these options we've built prototype workflows, stood up test integrations, and opened a dialogue with the creators of these toolsets about any gaps we identified. You'll see a summary of how each tooling option does or does not live up to the criteria we outlined in our last post.

Issue Workspaces

At DrupalCon Los Angeles in 2015, Mixologic from the DA Engineering team proposed the concept of Issue Workspaces. While we've historically talked about this idea as a single concept, it actually breaks down into two key components:

  1. The definition of a modern, idealized workflow for the Drupal project.
  2. A technical implementation using a bespoke tooling solution built on Git Namespaces to implement this workflow.

Criteria (for a bespoke implementation):

  • Familiar workflow.
    • ❌/ While an implementation of workspaces based on a bespoke git-namespaces implementation has several technical advantages, it will not appear nearly as familiar to outside developers as a more standard 'pull request' implementation. (Though it would be functionally very similar).
  • Preserve the most valuable parts of Drupal's workflow.
    • By implementing a bespoke solution we could exactly tailor the solution to the needs of the Drupal community.
    • ❌ However, the needs of that community are great, and our capacity to build a 100% bespoke solution that meets all the community's needs is not clear.
  • Leverage a partner who will keep evolving their code collaboration featureset.
    • ❌ Building issue workspaces using a bespoke git namespaces implementation would require us to continuously compete on features with major third party tooling providers that have much greater resources.

A purely bespoke implementation misses the mark on two of our three primary criteria. Fortunately, however, two years after the original proposal, it is no longer necessary to build a bespoke solution in order to implement an idealized workflow for the Drupal project. It is something that can be done by integrating the existing Drupal.org project pages and issue queues with third-party tooling solutions. Leveraging third party solutions provides several advantages including: familiarity to a wider audience of developers, a continually evolving featureset, and mitigation of the risks of maintaining a bespoke solution with a small team of engineers.

Here is the idealized 'Drupal-flow' visualized:

Drupal Flow - the interaction between issues and workspaces

  • Every issue should be associated with one or more canonical workspaces (git repos+merge requests) which can be collaborated on by any contributor, and merged into the canonical parent by a project maintainer.
  • Contributors should be able to modify the code in these workspaces by:
    • Checking out the code and committing/pushing changes to the workspace.
    • Inline editing files within a workspace.
    • Legacy: uploading a patch.
    • Any contribution method should trigger the appropriate tests.
  • Workspaces should be rebased (manually or automatically) on any changes to the canonical parent.

This workflow is not dissimilar to the default GitHub flow, GitLab flow, or even the suggested Bitbucket flow. These are all variations of the generic concept of 'Git flow' created by Vincent Driessen. However, the important difference comes in how each of these solutions tries to integrate a gitflow model into their issue/pull-request model.

The dominant model of these large third party tooling providers is not particularly collaborative. It encourages forks-of-forks, separates conversation about potential solutions onto multiple branch comments or merge requests threads, and generally caters to single developers working on individual proposed solutions, with one ultimately selected by the maintainer and the rest thrown away.

This is a key area in which the 'Drupal-flow' differs from the standard workflow that is implicitly enforced by these toolsets. Our ethos encourages collaboration of many developers on a single solution, and a single threaded conversation.

The gitflow behind the scenes may be identical, but the user experience wrapped around that flow makes very different assumptions about how people will collaborate.

GitHubGitHub

GitHub is of course the dominant player in the open-source tooling space. For many years though, their toolset was very much targeted at smaller open-source projects, and their default workflow highly encourages a many-to-many, forks-of-forks workflow, rather than the many-to-one single-threaded collaboration that is part of the Drupal community's ethos. More recently, GitHub has been improving their featureset, providing better issue management tools, more control over organization-level namespaces, and a new app marketplace that allows developers to extend GitHub's core featureset.

We reached out to GitHub and spoke with members of the technical and partnership teams, and while they were excited by the idea of bringing a project like Drupal to GitHub, we did run into some unresolvable blockers - most critically, by policy GitHub does not allow users of GitHub Enterprise to run their own public instances. This means we would have to migrate Drupal projects to GitHub.com, rather than self-hosting our instance.

Criteria:

  • Familiar workflow
    • GitHub is unquestionably the dominant platform for code collaboration on the web today.
  • Preserve the most valuable parts of Drupal's unique workflow
    • ❌ No way to enforce Drupal.org as the home of projects. While we can still keep project pages on Drupal.org there is nothing to encourage visitors who find the project on GitHub to treat the Drupal.org project page as canonical.
    • ❌ ;No way to enforce our workflow - forks of forks of forks of Drupal core into personal namespaces would be one click away.
    • ❌ Issue management tools are not setup for large, crowd-sourced teams.
      • Even for large community projects, it's more typical for only one or two developers to work on a single github issue at a time. Collaboration tends to be divided across issues, rather than within them.
    • ❌ Hard blocker Changing issue metadata requires maintainer permissions. (i.e: No way for non-maintainer collaborators to set an issue to RTBC)
    • ❌ Limited ability to manage project maintainers, especially in the case of abandoned or insecure projects.
  • Leverage a partner who will keep evolving their code collaboration featureset.

Other issues:

  • License/Source availability
    • ❌ GitHub is closed-source.
  • API availability
  • Hybrid integration
    • A hybrid integration between Drupal.org and GitHub is possible with SSO, automatic repository syncing, and a choice of using our issue queue or theirs.
    • ❌ However, there would be no portability of issues across a project that uses GitHub to one that does not.
    • ❌ Projects choosing to use the GitHub issues would not have access to DrupalCI
    • /❌ Projects using GitHub issues would not have access to contribution credit (though it may be possible to develop an app).
  • Data portability/vendor lock-in

    • /❌ GitHub's data migration feature is only available as part of GitHub enterprise.
    • ❌ GitHub policy will not allow us to self-host, which would allow us to work around some of the issues of preserving our Drupal-flow.
  • Developer support
    • GitHub is well-funded and has a large team of developers improving the product. However, it is not open source, and so there is no capacity for the community to fill in the gaps on anything GitHub fails to provide.
    • GitHub has recently added a new 'apps' architecture, enabling deeper integration.
  • Maintainability for an engineering team of limited resources
    • Repositories hosted on GitHub would be easy to maintain, but we would have little access or control in the event of issues, and little ability to customize the workflow for the community's needs.
  • Cost
    • The GitHub team would be willing to provide the Drupal project an enterprise account.
  • Unintended consequences
    • /❌ Moving most of our developer traffic to hosted GitHub could have a significant impact on Drupal.org traffic, affecting everything from our search presence to revenue programs.

GitHub is a no-go.

GitLabGitLab

GitLab is the up-and-comer, and is becoming exceptionally popular in the open-source space - even beginning to be adopted by larger open source projects such as GNOME and Debian.

We've worked closely with the GitLab team as well. In contrast to GitHub, GitLab's terms would allow us to run our own instance of the service. GitLab's fundamental workflow is very much like GitHub's - a one-to-many model of forks of forks. After discussing our workflow with their engineers GitLab has begun implementing their new 'fork network' feature, allowing merge requests across forks. However, this doesn't quite achieve the goal of allowing our many-to-one collaboration model just yet.

GitLab is very close to being a good solution for an Open Source project of Drupal's scale, and the rate at which they are improving the service is impressive. GitLab is also already a favorite of many in our community for personal or professional projects. Given some more time, it may ultimately be able to tick all the boxes.

Criteria:

  • Familiar workflow
    • GitLab's workflow is very similar to GitHub's and it is the second most widely used code collaboration toolset.
  • Preserve the most valuable parts of Drupal's unique workflow
    • ❌ Hard Blocker - Maintainers cannot yet collaborate on downstream merge requests
    • ❌ Monolithic toolset - GitLab's featureset is designed to be used in an all or nothing way. There is no provision for using specific elements while disabling others.
      • For example, there is no way to disable GitLab project pages in order to keep the canonical project pages on Drupal.org.
    • No plugin architecture - changes must be accepted as merge requests upstream in GitLab, or re-rolled as patches with each release (aka, "hacking core").
      • No easy way to turn on or off elements of the GitLab UI.
      • No way to extend the GitLab featureset.
      • Very limited issue management states, and no ability to customize them.
  • Leverage a partner who will keep evolving their code collaboration featureset.
    • GitLab's roll-out of new features is the fastest paced of the tooling providers that exist today.

Other issues:

  • License/Source availability
    • / GitLab Community Edition is open source, but GitLab Enterprise Edition (which we would need for a project of our scale and with our workflow needs) is not open source, only source-available.
  • API availability
    • GitLab has a fairly robust api, but in our evaluation we found that many api features were undocumented, and others were deprecated without notification between minor versions.
  • Hybrid integration
    • A hybrid integration between Drupal.org and GitLab is possible with SSO, automatic repository syncing, and a choice of using our issue queue or theirs.
    • ❌ However, there would be no portability of issues across a project that uses GitLab to one that does not.
    • ❌ Projects choosing to use the GitLab issues would not have DrupalCI integration, or issue credits.
  • Data portability/vendor lock-in
    • GitLab's API and documented data model should make it possible to migrate our data to another system should it become necessary.
  • Developer support
    • As an open-source project, GitLab has a growing community of contributors, as well as the support of a venture funded mothership organization.
  • Maintainability for an engineering team of limited resources
    • ❌ Hard Blocker - GitLab's current file-handling makes full copies of a repository for every fork. Taking the number of open core issues as a representative example, Drupal.org would need a new 200 TB+ redundant storage apparatus, just to accommodate Core. GitLab needs to transition to a shared git-object model, or an alternative like git namespaces.
  • Cost
    • The GitLab team is willing to provide the Enterprise Edition to the Drupal community for free.
  • Unintended consequences
    • n/a

GitLab: not-yet.

BitbucketBitbucket

Bitbucket was the dark horse candidate. Atlassian products are well known in software development circles, but they are often more tolerated than beloved. However, when we sat down at Midwest Drupal Summit in August and looked at the options we explored so far and found them to be not quite baked - Bitbucket began to show some advantages as a solution.

For one thing, Bitbucket has recently put a heavy focus on user experience and collaboration features. It also has the most flexible permissions model for control over branch permissions, forking, and automatic rebasing of any of the toolsets we evaluated. Finally, it can be implemented modularly - allowing us to use the components that serve our workflow and disable the ones that don't.

Criteria:

  • / Familiar workflow
    • Bitbucket implements a very standard pull request workflow. It is not as widely popular as GitHub or GitLab, but should be easily learned by developers familiar with either of those systems.
  • Preserve the most valuable parts of Drupal's unique workflow
    • Of all the options we prototyped, Bitbucket is the only one which provides the flexibility needed to preserve key elements of our workflow - in particular the ability to cleanly use only the parts of Bitbucket that we need (specifically, merge requests, inline editing, and code commenting) without having to use issues or project pages that are less sophisticated than our existing solutions.
  • Leverage a partner who will keep evolving their code collaboration featureset.
    • Though it's flown under the radar, Atlassian has been making some significant improvements to Bitbucket from a feature and user experience point of view over the course of the past year.

Other issues:

  • License/Source availability
    • /❌ Bitbucket is not open-source, only source-available, though they have an open source licensing program.
  • API availability
  • Hybrid integration
    • Bitbucket is much more flexible in terms of choosing which features to use, and which to disable- allowing a fairly tight hybrid integration.
  • Data portability/vendor lock-in
    • / Bitbucket stores repositories as standard git objects on the file system, making it relatively straightforward in case of a future migration to another system.
  • Developer support
    • Bitbucket has the professional support of Atlassian, but does not have a robust open source community driving feature development.
  • Maintainability for an engineering team of limited resources
    • Bitbucket is implemented with an api-first methodology, and is well documented, making it straightforward for a relatively small team to support a selective integration such as what we propose here.
  • Cost
  • Unintended consequences
    • n/a

Bitbucket: the best tool in the current landscape given our community's needs.

To sum up, each of the toolsets we evaluated have advantages and disadvantages. However, if we want to move forward with making improvements for our developers TODAY, we have determined that the best way to do that is to create modular integration points, and use the toolset that currently meets our criteria: Bitbucket. As these toolsets continue their rapid development, we will continue to periodically re-evaluate them.

In our next post:

An illustration of what the future of Drupal.org's developer tools could look like.

Dec 20 2017
Dec 20

The initiative to improve Drupal.org's developer tools is part of a broader effort to broaden the reach of Drupal, not just to end-users and evaluators, but to a wider audience of developers as well. Improvements to our tools are an opportunity to remove friction to changes, increase the quality of changes, improve velocity of changes, and make contributing to Drupal delightful.

The initiative began in coordination with Drupal Association Staff, the Technical Advisory Committee, and a small group of volunteers from the community. We first announced the initiative in October of 2016, and provided our last update at the end of April of this year. Since that time a great deal of work has been underway to evaluate and prototype our options, collaborate with the vendors who supply these toolsets, and make some decisions about the direction we want to move in, with more data in hand.

Buckle up! Because this is such a large topic we've broken it into multiple posts:

Want a TLDR?

Where do we stand now?

Drupal is one of the longest-running open source projects on the web, and for more than 15 years (and many more to come), Drupal.org has been the home of the project. Over the years, we've built a developer toolset to serve the unique needs of the project. These tools have evolved as the open source environment changed. Right now, we use a combination of best-of-breed third party technologies, such as Git, Jenkins, Fastly, OpsGenie, integrated with our own bespoke tools, including issue queues, project pages, etc.

Right now, some of the third party and bespoke tools that we are using are market leading, whereas others have fallen behind. For example, CGIT (third party) and the patch workflow (bespoke) have both fallen behind compared to toolsets that can be found on other tooling providers. On the other hand, our issue crediting system (bespoke) is market leading, and a model for other projects to follow.

In the end, we will always be negotiating a balance between what we can do uniquely well with bespoke solutions, and integrating the latest and greatest of third party solutions as they coalesce around best practices that we want to adopt as well.

At DrupalCon Vienna it was decided that the Technical Advisory Committee had fulfilled their threefold mandate:

  • Helping the technical leadership transition on the DA engineering team after the downsizing in summer of 2016.
  • Helping manage the prioritization and scope of work.
  • Making recommendations to the board and DA staff on key initiatives, such as the project application process changes, and the direction to take with our developer tools.

We want to thank Angie Byron, Steve Francia, and Moshe Weitzman for serving on the committee. From here, DA engineering staff will carry the torch, though we will continue to rely on the individuals who were part of the TAC and other volunteers for feedback and help from time to time as we move forward.

Our evaluation

The most recent phase of this initiative was an evaluation of the leading contenders for open source tooling providers that Drupal.org could integrate with. These options were GitHub and GitLab, and after setting out to develop prototypes for these options, we added BitBucket to the list as well. For each of these options we built MVP prototypes, either as integrations with Drupal.org development environments, or with private organizations/repositories. Finally, we wanted to compare these options to the bespoke Issue Workspaces solution that we had proposed several years ago.

The Criteria

For the Developer Tools initiative to be successful we have to understand the criteria for improvement. From a user requirements point of view, contrib and core developers are unique stakeholders with unique requirements. Ideally we want a solution that increases velocity for both types of users, without fundamentally sacrificing the needs of one or the other.

  • Adopt a developer workflow that will be familiar to the millions of developers outside our community.
  • Preserve those unique elements of how we collaborate that have made the Drupal project so successful.
    • Many-to-one collaboration: that is to say, many developers collaborating on a single solution to a problem.
    • Issue workflow.
    • Picking up on long standing issues where other collaborators left off.
    • Contribution credit.
  • If possible, leverage an expert partner who will help keeping our tooling up to date as open source collaboration tools continue to evolve.

There are also some technical requirements that came out of our evaluation process.

  • Retention of our data/ability to migrate.
    • Where possible, retain existing Git remote urls for projects.
  • Maintainability for a small staff.
  • Project maintainer management: including abandoned project reassignment, fork control, security release management, etc.

In terms of features, we were looking for:

  • Merge/pull requests.
  • Code review.
  • Inline editing.
  • Branch permissioning to allow collaboration on merge/pull requests.
  • Administrative tools for managing project maintainership.
  • Project management tools that equal or exceed what we have with the issue queues.*
  • Extensibility, so that we can preserve areas where the Drupal project is a market leader, such as with our contribution credit system.

*Our tools for issues are very sophisticated on an individual issue level, however we are sorely lacking in tools for grouping and prioritizing sets of issues, ie: issue boards.

Finally, cost:

  • Given the care with which we must use our funding from the community, any option we consider must be cost neutral with the current cost of maintaining our tools.

It's difficult to condense the scope of our evaluation into a short blog series, but in the following posts we'll talk about how the options we considered measured up to these criteria, what implementing one of these options could look like for Drupal.org, and our suggested implementation roadmap.

Dec 12 2017
Dec 12

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Announcements

Clarifications to Drupal licensing policy

There have been some long-standing questions about Drupal project licensing policy. In collaboration with the Licensing Working Group and Dries, we have updated the official licensing policy with the following clarifications and changes:

  • We explicitly affirm that code with a GPL-compatible license can be included in a Drupal.org hosted project, but will be redistributed under our standard "GPL2 or later" policy.
  • We clarify that GPL-incompatible non-code assets may be packaged and/or distributed "in aggregate" with GPL code in Drupal.org projects, per the final stipulation of Sec 3-2 of the GPL, so long as the maintainer has the rights to do so.
  • We clarify that Drupal.org hosted projects can depend on and/or link to GPL-incompatible code (via composer, for example), but that Drupal.org cannot host or distribute those GPL-incompatible dependencies.
  • We explicitly affirm our interpretation of the GPL that a Drupal service provider's act of assembling a codebase while under contract to a client does not fall under the more restrictive terms of 'distribution' per the GPL.

You can find more detail about these changes in the Repository Usage Policy and Licensing FAQ.

Welcoming new staff members

Brooke CandelariaWe want to welcome two new members to the Drupal Association team. Brooke Candelaria who will be taking over as Conference Director, joins us from Houston, TX. Brooke has a background in PR and event management, and has worked with other open source communities in the past, for example working on LinuxWorld and, most recently, with the Python community. Brooke's focus will be on evolving DrupalCon to meet the needs of every persona within our community, and helping to make the Con more sustainable and scalable.

Rachel Lawson in MoldovaRachel Lawson is taking on the role of Community Liaison. Based in the UK, Rachel has been a tremendous member of the community in her own right, participating in Camps, helping to organize Mentored Sprints, and serving on the Community Working Group. Rachel will be helping the community to understand the role and the functioning of the Drupal Association, while also keeping the Drupal Association more closely tied to all parts of our diverse community. We hope you'll welcome them!

DrupalCon Updates

DrupalCon | Be Human, Think DigitalLaunched new DrupalCon brand

We've just launched a new unified look and feel for DrupalCon, that will carry into all of our events moving forward. This will help give DrupalCon a stronger identity among OSS events, and also give us a place to put centralized content that applies to DrupalCon as a whole, rather than just a singular event.

Launched DrupalCon Nashville site

Alongside our new brand, we've launched the DrupalCon Nashville website, the first of the events to use the new unified look and feel. Registration is open now, as well as the call for papers. Take a look at some of the new tracks! \

Nashville guitar

We'll see you there!

Sponsorship makes DrupalCon possible - learn why you should be a part of it.

Drupal.org Updates

Reminder: New issue shortcuts and friendly url structure

Last month we announced that we would be implementing some changes to the issue url structure, as well as some shortcuts to help users navigate to issues more easily on Drupal.org. These changes went live in November. Here's a primer:

URL Pattern for issues:

https://drupal.org/project/drupal/issues/2922626 When an issue is moved between projects the alias will be updated.

Shortcuts

The search bar will now automatically redirect you to a node if you enter its id directly: Search Bar Node ID shortcut A new menu callback will help you get to issues with a shorter url string: https://drupal.org/i/

New individual member directory

We've been overhauling the Drupal Association membership experience, and as part of that process we now have a new directory of Drupal Association members. This new directory also includes new filters and sorting options so that you can see the latest community members to join the Association and filter by country, username, or first and last name.

In development: new organization member directory

We're working on a new directory of organization members as well. The current directory allows you to filter by organization type, and we will be adding a filter for Association membership as well. If you have feedback on this directory, please let us know!

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association. Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Nov 17 2017
Nov 17

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Announcement

New issue shortcuts and friendly url structure

Drupal contributors have been managing bug fixes, feature requests, and code reviews on Drupal.org for around 15 years now. Passing an issue node id around a sprint table is a stable of DrupalCon and camps around the world. In October we announced that we would be implementing some changes to the issue url structure, as well as some shortcuts to help users navigate to issues more easily.

URL Pattern for issues:

https://drupal.org/project/drupal/issues/2922626
When an issue is moved between projects the alias will be updated.

Shortcuts

The search bar will now automatically redirect you to a node if you enter its id directly: Search Bar Node ID shortcut

A new menu callback will help you get to issues with a shorter url string:
https://drupal.org/i/<nid>

And of course you can still use the old https://drupal.org/node/<nid> urls if you still have them bookmarked.

Spoiler alert: These shortcuts and new url patterns were deployed in November and you can use them right now!

Drupal.org updates

Composer instructions on release pages

To make it easier for site builders to figure out how to use a release with composer we've added the composer command line instructions to release pages.

Composer instructions on release nodes

This command installs the package with the current release number specified as a minimum version parameter. We also provide a link to the documentation on using Composer to manage Drupal site dependencies, to help users who may be unfamiliar with Composer learn how to use it.

The Community Section

The community is the heart of the Drupal project, but until now community news has not had it's own place to live. We've now made the community page a proper section with its own blog, so that community posts and CWG information has a dedicated place to live.

When the first posts in this new section go live, we'll add this blog feed to Drupal Planet as well. Over time, we hope to further refine the community section and improve the tools we provide for the community to connect with each other.

WYSIWYG for Forums (CKEditor)

We're always looking for ways to make improvements to the site that have a high impact to effort ratio. One such change was enabling the CKEditor for editing in the forums. CKEditor has been in the wild as a WYSIWYG editor on Drupal.org for other content types for quite a while now, and we felt confident it was ready for use on forums as well.

CKEditor WYSIWYG for the Forums

Bug-fix: Dev releases on project pages

In the runup to DrupalCon Vienna we made a number of improvements to project pages - however a bug or two crept in as well. A race condition was causing dev releases not to display in some cases, and we resolved this issue in October. If you're a project maintainer on Drupal.org and see anything else go missing, please let us know!

Dev Releases

Infrastructure

DrupalCI: Faster, more affordable testing

DrupalCI uses spot requests on Amazon Web Services to spin up testbots on-demand for the Drupal project. In the past, instances were provisioned in minimum increments of one hour, meaning to make the most of testing we had to queue up tests to reuse the remainder of any paid-for instance-hours.

Because AWS has enabled per-second billing, we no longer have to try to fill instance-hours, and so we have reconfigured our spot instance requests to provision testbots faster, while still saving money overall compared to the previous configuration.

DrupalCI: More efficient RTBC testing

We also discovered that a bug in our automated RTBC retesting system was triggering more tests than necessary. We've fixed the bug, and now only the most appropriate recent test/environment will be retested for RTBC issues.

Server Maintenance Windows

Finally, we scheduled several maintenance windows in cooperation with our infrastructure services partner to schedule updates/restarts of our servers.

If you want to keep up-to-date with Drupal.org-related changes and maintenance windows you can subscribe to our Change Notices, or follow us on Twitter.

https://t.co/57fE1VZHTn db maintenance window is complete. We may schedule a follow up maint window, and will notify here if needed.

— Drupal infra (@drupal_infra) October 31, 2017

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Nov 10 2017
Nov 10

Drupal developers have been shouting node id numbers across tables at sprints for almost 15 years now. The Drupal Association has added some shortcuts and friendly urls for issues to make this a little easier.

Shortcuts

The first shortcut we added is a searchbar feature. If you enter any node id in the search bar you'll be automatically redirected to that node. This works for more than just issues!

Searchbar Node Shortcut

The second shortcut we added is a quick url pattern for finding issues. If you know your node id, just type:

https://drupal.org/i/<node-id>

 and you're there!

Friendly-Urls - Updated 2017-11-12

We've implemented a new friendly url pattern on Drupal.org to make the canonical urls more sensible and search engine friendly.

Project pages:  Unchanged https://drupal.org/project/drupal

Issue Queue:  Unchanged (for now*) https://drupal.org/project/issues/drupal

UPDATED - Specific Issue: https://drupal.org/project/drupal/issues/2922626

This new pattern should be backfilled to all issues by the beginning of next week.

And of course, if you have old bookmarks linked, you can still get there by going to: https://drupal.org/node/<node-id>

These are small, but powerful improvements that should make the lives of everyone who contributes to the Drupal project easier.

_________________
UPDATED - *After this change to specific issue urls will we create a follow up to change issue queue urls to match the pattern: https://drupal.org/project/drupal/issues

Oct 17 2017
Oct 17

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

DrupalCon Vienna

We're back from DrupalCon Vienna, with updates on what's new from the month of our European event.

Announcement

TLS 1.0 and 1.1 deprecated

Drupal.org uses the Fastly CDN service for content delivery, and Fastly has depreciated support for TLS 1.1, 1.0, and 3DES on the cert we use for Drupal.org, per the mandate by the PCI Security Standards Council. This change took place on 9 Aug 2017. This means that browsers and API clients using the older TLS 1.1 or 1.0 protocols will no longer be supported. Older versions of curl or wget may be affected as well.

Drupal.org updates

DrupalCon Calendar syncing

In our last update, we teased a new feature for DrupalCon attendees - the ability to sync your personal schedule to a calendar program. We're pleased to report that this feature made it in time for the event, and was used by attendees throughout the week. If you've already synced your calendar for DrupalCon Vienna, you're already set up to use the same feed for DrupalCon Nashville next April!

Keynote simulcast to Youtube

This year at DrupalCon, in addition to live streaming on Events.Drupal.org itself, we simulcast the keynotes to YouTube. We also embedded the keynote on the Drupal.org homepage - to spread the latest news about Drupal beyond DrupalCon attendees.

In fact, if you couldn't attend DrupalCon or just missed the keynotes, you can watch Dries' update on the Drupal project here:

[embedded content]

Industry Pages promoted in the front page Call-to-Action

We've also made some updates to how the industry pages are promoted. In addition to the dedicated block with icons linking to each industry, we now also promote the industry solutions landing page in the main CTA under the homepage header.

Industry Page CTA

We hope to further encourage users evaluating Drupal to explore some of the tremendous solutions that are already out there, and take inspiration from their success.

First-in/First-out issue sorting

To make sure that issues are reviewed by maintainers in the order they are received, it is now possible to sort the issue queues by when the issue status last changed. This means RTBC issues can be reviewed on a first-in/first-out basis!

This 'status changed' date field is available on the advanced search view for any issue queue. Here's what it looks like for Drupal core:

Issue Status Sort

Project creation analysis

About six months ago we opened up project creation on Drupal.org to allow any confirmed user to create a full project. We've put together a blog post outlining the impact these changes have had on the contrib landscape. In short, we've seen a tremendous increase in the rate of project creation, and the rate of applications for security advisory coverage, and a modest increase in projects receiving stable releases without yet opting in coverage. We're continuing to monitor project creation and work with the Security Working Group and others on next steps.

Displaying orphan dev releases

In last month's update we talked about a variety of changes we made to project pages, to provide better signals about project quality to evaluators. In response to feedback, we've restored the visibility of dev releases, even when they aren't associated with a tagged release.

Dev releases

This is particularly helpful for project maintainers trying to bring visibility to the next major development version of their modules, such as their Drupal 8 module port efforts.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Oct 07 2017
Oct 07

About six months ago we made a significant change to the way that modules, themes, and distributions are created on Drupal.org.

In the past, contributors had to first create a sandbox project, and then request manual review of their project in the Project Applications issue queue. The benefit of this community-driven moderation process was that modules were vetted for code quality and security issues by a group of volunteers. Project maintainers who completed this process also received the benefit of security advisory coverage from the Security Team for stable releases of their projects.

Unfortunately, the rate of project applications outpaced what volunteers could keep up with, and many worthy projects were never promoted to full project status, or moved off of Drupal.org to be hosted elsewhere.

To ameliorate this issue, we changed the process so that any confirmed user on Drupal.org may now make full projects.

To mitigate the risks of low code quality or security vulnerabilities we added new signals to project pages: including highlighting which release is recommended by the maintainer, displaying recent test results, and indicating whether the project receives security coverage both on the project page and in the composer 'extra' attribute. We're continuing to work on identifying additional signals of project quality that we can include, as well as surfacing some of this information in Drupal core. We also converted the project applications issue queue into a 'request security advisory coverage' issue queue.

What we hoped to see

We knew this would be a significant change for the project and the community. While many community members were excited to see the gates to contribution opened, others were concerned about security issues and Drupal's reputation for code quality.

Our prediction was that the lower barrier to contribution would result in an increase in full projects created on Drupal.org. This would indicate that new contributors or third party technology providers were finding it easier to integrate with Drupal and contribute those integrations back for use by others.

At the same time, we also expected to see an increase in the number of full projects that do not receive coverage from the security team. The question was whether this increase would be within an acceptable range, or represent a flood of low quality or insecure modules.

The results

The table below provides statistics about the full projects created on Drupal.org in the 5 months before March 17th, 2017 - when we opened the creation of full projects to all confirmed users.

Full projects created from 2016-10-16 to 2017-03-17…

#

% of projects created in this period

… without stable release

431

55.76%

… with stable releases

342

44.24%

… with usage >= 50 sites

237

30.66%

… with usage >= 50 sites and without stable release

68

8.80%

… with usage >= 50 sites and with stable release

169

21.86%

… with an open security coverage application*

18

2.33%

Sub-total with security coverage

342

44.24%

Sub-total without security coverage

431

55.76%

Sub-total with security coverage and >=50 usage

169

21.86%

Sub-total without security coverage and >= 50 usage

68

8.80%

Total

773

* note: full projects that did not have stable releases were not automatically opted in to security coverage when we opened the full project creation gates.

… and this table provides statistics about the projects created in the 5 months after we opened the creation of full projects to all confirmed users:

Full projects created from 2017-03-17 to 2017-08-16…

#

Diff

% of projects created

Diff %

… without stable release

851

+420

69.53%

+97%

… with stable releases

373

+31

30.47%

+9%

… with usage >= 50 sites

156

-81

12.75%

-34%

… with usage >= 50 sites and without stable release

64

-4

5.23%

-6%

… with usage >= 50 sites and with stable release

92

-77

7.52%

+46%

… with an open security coverage application

62

+44

5.07%

+344%

Sub-total with security coverage

182

-160

14.87%

-53%

Sub-total without security coverage

1,042

+611

85.13%

+242%

Sub-total with security coverage and >=50 usage

54

-115

4.41%

-32%

Sub-total without security coverage and >= 50 usage

102

+34

8.33%

+150%

Total

1,224

+451

+58%

As you can see, we have an almost 58% increase in the rate of full projects created on Drupal.org. We can also see a significant proportional increase in two key areas: projects with greater than 50 site usage and no security coverage(up 150% compared to the previous period), and projects that have applied for security coverage(up 344% compared to the previous period). Note: this increase in applications is for projects *created in these date ranges* not necessarily applications created overall.

This tells us that reducing friction in applying for security coverage, and encouraging project maintainers to do so should be a top priority.

Finally, this last table gives statistics about all of the projects currently on Drupal.org, regardless of creation date:

Full projects (7.x and 8.x)

#

% of Total

Rate of change after 2017-03-17

… with the ability to opt into security coverage

8,718

36.15%

-1.33%

… with security coverage and stable releases

8,377

34.74%

-1.49%

… without security coverage

15,396

63.85%

+1.33%

… without security coverage and with stable releases

464

1.92%

+1.04%

… with security coverage and >=50 usage
 

6,475

66.91 / 26.85%

-0.54%

… with security coverage and stable releases and >=50 usage

6,308

65.19 /26.16%

-0.65%

… without security coverage and >=50 usage

3,202

33.09 /13.28%

+0.54%

… without security coverage and with stable releases and >=50 usage

130

1.34 /0.54%

+0.51%

Sub-total with >=50 usage

9,677

40.13%

-1.72%

Total

24,114

From the overall data we see approximately what we might expect. The increase in growth of full projects on Drupal.org has lead to a modest increase in projects without security coverage.

Before the project application change, all full projects with stable releases received security advisory coverage. After this change, only those projects that apply for the ability to opt in(and then do so) receive coverage.

What has this meant for security coverage of projects hosted on Drupal.org?

1.92% of all full 7.x and 8.x projects have stable releases, but do not receive security advisory coverage. It is likely no accident that this translates into 464 projects, which is nearly equivalent to the number of projects additional projects added compared to our old growth rate.

Of those only 130 of those projects report more than 50 sites usage(or .54% of all 7.x and 8x full projects).

Next steps

From this analysis we can conclude the following:

  1. The opening of the project application gates has dramatically increased the number of projects contributed to Drupal.org.

  2. It has also increased the number of projects without security coverage, and the number of applications for the ability to opt in to coverage among new projects.

In consultation with the Security Working Group, we recommend the following:

  • For now, leave the project creation projects as it stands today - open to contribution from any confirmed user on Drupal.org.

  • Solve the problem of too many security advisory coverage applications. The security advisory application queue has the same problem that the old project applications queue had - not enough volunteers to manually vet all of the applications - and therefore a significant backlog of project maintainers waiting on the ability to opt into coverage.

    • Recommendation: Implement an automated best practices quiz that maintainers can take in order to be granted the ability to opt into security advisory coverage. If this process is as successful as we hope, we may want to consider making this a gate on stable releases for full projects as well.

We look forward to working with the Security Working Group to implement this recommendation and continue to improve the contribution experience on Drupal.org, while preserving code quality and security.

Sep 19 2017
Sep 19

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Announcement

TLS 1.0 and 1.1 deprecated

Drupal.org uses the Fastly CDN service for content delivery, and Fastly has depreciated support for TLS 1.1, 1.0, and 3DES on the cert we use for Drupal.org, per the mandate by the PCI Security Standards Council. This change took place on 9 Aug 2017. This means that browsers and API clients using the older TLS 1.1 or 1.0 protocols will no longer be supported. Older versions of curl or wget may be affected as well.

Almost time for DrupalCon Vienna

DrupalCon Vienna

DrupalCon Vienna is almost here! From September 26-29 you can join us for keynotes, sessions, and sprinting. Most of the Drupal Association engineering team will be on site, and we'll be hosting a panel discussion about recent updates to Drupal.org, and our plans for the future.

We hope to see you there!

Drupal.org updates

8.4.0 Alpha/Beta/Release Candidate 1

On August 3rd, Drupal 8.4.0 received its alpha release, followed on the 17th by a beta release, and on September 6th by the first release candidate. Several new stable API modules are now included in core for everything from workflow management to media management. Core maintainers hope to reach a stable release of Drupal 8.4 soon.

Improvements to Project Pages

We made a number of improvements to project pages in August, one of which was to clean up the 'Project information' section and add new iconography to make signals about project quality more clear to site builders.

Project information improvements

In the same vein, we've also improved the download table for contrib projects, by making it more clear which releases are recommended by the maintainer, providing pre-release information for minor versions, and displaying recent test results.

Download table improvements

Metadata about security coverage available to Composer

Developers who build Drupal sites using Composer may miss some of the project quality indicators from project pages on Drupal.org. Because of this, we now include information about whether a project receives security advisory coverage in the Composer 'extra' attribute. By including this information in the composer json for each project, we hope to make it easier for developers using Composer to ensure they are only using modules with security advisory coverage. This information is also accessible for developers who may want to make additional tools for managing composer packages.

Automatic issue credit for committers

Just about the last step in resolving any code-related issue is for a project maintainer to commit the changes. To make sure these maintainers are credited for the work they do to review these code changes, we now automatically add issue credit for committers.

Performance Improvements for Events.Drupal.org

With DrupalCon coming up in September we spent a little bit of time tuning the performance of Events.Drupal.org. We managed to resolve a session management bug that was the root cause of a significant slow down, so now the site is performing much better.

Syncing your DrupalCon schedule to your calendar

A long requested feature for our DrupalCon websites has been the ability to sync a user's personal schedule to a calendar service. In August we released an initial implementation of this feature, and we're working on updating it in September to support ongoing syncing - stay tuned!

Membership CTA on Download and Extend

We've added a call to action for new members on the Drupal.org Download and Extend page, which highlights some great words and faces from the community. Membership contributions are a crucial part of funding Drupal.org and DrupalCon, but much the majority of traffic we receive on Drupal.org is anonymous, and may not reach the areas of the site where we've promoted membership in the past. We're hoping this campaign will help us reach a wider audience.

Membership CTA on the Download page

DrupalCI sponsorship

DrupalCI is one of the most critical services the Drupal Association provides to the project, and also one of the more expensive. We've recently added a very small section to highlight how membership contributions help provide testing for the project - and in the future we hope to highlight sponsors who will step up specifically to subsidize testing for the Drupal project.

Infrastructure

More semantic labels for testing

In August we added more semantic labels for DrupalCI test configuration. This means that project maintainers no longer have to update their testing targets with each new release of Drupal, they can instead test against the 'pre-release' or 'supported' version, etc. More information can be found in the DrupalCI documentation.

Semantic Labels for Testing

Started PCI audit

In August we also began a PCI audit, and developed a plan of action to reduce the Drupal Association's PCI scope. Protecting our community's personal and financial information is critically important, and with a small engineering team, the more we can offload PCI responsibility onto our payment vendors the better. We'll be continuing to work on these changes into the new year.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

May 24 2017
May 24

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

DrupalCon Baltimore logo Apr 24-28

At the end of April we joined the community at DrupalCon Baltimore. We met with many of you there, gave our update at the public board meeting, and hosted a panel detailing the last 6 months worth of changes on Drupal.org. If you weren't able to join us for this con, we hope to see you in Vienna!

Drupal.org updates

DrupalCon Vienna Full Site Launched!

DrupalCon Vienna logo Sep 26-29 2017

Speaking of Vienna, in April we launched the full site for DrupalCon Vienna which will take place from September 26-29th, 2017. If you're going to join us in Europe you can book your hotel now, or submit a session. Registration for the event will be opening soon!

DrupalCon Nashville Announced with new DrupalCon Brand

DrupalCon Nashville logo Apr 9-13 2018

Each year at DrupalCon the location of the next conference is held as closely guarded secret; the topic of speculation, friendly bets, and web crawlers looking for 403 pages. Per tradition, at the closing session we unveiled the next location for DrupalCon North America - Nashville, TN taking place from April 9-13th in 2018. But this year there was an extra surprise.

We've unveiled the new brand for DrupalCon, which you will begin to see as the new consistent identity for the event from city to city and year to year. You'll still see the unique character of the city highlighted for each regional event, but with an overarching brand that creates a consistent voice for the event.

Starring Projects

Users on Drupal.org may now star their favorite projects - making it easier to find favorite modules and themes for future projects, and giving maintainers a new dimension of feedback to judge their project's popularity. Users can find a list of the projects they've starred on the user profile. Over time we'll begin to factor the number of star's into a project's ranking in search results.

Starring Projects

At the same time that we made this change, we've also added a quick configuration for managing notification settings on a per-project basis. Users can opt to be notified of all issues for a project, only issues they've followed, or no issues. While these notification options have existed for some time, this new UI makes it easier than ever to control issue notifications in your inbox.

Project Browsing Improvements

One of the important functions of Drupal.org is to help Drupal site builders find the distributions, modules, and themes, that are the best fit for their needs. In April, we spent some time improving project browsing and discovery.

Search is now weighted by project usage so the most widely used modules for a given search phrase will be more likely to be the top result.

We've also added a filter to the project browsing pages to allow you to filter results by the presence of a supported, stable release. This should make it easier for site builders to sort out mature modules from those still in initial development.

Better visual separation of Documentation Guide description and contents

Better Documentation Guide Display

In response to user feedback, we've updated the visual display of Documentation Guides, to create a clearer distinction between the guide description text and the teaser text for the content within the guides.

Promoting hosting listings on the Download & Extend page

To leverage Drupal to the fullest requires a good hosting partner, and so we've begun promoting our hosting listings on the Download and Extend page. We want Drupal.org to provide every Drupal evaluator with all of the tools they need to achieve success—from the code itself, to professional services, to hosting, and more.

Composer

Sub-tree splits of Drupal are now available

Composer Façade

For developers using Composer to manage their projects, sub-tree splits of Drupal Core and Components are now available. This allows php developers to use components of Drupal in their projects, without having to depend on Drupal in its entirety.

DrupalCI

Automatic Requeuing of Tests in the event of a CI Error

DrupalCI logo

In the past, if the DrupalCI system encountered an error when attempting to run a test, the test would simply return a "CI error" message, and the user who submitted the test had to manually submit a new test. These errors would also cause the issues to be marked as 'Needs work' - potentially resetting the status of an otherwise RTBC issue.

We have updated Drupal.org's integration with DrupalCI so that instead of marking issues as needs work in the event of a CI Error, Drupal.org will instead automatically queue a retest.

Bugfix: Only retest one environment when running automatic RTBC retests

Finally, we've fixed a bug with the DrupalCI's automatic RTBC retest system. When Drupal HEAD changes, any RTBC patches are automatically retested to ensure that they still apply. It is only necessary to retest against the default or last-used test environment to ensure that the patch will work, but the automatic retests were being tested against every configured environment. We've fixed this issue, shortening queue times during a string of automatic retests and saving testing resources for the project.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Apr 18 2017
Apr 18

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

DrupalCon Baltimore logo Apr 24-28

The Drupal Association team is gearing up for DrupalCon Baltimore. We're excited to see you there and we'll presenting a panel giving an update on our work since Dublin, and our plans for the coming months.

Drupal.org updates

Project application revamp

As we announced in mid-March, new contributors on Drupal.org can now create full projects and releases! Contributors no longer have to wait in the project application queue for a manual review before they are able to contribute projects.

This is a very significant change in the Drupal contribution landscape, and it's something we approached carefully and will continue to monitor over the coming months. Drupal has always had a reputation for a high quality code, and we want to make sure that reputation is preserved with good security signals, project quality signals, and continued incentives for peer code review.

That said, we're very excited to see how this change opens up Drupal to a wider audience of contributors.

Please note that the removal of project applications to create full projects and releases means a change in the security advisory policy (see below for details).

Security Advisory Opt-in and new Security Signals for Projects

Are you responsible for the security of your clients' Drupal sites?

Please note that Drupal's security advisory coverage policy has changed. Security advisory coverage for contributed projects is now only available for projects that have both opted in to receive coverage and made a stable release. You can see which projects have opted in by checking their project pages. If you have questions, please contact [email protected].

Because users may now create full projects and releases without opting in to security advisory coverage, it's critically important that we provide good security signals to users evaluating projects on Drupal.org. This is why we've added a security coverage warning to projects that aren't opted in to coverage.

We've also:

2017 Community Elections Update

The 2017 elections for the community-at-large seat on the board were held successfully in March. Drupal Association community board elections are conducted with the Instant Runoff Voting system. This voting methodology requires that voters rank their preferred candidates on their ballot, and we've heard that this system has been somewhat unwieldy in the past.

Each year we try to improve the voter experience and so this year we deployed a new drag-and-drop ballot.

Drag and Drop Ballot

Finally, we want to congratulate our newest board member Ryan Szrama!

Better international datetime support throughout Drupal.org

Drupal.org has grown organically over the course of more than a decade, and as features have been built out they were not always consistent in their display of datetime information. While it sometimes makes sense to have a few different formats for displaying date and time, many of the formats in use were simply arbitrary historical decisions.

As a quality of life improvement, especially for users outside of the USA, we've standardized the datetime format used on Drupal.org. That format is: DD MMM YYYY - hh:mm (UTC±h). For example: 11 Aug 2016 - 16:42 (UTC+8)

DrupalCI

CSS Lint check style results

DrupalCI logo

When we implemented coding standards testing in DrupalCI in February we were not able to add CSS Lint testing until the CSSLint configuration file in core was fixed. That issue was fixed in late February and so we added CSSLint to support coding standards testing for CSS at the beginning of March.

Cleaning up coding standards results

The addition of coding standards results to DrupalCI means that Drupal.org is now storing even more test data about the code we test on Drupal.org. Our initial implementation of coding standards testing did not include clean up of older results, and so to preserve database space and testing resources, we implemented some clean-up routines in March. In particular we are now:

  • Cleaning up all results for closed issues
  • For custom one-off tests, keeping results for 30 days to match what is shown on project’s automated testing tab
  • For tests triggered on a schedule or commit, keeping the most recent per-environment per-branch, and keeping anything less than 24h old

Infrastructure

Protecting Git services

We experienced some minor Git outages in March, due to malicious authentication attempts. To mitigate these issues in the future, we've implemented fail2ban rules to protect Git authentication. This should improve the stability and uptime of Git services for all developers on Drupal.org.

We want to thank Drupal.org infrastructure volunteer mlhess for his assistance with this.

Community Initiatives

Contrib Documentation Migration

New tools for Documentation have been available on Drupal.org for more than half a year. While most of the core documentation has been migrated to the new system, we are still encouraging Contrib maintainers to migrate their docs.

To make it easier for contrib project maintainers to migrate their documentation to the new documentation tools, we've made two improvements:

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Apr 11 2017
Apr 11

Workflow iconIn October of last year the Technical Advisory Committee was formed to evaluate options for the developer tools we use on Drupal.org. The TAC consists of Angie Byron, Moshe Weitzman, and Steve Francia, acting as advisors to Megan Sanicki, the Executive Director of the Drupal Association.

The TAC's mandate is to recommend a direction for the future of our tools on Drupal.org. Megan will evaluate this recommendation, make a decision, and prioritize that work in the development roadmap of the Drupal Association engineering team.

What is the motivation behind looking at our developer tools now?

Close followers of the Drupal project will have noticed a trend in the last several months. From Dries' announcement of easy upgrades forever, to the revamp of the project application process, to the discussion about making tools for site builders— there is a unifying theme: broadening the reach of Drupal.

This is the same motivation that underlies this evaluation of our developer tools, and defines the goals and constraints of this initiative:

  • Adopt a developer workflow that will be familiar to the millions of developers outside our community
  • Preserve those unique elements of how we collaborate that have made the Drupal project so successful
  • If possible, leverage an expert partner who will help keeping our tooling up to date as open source collaboration tools continue to evolve

This means looking at a number of elements of the Drupal.org developer tool stack:

  • The underlying git service
  • How we tag and package releases
  • The contribution workflow (patch vs. pull request)
  • Project management workflows (the issue queues and tags)
  • CI integration
  • Maintainership
  • Project pages

If this looks like a tremendous undertaking - that's because it is. But there are some things we already know:

  • Drupal.org should continue to be the home of project pages
  • We should adopt a pull request workflow (and ideally we want to be able continue to accept patches as well, at least in the interim)
  • We should move contrib projects to semver, following core's lead
  • We want to preserve our familiar understanding of maintainership
  • We want to avoid forked code and forked conversation
  • We want to ensure the security team still has the tools they need to provide their service to the community

We also know that whatever decision is made, these changes cannot happen all at once. We'll need to take a progressive approach to the implementation, and focus on the parts of the stack that need to change together, so that we don't bite off more than we can chew.

What options are being considered?

At this time, the technical advisory committee is considering three options as they prepare to make their recommendation. The options are: GitLab, which offers both self-hosted and SaaS options; GitHub, which has recently been adding long-requested new features; or continuing to evolve our custom-built tooling, perhaps via issue workspaces.

GitLab

GitLab is the up-and-comer among Git hosts. GitLab can be self hosted using either their community or enterprise editions, or repositories can be hosted at GitLab.com. Though they recently stumbled, they have been notably open and transparent about their efforts to build a leading collaboration platform.

Gitlab is itself open-source, and has just released its 9.0 edition. GitLab has aggressively pursued the latest in development tools and workflow features, including project management tools, a ui for merge conflict resolution with in-line commenting and cherry-picking, docker registries for projects, integration with CI tools, and even Gitter, an IRC alternative for real-time collaboration.

GitHub

For quite some time, GitHub was the only real player in git repository hosting outside of rolling a custom solution (as we did for Drupal.org). Over the years it has become the home of many open source projects, and while most of those don't match the sheer scale of Drupal in terms of codebase size and number of contributors, there are certainly other major projects that have made their home there.

However, for all of its presence and longevity in the open source world, there are very few options for customizing its toolset for our needs, and we could no longer self-host our canonical repositories. The Drupal project would need to adapt to GitHub, rather than the other way around.
 

That said, in recent months, GitHub has been putting a strong focus on feature development, adding a number of new features including: automated licensing information,  protected branches, and review requests.

Custom tooling

We also must consider that the tools we have now are what built Drupal into what it is today. A great deal of work has gone into our existing developer tools over the years, and we have some unique workflows that would have to be given up if we switched over to a tooling partner. An idea like the issue workspaces proposal would allow us to achieve the goal of modernizing our tools, and likely do so in a way that is better tailored to those unique things about the Drupal workflow that we may want to preserve. However, doubling down on building our own tooling would come at a cost of continuing to be unfamiliar to the outside development community, and dependent on an internal team's ability to keep up with the featureset of these larger players.

Each of these three options would be a compromise between reaching outward and creating familiarity, and looking inward to preserve the Drupal specific workflows that have brought the project to where it is today.

What have we learned so far?

The TAC has conducted their own internal evaluation of the options as well as worked with Drupal Association staff in a two day exploratory session at the end of last year. The primary focus was to identify and triage gaps between the different toolsets in the following areas:

  • Migration effort
  • Project management
  • Code workflow
  • Project handling
  • Testing
  • Git Back-end/Packaging
  • Integrations beyond tools

This initial study also looked at the impact of each option on Drupal community values, and some key risks associated with each.

What comes next?

The next step for the TAC is to make their formal recommendation to the Executive Director of the Drupal Association. At that point, she will direct staff to validate the recommendation by prototyping the recommended solution. Once the recommendation has been validated, Megan will make a final decision and prioritize the work to fully implement this option, relative to other Drupal Association imperatives.

In the comments below, we would love to hear from the community:

What aspects of the way the Drupal community collaborates are the most important to you? What workflow elements do you feel need to be preserved in the transition to any new tooling option?

Mar 17 2017
Mar 17

Any user on Drupal.org who has accepted our Git usage policy may now create full projects with releases. This is a big change in policy for the Drupal project, representing an evolution of the contribution ecosystem in the past half a decade.

What was the Project Application Process?

Ever since the days when Drupal's code was hosted in CVS there has been some form of project application process in the Drupal Community. To prevent duplicate, low-quality, insecure, or otherwise undesirable projects from flooding Drupal, users would submit sandbox projects to an application queue to be reviewed by a group of volunteers.

After resolving any issues raised in this review process, the user would be given the git vetted role, allowing them to promote their sandbox to a full project, claim a namespace, and create releases. Once a user had been vetted for their first project, they would remain vetted and be able to promote any future projects on their own, without submitting an additional application.

The Problem

Unfortunately, though the project application process was created with the best of intentions, in the long term it proved not to be sustainable. Drupal grew too fast for a group of volunteer reviewers to keep up with reviewing new projects, and at times there were applications waiting in queue for 6 months to 1 year, or even more. That is much too slow in the world of software development.

This put Drupal in a difficult situation. After years of subjecting new projects and contributors to a rigorous standard of peer review, Drupal has a well-deserved reputation for code quality and security. Unlike many open source projects, we largely avoided the problem of having many duplicate modules that exist to serve the same purpose. We unified our community’s effort, and kept up a culture of collaboration and peer review. At the same time, many would-be contributors were unable or unwilling to navigate the application process and so simply chose not to contribute.

The question became, how could we preserve the emphasis on quality while at the same time removing the barrier to contribution that the application process had become?

Constraints on a solution

Opening the contribution gates while retaining strong signals about code quality and security was a tricky problem. We established three constraints on a solution:

  1. We need to welcome new contributors, and eliminate the walls that prevent contribution.
  2. We need to continue to send strong signals about security coverage to users evaluating whether to use modules from Drupal.org.
  3. We need to continue our strong emphasis on quality and collaboration through changes to project discovery that will provide new signals about code quality, and by providing incentives and credit for peer review.

The Solution

In collaboration with the community, the security team, members of the board, and staff we outlined a solution in four phases:

Phase 1: Send strong signals about security advisory coverage.

  • We updated project pages to include messaging and a shield icon to indicate whether a project received security advisory coverage from the security team.
  • We now serve security advisory coverage information in the Updates status information provided by Drupal.org, and we're working on a patch to display that information directly on the updates page of users' Drupal sites.

Here are some examples of what these security signals look like on project pages:

If a project is not opted in to security advisory coverage, this message will appear at the top of the project page:

Warning at the top of Project pages

And this one will appear near the download table:

Warning above download table

If a project has opted in, this message will appear near the download table:

Project opt in notice

And covered releases will show the coverage icon (note how the stable 7.x release has coverage and the 8.x release candidate does not):

Release coverage icon

Phase 2: Set up an opt-in process for security advisory coverage

  • Previously any project with a stable release would receive security advisory coverage from the security team. As we opened the gates for anyone to promote full projects, the security team needed an opt in process so that they could enforce an extra level of vetting on projects that wish to receive advisory coverage.
  • We agreed to repurpose the project application queue to be a queue for vetting users for the ability to opt their projects in to receive security advisory coverage. Now that this process has been decoupled from creating full projects, the security team may revise it in future–in collaboration with staff and the community.
  • Now a project maintainer must opt in their project to receive advisory coverage and make a stable release in order to receive security advisory coverage from the security team.

Once a maintainer has been vetted by the security advisory opt in process, they can edit their project and use this field set to opt-in:

Project opt-in field

Phase 3: Open the gate to allow users to create full projects with releases without project applications.

This is the milestone we've just reached!

Phase 4: Provide both automated code quality signals, as well as incentives for peer review of projects - and factor these into project discovery

  • We are working on this phase of the project in the issue queues, and we appreciate your feedback and ideas!

What is the new process?

So in the end - what is the new process if you want to make a contribution by hosting a project on Drupal.org?

  1. You must have a Drupal.org account, and you must accept the git terms of service.
  2. You can create a sandbox or a full project
  • Note: We still strongly recommend that project maintainers begin with sandbox projects, until they are sure they will be able to commit to supporting the project as a full project, and until the code is nearly ready for an initial release.
  • That said, you can promote a sandbox project to a full project at any time, to reserve your name space and begin making releases.

At this point, you will have a full project on Drupal.org, and will be able to make releases that anyone can use on their Drupal site. The project will not receive security advisory coverage, and a warning that the project is not covered will appear on the project page and in the updates information.

If you want to receive security advisory coverage for your project, you will need to take these additional steps:

  1. You must apply for vetted status in the security advisory coverage queue.
  2. Members of the security team or other volunteers will review your application - and may suggest changes to your project.
  3. Once feedback is resolved, you will be granted the vetted role and be able to opt in this project, and any future projects you create, to receive security advisory coverage.
    • Note: Only *stable* releases receive security advisory coverage, so even after opting your project in you will not receive the advisory coverage shield except on stable releases.

What comes next?

Now that the project application process is no more, the gates are open. We are already seeing an uptick in projects created on Drupal.org, and have seen some projects that had migrated to other places (like GitHub) migrate back to Drupal.org. We can expect to see contributions from some great developers who previously felt gate-kept out of the community. We will also see an uptick in contributions that need work, from new developers and others who are still learning Drupal best practices.

That is why our next focus will be on providing good code quality signals for projects on Drupal.org. We want to provide both automated signals of code quality, and new incentives for peer review from existing members of the community. We're outlining that plan in the issue queues, and we welcome your feedback and contributions.

We also still have work to do to communicate this well. This is a big change for the Drupal community and so we want to make people aware of this change in every channel that we can.

Finally, after such a significant change, we're going to need to monitor the contrib ecosystem closely. We're going to learn a lot about the project in the next several months, and it's likely there will be additional follow ups and other changes that we'll need to make.  

Special Thanks

There are many, many contributors on Drupal.org who have put in time and effort to help make the contribution process better for new contributors to Drupal - the deepest thanks to all of you for your insight and feedback. We'd also like to specifically thank those who participated in the Project Application Revamp, including:

Mar 09 2017
Mar 09

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Drupal.org updates

Industry Pages Launched

After a great deal of preparation, user research, and content development we've launched the first three 'Drupal in your Industry' pages. These first three pages highlight the power of Drupal in Media and Publishing, Higher Education, and Government. Each of these pages uses geo-targeted content to reach audiences in: the Americas; Europe, the Middle East, and Africa; and the Asia Pacific, Australia and New Zealand regions.

These pages are targeted at evaluators of Drupal in these specific industries. From our research, we've found that these evaluators typically have Drupal on their short list of technology choices, but are not familiar with how a complete solution is built on Drupal, and they're eager to see success stories from their industry peers.

We'll be expanding on this initiative with additional industry pages as time goes on.

Project Application Revamp

In February we completed phases 1 and 2 of the Project Application Process Revamp. This has meant polishing up the security advisory coverage messages that are provided on project pages, adding a new field for vetted users to opt-in to advisory coverage for their projects, and adding security advisory coverage information to the updates xml served from Drupal.org. With these issues complete we'll be able to move forward with Phase 3 (opening the project promotion gates) and Phase 4 (improving code quality signals and incentivizing peer review) as we roll into March.

[Author's note] The project application revamp hit a major milestone in early March with the completion of Phase 3. Now, any user who has accepted the git terms of service may now promote sandbox projects to full projects with releases, and the application process has been re-purposed for vetting users who want the ability to opt into security advisory coverage for their projects. Look for more information in our upcoming March post.

2017 Community Elections are Live

On February 1 we opened self-nominations for one of the two community-at-large seats on the Drupal Association Board of Directors. At the time of this post, self-nominations have closed and now it's time to vote!.

Each year we make incremental improvements to the elections process. This year we've allowed each candidate to present a short 'statement of candidacy' video - and we've updated the ballot to allow easy drag-and-drop ranking of candidates.

Voting closes on March 18th, so make sure to vote soon!

Documentation polish, and new "call-out" templates

As the migration of content into the new documentation system continues, we've continued to polish and improve the tools. In February we made a few small improvements including: help text for maintainers and fixes for links to the discuss page in email notifications. We also made one large improvement: Call-out templates for highlighting warning information or version-specific notes within a documentation page. These templates are available using the CKEditor Templates button when editing any documentation page.

The documentation editor may select from the 'Warning note' template, which will highlight cautionary information in a visually distinct orange section on the page, or the 'Version-specific note' template, which allows users to highlight information that may only be relevant to a specific minor release of Drupal.

Here are two examples of what the call-outs will look like to a documentation reader.

DrupalCI

Coding standards testing

DrupalCI logo

DrupalCI continues to accelerate the pace of Drupal development as we make the system more efficient and add new features. In February we enhanced the coding standards testing that was added DrupalCI in January. Using PHPCodeSniffer, ESlint, and CSSlint coding standards results are available in the test results' Build Artifacts directory, including automatically generated patches to fix found issues. We've also begun displaying summary information about coding standards testing on Drupal.org test results. Again we'd like to thank community contributor mile23 for his work on this feature.

More useful error output

We also made DrupalCI's error output more detailed, to make it more immediately clear to developers what the issue with a particular patch might be. Developers will now see messages on the test result bubbles, for example a 'patch failed to apply' error rather than a generic 'CI error' message.

Community Initiatives

Contrib Documentation Migration

We want to continue to encourage Project maintainers to create documentation guides on their projects using the new documentation content types. Maintainers can then migrate their old documentation content into these new guides, or create new documentation pages. For more information about this process, please consult our guide to contrib documentation.

Help port Dreditor features to Drupal.org

Are you a Drupal.org power user who relies on Dreditor? Markcarver is currently leading the charge to port Dreditor features to Drupal.org, and invites anyone interested in contributing to join him in #dreditor on freenode IRC or the Dreditor GitHub.

Infrastructure

Special note: Drupal Association seeks Infrastructure Services vendor

We'd also like to announce a Request for Information. The Drupal Association seeks an infrastructure services vendor to help us manage the underlying infrastructure that supports Drupal.org, our sub-sites, and the services we maintain. Our internal engineering team will continue to manage the sites and services themselves, while this vendor will help us with systems administration, virtual machine management, monitoring and pager responsibilities, disaster recovery, etc.

For more details about this request for information, please see our post on the Association blog.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects. If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association. Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Feb 14 2017
Feb 14

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Drupal.org updates

Recognizing more types of contribution in the Drupal.org Marketplace

We were very pleased to announce an expansion of the issue credit system into a broader contribution credit system which recognizes more than just code contributions for the purposes of ranking organizations in the marketplace.

We now calculate the following 4 types of contribution into overall contribution credit:

User research for the upcoming industry pages

In a previous blog post on Drupal.org, we talked about our increasing focus on the adoption journey and our plans to create industry specific landing pages on Drupal.org. In January we did extensive user research with people in media and publishing, higher education, and government, which will be the first industries we promote. We're hoping to launch these pages very soon, so keep an eye on the home page.

Preparing for community elections for the Drupal Association board

Elections

The elections process for the community seats on the Drupal Association board kicks off with self-nominations in February each year. This means that we dedicated some time in January to making small refinements and improvements to the nomination process. In particular we've added more in-context educational materials about the board to the self-nomination form, including a video by executive director Megan Sanicki. We've also refined our candidate questions to help candidates express their unique qualifications.

If you're interested in bringing your perspective to the Drupal Association board, please nominate yourself.

Membership history messaging

To make it easier for members to understand their membership history, we've added new messaging to the membership join and renew pages. Users who go to join or renew their Drupal Association membership will now see a message indicating their current membership expiration date, their last contribution amount, a link to contribute again, and their auto-renewal status.

Membership messaging

Migration of Drupal Association content to Drupal.org

Drupal Association

In January we also migrated the majority of content from assoc.drupal.org to a new section on Drupal.org itself. This effort is part of our larger content restructure initiative. By moving Drupal Association content into Drupal.org we hope to increase discoverability of information about the DA, and create a tighter integration between Drupal Association news and the front-page news feed.

DrupalCI

Checkstyle results now available on the DrupalCI dispatcher

DrupalCI logo

Thanks to community member mile23, DrupalCI now supports automated code style testing. To see checkstyle results for any test on Drupal.org, click on the test result bubble and then click the 'view results' link to view the detailed test results on DrupalCI's jenkins dispatcher.

We're still gathering input and feedback for this initial release of the checkstyle feature, as we decide how to integrate the checkstyle results more tightly with Drupal.org. If you have feedback or suggestions please leave your comments in this issue: #1299710: [meta] Automate the coding-standards part of patch review.

Updated testing environments

DrupalCI supports testing code against a matrix of php and database versions. In January we updated the php environments that DrupalCI supports, so that you can test against the minimum supported versions or the latest point releases. Our 5.X containers have been upgraded to the latest version for each minor release (5.3.29, 5.4.45, 5.5.38, 5.6.29). The singular PHP 7 environment that we were using was following the 7.0.x branch of php7. This has now been expanded into four php 7 environments, 7.0 (7.0.14), 7.1 (7.1.0), 7.0.x, and 7.1.x.

The dev versions of php are primarily intended for Core to sense upstream changes to php before they become released, as our comprehensive test suite often finds unanticipated bugs in php7. Additionally some missing features in the php7 containers were added, specifically apcu.

Local testing improvements

DrupalCI has always supported local testing, in order to allow developers to test changes on their own machines. This is helpful for several reasons: it allows people to test on their own machines before triggering one of the DrupalCI test bots, it lets users troubleshoot failing tests, and it helps to eliminate the 'works on my machine' problem where code appears to work in a local environment, but fails on the test bots.

To make local testing even easier, DrupalCI now automatically generates a vagrant environment for local testing. To use this functionality simply clone the drupalci_testrunner.git repo and then run $ vagrant up from within the directory. Furthermore, DrupalCI can download a build.yml file from a dispatcher.drupalci.org url to replicate any test that has been run on Drupal.org. More information about this will be added to the DrupalCI documentation soon.

Adding test priority

DrupalCI runs thousands of tests of the Drupal codebase for core and contrib modules every month. These tests include commit and patch testing for the active development which may be occurring at any time day or night, as well as the hundreds of daily regression tests run for both core and contrib projects. To help make testing more responsive, we've added a notion of testing priority. When there is a queue of waiting tests, Drupal 8 core patch tests will take priority; followed by D8 branch tests; followed by D8 contrib tests; followed by Drupal 7 patch, branch, and contrib tests.

Community Initiatives

Project Applications Revamp

Our primary community initiative priority for the first quarter of the new year is the Project Application Revamp. There are four phases to the revamp: 1) preserving security advisory coverage signals about projects, 2) transitioning security advisory coverage to an opt-in process, 3) opening the gates to allow any user to promote a project to full and create releases, 4) building new tools to incentivize code review and provide code quality signals on project pages. One of the changes we made as part of phase 1 was to adjust the way recommended releases are highlighted on Drupal.org project pages.

Contrib Documentation Migration

Project maintainers are now able to create documentation guides on their projects using the new documentation content types. Maintainers can then migrate their old documentation content into these new guides, or create new documentation pages. For more information about this process, please consult our guide to contrib documentation.

Help port Dreditor features to Drupal.org

Are you a Drupal.org power user who relies on Dreditor? Markcarver is currently leading the charge to port Dreditor features to Drupal.org, and invites anyone interested in contributing to join him in #dreditor on freenode IRC or the Dreditor GitHub.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects.

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web