Oct 05 2018
Oct 05

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Drupal.org Updates

Drupal Europe 10-14 Sep 2018Thank you to the Drupal Europe team and attendees!

Members of the Drupal Association team joined the community to attend Drupal Europe in September. It was a fantastic event, and we had many great conversations with local community leaders, Supporting Partners, and others about the challenges and opportunities of the European market.

The Drupal.org Engineering Team also met with a number of contributors at Drupal Europe to move forward initiatives like improving Composer support for core, automatic updates, and more.

Reminder: DrupalCon Seattle Early Bird Registration is open nowDrupalCon Seattle April 8-12 2019

DrupalCon Seattle general registration is open now. The programming has been transformed to address the needs of Builders, Content and Digital Marketers, Agency Leaders, and Executives, while preserving that feeling of homecoming for the community that is central to every DrupalCon.

Have questions about the next evolution of DrupalCon? Head of Events Amanda Gonser was recently interviewed by Lullabot for their podcast and explains what's new and what's staying the same.

A video prototype of our integration with GitLab

Are you as excited as we are about the upcoming migration to GitLab? Watch this video for a visual prototype of the integration we're planning.

[embedded content]

We should be announcing a window for our Phase 1 migration shortly.

Phase 1 of Improved Support for Composer begins

In September we moved forward with our multi-phase proposal for improving Drupal core's support for Composer workflows. There are still considerations under discussion, such as how to handle multi-site support, and the implementation details of the later phase. However, Phase 1 has now been broken into its own meta issue, with a goal of bringing these changes into Drupal as of release 8.7.

Seeking a Technical Program Manager

The Drupal Association seeks a Technical Program Manager (TPM) to join our Engineering team and shepherd key programs for Drupal.org that empower our global community to collaborate and build the Drupal project. A TPM is expected to be technically fluent, have excellent project management skills, and excel in internal and external communication. The Drupal Association serves one of the largest global open source communities — Drupal has pioneered open source for 17 years. Join our incredible, mission-driven team and make an important impact by building the tools that help our community build Drupal.

Join Promote Drupal

At Drupal Europe we kicked off the volunteer coordination for Promote Drupal. We've put together an introductory video that explains how to get your marketing teams involved.

[embedded content]

Just go to the Promote Drupal landing page to sign up!

Further improvements for inclusivity on Drupal.org

Thanks to the community contributed work of @justafish, among others, Drupal.org user profiles now include fields for pronouns, primary language, and location, to help give people cultural context as they interact with each other online. We’ll be adding options to show this information with comments throughout Drupal.org.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Sep 06 2018
Sep 06

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Announcements

The Drupal Association announces our partnership with GitLab

In August we were pleased to announce a migration of the Drupal.org developer tools to GitLab.

This will be a three phase process, and we'll be announcing more soon at Drupal Europe and in future blog posts.

We want to say a special thank you to contributor Michael Hess, who volunteered hundreds of hours toward this initiative over the past year (and more).

Git remote URL changes for full projects and sandboxes

To support the migration to GitLab, Git authentication methods for Drupal.org hosted projects are changing.

In particular we have:

  • Deprecated password authentication for Git
  • Deprecated the full project Git remote format
    <username>@git.drupal.org/project/<yourproject>.git
    in favor of:
    [email protected]:project/<yourproject>.git
  • HTTPS clone urls for full projects will not be changing.
  • Sandbox URLs have changed from: <username>@git.drupal.org:sandbox/<author-username>/<project-node-id>.git
    to
    [email protected]:sandbox/<author-username>-<project-node-id>.git
  • HTTPS clones sandbox URLs have changed from:
    https://git.drupal.org/sandbox/<author-username>/<project-node-id>.git
    to
    https://git.drupal.org/sandbox/<author-username>-<project-node-id>.git

We have updated the version control instructions for Drupal.org projects, and put a message in our Git server for any user who makes a push using the deprecated format.

For more information, please review: https://drupal.org/gitauth

Drupal Europe 10-14 Sep 2018

Reminder: Drupal Europe is coming up soon

Drupal Europe is coming up in less than a week! Drupal Europe will be the largest gathering of the Drupal community in Europe and is a reimagining of this important community event as both technical conference and family reunion. The Drupal Association engineering team will be attending to connect with the community, provide updates on Drupal.org, and listen to some of the incredible speakers who will be in attendance.

Join the community in Darmstadt, Germany, from September 10-14, 2018. Make sure to register, book your travel, and secure accommodation: http://drupaleurope.org/

DrupalCon Seattle April 8-12 2019

Reminder: DrupalCon Seattle Early Bird Registration is open now

DrupalCon Seattle general registration is open now. The programming has been transformed to address the needs of Builders, Content and Digital Marketers, Agency Leaders, and Executives: https://events.drupal.org/seattle2019/registration

Please note that ticket prices have changed—and now Supporting Partners will receive a discounted rate compared to non-partner attendees. Joining the Supporting Partner program is a great way to support the Drupal project and the Drupal Association.

Drupal.org Updates

Drupal.org issues can be sorted by # of followers

To help Drupal.org project maintainers understand what issues are most important to people, the issue queues can now be sorted by the number of people following each issue. Here's an example of the Drupal Core issue queue sorted by follower count.

Documentation landing page updated

The Documentation Initiative was formed at DrupalCon Nashville to help improve the onboarding experience for users new to Drupal. A lot of work has been going on in the background, but one of the more visible changes is the new Documentation landing page.

Stable Download link and updated Try Drupal menu

Drupal 8.6 ships with an incredible new Quickstart feature—and to make the process of trying out Drupal as simple as possible, there is now a stable url to retrieve Drupal's latest release.

Similarly, we've updated the top navigation of Drupal.org. The Try Drupal button which appears at the top of every page now offers users the choice of trying an online demo or downloading the software directly.

Travel and Tourism industry page launched

Drupal has a powerful place in the Travel and Tourism space, in use everywhere from Princess Cruises to AAA. To promote Drupal's power in this industry we've launched our Travel and Tourism page.

Decoupled Solutions industry page launched

Perhaps the most powerful implementations of Drupal are Decoupled solutions, relying on Drupal as a stable back-end for content delivery to multiple endpoints with different front-end architectures. This approach is still new to many, our Decoupled industry page helps educate the market.

Staff Updates

Welcome Angie Sabin, Director of Operations

We're also very pleased to welcome Angie Sabin to the team. Angie comes from a non-profit background and is joining the Association to help us take finance and operations to the next level.

Welcome Kelly Delaney, Customer Success Coordinator

The Drupal Association is pleased to announce that Kelly Delaney has joined our team. As our new customer success coordinator, she'll work closely with supporting partners and sponsors to ensure that their needs are met and benefits are fulfilled.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.
Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Aug 16 2018
Aug 16

This is the fifth post in our series about integrating Drupal.org with a 3rd party developer tooling provider:

In this post we are announcing our migration to a new tooling provider, and outlining the phases of that migration process to take place over the next several months.

Announcing our partnership with GitLab

Wait, what?

Drupal Association + GitLab

Yes, in our four part series from December of last year it certainly looked like we were going in a different direction for the future of Drupal's developer stack.

So what changed?

Last year we laid out a model for integrating Drupal.org with a third party tooling provider, which we described as "Drupal Flow". This model was deliberately laid out to be agnostic to the provider we chose, so long as certain requirements were met. We worked with representatives from three potential providers at the time: GitHub, GitLab, and BitBucket, and each one had pros and cons. Once we had completed our evaluation, BitBucket was the only provider without hard blockers to the integration we wanted to build.

However, following our blog series, the GitLab team reached out directly to the Drupal Association team, and asked us to give them the chance to resolve the blockers and close the gaps in our integration.

At the same time, we saw an outpouring of feedback from our community asking us to see if we could find a way to make GitLab work.

And so we did.

The Agreement

For the past six months we've been working closely with Eliran Mesika, the Director of Partnerships at GitLab, in addition to CEO Sid Sijbrandij and members of GitLab's engineering team. They've escalated the internal priority of issues that blocked our adoption of GitLab, offered technical and financial support for the migration, and made a commitment to ongoing support for the Drupal project.

And so we're happy to announce that Drupal.org is going to be moving our code collaboration tools for our forty-five thousand projects to GitLab over the course of the coming months.

Three Phases to the Migration

Phase 1: Replacing Drupal.org's Git backend

The first phase of the Drupal.org migration

  • Transparently replace Drupal’s current underlying Git infrastructure (for repository hosting, maintainer permissions, code viewing) with GitLab repositories, GitLab roles and permissions for maintainers, and the GitLab code viewing UI.
  • Enable inline code editing (only for maintainers for this phase).
  • During this phase, Drupal.org will remain the primary source of information.  SSH keys, new projects, etc. will be created on Drupal.org.

This first phase, while modest, will bring some concrete benefits to the project:

  • Maintainers will be able to begin familiarizing themselves with GitLab's code collaboration tools.
  • Code viewing will receive a significant upgrade from CGIT to GitLab's built-in code viewer.
  • And Drupal.org's old Git server will be phased out.

Phase 2: Enabling Merge Requests, Inline Code Editing, and Web-based Code Review

  • The timeline for Phase 2 is dependent on GitLab’s resolution of a diskspace deduplication issue, which they have committed to on our behalf: https://gitlab.com/gitlab-org/gitlab-ce/issues/23029
  • Enable GitLab Merge Requests, GitLab inline code editing in the web UI, and GitLab web-based code review.
  • During this phase, Drupal.org will handle any 'create branch/merge request' integrations from the Drupal.org Issue queues, and related call-backs from GitLab into the Drupal.org issue comment stream.

Phase 2 is where we realize some tremendous benefits to developer velocity and collaboration:

  • By adding merge requests, contributing to Drupal will become much more familiar to the broad audience of open source contributors who learned their skills in the post-patch era.
  • By adding inline editing and web-based code review, it will be much easier to make quick contributions. This not only lowers the barrier to contribution for people new to our community, it also saves significant effort for our existing community members, as they'll no longer need to clone work locally and generate patches.
  • Finally, by creating a tight integration between the Drupal.org issue queues and GitLab's development tools, we'll be able to transition to this new toolset without disrupting the community's existing way of collaborating.

Phase 3: Evaluating Additional Features

Phase 3 has no strict timeline, but will be dependent on feedback from the community as they get up to speed on using the new GitLab-based contribution workflow for Drupal.

These additional features may allow us to further improve the velocity of the Drupal project, or realize additional cost savings for the association. For example, we may be able to use GitLab's test runner integration to orchestrate tests across a wider variety of cloud platforms, helping us find the best pricing. We may be able to entirely replace security.drupal.org with a private issue tracker, eliminating an entire sub-site for the Drupal.org team to maintain. We may even be able to enhance existing community services like SimplyTest.me by integrating features like GitLab's AutoDevops tools to automatically create review environments for issues or branches.

We won't really know what's possible within the scope of our resources until the first two phases are completed, but this helps to show that by hitching our toolset to a partner that specializes in collaboration, we may be able to realize even more benefits for our community.

Changes to Git Remotes

  • Git remote urls for pushes to full projects have changed:
  • HTTPS clone urls for full projects are unchanged.
  • HTTPS clone urls and Git remote urls for sandbox projects have changed:
    • For remotes of the format:
      <username>@git.drupal.org:sandbox/<username>/<node-id>.git
      the format should be changed to:
      [email protected]:sandbox/<username>-<nodeid>.git
    • Clone urls will be changing from:
      https://git.drupal.org/sandbox/<username>/<nodeid>.git
      to the format:
      https://git.drupal.org/sandbox/<username>-<nodeid>.git

Important: If you have any automated systems which authenticate to Git, such as CI pipelines or repo mirroring, ensure they are updated as well.

For more detailed information about these changes, as well as instructions for changing your Git remotes or setting ssh keys, please consult these instructions: https://drupal.org/gitauth

How to follow our progress

Issues for the Drupal.org migration to GitLab will be opened in the Drupal.org Infrastructure queue and tagged 'GitLab'.

For questions or concerns, please create an issue at https://www.drupal.org/node/add/project-issue/infrastructure

Learn more on the GitLab Blog

You can learn more about the decisions that brought us to this point on the GitLab blog.

Aug 07 2018
Aug 07

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

AnnouncementsDrupal Association Logo

Git remote URL changes for full projects and sandboxes

Git authentication methods for Drupal.org hosted projects are changing as we approach upgrading our developer tooling stack.

In particular we are:

We have updated the version control instructions for Drupal.org projects, and put a message in our Git server for any user who makes a push using the deprecated format.

For more information, please review: https://drupal.org/gitauth

Drupal Europe

Reminder: Drupal Europe is coming up soon

Drupal Europe is coming up in less than 40 days! Drupal Europe will be the largest gathering of the Drupal community in Europe and is a reimagining of this important community event as both technical conference and family reunion. The Drupal Association engineering team will be attending to connect with the community, provide updates on Drupal.org, and listen to some of the incredible speakers who will be in attendance.

Join the community in Darmstadt, Germany, from September 10-14, 2018. Make sure to register, book your travel, and secure accommodation: http://drupaleurope.org/

We want your feedback on ideas for Drupal Core

The Drupal Association has proposed several initiatives for Drupal Core - but before they can be officially adopted they need feedback from stakeholders in the community (even if it's just a "+1") and to reach community RTBC. Here are the proposals:

Drupal.org Updates

Staff retreat

In July the Drupal Association gathered together in Portland Oregon for our bi-annual staff retreat. At these retreats we discuss the progress made in the last six months, and our prioritization as an organization going into the next six month period.

Hightech users of Drupal

Hightech industry page launched

Drupal is the CMS of choice for a variety of companies in the high tech space, including organizations like Redhat, Cisco, and Tesla. Whether it is used in a front-facing application, as a decoupled back-end, or for an internal intranet experts in hightech defer to Drupal's example for their needs.

We launched a new industry page featuring these stories from high tech in July.

Drupal.org API updated for security advisories

To improve the automated toolchains built by organizations and individuals in the community to watch for new security advisories, we've updated the Security Advisory API. One of these changes ensures that the full canonical identifier for each advisory is included in the API data, which is a small but valuable change for anyone monitoring the API for advisory information.

Social Media Sharing for Events News

The DrupalCon news feed now includes social media sharing icons, so that you can better promote DrupalCon news and announcements to your networks. Word of mouth has always been a critical part of Drupal's success - so we hope that as featured speakers are announced, early bird registration begins, or the schedule is published, you will help us get the word out!

DrupalCon Seattle is coming up from April 8-12 2019, and we're featuring some bold new changes to support a variety of audiences from our traditional core of those people who build Drupal, to marketers and content editors, and to the agency sales forces that sell Drupal to the world.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Jul 13 2018
Jul 13

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Announcements

Last chance to vote in the Drupal Association board election

Elections for the Drupal Association board end on July 13th, 2018 at 5pm Pacific (in just over an hour at the time this is posted). There are nine candidates from 7 countries across six continents representing a wide variety of perspectives from the Drupal community. Anyone user who has been active in the last year and registered before the elections began is welcome to cast a ballot.

We encourage you to vote today and help guide the future of the Drupal Association.  

Reminder: Drupal Europe is coming up soon

Drupal Europe is coming up in less than 60 days! Drupal Europe will be the largest gathering of the Drupal community in Europe and is a reimagining of this important community event as both technical conference and family reunion. The Drupal Association engineering team will be attending to connect with the community, provide updates on Drupal.org, and listen to some of the incredible speakers who will be in attendance.

Join the community in Darmstadt, Germany from September 10-14, 2018. Make sure to register, book your travel, and secure accommodation: http://drupaleurope.org/

Project maintainers: Change your git remote configuration

Git authentication methods for Drupal.org hosted projects are changing as we approach upgrading our developer tooling stack. In particular we will be:

  • Deprecating password authentication for git

  • Deprecating the git remote format <username>@git.drupal.org/project/<yourproject>.git in favor of [email protected]:project/<yourproject>.git

We have updated the version control instructions for Drupal.org projects, and put a message in the git daemon for any user who makes a push using the deprecated format.

For more information, please review: https://drupal.org/gitauth

Drupal.org Updates

Ecommerce industry page launched

Since last year, one of our ongoing initiatives has been to develop more content on Drupal.org focused on specific industries. Drupal is an incredible powerful tool for building ambitious digital experiences, but it's flexibility can sometimes be overwhelming. These industry specific pages help Drupal evaluators discover how Drupal can be tailored for their specific needs, and highlight successful case studies of Drupal in the wild.

The Drupal Association has launched our sixth industry page promoting the power of Drupal for Ecommerce. We want to thank Commerce Guys for their contributions to getting this page off the ground.

Display project screenshots in a more user friendly way

For every new Drupal project that a developer or site-builder undertakes, time is spent evaluating distributions, modules, and themes to find integrations that will accelerate launching the project.

To improve the user experience for users evaluating modules on Drupal.org, we've implemented a new lightbox-style display for project screenshots.

Here's an example of a screenshot from the Token project:

Granted more maintainers the ability to give contribution credit

Since the introduction of contribution credits at the end of 2015, they've become an important part of the way the Drupal community recognizes individual and organizational contributions to the project. The Drupal Association Engineering team regularly reviews the contribution credit system to make small tweaks and adjustments to make the experience even better.

For our most recent update, Drupal.org now grants all project maintainers with the 'maintain issues' permission the ability to grant contribution credit, instead of just those users with 'Write to Version Control' permissions. This means that a much wider group of maintainers can now participate in granting credit.

Showing maintainer photos on top level Docs guides

Documentation is critically important to the Drupal project To make it easier for potential contributors to find out who they should reach out to for issues that affect the top levels of documentation, we've added maintainer information to the top level documentation guides.

Email confirmation when creating an organization node

To help more organizations that work with Drupal join our community, we now send an email confirmation to any user who creates an organization profile with information about becoming listed as a service provider, details about the contribution credit system, and information about becoming a Drupal Association member or supporting partner.

We encourage everyone in the Drupal community to ask your clients to create a Drupal.org organization profile. Bringing end-users into the contribution journey will be a key part of Drupal's long term health and success.

Contributing to the Open Demographics Initiative

One of our goals on the Drupal Association engineering team is to adopt the Open Demographics Initiative in our user registration process. As part of our effort to work towards that goal, we have contributed a machine readable version of the demographic questions and and answers to the ODI project.

We're hopeful that can be reviewed and committed soon, and be used as the basis for an ODI Drupal module.

Security Improvements

Added PSA and SAs to the /news feed

To increase the visibility of security notifications, Public Security Announcements and Security Advisories will now be included in the https://drupal.org/news feed.

Multi-value CVE field for Security Advisories

We've also updated the security advisory content type so that an advisory can be associated with multiple CVEs.

Infrastructure Updates

DrupalCI: Converted core javascript tests to use Chrome driver

The DA Engineering team has worked together with Core to convert the Core javascript tests from using PhantomJS to using Chrome Webdriver. This provides much more powerful and better supported tools for javascript development in Drupal.

DrupalCI: Reduced disk space usage of the DrupalCI dispatcher

One of the most important services the Drupal Association provides for the project is DrupalCI, the suite of tools used to test all of Drupal's code. These tools are very powerful, but also expensive to maintain, and something we have to monitor carefully. In June, we spent some time automating disk space management for the DrupalCI dispatcher, to help reduce the maintenance cost of keeping it running smoothly.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

  • OPIN - Renewing Signature Supporting Partner
  • Srijan - Renewing Signature Supporting Partner
  • Lullabot - Renewing Premium Supporting Partner
  • Aten - Renewing Premium Supporting Partner
  • Phase2 - Renewing Premium Supporting Partner
  • WebEnertia - *NEW* Premium Supporting Partner
  • Pantheon - Renewing Premium Hosting Supporter
  • Datadog - Renewing Premium Technology Supporter
  • Promet Source - Renewing Classic Supporting Partner
  • Evolving Web - Renewing Classic Supporting Partner
  • ImageX - Renewing Classic Supporting Partner
  • Adapt - Renewing Classic Supporting Partner
  • Green Geeks - Renewing Hosting Supporter
  • Microserve - Renewing Classic Supporting Partner
  • ThinkShout - Renewing Classic Supporting Partner
  • Amazee Labs - Renewing Classic Supporting Partner
  • Four Kitchens - Renewing Classic Supporting Partner
  • Access - Renewing Classic Supporting Partner
  • Appnovation - Renewing Classic Supporting Partner
  • Studio Present - Renewing Classic Supporting Partner
  • undpaul - Renewing Classic Supporting Partner
  • Position2 - Renewing Classic Supporting Partner
  • Blend Interactive - Renewing Classic Supporting Partner

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Jun 11 2018
Jun 11

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

AnnouncementsDrupal Association logo

Change your git remote configuration

We will be deprecating the git remote format <yourusername>@git.drupal.org/project/<yourproject>.git in favor of [email protected]rupal.org/project/<yourproject>.git in preparation for changes to our developer tooling stack. If you used the <username>@ format for your git remotes, you should change your remote to the [email protected] format. You can use $ git remote set-url to make this change for existing repositories you have cloned.

We have updated the version control instructions on Drupal.org to reflect this change, and will be updating the git daemon to warn developers who are using the deprecated remote format.

Proposal: Improving Core's Relationship with Composer

In May, Mixologic from the Drupal Association engineering team worked with community members Mile23, Bojanz, Webflo, and others in the community to develop a proposal for improving Drupal Core's relationship with Composer.

In its simplest form, the proposal is to: Conceptually separate Drupal, the product, from Drupal's git repository, and provide a mechanism that creates a composer ready Drupal installation.

Going into early June, we've been circulating this proposal to the Core Committers, the Auto-Updates Initiative team, Contrib maintainers, Distribution maintainers etc.

Credit for non-code projects on Drupal.org

We're excited to announce that we've created a 'Community Projects' section in the Drupal.org issue queues. This section exists to record all the tremendous community labor exercised to promote the Drupal project in ways other than code. This format was pioneered by the Drupal Diversity and Inclusion group, who started recording their meeting minutes in the issue queues so they could provide contribution credit for attendees. This same model can be used by initiative coordinators, camp organizers, or any other Drupal community group that would like a place to recognize their work with the official contribution credit system.

The Contribution Credit system is one of the Drupal projects most successful innovations in the way that open source projects are managed, and it will continue to evolve and grow as time goes on.

Updates for GDPR

AEU GDPRre the words "We've updated our privacy policy" burned into your laptop screen yet? Well in May we did the same.  In particular, we've updated our Terms of Service, Privacy Policy, Digital Advertising Policy, and Git Contributor Agreement to clarify our compliance with the EU's GDPR. We also initiated a re-consent campaign for our marketing lists. If you have not re-consented to communications we strongly encourage you to do so

Launched the Customer Supporter program

Have you built great relationships with your Drupal customers? Help them contribute back to the project by becoming part of our Customer Supporter program.

Drupal.org Updates

Self-nominations for the Drupal Association board are live

Each year one of the two community-held seats on the Drupal Association board comes up for election. We opened the self-nomination process for this year, and some passionate and dedicated members of the community have already stepped forward with their candidacy.

To learn more, you can view our portal for the 2018 Elections Process. Key dates to remember:

  • Self nominations: 1-11 June, 2018
  • Meet the candidates: 12-29 June 2018
  • Voting: 2-13 July, 2018
  • Votes ratified, Winner announced: 25 July, 2018

Better landing pages for Drupal's strategic initiatives

Did you know there are 12 active Drupal strategic initiatives right now?

To help the initiative coordinators promote this work, and recruit more open source contributors to the cause, we've given initiative coordinators new landing page tools. Check out the first initiative to use this landing page: the Admin UI and Javascript Modernization Initiative.

These tools are the first step in improving the project management tools available to initiative coordinators to help move the Drupal project forward.

Historical user and organization contribution data is now available.

Drupal.org user profiles show the last year's worth of contributions by users. We chose the one year window deliberately, to promote the importance of a user's more recent activity. However, seeing a user's complete contribution history can be valuable as well. We've recently added a link to the bottom of this view to display that history.

Similarly, organization profiles have shown the last 90 days of contributions by organization. Again, we chose this very deliberately to emphasize the  importance of recent and ongoing contribution. However, as with user accounts, these profiles now also include a link to the organization's complete contribution history. You can see an example of where to find this link below:

Organizations - View all credit history

Expanded spam protections

After the sunsetting of Mollom in March of this year, we've been implementing a new set of tools to mitigate spam on Drupal.org. We expanded these protections in May, using a combination of bot detection, content analysis, rate limiting, and more to try and reduce the impact of spam attacks on Drupal.org. The less time the community spends wading through spam, the greater the velocity of the Drupal project.

And a Thank You

We'd also like to give a special shout out to contributor Wim Leers, for his incredibly kind 'Ode to the Drupal Association' about our work on the testing infrastructure. The nature of software engineering has a tendency to draw our attention to things that are broken, buggy, or unoptimized, and so when things are working well that success can sometimes feel invisible.

Fortunately, the Drupal community puts people first, and celebrates our collective success, and Wim's words are a tremendous example of that ethos.

Thank you, Wim - and thank you to everyone who takes the time to recognize the hard work and dedication of your fellow contributors.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

May 23 2018
May 23

Our global community includes many EU citizens and residents of the EEA, and we have taken steps to comply with the GDPR which takes effect on May 25, 2018.

Your rights under this law and how Drupal.org complies with GDPR

We've updated our Terms of Service, Privacy Policy, Git Contributor Agreement, and Digital Advertising Policy based on the requirements of the EU General Data Protection Regulation. We've also begun a campaign to reconfirm your consent to our marketing messages.

For easy and clear access to the changes: 

Human Readable Summary

Disclaimer: This summary is not itself a part of the Terms of Service, Privacy Policy, Git Contributor Agreement, or Digital Advertising Policy, and is not a legal document. It is simply a handy reference for understanding privacy rights and regulations. Think of it as the user-friendly interface to the legal language.

In plain language, regulations such as GDPR define the following roles, rights, and responsibilities:

  • Data Subject - this is you, the end user.
  • Data Controller - this is us, the Drupal Association as the owners and operators of Drupal.org and its sub-sites.
  • Data Processor - any other organization that processes personal data on behalf of the Data Controller.

Rights of the Data Subject

  • Right to be Informed - A data subject has the right to know whether personal information is being processed; where; and for what purpose.
     
  • Right to Access - A data subject has a right to access the information about them that is stored by the Data Controller.
     
  • Right to Rectification - A data subject has the right to correct any errors in the data about them. This can be done by editing your user account, or contacting the Drupal Association directly.
     
  • Right to Restrict Processing - A data subject has the right to request that data not be processed, and yet also not be deleted by the Data Controller.
     
  • Right to Object - A data subject has the right to opt out of marketing, processing based on legitimate interest, or processing for research or statistical purposes.
     
  • Right to be Forgotten - Also known as the right to revoke consent, the right to be forgotten states that a data subject has the right to request erasure of data, the cessation of processing by the controller, and halting processing of the data by third party processors.

    The conditions for this, as outlined in article 17, include the data no longer being relevant to original purposes for processing, or a data subjects withdrawing consent.

    It should also be noted that this right requires controllers to compare the subjects' rights to "the public interest in the availability of the data" when considering such requests.

  • Data Portability - A data subject has the right to receive a copy of their data in a 'commonly used and machine readable format.'

    This information is outlined in the sections below titled "Your Choices About Use and Disclosure of Your Information" and "Accessing and Correcting Your Information".

Responsibilities of the Data Controller and Data Processors

  • Privacy by Design - 'The controller shall..implement appropriate technical and organisational measures..in an effective way.. in order to meet the requirements of this Regulation and protect the rights of data subjects'. Article 23 of the GDPR calls for controllers to hold and process only the data absolutely necessary for the completion of its duties, as well as limit the access to personal data to those who need it to carry out these duties.
     
  • Breach Notification - The Data Controller must notify the appropriate data processing authority and any affected end user of any breach that might result in 'risk to the rights and freedoms of individuals' within 72 hours of becoming aware of the breach.

    A Data Processor must notify the Data Controller of any breach 'without undue delay.'

  • Data protection officer - A Data Controller or Processor must appoint a Data Protection Officer when: a Data Controller represents a public authority; or the core operations of the Controller require regular and systematic monitoring of Subjects on a large scale; or when the Controller's core operations depend on processing a large scale of special categories of data (including but not limited to health data, criminal conviction information, etc).
     

    The Drupal Association's core operations do not require the Association to establish a Data Protection Officer.

We take privacy and security very seriously, as all Drupal professionals do! We will continue analyzing the legal landscape and collecting feedback for future revisions.

If you have any questions or concerns about our GDPR compliance, or if you want to point out a mistake or provide a suggestion for the Terms of Service, Privacy Policy, Git Contributor Agreement, or Digital Advertising policy, you can send an email to [email protected].

Apr 30 2018
Apr 30

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Drupal.org UpdatesDrupal Association Logo

Drupal.org's new front page and persona pages launched

As you've probably seen by now, just before DrupalCon Nashville we launched a makeover of the Drupal.org front page. This was a research-based redesign focused on addressing the three key personas that come to Drupal.org: Developers, Marketers/Content Editors, and Agencies.

The new redesign simplifies the number of calls to action on the front page, and directs each of these personas into a more focused funnel, to ensure they are more likely to find the information they really need. To learn more about this redesign and the Promote Drupal initiative, read our recent blog post. We want to thank SixEleven for their help with this new design initiative.

Promote Drupal Initiative

Redesigning the front page was just the start, we kicked off DrupalCon by announcing a new 'Promote Drupal' initiative, asking the community to come together to help bring Drupal to new audiences, and to convince people who've used older versions in the past to give Drupal 8 another look.

We need your support to make the Promote Drupal initiative happen!

Updated top navigation and IA

Along with the front page changes, we've updated Drupal.org's top level IA, providing a more logical structure for navigating to the major areas of the site depending on a user's persona.

Drupal.org Top Nav Redesign

Promoting Nonprofit solutions built with Drupal

And last, but not least, in our efforts to #PromoteDrupal we've launched a new Nonprofit solution page, promoting the power of Drupal for Nonprofits and NGO's around the globe. Drupal has long been the choice for well-recognized, global nonprofit organizations to extend their reach and maximize their impact.

Drupal nonprofit

Simplify Drupal Initiatives

In project founder Dries Buytaert's keynote at DrupalCon Nashville he proposed a series of initiatives to simplify Drupal - lowering the barriers to adoption and improving the user experience of site administrators and content editors. Some of these initiatives are to improve features of Drupal core itself, whereas others are focused on the evaluator experience and will be managed in collaboration with the Drupal Association.

In particular, the Drupal Association will collaborate with the core initiatives teams on:

These initiatives are not going to be quick or easy. They rely on collaboration between the Drupal Association, Drupal's core committer team, and a variety of volunteers throughout the community. We'll need your help.

Drupal.org and GDPR

GDPR, the General Data Protection Regulation passed by the EU last year, begins enforcement on May 25th, 2018. We've been preparing for this new regulation for some time, and will be implementing a few changes in the coming weeks:

  • Updates to our:
  • Updates to our mailing lists
  • Publishing our data retention policy
  • Publishing our data portability policy
  • Updating our breach notification policy

Drupal securitySecurity Release

SA-CORE-2018-003

Drupal Core coordinated a security release with the CKEditor team to ensure that the security fix for CKEditor was immediately available in Drupal 8. As Drupal becomes further integrated into a world of third party dependencies, this kind of coordination between open source projects becomes increasingly important. We want to thank the CKEditor team and the volunteer Drupal Security team for their hard work and careful collaboration.

SA-CORE-2018-004

After the release of SA-CORE-2018-002 in March, a related vulnerability was discovered and an additional security advisory for Drupal 7 and 8 released in April. If you have not yet updated your Drupal sites to address these vulnerabilities they may already be compromised. If that is the case, we encourage you to read this PSA, which provides some steps you can take.

Security releases tend to spark quite a bit of conversation in the community about the nature of software security, proprietary vs open source, and related issues. Community member @rickmanelius provided some much-needed context to keep these security focused efforts in perspective:

The recent SA-CORE-2018-004 and SA-CORE-2018-002 security advisories have sparked a lot of conversations in the Drupal community regarding all things security. IMHO, it's important to highlight several talking points to keep things in perspective.

— Rick Manelius, PhD (@rickmanelius) April 26, 2018

DrupalCI: Support for DrupalCI.yml

DrupalCI now supports the use of Drupalci.yml files in projects to customize and override elements of testing. This makes the testing capability of DrupalCI much more powerful and flexible for project maintainers. We're still working on documenting these new features, but you can read about the new features here.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Apr 27 2018
Apr 27

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

This month's update comes a bit later than usual, as we return from DrupalCon Nashville. Expect our April update to follow after shortly.

Representing Drupal at Google's CMS Leadership Summit

In mid-March, we attended the Google CMS Leadership Summit, as representatives of the Drupal project. The Summit was a one-day event hosted by Google to unite the 20 or so projects in the CMS space responsible for more than 50% of the content on the web.

The goal was to understand how to preserve an open web, by empowering better authoring experiences, content consumption, and performance in our CMS platforms.

This level of dialogue and engagement with an organization like Google is new and exciting for us, and we're looking forward to ongoing conversations, both with Google and with the other CMS project leaders they assembled at the event.

Drupal.org UpdatesDrupal Association Logo

Researching the anonymous traffic to Drupal.org

One of the focuses of the Drupal Association in 2018 has been to better understand our audience. When it comes to users who register on Drupal.org, and our DrupalCon attendees, we have quite a bit of information about who our users are.

However, when it comes to the wider ecosystem of Drupal users (evaluators and end-users who do not have Drupal.org accounts) we've been largely in the dark for most of the project's history. One way we want to improve this is by working with Drupal Core to add telemetry to Drupal, but that is an effort that will take some time.

In the meantime, we've implemented several Audience Insight tools to help us learn more about our anonymous users. Privacy is always a paramount concern, so we chose only insight tools which provide aggregate, anonymized data, and we wrapped those tools in our own implementation of Do-Not-Track so that we could ensure that user privacy preferences are respected.

The table below demonstrates the job functions held by the anonymous visitors to Drupal.org. (Please note: these job functions might be held within any kind of industry, this data is about the user's role, not their target market).

Job Function

D.O Front Page Visitors

All D.O Visitors

Diff

Engineering

26.20%

44.90%

-18.70%

Information Technology

14%

16.40%

-2%

Business Development

11.60%

9.60%

2.00%

Entrepreneurship

10.30%

10.70%

-0.40%

Arts and Design

7.70%

5.30%

2.40%

Media and Communication

6.60%

6%

0.60%

Marketing

6.30%

3.70%

2.60%

Education

6.10%

4.60%

1.50%

Operations

5.20%

3.50%

1.70%

Program and Project Management

4.30%

4.20%

0.10%

Sales

4.20%

2.60%

1.60%

Consulting

2.90%

3.30%

-0.40%

Research

2.40%

1.70%

0.70%

Community and Social Services

2.40%

1.80%

0.60%

Administrative

2.30%

1.30%

1.00%

We've used this data to inform the redesign of Drupal.org, as well as our new persona pages. Learn more about that process in our April update. The redesign work was carried out in collaboration with SixEleven, who also produced the DrupalCon brand and design for DrupalCon Nashville.

Nashville guitar

Preparing for DrupalCon Nashville

In the lead up to DrupalCon Nashville in April, the team was in high-gear preparing for the event. We participated in a panel about the future of pull requests on Drupal.org, the public board meeting, and handled the keynote livestream process.

DrupalCon is always an incredible opportunity for the team to connect with the community about upcoming initiatives, drupal.org support requests, and to plan for the future.

We were happy to see so many of you there, and we'll talk more about this in our April update.

Documentation enhancements

Our efforts to improve the quality of Drupal's documentation continued in March and April, as we added features:

  • New D7/D8 guides are now automatically approved, so new project contributors aren't blocked on documenting their projects.
  • Added Drupal version to page title for better searchability.
  • Follow/Unfollow links are available directly on the discuss page of any documentation.

In-context links to newer and older releases

To ensure that users are aware when there are newer releases than the one they may be looking at, we now provide newer and older release history on release nodes. In the sidebar of any release page you will see links and dates to related releases. Among other things, we hope this will prevent users from accidentally installing an older release when another new one has just come out!  Other releases

Email notifications for new maintainers

Encouraging succession planning in module maintership is an incredible challenge for any open source project. We want to encourage maintainers to invite contributors to their projects to help maintain those projects, but we recognize that we also have to make sure the appropriate tools are in place to make this a smooth process.

To make sure that new maintainers of projects are welcomed into the fold, we've added email notifications to let a user know when they've been added as a project co-maintainer. If you are invited by an existing maintainer of a project to help maintain it, you should now receive a warm welcome.

Drupal Security Team

Security Release

SA-CORE-2018-002

The Drupal Association Engineering Team collaborated with the Security Working Group and Security Team to coordinate 3 significant security releases in March and April.

The primary release was SA-CORE-2018-002, a highly critical security release for Drupal 7 and 8. For more information about all Drupal security releases and PSAs, please visit our security portal.

The volunteer Security Team has always been a tremendous asset to our community, and the Drupal Association is proud to support their work.

Infrastructure Updates

DrupalCI: Support for testing themes

DrupalCI has enabled support for testing Themes, so now Theme projects on Drupal.org can include tests. This has become more and more necessary as javascript becomes critical to modern web design, and we hope this will help accelerate the build out of themes for Drupal 8 and increase their quality.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Mar 07 2018
Mar 07

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Drupal.org UpdatesDrupal Association

Reimagining Drupal.org's front pages to serve distinct personas

Drupal serves a wide audience of users, from developers to marketers to content editors and beyond. Historically, Drupal.org has been focused on our community of contributors, whether those contributions are in code, documentation, volunteer support, camp organizing, etc. However, only 1 in 15 visitors to Drupal.org are an authenticated user, and the rest are primarily visiting Drupal.org as representatives of an end-user organization that is evaluating Drupal. We want to serve these visitors better.

In February we held an off-site in Portland to consolidate our research about the personas within end-user organizations who make the decision to adopt Drupal. We identified three key roles:

  • Technical evaluators - who are often developer evangelists within their organization

  • Marketing and business users -who are evaluating Drupal as a platform. They are interested in the editorial experience and time-to-market for building a solution that integrates with tools they already use

  • Agencies - who are already using Drupal for their clients, or are considering making it central to their business.

From there, we developed some initial concepts for a reimagining of the front page of Drupal.org to better serve these first three personas.

This work will carry us through DrupalCon Nashville and beyond, so expect additional updates over the coming months.

Contribution credit update

The Drupal project has an innovative system of crediting users and sponsoring organizations for the work they contribute. However, as a system that we've pioneered, there is always room for additional improvements. One area that needed improvement was the date used for the assignment of credit. In the past, the credit for a user or organization would be tracked to the timestamp of the latest activity on the issue. This was a good approximation, but additional comments after issue resolution would bump the date of the credit.

We've updated the way that contribution credits are calculated - so that it is now based on the date that the issue was closed(status last changed) instead of the date of the last change to the issue. This change affects both individual contribution credits and the marketplace ranking.

Documentation improvements

As our new team member Dhanya has come on board, she has helped make some great improvements to the documentation system, including: fixing the display of sidebar lists of guide contents, increasing the visibility of the current page indicators, and swapping the grid treatment for a more readable guide contents layout

Doc Guide Preview

Accessibility and readability

We've made two additional small fixes.

One for accessibility - improving the keyboard 'skip to…' links in the Drupal.org top navigation.

Skip to main

Skip to search

… And one for navigating issues, fixing a bug that prevented links to comments on multi-page threads. Now, any Drupal.org user who receives an email notification about a multi-page issue should be properly linked to the correct comment.

Preparing our live-streaming capability for DrupalCon Nashville

For DrupalCon Nashville, rather than relying on a vendor, we are going to be managing the live stream of the keynotes and closing session ourselves, together with the AV staff of the venue. In February we spent some time putting together our equipment and running some streaming tests.

Continued work to reduce our PCI scope

In February we finished migrating our donation process for both USD and Euro donations to new payment processors to reduce our PCI scope and thus maintenance costs. We've also launched the beta of the membership system, and will hopefully complete the migration of existing memberships soon.

If you are not yet a member of the Drupal Association, and would like to support us both by joining and helping us test the new membership system, you can sign up here. (If you are an existing member, please continue to process your renewals on the original system for now).

Infrastructure Updates

Git servers updated

We migrated our existing git infrastructure from bare metal servers to virtual machines, which will help to make our infrastructure more flexible and portable in the future. This has been an ongoing effort, and the git servers are among the last of the servers to be migrated.

Continued tuning of Perimeter XPerimeterX

PerimeterX helps to identify bad actor behavior and DDOS attempts and mitigate them at the edge of our network. We established our relationship in January, and throughout February have been monitoring and tuning our configuration to better protect Drupal.org. We've already managed to mitigate a persistent DDOS attack which has recurred every couple of months, and hopefully we can make more improvements to protect Drupal.org, and reduce the pager burden on our team.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Feb 26 2018
Feb 26

JetBrainsWe are happy to announce today that JetBrains is continuing their support of the Drupal Association and the Drupal project in a new way:

It has been part of JetBrains mission to give back to key contributors in the open source world, and we're grateful that they want to offer this opportunity as a benefit for Drupal.org contribution credits.

To receive the free PhpStorm Open Source license, please fill out this open source license request form. Be sure to include:

  •  Project Name: Drupal
  •  # of Developers: 1 - since you are requesting only a single license for yourself
  • Project role: Contributor
  • Your role in project: <-- This should include the link to your Drupal.org profile.
  • Fill in the rest of the fields as you are best able - the ones above are the key fields for this offer.

The Jetbrains team will be checking the profile to ensure you meet the 35 contributions threshold, they will be looking for the line which reads Credited on XXX issues fixed in the past 1 year. You can read more about PhpStorm and the Open Source support program by JetBrains.

Many thanks to JetBrains, we’re excited for their continued partnership.

Feb 20 2018
Feb 20

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

In January the Drupal Association kicked off the new year with our Winter Staff Retreat, where the whole organization came together to review the past 6 months and lay out our strategy for the new year. In addition to taking a big picture look at our upcoming priorities, we also made some great improvements this past month.

Before we dive into those, we'd like to welcome Dhanya Girish to the Drupal Association team. Her time has been generously sponsored by Zyxware and she joins us as a skilled engineer. Please welcome Dhanya!

Drupal.org Updates

Celebrating Drupal's 17th Birthday

To celebrate the 17th birthday of Drupal, we've embedded a wonderful video on the home page, celebrating some of the incredible things that have been built with Drupal. Drupal has grown from being a dorm-room experiment to being the driver for some of the most powerful digital experiences on the web. We can't wait to see what the next 17 years bring.

[embedded content]

Proposed new initiatives for collaboration with Core

The Drupal Association engineering team meets monthly with the Drupal Core committers to align our goals and ensure that we're on the same page about what the project needs moving forward. At the beginning of this year we outlined some new proposed initiatives that, in collaboration with Core, we believe could be a tremendous value to the project.

None of these initiatives can be accomplished quickly, nor can they be moved forward without the collaboration of key contributors and the core committers, but we strongly believe they will be important next steps for the future of Drupal.

Refresh of the Hosting Listings program

A major focus in January was the refresh of our hosting listings program. The new hosting listings now display all providers on the primary view, with a series of filters to help users find exactly the right hosting partner. This update also brings a much needed visual refresh to the listings. New filters and search facets will help end-users find the hosting partner that is right for their needs, industry, and budget, and will enable users to identify and support those platforms that support the project.

Promoting industry solutions in the marketplace

Over the last year, we've been focused on pushing a message about crafting the perfect solution for particular industries with Drupal. We link to this information off of the front page, but we've also added a contextual block in the sidebar that will appear whenever users filter the marketplace by the relevant industry.

These efforts are an ongoing part of better serving the various personas who come to Drupal.org. Look forward to hearing about even more changes on this front as we approach DrupalCon Nashville in April.

In-context issue tag explanations

One of the cornerstones of Drupal.org contribution and issue management is our issue tagging system. However, it's been difficult in the past to understand what each tag is used for, especially for tags that are carefully monitored and curated by project maintainers or the core committer team. We've enhanced the tagging system by providing in-context hover-states that describe what tags are used for.

Infrastructure Updates

Implemented PerimeterX for increased protection from DDOS, crawlers, badbots etc.

Would you believe that Drupal.org is the target of a bad crawler or outright DDOS attack at least once every two months? The team does an incredible job reducing the impact on end users of Drupal.org, but to make that job easier, we've partnered with PerimeterX for bad bot protection integrated with our CDN. PerimeterX integrates closely with Fastly, our CDN, and so can provide intelligent logic and protection at the edge.

We're starting off using this system simply for greater introspection into the nature of traffic patterns and the attacks of bad actors. We're gradually enabling the protection as we tune the system for Drupal.org's unique needs. We want to thank PerimeterX for supporting the project, and for helping to make Drupal.org a better place for our community.

DrupalCI: Support for yarn-based core tests

We've added a yarn build-step plugin to DrupalCI, allowing us to support new testing types, including nightwatch javascript tests in core and contrib. We want to provide a special thanks to community contributors who helped: justafish, dawhener, and mile23.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

  • PerimeterX - *NEW* Signature Technology Supporting Partner
  • publicplan - *NEW* Premium Supporting Partner
  • WebEnertia - *NEW* Premium Supporting Partner
  • Electric Citizen - *NEW* Classic Supporting Partner
  • Factorial - *NEW* Classic Supporting Partner
  • One Shoe - Renewing Classic Supporting Partner
  • DRUD - Renewing Classic Hosting Supporter

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Jan 05 2018
Jan 05

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Announcements

DrupalCon License program launched

DrupalCon In 2018, the Drupal Association made the difficult decision to pause DrupalCon Europe for the year, so that we could re-envision the program for greater scalability and sustainability. We're pleased to announce that we have launched a new program for licensing DrupalCon, so that local entities can bring this great event to their area. Read the announcement by Executive Director Megan Sanicki for more information.

Analysis of Developer Tooling options published

For more than a year, the Drupal Association has been evaluating options for improving the tooling used by developers on Drupal.org. We have recently concluded our study, and published a detailed analysis of the options, as well as our next steps. As expected, this blog series has sparked ongoing conversations about the future of our tools both among the community and with our potential partners - so look out for more updates.

Drupal.org Updates

Easier management of Drupal Association Membership

Drupalcon Individual Member Badge In December we worked on updates to membership management, so that Drupal Association members will be able to manage their membership information.

New DA Membership Directories launched

Last month we mentioned the launch of the new individual member directory on Drupal.org. In December we expanded on this work to update our directory of organization members as well. Drupalcon Organization Member Badge You can explore the directory of Drupal Association members here, as well as the new directory of organization members. If you have feedback on either of these directories, please let us know!

Akismet for spam protection

The content analysis tool Mollom is rapidly approaching its end of life. And so to continue to protect the Drupal community from spam, we have implemented Akismet on Drupal.org. We are currently running it in silent mode, side-by-side with Mollom and our other protection methods, to ensure a smooth transition.

DrupalCI: Chrome Webdriver available for JS testing

DrupalCI One of the major services provided by the Drupal Association is continuous integration testing for the Drupal project. An increasingly important component of this is our javascript testing stack. Previously we tested javascript for the project using PhantomJS. However, that library is now deprecated. We have since created a new testing environment running Chrome Webdriver, and are working with core and contrib developers to ensure it meets their needs.

Drupal.org Updates

Mitigating the risks of Spectre and Meltdown

By now everyone in the technology industry is likely aware of Meltdown and Spectre, the two major security vulnerabilities recently disclosed in major CPU architectures. Drupal Association staff are in close coordination with our infrastructure partners at Tag1Consulting, to ensure that any vulnerable machines in our infrastructure are protected as soon as possible, and our community's data is kept safe. ——— As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association. Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Dec 20 2017
Dec 20

This is the fourth post in our series about integrating Drupal.org with a 3rd party developer tooling provider:

With our plan to create modular integration points for our tooling options, we have a few clear steps for moving forward:

Phase 1: Prep work

  • Deprecation of password authentication for Git, since many external tooling services no longer support it.
  • Working with core to provide compatibility for semver versioning for contrib, both because this is needed for Composer, and because all of the third party developer toolsets we are considering have begun to standardize on semver.

Phase 2: Initial implementation, replacing current features

  • Replacement of custom, bespoke Twisted Git daemon with standard Bitbucket repositories.
  • Replacement of unmaintained CGit with supported Bitbucket code viewing.

Phase 3: New features

  • Integration of merge request 'hook' into issue queues, to allow contributors to use a pull request workflow instead of patches.
    • Modular - to be used with Bitbucket for now, but potentially another solution when more mature.
  • Integration of code review 'hook' into issue queues, to give us powerful inline code commenting tools.
    • Modular - to be used with Bitbucket for now, but potentially another solution when more mature.

Phase 4: Implement Hybrid Integrations for other toolsets

  • Updating project page integrations such that those projects which are already hosted on third party tools such as GitHub or GitLab (for example, Drush) can easily login with SSO, synchronize their repositories, and choose the canonical home of their issues.

On-going: Evaluation

  • Re-evaluate other tooling solutions as blocking issues are resolved and their feature-sets evolve.

So that's the update!

In short: after more than a year's evaluation of the current leaders in open source tooling solutions, including direct collaboration with several of those teams, we are going to focus on making Drupal.org modular to integrate with the best tooling solution as it develops. For now, we will be implementing key improvements for Drupal developers using Bitbucket for our repository hosting, code viewing/review, inline editing, and merge requests - integrated with the existing project pages and issue queues.

We'd like to thank the following people for their involvement in this initiative at various times through the process:

Drupal Association Staff

The Technical Advisory Committee (TAC)

Members of the community

The teams of our potential partner organizations

Each of these teams is committed to serving open source and we thank them for their collaboration during our evaluation process:

  • GitHub
  • GitLab
  • Atlassian/BitBucket
Dec 20 2017
Dec 20

This is the third post in our series about integrating Drupal.org with a 3rd party developer tooling provider:

Below are some rough mockups of what a modular integration between Drupal.org's issue queues and a third party toolset could look like. For the sake of this example, I've used Bitbucket as the tooling provider, as that is likely to be our interim toolset for the reasons outlined above. However, one can easily imagine these functions being substituted for their equivalents in other toolsets.

The following is only an early concept. These are by no means final designs for each of these integration points, but they will help to visualize how these tools will be integrated with Drupal.org.

Mockup: Workspaces in the issue summary

The biggest change for contributors would be the addition of a new place in the issue summary to contain these workspaces. For most issues we anticipate a single workspace, as most issues are resolved via collaboration on a single code path, however we would be able to generate multiple workspaces per issue for multiple solutions being proposed in parallel:

Code workspace in issue summary

The key functions of the Proposed code workspace are: the ability to make a new workspace, the ability to clone an existing workspace, the ability to create/view/edit a workspace merge request. The latest test result and ad-hoc testing options for each workspace would be provided here as well.

Mockup: Propose new Code

The "Propose new code" button is very simply a way to automatically generate a new workspace. This would only be used when a contributor wants to propose an alternate solution to whatever is currently under development. When pressed, we would generate a new workspace with the back-end tooling provider (whether as a fork, branch, or Git namespace) and add it to the table to be cloned, viewed for comments/inline editing, or ultimately to create a new merge request.

A second workspace in the issue summary

The new workspace will be generated with a name based on the issue id, and will present its own test result, clone, and merge request features.

If a user wants to collaborate on an existing solution instead, they can simply clone that workspace locally, or view it to make inline edits. Because some issues can languish for months, or even years, we also want to automate the process of rebasing these workspaces from their parents.

Mockup: Automatic comments as workspaces are updated

Whenever a change is made within the third party toolset, we would use the API to call back to Drupal.org and leave a system message describing the change, as well as linking to the relevant part of the third party toolset UI.

This is what an automated issue comment for code review might look like:

hestenet commented on 2893061-0 - "This is just a test comment" Read more...

This is what a comment for recently pushed changes would look like:

hestenet pushed code to: 2893061-0: View the diff This is a commit messag. Read more...

And this is what a comment for inline edits could look like:

hestenet used the inline editor to update 2893061-1: drupal/core/modules/update/update.api.php - View the diff

Mockup: Clone

The clone button would simply provide a basic modal pop-up with instructions for collaborators to clone the workspace locally with Git. This is not the final url pattern (one of our goals is to avoid changes in canonical git urls where possible) but it could look something like this:

$ git clone ssh://[email protected]/is/drupal.git

Mockup: Create Merge Request (in Bitbucket)

Merge requests allow contributors to propose changes to projects they don't maintain, and provide maintainers an easy way to view proposed changes.

The create merge request button would allow the user to view the current workspace and request that it be merged into the canonical parent. Only a project maintainer would have the permissions to complete the merge.

Create pull request

Code in a workspace with an open pull request can continue to be iterated on by the contributors. Code comments, diffs, and reviews are summarized on the request.

Mockup: View Merge Request (in Bitbucket)

View existing merge request

View merge request, like 'create merge request' would allow any collaborator to view the current workspace. This could also be used to initiate inline code edits, or leave code comments.

Mockup: Inline editing

Inline editing allows a quick and easy way for people to propose "standalone" changes to e.g. documentation, markup, etc. without having to have an entire development stack.

We would rely on the third party tooling provider's inline editing functionality. In the example below, we can see how an inline edit is made using Bitbucket:

Inline editor

Mockup: Code review

As with inline editing, we would rely on the third-party tooling provider's tools for code review. In the example below you can see the code review options provided by Bitbucket:

Code review tools and comments

As indicated above, these mockups are simply an illustration of what this integration could look like, not final designs. However, this has hopefully shown how we can introduce a hybrid model to integrate the best features of a third party tooling provider (pull requests, code review, inline editing), while retaining the essential nature of the drupal-flow.

Dec 20 2017
Dec 20

This is the second post in our series about integrating Drupal.org with a 3rd party developer tooling provider:

In this update we'll talk about the specific options we've been evaluating to improve the tools for developers on Drupal.org. For each of these options we've built prototype workflows, stood up test integrations, and opened a dialogue with the creators of these toolsets about any gaps we identified. You'll see a summary of how each tooling option does or does not live up to the criteria we outlined in our last post.

Issue Workspaces

At DrupalCon Los Angeles in 2015, Mixologic from the DA Engineering team proposed the concept of Issue Workspaces. While we've historically talked about this idea as a single concept, it actually breaks down into two key components:

  1. The definition of a modern, idealized workflow for the Drupal project.
  2. A technical implementation using a bespoke tooling solution built on Git Namespaces to implement this workflow.

Criteria (for a bespoke implementation):

  • Familiar workflow.
    • ❌/ While an implementation of workspaces based on a bespoke git-namespaces implementation has several technical advantages, it will not appear nearly as familiar to outside developers as a more standard 'pull request' implementation. (Though it would be functionally very similar).
  • Preserve the most valuable parts of Drupal's workflow.
    • By implementing a bespoke solution we could exactly tailor the solution to the needs of the Drupal community.
    • ❌ However, the needs of that community are great, and our capacity to build a 100% bespoke solution that meets all the community's needs is not clear.
  • Leverage a partner who will keep evolving their code collaboration featureset.
    • ❌ Building issue workspaces using a bespoke git namespaces implementation would require us to continuously compete on features with major third party tooling providers that have much greater resources.

A purely bespoke implementation misses the mark on two of our three primary criteria. Fortunately, however, two years after the original proposal, it is no longer necessary to build a bespoke solution in order to implement an idealized workflow for the Drupal project. It is something that can be done by integrating the existing Drupal.org project pages and issue queues with third-party tooling solutions. Leveraging third party solutions provides several advantages including: familiarity to a wider audience of developers, a continually evolving featureset, and mitigation of the risks of maintaining a bespoke solution with a small team of engineers.

Here is the idealized 'Drupal-flow' visualized:

Drupal Flow - the interaction between issues and workspaces

  • Every issue should be associated with one or more canonical workspaces (git repos+merge requests) which can be collaborated on by any contributor, and merged into the canonical parent by a project maintainer.
  • Contributors should be able to modify the code in these workspaces by:
    • Checking out the code and committing/pushing changes to the workspace.
    • Inline editing files within a workspace.
    • Legacy: uploading a patch.
    • Any contribution method should trigger the appropriate tests.
  • Workspaces should be rebased (manually or automatically) on any changes to the canonical parent.

This workflow is not dissimilar to the default GitHub flow, GitLab flow, or even the suggested Bitbucket flow. These are all variations of the generic concept of 'Git flow' created by Vincent Driessen. However, the important difference comes in how each of these solutions tries to integrate a gitflow model into their issue/pull-request model.

The dominant model of these large third party tooling providers is not particularly collaborative. It encourages forks-of-forks, separates conversation about potential solutions onto multiple branch comments or merge requests threads, and generally caters to single developers working on individual proposed solutions, with one ultimately selected by the maintainer and the rest thrown away.

This is a key area in which the 'Drupal-flow' differs from the standard workflow that is implicitly enforced by these toolsets. Our ethos encourages collaboration of many developers on a single solution, and a single threaded conversation.

The gitflow behind the scenes may be identical, but the user experience wrapped around that flow makes very different assumptions about how people will collaborate.

GitHubGitHub

GitHub is of course the dominant player in the open-source tooling space. For many years though, their toolset was very much targeted at smaller open-source projects, and their default workflow highly encourages a many-to-many, forks-of-forks workflow, rather than the many-to-one single-threaded collaboration that is part of the Drupal community's ethos. More recently, GitHub has been improving their featureset, providing better issue management tools, more control over organization-level namespaces, and a new app marketplace that allows developers to extend GitHub's core featureset.

We reached out to GitHub and spoke with members of the technical and partnership teams, and while they were excited by the idea of bringing a project like Drupal to GitHub, we did run into some unresolvable blockers - most critically, by policy GitHub does not allow users of GitHub Enterprise to run their own public instances. This means we would have to migrate Drupal projects to GitHub.com, rather than self-hosting our instance.

Criteria:

  • Familiar workflow
    • GitHub is unquestionably the dominant platform for code collaboration on the web today.
  • Preserve the most valuable parts of Drupal's unique workflow
    • ❌ No way to enforce Drupal.org as the home of projects. While we can still keep project pages on Drupal.org there is nothing to encourage visitors who find the project on GitHub to treat the Drupal.org project page as canonical.
    • ❌ ;No way to enforce our workflow - forks of forks of forks of Drupal core into personal namespaces would be one click away.
    • ❌ Issue management tools are not setup for large, crowd-sourced teams.
      • Even for large community projects, it's more typical for only one or two developers to work on a single github issue at a time. Collaboration tends to be divided across issues, rather than within them.
    • ❌ Hard blocker Changing issue metadata requires maintainer permissions. (i.e: No way for non-maintainer collaborators to set an issue to RTBC)
    • ❌ Limited ability to manage project maintainers, especially in the case of abandoned or insecure projects.
  • Leverage a partner who will keep evolving their code collaboration featureset.

Other issues:

  • License/Source availability
    • ❌ GitHub is closed-source.
  • API availability
  • Hybrid integration
    • A hybrid integration between Drupal.org and GitHub is possible with SSO, automatic repository syncing, and a choice of using our issue queue or theirs.
    • ❌ However, there would be no portability of issues across a project that uses GitHub to one that does not.
    • ❌ Projects choosing to use the GitHub issues would not have access to DrupalCI
    • /❌ Projects using GitHub issues would not have access to contribution credit (though it may be possible to develop an app).
  • Data portability/vendor lock-in

    • /❌ GitHub's data migration feature is only available as part of GitHub enterprise.
    • ❌ GitHub policy will not allow us to self-host, which would allow us to work around some of the issues of preserving our Drupal-flow.
  • Developer support
    • GitHub is well-funded and has a large team of developers improving the product. However, it is not open source, and so there is no capacity for the community to fill in the gaps on anything GitHub fails to provide.
    • GitHub has recently added a new 'apps' architecture, enabling deeper integration.
  • Maintainability for an engineering team of limited resources
    • Repositories hosted on GitHub would be easy to maintain, but we would have little access or control in the event of issues, and little ability to customize the workflow for the community's needs.
  • Cost
    • The GitHub team would be willing to provide the Drupal project an enterprise account.
  • Unintended consequences
    • /❌ Moving most of our developer traffic to hosted GitHub could have a significant impact on Drupal.org traffic, affecting everything from our search presence to revenue programs.

GitHub is a no-go.

GitLabGitLab

GitLab is the up-and-comer, and is becoming exceptionally popular in the open-source space - even beginning to be adopted by larger open source projects such as GNOME and Debian.

We've worked closely with the GitLab team as well. In contrast to GitHub, GitLab's terms would allow us to run our own instance of the service. GitLab's fundamental workflow is very much like GitHub's - a one-to-many model of forks of forks. After discussing our workflow with their engineers GitLab has begun implementing their new 'fork network' feature, allowing merge requests across forks. However, this doesn't quite achieve the goal of allowing our many-to-one collaboration model just yet.

GitLab is very close to being a good solution for an Open Source project of Drupal's scale, and the rate at which they are improving the service is impressive. GitLab is also already a favorite of many in our community for personal or professional projects. Given some more time, it may ultimately be able to tick all the boxes.

Criteria:

  • Familiar workflow
    • GitLab's workflow is very similar to GitHub's and it is the second most widely used code collaboration toolset.
  • Preserve the most valuable parts of Drupal's unique workflow
    • ❌ Hard Blocker - Maintainers cannot yet collaborate on downstream merge requests
    • ❌ Monolithic toolset - GitLab's featureset is designed to be used in an all or nothing way. There is no provision for using specific elements while disabling others.
      • For example, there is no way to disable GitLab project pages in order to keep the canonical project pages on Drupal.org.
    • No plugin architecture - changes must be accepted as merge requests upstream in GitLab, or re-rolled as patches with each release (aka, "hacking core").
      • No easy way to turn on or off elements of the GitLab UI.
      • No way to extend the GitLab featureset.
      • Very limited issue management states, and no ability to customize them.
  • Leverage a partner who will keep evolving their code collaboration featureset.
    • GitLab's roll-out of new features is the fastest paced of the tooling providers that exist today.

Other issues:

  • License/Source availability
    • / GitLab Community Edition is open source, but GitLab Enterprise Edition (which we would need for a project of our scale and with our workflow needs) is not open source, only source-available.
  • API availability
    • GitLab has a fairly robust api, but in our evaluation we found that many api features were undocumented, and others were deprecated without notification between minor versions.
  • Hybrid integration
    • A hybrid integration between Drupal.org and GitLab is possible with SSO, automatic repository syncing, and a choice of using our issue queue or theirs.
    • ❌ However, there would be no portability of issues across a project that uses GitLab to one that does not.
    • ❌ Projects choosing to use the GitLab issues would not have DrupalCI integration, or issue credits.
  • Data portability/vendor lock-in
    • GitLab's API and documented data model should make it possible to migrate our data to another system should it become necessary.
  • Developer support
    • As an open-source project, GitLab has a growing community of contributors, as well as the support of a venture funded mothership organization.
  • Maintainability for an engineering team of limited resources
    • ❌ Hard Blocker - GitLab's current file-handling makes full copies of a repository for every fork. Taking the number of open core issues as a representative example, Drupal.org would need a new 200 TB+ redundant storage apparatus, just to accommodate Core. GitLab needs to transition to a shared git-object model, or an alternative like git namespaces.
  • Cost
    • The GitLab team is willing to provide the Enterprise Edition to the Drupal community for free.
  • Unintended consequences
    • n/a

GitLab: not-yet.

BitbucketBitbucket

Bitbucket was the dark horse candidate. Atlassian products are well known in software development circles, but they are often more tolerated than beloved. However, when we sat down at Midwest Drupal Summit in August and looked at the options we explored so far and found them to be not quite baked - Bitbucket began to show some advantages as a solution.

For one thing, Bitbucket has recently put a heavy focus on user experience and collaboration features. It also has the most flexible permissions model for control over branch permissions, forking, and automatic rebasing of any of the toolsets we evaluated. Finally, it can be implemented modularly - allowing us to use the components that serve our workflow and disable the ones that don't.

Criteria:

  • / Familiar workflow
    • Bitbucket implements a very standard pull request workflow. It is not as widely popular as GitHub or GitLab, but should be easily learned by developers familiar with either of those systems.
  • Preserve the most valuable parts of Drupal's unique workflow
    • Of all the options we prototyped, Bitbucket is the only one which provides the flexibility needed to preserve key elements of our workflow - in particular the ability to cleanly use only the parts of Bitbucket that we need (specifically, merge requests, inline editing, and code commenting) without having to use issues or project pages that are less sophisticated than our existing solutions.
  • Leverage a partner who will keep evolving their code collaboration featureset.
    • Though it's flown under the radar, Atlassian has been making some significant improvements to Bitbucket from a feature and user experience point of view over the course of the past year.

Other issues:

  • License/Source availability
    • /❌ Bitbucket is not open-source, only source-available, though they have an open source licensing program.
  • API availability
  • Hybrid integration
    • Bitbucket is much more flexible in terms of choosing which features to use, and which to disable- allowing a fairly tight hybrid integration.
  • Data portability/vendor lock-in
    • / Bitbucket stores repositories as standard git objects on the file system, making it relatively straightforward in case of a future migration to another system.
  • Developer support
    • Bitbucket has the professional support of Atlassian, but does not have a robust open source community driving feature development.
  • Maintainability for an engineering team of limited resources
    • Bitbucket is implemented with an api-first methodology, and is well documented, making it straightforward for a relatively small team to support a selective integration such as what we propose here.
  • Cost
  • Unintended consequences
    • n/a

Bitbucket: the best tool in the current landscape given our community's needs.

To sum up, each of the toolsets we evaluated have advantages and disadvantages. However, if we want to move forward with making improvements for our developers TODAY, we have determined that the best way to do that is to create modular integration points, and use the toolset that currently meets our criteria: Bitbucket. As these toolsets continue their rapid development, we will continue to periodically re-evaluate them.

In our next post:

An illustration of what the future of Drupal.org's developer tools could look like.

Dec 20 2017
Dec 20

The initiative to improve Drupal.org's developer tools is part of a broader effort to broaden the reach of Drupal, not just to end-users and evaluators, but to a wider audience of developers as well. Improvements to our tools are an opportunity to remove friction to changes, increase the quality of changes, improve velocity of changes, and make contributing to Drupal delightful.

The initiative began in coordination with Drupal Association Staff, the Technical Advisory Committee, and a small group of volunteers from the community. We first announced the initiative in October of 2016, and provided our last update at the end of April of this year. Since that time a great deal of work has been underway to evaluate and prototype our options, collaborate with the vendors who supply these toolsets, and make some decisions about the direction we want to move in, with more data in hand.

Buckle up! Because this is such a large topic we've broken it into multiple posts:

Want a TLDR?

Where do we stand now?

Drupal is one of the longest-running open source projects on the web, and for more than 15 years (and many more to come), Drupal.org has been the home of the project. Over the years, we've built a developer toolset to serve the unique needs of the project. These tools have evolved as the open source environment changed. Right now, we use a combination of best-of-breed third party technologies, such as Git, Jenkins, Fastly, OpsGenie, integrated with our own bespoke tools, including issue queues, project pages, etc.

Right now, some of the third party and bespoke tools that we are using are market leading, whereas others have fallen behind. For example, CGIT (third party) and the patch workflow (bespoke) have both fallen behind compared to toolsets that can be found on other tooling providers. On the other hand, our issue crediting system (bespoke) is market leading, and a model for other projects to follow.

In the end, we will always be negotiating a balance between what we can do uniquely well with bespoke solutions, and integrating the latest and greatest of third party solutions as they coalesce around best practices that we want to adopt as well.

At DrupalCon Vienna it was decided that the Technical Advisory Committee had fulfilled their threefold mandate:

  • Helping the technical leadership transition on the DA engineering team after the downsizing in summer of 2016.
  • Helping manage the prioritization and scope of work.
  • Making recommendations to the board and DA staff on key initiatives, such as the project application process changes, and the direction to take with our developer tools.

We want to thank Angie Byron, Steve Francia, and Moshe Weitzman for serving on the committee. From here, DA engineering staff will carry the torch, though we will continue to rely on the individuals who were part of the TAC and other volunteers for feedback and help from time to time as we move forward.

Our evaluation

The most recent phase of this initiative was an evaluation of the leading contenders for open source tooling providers that Drupal.org could integrate with. These options were GitHub and GitLab, and after setting out to develop prototypes for these options, we added BitBucket to the list as well. For each of these options we built MVP prototypes, either as integrations with Drupal.org development environments, or with private organizations/repositories. Finally, we wanted to compare these options to the bespoke Issue Workspaces solution that we had proposed several years ago.

The Criteria

For the Developer Tools initiative to be successful we have to understand the criteria for improvement. From a user requirements point of view, contrib and core developers are unique stakeholders with unique requirements. Ideally we want a solution that increases velocity for both types of users, without fundamentally sacrificing the needs of one or the other.

  • Adopt a developer workflow that will be familiar to the millions of developers outside our community.
  • Preserve those unique elements of how we collaborate that have made the Drupal project so successful.
    • Many-to-one collaboration: that is to say, many developers collaborating on a single solution to a problem.
    • Issue workflow.
    • Picking up on long standing issues where other collaborators left off.
    • Contribution credit.
  • If possible, leverage an expert partner who will help keeping our tooling up to date as open source collaboration tools continue to evolve.

There are also some technical requirements that came out of our evaluation process.

  • Retention of our data/ability to migrate.
    • Where possible, retain existing Git remote urls for projects.
  • Maintainability for a small staff.
  • Project maintainer management: including abandoned project reassignment, fork control, security release management, etc.

In terms of features, we were looking for:

  • Merge/pull requests.
  • Code review.
  • Inline editing.
  • Branch permissioning to allow collaboration on merge/pull requests.
  • Administrative tools for managing project maintainership.
  • Project management tools that equal or exceed what we have with the issue queues.*
  • Extensibility, so that we can preserve areas where the Drupal project is a market leader, such as with our contribution credit system.

*Our tools for issues are very sophisticated on an individual issue level, however we are sorely lacking in tools for grouping and prioritizing sets of issues, ie: issue boards.

Finally, cost:

  • Given the care with which we must use our funding from the community, any option we consider must be cost neutral with the current cost of maintaining our tools.

It's difficult to condense the scope of our evaluation into a short blog series, but in the following posts we'll talk about how the options we considered measured up to these criteria, what implementing one of these options could look like for Drupal.org, and our suggested implementation roadmap.

Dec 12 2017
Dec 12

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Announcements

Clarifications to Drupal licensing policy

There have been some long-standing questions about Drupal project licensing policy. In collaboration with the Licensing Working Group and Dries, we have updated the official licensing policy with the following clarifications and changes:

  • We explicitly affirm that code with a GPL-compatible license can be included in a Drupal.org hosted project, but will be redistributed under our standard "GPL2 or later" policy.
  • We clarify that GPL-incompatible non-code assets may be packaged and/or distributed "in aggregate" with GPL code in Drupal.org projects, per the final stipulation of Sec 3-2 of the GPL, so long as the maintainer has the rights to do so.
  • We clarify that Drupal.org hosted projects can depend on and/or link to GPL-incompatible code (via composer, for example), but that Drupal.org cannot host or distribute those GPL-incompatible dependencies.
  • We explicitly affirm our interpretation of the GPL that a Drupal service provider's act of assembling a codebase while under contract to a client does not fall under the more restrictive terms of 'distribution' per the GPL.

You can find more detail about these changes in the Repository Usage Policy and Licensing FAQ.

Welcoming new staff members

Brooke CandelariaWe want to welcome two new members to the Drupal Association team. Brooke Candelaria who will be taking over as Conference Director, joins us from Houston, TX. Brooke has a background in PR and event management, and has worked with other open source communities in the past, for example working on LinuxWorld and, most recently, with the Python community. Brooke's focus will be on evolving DrupalCon to meet the needs of every persona within our community, and helping to make the Con more sustainable and scalable.

Rachel Lawson in MoldovaRachel Lawson is taking on the role of Community Liaison. Based in the UK, Rachel has been a tremendous member of the community in her own right, participating in Camps, helping to organize Mentored Sprints, and serving on the Community Working Group. Rachel will be helping the community to understand the role and the functioning of the Drupal Association, while also keeping the Drupal Association more closely tied to all parts of our diverse community. We hope you'll welcome them!

DrupalCon Updates

DrupalCon | Be Human, Think DigitalLaunched new DrupalCon brand

We've just launched a new unified look and feel for DrupalCon, that will carry into all of our events moving forward. This will help give DrupalCon a stronger identity among OSS events, and also give us a place to put centralized content that applies to DrupalCon as a whole, rather than just a singular event.

Launched DrupalCon Nashville site

Alongside our new brand, we've launched the DrupalCon Nashville website, the first of the events to use the new unified look and feel. Registration is open now, as well as the call for papers. Take a look at some of the new tracks! \

Nashville guitar

We'll see you there!

Sponsorship makes DrupalCon possible - learn why you should be a part of it.

Drupal.org Updates

Reminder: New issue shortcuts and friendly url structure

Last month we announced that we would be implementing some changes to the issue url structure, as well as some shortcuts to help users navigate to issues more easily on Drupal.org. These changes went live in November. Here's a primer:

URL Pattern for issues:

https://drupal.org/project/drupal/issues/2922626 When an issue is moved between projects the alias will be updated.

Shortcuts

The search bar will now automatically redirect you to a node if you enter its id directly: Search Bar Node ID shortcut A new menu callback will help you get to issues with a shorter url string: https://drupal.org/i/

New individual member directory

We've been overhauling the Drupal Association membership experience, and as part of that process we now have a new directory of Drupal Association members. This new directory also includes new filters and sorting options so that you can see the latest community members to join the Association and filter by country, username, or first and last name.

In development: new organization member directory

We're working on a new directory of organization members as well. The current directory allows you to filter by organization type, and we will be adding a filter for Association membership as well. If you have feedback on this directory, please let us know!

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association. Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Nov 17 2017
Nov 17

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Announcement

New issue shortcuts and friendly url structure

Drupal contributors have been managing bug fixes, feature requests, and code reviews on Drupal.org for around 15 years now. Passing an issue node id around a sprint table is a stable of DrupalCon and camps around the world. In October we announced that we would be implementing some changes to the issue url structure, as well as some shortcuts to help users navigate to issues more easily.

URL Pattern for issues:

https://drupal.org/project/drupal/issues/2922626
When an issue is moved between projects the alias will be updated.

Shortcuts

The search bar will now automatically redirect you to a node if you enter its id directly: Search Bar Node ID shortcut

A new menu callback will help you get to issues with a shorter url string:
https://drupal.org/i/<nid>

And of course you can still use the old https://drupal.org/node/<nid> urls if you still have them bookmarked.

Spoiler alert: These shortcuts and new url patterns were deployed in November and you can use them right now!

Drupal.org updates

Composer instructions on release pages

To make it easier for site builders to figure out how to use a release with composer we've added the composer command line instructions to release pages.

Composer instructions on release nodes

This command installs the package with the current release number specified as a minimum version parameter. We also provide a link to the documentation on using Composer to manage Drupal site dependencies, to help users who may be unfamiliar with Composer learn how to use it.

The Community Section

The community is the heart of the Drupal project, but until now community news has not had it's own place to live. We've now made the community page a proper section with its own blog, so that community posts and CWG information has a dedicated place to live.

When the first posts in this new section go live, we'll add this blog feed to Drupal Planet as well. Over time, we hope to further refine the community section and improve the tools we provide for the community to connect with each other.

WYSIWYG for Forums (CKEditor)

We're always looking for ways to make improvements to the site that have a high impact to effort ratio. One such change was enabling the CKEditor for editing in the forums. CKEditor has been in the wild as a WYSIWYG editor on Drupal.org for other content types for quite a while now, and we felt confident it was ready for use on forums as well.

CKEditor WYSIWYG for the Forums

Bug-fix: Dev releases on project pages

In the runup to DrupalCon Vienna we made a number of improvements to project pages - however a bug or two crept in as well. A race condition was causing dev releases not to display in some cases, and we resolved this issue in October. If you're a project maintainer on Drupal.org and see anything else go missing, please let us know!

Dev Releases

Infrastructure

DrupalCI: Faster, more affordable testing

DrupalCI uses spot requests on Amazon Web Services to spin up testbots on-demand for the Drupal project. In the past, instances were provisioned in minimum increments of one hour, meaning to make the most of testing we had to queue up tests to reuse the remainder of any paid-for instance-hours.

Because AWS has enabled per-second billing, we no longer have to try to fill instance-hours, and so we have reconfigured our spot instance requests to provision testbots faster, while still saving money overall compared to the previous configuration.

DrupalCI: More efficient RTBC testing

We also discovered that a bug in our automated RTBC retesting system was triggering more tests than necessary. We've fixed the bug, and now only the most appropriate recent test/environment will be retested for RTBC issues.

Server Maintenance Windows

Finally, we scheduled several maintenance windows in cooperation with our infrastructure services partner to schedule updates/restarts of our servers.

If you want to keep up-to-date with Drupal.org-related changes and maintenance windows you can subscribe to our Change Notices, or follow us on Twitter.

https://t.co/57fE1VZHTn db maintenance window is complete. We may schedule a follow up maint window, and will notify here if needed.

— Drupal infra (@drupal_infra) October 31, 2017

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Nov 10 2017
Nov 10

Drupal developers have been shouting node id numbers across tables at sprints for almost 15 years now. The Drupal Association has added some shortcuts and friendly urls for issues to make this a little easier.

Shortcuts

The first shortcut we added is a searchbar feature. If you enter any node id in the search bar you'll be automatically redirected to that node. This works for more than just issues!

Searchbar Node Shortcut

The second shortcut we added is a quick url pattern for finding issues. If you know your node id, just type:

https://drupal.org/i/<node-id>

 and you're there!

Friendly-Urls - Updated 2017-11-12

We've implemented a new friendly url pattern on Drupal.org to make the canonical urls more sensible and search engine friendly.

Project pages:  Unchanged https://drupal.org/project/drupal

Issue Queue:  Unchanged (for now*) https://drupal.org/project/issues/drupal

UPDATED - Specific Issue: https://drupal.org/project/drupal/issues/2922626

This new pattern should be backfilled to all issues by the beginning of next week.

And of course, if you have old bookmarks linked, you can still get there by going to: https://drupal.org/node/<node-id>

These are small, but powerful improvements that should make the lives of everyone who contributes to the Drupal project easier.

_________________
UPDATED - *After this change to specific issue urls will we create a follow up to change issue queue urls to match the pattern: https://drupal.org/project/drupal/issues

Oct 17 2017
Oct 17

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

DrupalCon Vienna

We're back from DrupalCon Vienna, with updates on what's new from the month of our European event.

Announcement

TLS 1.0 and 1.1 deprecated

Drupal.org uses the Fastly CDN service for content delivery, and Fastly has depreciated support for TLS 1.1, 1.0, and 3DES on the cert we use for Drupal.org, per the mandate by the PCI Security Standards Council. This change took place on 9 Aug 2017. This means that browsers and API clients using the older TLS 1.1 or 1.0 protocols will no longer be supported. Older versions of curl or wget may be affected as well.

Drupal.org updates

DrupalCon Calendar syncing

In our last update, we teased a new feature for DrupalCon attendees - the ability to sync your personal schedule to a calendar program. We're pleased to report that this feature made it in time for the event, and was used by attendees throughout the week. If you've already synced your calendar for DrupalCon Vienna, you're already set up to use the same feed for DrupalCon Nashville next April!

Keynote simulcast to Youtube

This year at DrupalCon, in addition to live streaming on Events.Drupal.org itself, we simulcast the keynotes to YouTube. We also embedded the keynote on the Drupal.org homepage - to spread the latest news about Drupal beyond DrupalCon attendees.

In fact, if you couldn't attend DrupalCon or just missed the keynotes, you can watch Dries' update on the Drupal project here:

[embedded content]

Industry Pages promoted in the front page Call-to-Action

We've also made some updates to how the industry pages are promoted. In addition to the dedicated block with icons linking to each industry, we now also promote the industry solutions landing page in the main CTA under the homepage header.

Industry Page CTA

We hope to further encourage users evaluating Drupal to explore some of the tremendous solutions that are already out there, and take inspiration from their success.

First-in/First-out issue sorting

To make sure that issues are reviewed by maintainers in the order they are received, it is now possible to sort the issue queues by when the issue status last changed. This means RTBC issues can be reviewed on a first-in/first-out basis!

This 'status changed' date field is available on the advanced search view for any issue queue. Here's what it looks like for Drupal core:

Issue Status Sort

Project creation analysis

About six months ago we opened up project creation on Drupal.org to allow any confirmed user to create a full project. We've put together a blog post outlining the impact these changes have had on the contrib landscape. In short, we've seen a tremendous increase in the rate of project creation, and the rate of applications for security advisory coverage, and a modest increase in projects receiving stable releases without yet opting in coverage. We're continuing to monitor project creation and work with the Security Working Group and others on next steps.

Displaying orphan dev releases

In last month's update we talked about a variety of changes we made to project pages, to provide better signals about project quality to evaluators. In response to feedback, we've restored the visibility of dev releases, even when they aren't associated with a tagged release.

Dev releases

This is particularly helpful for project maintainers trying to bring visibility to the next major development version of their modules, such as their Drupal 8 module port efforts.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Oct 07 2017
Oct 07

About six months ago we made a significant change to the way that modules, themes, and distributions are created on Drupal.org.

In the past, contributors had to first create a sandbox project, and then request manual review of their project in the Project Applications issue queue. The benefit of this community-driven moderation process was that modules were vetted for code quality and security issues by a group of volunteers. Project maintainers who completed this process also received the benefit of security advisory coverage from the Security Team for stable releases of their projects.

Unfortunately, the rate of project applications outpaced what volunteers could keep up with, and many worthy projects were never promoted to full project status, or moved off of Drupal.org to be hosted elsewhere.

To ameliorate this issue, we changed the process so that any confirmed user on Drupal.org may now make full projects.

To mitigate the risks of low code quality or security vulnerabilities we added new signals to project pages: including highlighting which release is recommended by the maintainer, displaying recent test results, and indicating whether the project receives security coverage both on the project page and in the composer 'extra' attribute. We're continuing to work on identifying additional signals of project quality that we can include, as well as surfacing some of this information in Drupal core. We also converted the project applications issue queue into a 'request security advisory coverage' issue queue.

What we hoped to see

We knew this would be a significant change for the project and the community. While many community members were excited to see the gates to contribution opened, others were concerned about security issues and Drupal's reputation for code quality.

Our prediction was that the lower barrier to contribution would result in an increase in full projects created on Drupal.org. This would indicate that new contributors or third party technology providers were finding it easier to integrate with Drupal and contribute those integrations back for use by others.

At the same time, we also expected to see an increase in the number of full projects that do not receive coverage from the security team. The question was whether this increase would be within an acceptable range, or represent a flood of low quality or insecure modules.

The results

The table below provides statistics about the full projects created on Drupal.org in the 5 months before March 17th, 2017 - when we opened the creation of full projects to all confirmed users.

Full projects created from 2016-10-16 to 2017-03-17…

#

% of projects created in this period

… without stable release

431

55.76%

… with stable releases

342

44.24%

… with usage >= 50 sites

237

30.66%

… with usage >= 50 sites and without stable release

68

8.80%

… with usage >= 50 sites and with stable release

169

21.86%

… with an open security coverage application*

18

2.33%

Sub-total with security coverage

342

44.24%

Sub-total without security coverage

431

55.76%

Sub-total with security coverage and >=50 usage

169

21.86%

Sub-total without security coverage and >= 50 usage

68

8.80%

Total

773

* note: full projects that did not have stable releases were not automatically opted in to security coverage when we opened the full project creation gates.

… and this table provides statistics about the projects created in the 5 months after we opened the creation of full projects to all confirmed users:

Full projects created from 2017-03-17 to 2017-08-16…

#

Diff

% of projects created

Diff %

… without stable release

851

+420

69.53%

+97%

… with stable releases

373

+31

30.47%

+9%

… with usage >= 50 sites

156

-81

12.75%

-34%

… with usage >= 50 sites and without stable release

64

-4

5.23%

-6%

… with usage >= 50 sites and with stable release

92

-77

7.52%

+46%

… with an open security coverage application

62

+44

5.07%

+344%

Sub-total with security coverage

182

-160

14.87%

-53%

Sub-total without security coverage

1,042

+611

85.13%

+242%

Sub-total with security coverage and >=50 usage

54

-115

4.41%

-32%

Sub-total without security coverage and >= 50 usage

102

+34

8.33%

+150%

Total

1,224

+451

+58%

As you can see, we have an almost 58% increase in the rate of full projects created on Drupal.org. We can also see a significant proportional increase in two key areas: projects with greater than 50 site usage and no security coverage(up 150% compared to the previous period), and projects that have applied for security coverage(up 344% compared to the previous period). Note: this increase in applications is for projects *created in these date ranges* not necessarily applications created overall.

This tells us that reducing friction in applying for security coverage, and encouraging project maintainers to do so should be a top priority.

Finally, this last table gives statistics about all of the projects currently on Drupal.org, regardless of creation date:

Full projects (7.x and 8.x)

#

% of Total

Rate of change after 2017-03-17

… with the ability to opt into security coverage

8,718

36.15%

-1.33%

… with security coverage and stable releases

8,377

34.74%

-1.49%

… without security coverage

15,396

63.85%

+1.33%

… without security coverage and with stable releases

464

1.92%

+1.04%

… with security coverage and >=50 usage
 

6,475

66.91 / 26.85%

-0.54%

… with security coverage and stable releases and >=50 usage

6,308

65.19 /26.16%

-0.65%

… without security coverage and >=50 usage

3,202

33.09 /13.28%

+0.54%

… without security coverage and with stable releases and >=50 usage

130

1.34 /0.54%

+0.51%

Sub-total with >=50 usage

9,677

40.13%

-1.72%

Total

24,114

From the overall data we see approximately what we might expect. The increase in growth of full projects on Drupal.org has lead to a modest increase in projects without security coverage.

Before the project application change, all full projects with stable releases received security advisory coverage. After this change, only those projects that apply for the ability to opt in(and then do so) receive coverage.

What has this meant for security coverage of projects hosted on Drupal.org?

1.92% of all full 7.x and 8.x projects have stable releases, but do not receive security advisory coverage. It is likely no accident that this translates into 464 projects, which is nearly equivalent to the number of projects additional projects added compared to our old growth rate.

Of those only 130 of those projects report more than 50 sites usage(or .54% of all 7.x and 8x full projects).

Next steps

From this analysis we can conclude the following:

  1. The opening of the project application gates has dramatically increased the number of projects contributed to Drupal.org.

  2. It has also increased the number of projects without security coverage, and the number of applications for the ability to opt in to coverage among new projects.

In consultation with the Security Working Group, we recommend the following:

  • For now, leave the project creation projects as it stands today - open to contribution from any confirmed user on Drupal.org.

  • Solve the problem of too many security advisory coverage applications. The security advisory application queue has the same problem that the old project applications queue had - not enough volunteers to manually vet all of the applications - and therefore a significant backlog of project maintainers waiting on the ability to opt into coverage.

    • Recommendation: Implement an automated best practices quiz that maintainers can take in order to be granted the ability to opt into security advisory coverage. If this process is as successful as we hope, we may want to consider making this a gate on stable releases for full projects as well.

We look forward to working with the Security Working Group to implement this recommendation and continue to improve the contribution experience on Drupal.org, while preserving code quality and security.

Sep 19 2017
Sep 19

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Announcement

TLS 1.0 and 1.1 deprecated

Drupal.org uses the Fastly CDN service for content delivery, and Fastly has depreciated support for TLS 1.1, 1.0, and 3DES on the cert we use for Drupal.org, per the mandate by the PCI Security Standards Council. This change took place on 9 Aug 2017. This means that browsers and API clients using the older TLS 1.1 or 1.0 protocols will no longer be supported. Older versions of curl or wget may be affected as well.

Almost time for DrupalCon Vienna

DrupalCon Vienna

DrupalCon Vienna is almost here! From September 26-29 you can join us for keynotes, sessions, and sprinting. Most of the Drupal Association engineering team will be on site, and we'll be hosting a panel discussion about recent updates to Drupal.org, and our plans for the future.

We hope to see you there!

Drupal.org updates

8.4.0 Alpha/Beta/Release Candidate 1

On August 3rd, Drupal 8.4.0 received its alpha release, followed on the 17th by a beta release, and on September 6th by the first release candidate. Several new stable API modules are now included in core for everything from workflow management to media management. Core maintainers hope to reach a stable release of Drupal 8.4 soon.

Improvements to Project Pages

We made a number of improvements to project pages in August, one of which was to clean up the 'Project information' section and add new iconography to make signals about project quality more clear to site builders.

Project information improvements

In the same vein, we've also improved the download table for contrib projects, by making it more clear which releases are recommended by the maintainer, providing pre-release information for minor versions, and displaying recent test results.

Download table improvements

Metadata about security coverage available to Composer

Developers who build Drupal sites using Composer may miss some of the project quality indicators from project pages on Drupal.org. Because of this, we now include information about whether a project receives security advisory coverage in the Composer 'extra' attribute. By including this information in the composer json for each project, we hope to make it easier for developers using Composer to ensure they are only using modules with security advisory coverage. This information is also accessible for developers who may want to make additional tools for managing composer packages.

Automatic issue credit for committers

Just about the last step in resolving any code-related issue is for a project maintainer to commit the changes. To make sure these maintainers are credited for the work they do to review these code changes, we now automatically add issue credit for committers.

Performance Improvements for Events.Drupal.org

With DrupalCon coming up in September we spent a little bit of time tuning the performance of Events.Drupal.org. We managed to resolve a session management bug that was the root cause of a significant slow down, so now the site is performing much better.

Syncing your DrupalCon schedule to your calendar

A long requested feature for our DrupalCon websites has been the ability to sync a user's personal schedule to a calendar service. In August we released an initial implementation of this feature, and we're working on updating it in September to support ongoing syncing - stay tuned!

Membership CTA on Download and Extend

We've added a call to action for new members on the Drupal.org Download and Extend page, which highlights some great words and faces from the community. Membership contributions are a crucial part of funding Drupal.org and DrupalCon, but much the majority of traffic we receive on Drupal.org is anonymous, and may not reach the areas of the site where we've promoted membership in the past. We're hoping this campaign will help us reach a wider audience.

Membership CTA on the Download page

DrupalCI sponsorship

DrupalCI is one of the most critical services the Drupal Association provides to the project, and also one of the more expensive. We've recently added a very small section to highlight how membership contributions help provide testing for the project - and in the future we hope to highlight sponsors who will step up specifically to subsidize testing for the Drupal project.

Infrastructure

More semantic labels for testing

In August we added more semantic labels for DrupalCI test configuration. This means that project maintainers no longer have to update their testing targets with each new release of Drupal, they can instead test against the 'pre-release' or 'supported' version, etc. More information can be found in the DrupalCI documentation.

Semantic Labels for Testing

Started PCI audit

In August we also began a PCI audit, and developed a plan of action to reduce the Drupal Association's PCI scope. Protecting our community's personal and financial information is critically important, and with a small engineering team, the more we can offload PCI responsibility onto our payment vendors the better. We'll be continuing to work on these changes into the new year.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

May 24 2017
May 24

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

DrupalCon Baltimore logo Apr 24-28

At the end of April we joined the community at DrupalCon Baltimore. We met with many of you there, gave our update at the public board meeting, and hosted a panel detailing the last 6 months worth of changes on Drupal.org. If you weren't able to join us for this con, we hope to see you in Vienna!

Drupal.org updates

DrupalCon Vienna Full Site Launched!

DrupalCon Vienna logo Sep 26-29 2017

Speaking of Vienna, in April we launched the full site for DrupalCon Vienna which will take place from September 26-29th, 2017. If you're going to join us in Europe you can book your hotel now, or submit a session. Registration for the event will be opening soon!

DrupalCon Nashville Announced with new DrupalCon Brand

DrupalCon Nashville logo Apr 9-13 2018

Each year at DrupalCon the location of the next conference is held as closely guarded secret; the topic of speculation, friendly bets, and web crawlers looking for 403 pages. Per tradition, at the closing session we unveiled the next location for DrupalCon North America - Nashville, TN taking place from April 9-13th in 2018. But this year there was an extra surprise.

We've unveiled the new brand for DrupalCon, which you will begin to see as the new consistent identity for the event from city to city and year to year. You'll still see the unique character of the city highlighted for each regional event, but with an overarching brand that creates a consistent voice for the event.

Starring Projects

Users on Drupal.org may now star their favorite projects - making it easier to find favorite modules and themes for future projects, and giving maintainers a new dimension of feedback to judge their project's popularity. Users can find a list of the projects they've starred on the user profile. Over time we'll begin to factor the number of star's into a project's ranking in search results.

Starring Projects

At the same time that we made this change, we've also added a quick configuration for managing notification settings on a per-project basis. Users can opt to be notified of all issues for a project, only issues they've followed, or no issues. While these notification options have existed for some time, this new UI makes it easier than ever to control issue notifications in your inbox.

Project Browsing Improvements

One of the important functions of Drupal.org is to help Drupal site builders find the distributions, modules, and themes, that are the best fit for their needs. In April, we spent some time improving project browsing and discovery.

Search is now weighted by project usage so the most widely used modules for a given search phrase will be more likely to be the top result.

We've also added a filter to the project browsing pages to allow you to filter results by the presence of a supported, stable release. This should make it easier for site builders to sort out mature modules from those still in initial development.

Better visual separation of Documentation Guide description and contents

Better Documentation Guide Display

In response to user feedback, we've updated the visual display of Documentation Guides, to create a clearer distinction between the guide description text and the teaser text for the content within the guides.

Promoting hosting listings on the Download & Extend page

To leverage Drupal to the fullest requires a good hosting partner, and so we've begun promoting our hosting listings on the Download and Extend page. We want Drupal.org to provide every Drupal evaluator with all of the tools they need to achieve success—from the code itself, to professional services, to hosting, and more.

Composer

Sub-tree splits of Drupal are now available

Composer Façade

For developers using Composer to manage their projects, sub-tree splits of Drupal Core and Components are now available. This allows php developers to use components of Drupal in their projects, without having to depend on Drupal in its entirety.

DrupalCI

Automatic Requeuing of Tests in the event of a CI Error

DrupalCI logo

In the past, if the DrupalCI system encountered an error when attempting to run a test, the test would simply return a "CI error" message, and the user who submitted the test had to manually submit a new test. These errors would also cause the issues to be marked as 'Needs work' - potentially resetting the status of an otherwise RTBC issue.

We have updated Drupal.org's integration with DrupalCI so that instead of marking issues as needs work in the event of a CI Error, Drupal.org will instead automatically queue a retest.

Bugfix: Only retest one environment when running automatic RTBC retests

Finally, we've fixed a bug with the DrupalCI's automatic RTBC retest system. When Drupal HEAD changes, any RTBC patches are automatically retested to ensure that they still apply. It is only necessary to retest against the default or last-used test environment to ensure that the patch will work, but the automatic retests were being tested against every configured environment. We've fixed this issue, shortening queue times during a string of automatic retests and saving testing resources for the project.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Apr 18 2017
Apr 18

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

DrupalCon Baltimore logo Apr 24-28

The Drupal Association team is gearing up for DrupalCon Baltimore. We're excited to see you there and we'll presenting a panel giving an update on our work since Dublin, and our plans for the coming months.

Drupal.org updates

Project application revamp

As we announced in mid-March, new contributors on Drupal.org can now create full projects and releases! Contributors no longer have to wait in the project application queue for a manual review before they are able to contribute projects.

This is a very significant change in the Drupal contribution landscape, and it's something we approached carefully and will continue to monitor over the coming months. Drupal has always had a reputation for a high quality code, and we want to make sure that reputation is preserved with good security signals, project quality signals, and continued incentives for peer code review.

That said, we're very excited to see how this change opens up Drupal to a wider audience of contributors.

Please note that the removal of project applications to create full projects and releases means a change in the security advisory policy (see below for details).

Security Advisory Opt-in and new Security Signals for Projects

Are you responsible for the security of your clients' Drupal sites?

Please note that Drupal's security advisory coverage policy has changed. Security advisory coverage for contributed projects is now only available for projects that have both opted in to receive coverage and made a stable release. You can see which projects have opted in by checking their project pages. If you have questions, please contact [email protected].

Because users may now create full projects and releases without opting in to security advisory coverage, it's critically important that we provide good security signals to users evaluating projects on Drupal.org. This is why we've added a security coverage warning to projects that aren't opted in to coverage.

We've also:

2017 Community Elections Update

The 2017 elections for the community-at-large seat on the board were held successfully in March. Drupal Association community board elections are conducted with the Instant Runoff Voting system. This voting methodology requires that voters rank their preferred candidates on their ballot, and we've heard that this system has been somewhat unwieldy in the past.

Each year we try to improve the voter experience and so this year we deployed a new drag-and-drop ballot.

Drag and Drop Ballot

Finally, we want to congratulate our newest board member Ryan Szrama!

Better international datetime support throughout Drupal.org

Drupal.org has grown organically over the course of more than a decade, and as features have been built out they were not always consistent in their display of datetime information. While it sometimes makes sense to have a few different formats for displaying date and time, many of the formats in use were simply arbitrary historical decisions.

As a quality of life improvement, especially for users outside of the USA, we've standardized the datetime format used on Drupal.org. That format is: DD MMM YYYY - hh:mm (UTC±h). For example: 11 Aug 2016 - 16:42 (UTC+8)

DrupalCI

CSS Lint check style results

DrupalCI logo

When we implemented coding standards testing in DrupalCI in February we were not able to add CSS Lint testing until the CSSLint configuration file in core was fixed. That issue was fixed in late February and so we added CSSLint to support coding standards testing for CSS at the beginning of March.

Cleaning up coding standards results

The addition of coding standards results to DrupalCI means that Drupal.org is now storing even more test data about the code we test on Drupal.org. Our initial implementation of coding standards testing did not include clean up of older results, and so to preserve database space and testing resources, we implemented some clean-up routines in March. In particular we are now:

  • Cleaning up all results for closed issues
  • For custom one-off tests, keeping results for 30 days to match what is shown on project’s automated testing tab
  • For tests triggered on a schedule or commit, keeping the most recent per-environment per-branch, and keeping anything less than 24h old

Infrastructure

Protecting Git services

We experienced some minor Git outages in March, due to malicious authentication attempts. To mitigate these issues in the future, we've implemented fail2ban rules to protect Git authentication. This should improve the stability and uptime of Git services for all developers on Drupal.org.

We want to thank Drupal.org infrastructure volunteer mlhess for his assistance with this.

Community Initiatives

Contrib Documentation Migration

New tools for Documentation have been available on Drupal.org for more than half a year. While most of the core documentation has been migrated to the new system, we are still encouraging Contrib maintainers to migrate their docs.

To make it easier for contrib project maintainers to migrate their documentation to the new documentation tools, we've made two improvements:

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects. In particular we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Apr 11 2017
Apr 11

Workflow iconIn October of last year the Technical Advisory Committee was formed to evaluate options for the developer tools we use on Drupal.org. The TAC consists of Angie Byron, Moshe Weitzman, and Steve Francia, acting as advisors to Megan Sanicki, the Executive Director of the Drupal Association.

The TAC's mandate is to recommend a direction for the future of our tools on Drupal.org. Megan will evaluate this recommendation, make a decision, and prioritize that work in the development roadmap of the Drupal Association engineering team.

What is the motivation behind looking at our developer tools now?

Close followers of the Drupal project will have noticed a trend in the last several months. From Dries' announcement of easy upgrades forever, to the revamp of the project application process, to the discussion about making tools for site builders— there is a unifying theme: broadening the reach of Drupal.

This is the same motivation that underlies this evaluation of our developer tools, and defines the goals and constraints of this initiative:

  • Adopt a developer workflow that will be familiar to the millions of developers outside our community
  • Preserve those unique elements of how we collaborate that have made the Drupal project so successful
  • If possible, leverage an expert partner who will help keeping our tooling up to date as open source collaboration tools continue to evolve

This means looking at a number of elements of the Drupal.org developer tool stack:

  • The underlying git service
  • How we tag and package releases
  • The contribution workflow (patch vs. pull request)
  • Project management workflows (the issue queues and tags)
  • CI integration
  • Maintainership
  • Project pages

If this looks like a tremendous undertaking - that's because it is. But there are some things we already know:

  • Drupal.org should continue to be the home of project pages
  • We should adopt a pull request workflow (and ideally we want to be able continue to accept patches as well, at least in the interim)
  • We should move contrib projects to semver, following core's lead
  • We want to preserve our familiar understanding of maintainership
  • We want to avoid forked code and forked conversation
  • We want to ensure the security team still has the tools they need to provide their service to the community

We also know that whatever decision is made, these changes cannot happen all at once. We'll need to take a progressive approach to the implementation, and focus on the parts of the stack that need to change together, so that we don't bite off more than we can chew.

What options are being considered?

At this time, the technical advisory committee is considering three options as they prepare to make their recommendation. The options are: GitLab, which offers both self-hosted and SaaS options; GitHub, which has recently been adding long-requested new features; or continuing to evolve our custom-built tooling, perhaps via issue workspaces.

GitLab

GitLab is the up-and-comer among Git hosts. GitLab can be self hosted using either their community or enterprise editions, or repositories can be hosted at GitLab.com. Though they recently stumbled, they have been notably open and transparent about their efforts to build a leading collaboration platform.

Gitlab is itself open-source, and has just released its 9.0 edition. GitLab has aggressively pursued the latest in development tools and workflow features, including project management tools, a ui for merge conflict resolution with in-line commenting and cherry-picking, docker registries for projects, integration with CI tools, and even Gitter, an IRC alternative for real-time collaboration.

GitHub

For quite some time, GitHub was the only real player in git repository hosting outside of rolling a custom solution (as we did for Drupal.org). Over the years it has become the home of many open source projects, and while most of those don't match the sheer scale of Drupal in terms of codebase size and number of contributors, there are certainly other major projects that have made their home there.

However, for all of its presence and longevity in the open source world, there are very few options for customizing its toolset for our needs, and we could no longer self-host our canonical repositories. The Drupal project would need to adapt to GitHub, rather than the other way around.
 

That said, in recent months, GitHub has been putting a strong focus on feature development, adding a number of new features including: automated licensing information,  protected branches, and review requests.

Custom tooling

We also must consider that the tools we have now are what built Drupal into what it is today. A great deal of work has gone into our existing developer tools over the years, and we have some unique workflows that would have to be given up if we switched over to a tooling partner. An idea like the issue workspaces proposal would allow us to achieve the goal of modernizing our tools, and likely do so in a way that is better tailored to those unique things about the Drupal workflow that we may want to preserve. However, doubling down on building our own tooling would come at a cost of continuing to be unfamiliar to the outside development community, and dependent on an internal team's ability to keep up with the featureset of these larger players.

Each of these three options would be a compromise between reaching outward and creating familiarity, and looking inward to preserve the Drupal specific workflows that have brought the project to where it is today.

What have we learned so far?

The TAC has conducted their own internal evaluation of the options as well as worked with Drupal Association staff in a two day exploratory session at the end of last year. The primary focus was to identify and triage gaps between the different toolsets in the following areas:

  • Migration effort
  • Project management
  • Code workflow
  • Project handling
  • Testing
  • Git Back-end/Packaging
  • Integrations beyond tools

This initial study also looked at the impact of each option on Drupal community values, and some key risks associated with each.

What comes next?

The next step for the TAC is to make their formal recommendation to the Executive Director of the Drupal Association. At that point, she will direct staff to validate the recommendation by prototyping the recommended solution. Once the recommendation has been validated, Megan will make a final decision and prioritize the work to fully implement this option, relative to other Drupal Association imperatives.

In the comments below, we would love to hear from the community:

What aspects of the way the Drupal community collaborates are the most important to you? What workflow elements do you feel need to be preserved in the transition to any new tooling option?

Mar 17 2017
Mar 17

Any user on Drupal.org who has accepted our Git usage policy may now create full projects with releases. This is a big change in policy for the Drupal project, representing an evolution of the contribution ecosystem in the past half a decade.

What was the Project Application Process?

Ever since the days when Drupal's code was hosted in CVS there has been some form of project application process in the Drupal Community. To prevent duplicate, low-quality, insecure, or otherwise undesirable projects from flooding Drupal, users would submit sandbox projects to an application queue to be reviewed by a group of volunteers.

After resolving any issues raised in this review process, the user would be given the git vetted role, allowing them to promote their sandbox to a full project, claim a namespace, and create releases. Once a user had been vetted for their first project, they would remain vetted and be able to promote any future projects on their own, without submitting an additional application.

The Problem

Unfortunately, though the project application process was created with the best of intentions, in the long term it proved not to be sustainable. Drupal grew too fast for a group of volunteer reviewers to keep up with reviewing new projects, and at times there were applications waiting in queue for 6 months to 1 year, or even more. That is much too slow in the world of software development.

This put Drupal in a difficult situation. After years of subjecting new projects and contributors to a rigorous standard of peer review, Drupal has a well-deserved reputation for code quality and security. Unlike many open source projects, we largely avoided the problem of having many duplicate modules that exist to serve the same purpose. We unified our community’s effort, and kept up a culture of collaboration and peer review. At the same time, many would-be contributors were unable or unwilling to navigate the application process and so simply chose not to contribute.

The question became, how could we preserve the emphasis on quality while at the same time removing the barrier to contribution that the application process had become?

Constraints on a solution

Opening the contribution gates while retaining strong signals about code quality and security was a tricky problem. We established three constraints on a solution:

  1. We need to welcome new contributors, and eliminate the walls that prevent contribution.
  2. We need to continue to send strong signals about security coverage to users evaluating whether to use modules from Drupal.org.
  3. We need to continue our strong emphasis on quality and collaboration through changes to project discovery that will provide new signals about code quality, and by providing incentives and credit for peer review.

The Solution

In collaboration with the community, the security team, members of the board, and staff we outlined a solution in four phases:

Phase 1: Send strong signals about security advisory coverage.

  • We updated project pages to include messaging and a shield icon to indicate whether a project received security advisory coverage from the security team.
  • We now serve security advisory coverage information in the Updates status information provided by Drupal.org, and we're working on a patch to display that information directly on the updates page of users' Drupal sites.

Here are some examples of what these security signals look like on project pages:

If a project is not opted in to security advisory coverage, this message will appear at the top of the project page:

Warning at the top of Project pages

And this one will appear near the download table:

Warning above download table

If a project has opted in, this message will appear near the download table:

Project opt in notice

And covered releases will show the coverage icon (note how the stable 7.x release has coverage and the 8.x release candidate does not):

Release coverage icon

Phase 2: Set up an opt-in process for security advisory coverage

  • Previously any project with a stable release would receive security advisory coverage from the security team. As we opened the gates for anyone to promote full projects, the security team needed an opt in process so that they could enforce an extra level of vetting on projects that wish to receive advisory coverage.
  • We agreed to repurpose the project application queue to be a queue for vetting users for the ability to opt their projects in to receive security advisory coverage. Now that this process has been decoupled from creating full projects, the security team may revise it in future–in collaboration with staff and the community.
  • Now a project maintainer must opt in their project to receive advisory coverage and make a stable release in order to receive security advisory coverage from the security team.

Once a maintainer has been vetted by the security advisory opt in process, they can edit their project and use this field set to opt-in:

Project opt-in field

Phase 3: Open the gate to allow users to create full projects with releases without project applications.

This is the milestone we've just reached!

Phase 4: Provide both automated code quality signals, as well as incentives for peer review of projects - and factor these into project discovery

  • We are working on this phase of the project in the issue queues, and we appreciate your feedback and ideas!

What is the new process?

So in the end - what is the new process if you want to make a contribution by hosting a project on Drupal.org?

  1. You must have a Drupal.org account, and you must accept the git terms of service.
  2. You can create a sandbox or a full project
  • Note: We still strongly recommend that project maintainers begin with sandbox projects, until they are sure they will be able to commit to supporting the project as a full project, and until the code is nearly ready for an initial release.
  • That said, you can promote a sandbox project to a full project at any time, to reserve your name space and begin making releases.

At this point, you will have a full project on Drupal.org, and will be able to make releases that anyone can use on their Drupal site. The project will not receive security advisory coverage, and a warning that the project is not covered will appear on the project page and in the updates information.

If you want to receive security advisory coverage for your project, you will need to take these additional steps:

  1. You must apply for vetted status in the security advisory coverage queue.
  2. Members of the security team or other volunteers will review your application - and may suggest changes to your project.
  3. Once feedback is resolved, you will be granted the vetted role and be able to opt in this project, and any future projects you create, to receive security advisory coverage.
    • Note: Only *stable* releases receive security advisory coverage, so even after opting your project in you will not receive the advisory coverage shield except on stable releases.

What comes next?

Now that the project application process is no more, the gates are open. We are already seeing an uptick in projects created on Drupal.org, and have seen some projects that had migrated to other places (like GitHub) migrate back to Drupal.org. We can expect to see contributions from some great developers who previously felt gate-kept out of the community. We will also see an uptick in contributions that need work, from new developers and others who are still learning Drupal best practices.

That is why our next focus will be on providing good code quality signals for projects on Drupal.org. We want to provide both automated signals of code quality, and new incentives for peer review from existing members of the community. We're outlining that plan in the issue queues, and we welcome your feedback and contributions.

We also still have work to do to communicate this well. This is a big change for the Drupal community and so we want to make people aware of this change in every channel that we can.

Finally, after such a significant change, we're going to need to monitor the contrib ecosystem closely. We're going to learn a lot about the project in the next several months, and it's likely there will be additional follow ups and other changes that we'll need to make.  

Special Thanks

There are many, many contributors on Drupal.org who have put in time and effort to help make the contribution process better for new contributors to Drupal - the deepest thanks to all of you for your insight and feedback. We'd also like to specifically thank those who participated in the Project Application Revamp, including:

Mar 09 2017
Mar 09

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Drupal.org updates

Industry Pages Launched

After a great deal of preparation, user research, and content development we've launched the first three 'Drupal in your Industry' pages. These first three pages highlight the power of Drupal in Media and Publishing, Higher Education, and Government. Each of these pages uses geo-targeted content to reach audiences in: the Americas; Europe, the Middle East, and Africa; and the Asia Pacific, Australia and New Zealand regions.

These pages are targeted at evaluators of Drupal in these specific industries. From our research, we've found that these evaluators typically have Drupal on their short list of technology choices, but are not familiar with how a complete solution is built on Drupal, and they're eager to see success stories from their industry peers.

We'll be expanding on this initiative with additional industry pages as time goes on.

Project Application Revamp

In February we completed phases 1 and 2 of the Project Application Process Revamp. This has meant polishing up the security advisory coverage messages that are provided on project pages, adding a new field for vetted users to opt-in to advisory coverage for their projects, and adding security advisory coverage information to the updates xml served from Drupal.org. With these issues complete we'll be able to move forward with Phase 3 (opening the project promotion gates) and Phase 4 (improving code quality signals and incentivizing peer review) as we roll into March.

[Author's note] The project application revamp hit a major milestone in early March with the completion of Phase 3. Now, any user who has accepted the git terms of service may now promote sandbox projects to full projects with releases, and the application process has been re-purposed for vetting users who want the ability to opt into security advisory coverage for their projects. Look for more information in our upcoming March post.

2017 Community Elections are Live

On February 1 we opened self-nominations for one of the two community-at-large seats on the Drupal Association Board of Directors. At the time of this post, self-nominations have closed and now it's time to vote!.

Each year we make incremental improvements to the elections process. This year we've allowed each candidate to present a short 'statement of candidacy' video - and we've updated the ballot to allow easy drag-and-drop ranking of candidates.

Voting closes on March 18th, so make sure to vote soon!

Documentation polish, and new "call-out" templates

As the migration of content into the new documentation system continues, we've continued to polish and improve the tools. In February we made a few small improvements including: help text for maintainers and fixes for links to the discuss page in email notifications. We also made one large improvement: Call-out templates for highlighting warning information or version-specific notes within a documentation page. These templates are available using the CKEditor Templates button when editing any documentation page.

The documentation editor may select from the 'Warning note' template, which will highlight cautionary information in a visually distinct orange section on the page, or the 'Version-specific note' template, which allows users to highlight information that may only be relevant to a specific minor release of Drupal.

Here are two examples of what the call-outs will look like to a documentation reader.

DrupalCI

Coding standards testing

DrupalCI logo

DrupalCI continues to accelerate the pace of Drupal development as we make the system more efficient and add new features. In February we enhanced the coding standards testing that was added DrupalCI in January. Using PHPCodeSniffer, ESlint, and CSSlint coding standards results are available in the test results' Build Artifacts directory, including automatically generated patches to fix found issues. We've also begun displaying summary information about coding standards testing on Drupal.org test results. Again we'd like to thank community contributor mile23 for his work on this feature.

More useful error output

We also made DrupalCI's error output more detailed, to make it more immediately clear to developers what the issue with a particular patch might be. Developers will now see messages on the test result bubbles, for example a 'patch failed to apply' error rather than a generic 'CI error' message.

Community Initiatives

Contrib Documentation Migration

We want to continue to encourage Project maintainers to create documentation guides on their projects using the new documentation content types. Maintainers can then migrate their old documentation content into these new guides, or create new documentation pages. For more information about this process, please consult our guide to contrib documentation.

Help port Dreditor features to Drupal.org

Are you a Drupal.org power user who relies on Dreditor? Markcarver is currently leading the charge to port Dreditor features to Drupal.org, and invites anyone interested in contributing to join him in #dreditor on freenode IRC or the Dreditor GitHub.

Infrastructure

Special note: Drupal Association seeks Infrastructure Services vendor

We'd also like to announce a Request for Information. The Drupal Association seeks an infrastructure services vendor to help us manage the underlying infrastructure that supports Drupal.org, our sub-sites, and the services we maintain. Our internal engineering team will continue to manage the sites and services themselves, while this vendor will help us with systems administration, virtual machine management, monitoring and pager responsibilities, disaster recovery, etc.

For more details about this request for information, please see our post on the Association blog.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects. If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association. Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Feb 14 2017
Feb 14

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Drupal.org updates

Recognizing more types of contribution in the Drupal.org Marketplace

We were very pleased to announce an expansion of the issue credit system into a broader contribution credit system which recognizes more than just code contributions for the purposes of ranking organizations in the marketplace.

We now calculate the following 4 types of contribution into overall contribution credit:

User research for the upcoming industry pages

In a previous blog post on Drupal.org, we talked about our increasing focus on the adoption journey and our plans to create industry specific landing pages on Drupal.org. In January we did extensive user research with people in media and publishing, higher education, and government, which will be the first industries we promote. We're hoping to launch these pages very soon, so keep an eye on the home page.

Preparing for community elections for the Drupal Association board

Elections

The elections process for the community seats on the Drupal Association board kicks off with self-nominations in February each year. This means that we dedicated some time in January to making small refinements and improvements to the nomination process. In particular we've added more in-context educational materials about the board to the self-nomination form, including a video by executive director Megan Sanicki. We've also refined our candidate questions to help candidates express their unique qualifications.

If you're interested in bringing your perspective to the Drupal Association board, please nominate yourself.

Membership history messaging

To make it easier for members to understand their membership history, we've added new messaging to the membership join and renew pages. Users who go to join or renew their Drupal Association membership will now see a message indicating their current membership expiration date, their last contribution amount, a link to contribute again, and their auto-renewal status.

Membership messaging

Migration of Drupal Association content to Drupal.org

Drupal Association

In January we also migrated the majority of content from assoc.drupal.org to a new section on Drupal.org itself. This effort is part of our larger content restructure initiative. By moving Drupal Association content into Drupal.org we hope to increase discoverability of information about the DA, and create a tighter integration between Drupal Association news and the front-page news feed.

DrupalCI

Checkstyle results now available on the DrupalCI dispatcher

DrupalCI logo

Thanks to community member mile23, DrupalCI now supports automated code style testing. To see checkstyle results for any test on Drupal.org, click on the test result bubble and then click the 'view results' link to view the detailed test results on DrupalCI's jenkins dispatcher.

We're still gathering input and feedback for this initial release of the checkstyle feature, as we decide how to integrate the checkstyle results more tightly with Drupal.org. If you have feedback or suggestions please leave your comments in this issue: #1299710: [meta] Automate the coding-standards part of patch review.

Updated testing environments

DrupalCI supports testing code against a matrix of php and database versions. In January we updated the php environments that DrupalCI supports, so that you can test against the minimum supported versions or the latest point releases. Our 5.X containers have been upgraded to the latest version for each minor release (5.3.29, 5.4.45, 5.5.38, 5.6.29). The singular PHP 7 environment that we were using was following the 7.0.x branch of php7. This has now been expanded into four php 7 environments, 7.0 (7.0.14), 7.1 (7.1.0), 7.0.x, and 7.1.x.

The dev versions of php are primarily intended for Core to sense upstream changes to php before they become released, as our comprehensive test suite often finds unanticipated bugs in php7. Additionally some missing features in the php7 containers were added, specifically apcu.

Local testing improvements

DrupalCI has always supported local testing, in order to allow developers to test changes on their own machines. This is helpful for several reasons: it allows people to test on their own machines before triggering one of the DrupalCI test bots, it lets users troubleshoot failing tests, and it helps to eliminate the 'works on my machine' problem where code appears to work in a local environment, but fails on the test bots.

To make local testing even easier, DrupalCI now automatically generates a vagrant environment for local testing. To use this functionality simply clone the drupalci_testrunner.git repo and then run $ vagrant up from within the directory. Furthermore, DrupalCI can download a build.yml file from a dispatcher.drupalci.org url to replicate any test that has been run on Drupal.org. More information about this will be added to the DrupalCI documentation soon.

Adding test priority

DrupalCI runs thousands of tests of the Drupal codebase for core and contrib modules every month. These tests include commit and patch testing for the active development which may be occurring at any time day or night, as well as the hundreds of daily regression tests run for both core and contrib projects. To help make testing more responsive, we've added a notion of testing priority. When there is a queue of waiting tests, Drupal 8 core patch tests will take priority; followed by D8 branch tests; followed by D8 contrib tests; followed by Drupal 7 patch, branch, and contrib tests.

Community Initiatives

Project Applications Revamp

Our primary community initiative priority for the first quarter of the new year is the Project Application Revamp. There are four phases to the revamp: 1) preserving security advisory coverage signals about projects, 2) transitioning security advisory coverage to an opt-in process, 3) opening the gates to allow any user to promote a project to full and create releases, 4) building new tools to incentivize code review and provide code quality signals on project pages. One of the changes we made as part of phase 1 was to adjust the way recommended releases are highlighted on Drupal.org project pages.

Contrib Documentation Migration

Project maintainers are now able to create documentation guides on their projects using the new documentation content types. Maintainers can then migrate their old documentation content into these new guides, or create new documentation pages. For more information about this process, please consult our guide to contrib documentation.

Help port Dreditor features to Drupal.org

Are you a Drupal.org power user who relies on Dreditor? Markcarver is currently leading the charge to port Dreditor features to Drupal.org, and invites anyone interested in contributing to join him in #dreditor on freenode IRC or the Dreditor GitHub.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects.

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Feb 09 2017
tvn
Feb 09

I am happy and sad to share the news today that I am leaving the Drupal Association for an exciting new adventure.

I've been volunteering for Drupal.org for a while before I joined the Association as a staff member in April 2012. Before that I never actually thought you can be paid to do something you volunteer your free time for because you enjoy doing it so much.

It was also my first remote job. Back then it was not as common. I clearly remember first talking to Angie, and my initial response was "You are crazy. I am literally on the other side of the world." Which she didn't find to be of any consequence.

It's been a wonderful (almost) 5 years. I've learned a lot, and grew. I've traveled a lot and met lots of fantastic people. I watched the Association grow. Our engineering team grew as well from 2 to 11, which was exciting but also challenging to be a part of. We've done some great things on Drupal.org. Of course, there are so many many more I wish we'd done. :)

The time has come however to move on to the new challenges.

Thanks to all the volunteers who worked with me during all these years. You were so helpful and generous with your time.

Thanks to the staff members for being wonderful human beings, for your support, and laughs, for becoming a family.

I am incredibly sad to leave our staff, and especially the Engineering team. They are a smaller team now, and I know just how hard it will be to maneuver all the requests coming from all the different parts of the community with the limited resources they now have. So please be nice to them.

As for the next steps, I am looking forward to joining the DA alumni club, which includes some of my closest friends. And I will be around. You will probably see me at the next DrupalCon. Come say hi. And no, I will not fix your webmasters' queue issue. 
 

---
Photo from Flickr.

Jan 25 2017
Jan 25

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Our December update comes to you a bit later than our usual monthly posts, for all the usual practical reasons: holidays, vacations, and our staff retreat in early January. But also, because we've been reflecting on the past year, and planning for the year to come. You'll soon hear about our initiatives for 2017, but for now— let's dive into what we did in December.

Drupal.org updates

DrupalCon Baltimore

DrupalCon Baltimore logo Apr 24-28

At the beginning of December we launched the full site for DrupalCon Baltimore, which is coming up April 24-28. For the first time, we launched the full event site including the call for papers, scholarship applications, and registration all on the same day.

Early bird pricing is available for a limited time, so we encourage you to register today.

Stable release of the Composer Façade

Project Composer logo

Drupal.org's support for Composer has been in development since the beginning of last year. We released the public alpha of our composer endpoints at DrupalCon New Orleans, and then entered beta over the course of this past summer. After a period of feedback, bug fixes, and further refinement with the help of core and contrib developers we announced the stable release of Drupal.org's composer support on December 21st.

We'd like to thank the following community members for their help with this initiative: seldeak, webflo, timmillwood, dixon_, badjava, cweagans, tstoeckler, and mile23. We'd also like to thank Appnovation for sponsoring our initial Composer support work.

Improved messaging for new users

One of the innovations of Drupal.org's online community that we introduced about 2 years ago, is the process by which new users get confirmed by trusted users. As a user of Drupal.org, you know that when you see a new user with a 'confirm' button under their user icon, you can check their recent activity and help confirm for us that they're a real user (not a bot or spammer who managed to slip through).

However, we received some feedback from recently registered users, that this process was too opaque. New users did not have enough guidance to understand that they can only perform a sub-set of site activities until another user confirms them.

After hearing this feedback, we spent some time in December improving the messaging tonew users when they first sign up on Drupal.org— so they can better understand how to become confirmed.

DrupalCI refactored and updated to use composer

DrupalCI logo

In December we also completed a refactor of DrupalCI and updated the testing system to use Composer when testing Drupal. This means we can now test projects with external composer dependencies on Drupal.org. Other new features and bugfixes include: more available test artifacts; dependency changes can now be submitted in patches to composer json; the test runner produces a build file that can be downloaded and run locally to re-execute any test verbatim. There are more added features as well..

This work has continued into January, particularly around making more testing environments available, and adding new test types (such as code sniffer). Look for additional updates in the upcoming January report.

Special thanks to mile23 for collaborating with us on this work.

Jenkins upgraded to better manage our EC2 Instances

The cost of automated testing for the Drupal project is a significant expense for the Drupal Association. In December we updated Jenkins and several of the plugins that are used to orchestrate the creation and management of DrupalCI testbots, and now our enforcement of instance limits is much more reliable. In December this saved us nearly 50% on our testing bill, without a significant increase in testing wait times. In January we are projecting a similar savings.

The work of community member fabianX might also provide similar savings for the project, so we encourage contributors involved in core to help review: #2759197: [D7] Improve WebTestCase performance by 50% and #2747075: [meta] Improve WebTestCase / BrowserTestBase performance by 50%

HTTP/2 Support enabled

HTTP/2 is the next generation network protocol that decreases latency in page loads by using better data compression, pipelining, and server push. In December we enabled HTTP/2 support for Drupal.org, improving performance for all users with modern browsers that support the standard.

Community Initiatives

Preparing for the Project Applications Revamp

In November the Drupal 8 User Guide went live, so in December we prepared for the next community initiative on our roadmap - the Project Application Revamp. Over the course of the last several months we've been doing pre-work around this initiative to ensure that the appropriate signals about security advisory coverage and recommended releases are provided on project pages. This pre-work will help ensure that Drupal users still have good signals to project quality, even as we open up the creation of full projects.

Initiatives need your help

Are you a Drupal.org power user who relies on Dreditor? Markcarver is currently leading the charge to port Dreditor features to Drupal.org, and invites anyone interested in contributing to join him in #dreditor on freenode IRC or the Dreditor GitHub.

Is the written word your domain? Consider putting your skills to use by becoming a maintainer of Drupal documentation. If you are a developer interested in contributing code to the new documentation system, please contact tvn.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects.

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Jan 12 2017
Jan 12

Within weeks of introducing the contribution credit system on Drupal.org we realized we had created something powerful. Like all open source projects, Drupal has a behind-the-scenes economy of contribution in which individuals, organizations, and end users work together to maintain the software as a public good. That behind-the-scenes economy was brought to the fore when we chose to rank the Drupal Marketplace by issue credits. For the first time, Drupal.org gave businesses a direct financial incentive to contribute code.  

Being good stewards of these incentives is a sobering responsibility, but also a great opportunity. We can use this system to recognize the selfless effort of our community volunteers, to reward the organizations that sponsor their employees' time to give back to the project, and to connect end-users with the organizations that are the biggest contributors.

But as we often say in this community—contribution is more than code. It is the time provided by dedicated volunteers; the talent of community organizers, documentation maintainers, and developers; and the treasure provided by organizations that sponsor Drupal events and fund the operations and infrastructure that maintain the project.

What are we changing?

We’re updating the ranking algorithm for Drupal.org’s Marketplace of service providers and list of all organizations in the Drupal ecosystem. We've expanded on the issue credit system to create a more generic contribution credit system which lets us recognize more types of contribution. Each type of contribution is now weighted to give the organization an overall amount of contribution credit. We've built this system so that we can continuously evolve the incentives it creates by adjusting the weight given to each type of contribution as the project's needs change. To prevent gaming, we will not be publishing the exact weights or total contribution score, but those weights have been reviewed by the Association Board and Community Working Group.

We've carefully chosen a few new types of contribution to factor into the ranking. These were selected because they create incentives to reach specific goals: encouraging organizations to sponsor development of Drupal, gathering more Drupal 8 success stories that can be used to promote Drupal adoption, and recognizing the financial contributions that promote the fiscal health of the Drupal association.

We now calculate the following 4 types of contribution into overall contribution credit:

What about other types of contribution?

Of course, these new factors still don't include all types of contribution. This iteration aims to add measurable factors that reward the behavior of organizations that are good Drupal citizens, and incentivize some of the most important contributions that have a big impact in moving the project forward. But there are other factors we'd like to include in the future! We're keeping track of these additional kinds of contribution, such as sponsoring local user groups, organizing training days, writing documentation, and more, in this issue: #2649100: Improve contribution statistics on user and organization profiles.

There are two factors in particular that we are not yet including that we'd like to address.

The first is project application reviews. These reviews are a critical part of the lifecycle of a new project on Drupal.org, but because we are making the Project Application Revamp a key priority for the first part of 2017, this was not our focus in this initial update. We may revisit this factor as the Project Application Revamp initiative gets underway.

The second is camp organization. We know that there are many individuals and organizations who invest heavily in Drupal Camps, and this has been a critical part of the project's success. However, at this time our data about the individuals and organizations who participate in camp organization is purely self-reported, and therefore too vulnerable to manipulation to include in the algorithm at this time. In the future we hope we can find a responsible way to measure and credit this kind of contribution.

We’ll continue to look for other good factors to add, and do our best to weigh them fairly.

How often will the algorithm change? Who governs these changes?

As this is our first major change to the marketplace ranking system since the launch of issue credits, we may need to make some small adjustments in the first weeks following the launch. However, we know that too frequent changes to the incentive structure will be frustrating for the individuals and organizations who are contributing to the project. Therefore, after the initial tuning we intend to update the marketplace ranking system on a roughly 6 month cycle.

While the primary responsibility to manage the contribution credit system is ours, we have committed to vetting these and future changes with members of the Drupal Association Board and Community Working Group.

Dec 21 2016
Dec 21

Drupal.org Composer Logo

Drupal.org's Composer endpoints have been available in beta for some time now, and in that time we've begun to see many, many people use Composer to manage Drupal modules and themes. We first launched these repositories before DrupalCon New Orleans as an alpha release, and move into beta a few months later. After receiving your feedback and bug reports we've made updates, and are ready to call this service stable.

What is Composer?

Composer is a tool for dependency management in PHP. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you.

… Composer is strongly inspired by node's npm and ruby's bundler." - Source

In a nutshell, Composer allows you to declare the dependencies of your project in a composer.json file in the root of your PHP project. Those dependencies, which you then install through Composer, can have their own composer.json files and their own dependencies—all of which will be automatically managed and installed by Composer. When you need specific control over the versions of dependencies, you can use a composer.lock file.

You can read more about Composer at GetComposer.org.

How do Drupal.org's composer repositories work?

Drupal.org offers two Composer repositories—one for Drupal 7, and one for Drupal 8. Composer requires that packages adhere to semantic versioning, which Drupal 8 core does, but Drupal 8 contrib, and Drupal 7 core and contrib, don’t. To solve this problem, we've created a Composer façade, which takes all of the metadata about projects on Drupal.org and translates them into a format Composer can understand—including translating the Drupal-specific versioning for Drupal 7 and contrib into semantic versioning.

By creating this façade, we've made sure that Drupal.org is still the canonical source for metadata about Drupal.org projects, and that we can update this translation layer as the versioning schema changes. (Learn more about the effort to move Contrib projects to semantic versioning).

In addition to providing endpoints for building projects, Drupal's automated testing suite— DrupalCI—now uses Composer to test Drupal core and contributed projects. This allows developers to test any external dependencies.

How do I use Drupal.org's Composer repositories?

To begin using Drupal.org's Composer repositories, you'll need to update your composer.json file to include the appropriate Composer repository for the version of Drupal. To use Composer with Drupal 7, use the repository url:

https://packages.drupal.org/7

. To use Composer with Drupal 8, use the repository url:

https://packages.drupal.org/8

, as in this example.

After setting up composer, simply run the command:

$ composer config repositories.drupal composer https://packages.drupal.org/8

And your project's composer.json should be updated to look like the following:

{ 
    "repositories": { 
        "drupal": {
            "type": "composer",
            "url": "https://packages.drupal.org/8" 
        }
    }
}

Once you've made that change, you should be able to use Composer for Drupal modules and themes as you would for any other PHP package, using the drupal/ namespace:

$ composer require drupal/<modulename>

There is one caveat about the pattern: there are some namespace collisions among modules, and so it is on our roadmap to update Drupal.org project pages to specify the exact namespace to use to require a given project.

To learn more about how to use Drupal.org's Composer repositories, and for some troubleshooting tips, read the Project Composer documentation.

What about licensing?

All the projects hosted on Drupal.org are licensed GPLv2 or later or have an entry in the packaging whitelist. This means that you can rely on Drupal Core and contributed modules and themes to be licensed under the GPL or compatible. And if you need to redistribute your code created with Drupal projects, it must be redistributed as GPL-2.0 or GPL-3.0, but please note that Drupal.org will only host GPL-2.0 licensed projects.

However, because Composer is a tool that can manage packages in the wider PHP ecosystem, you might find that you want to require a non-GPL package in your project. Using GPL-licensed Drupal projects with external packages that are GPL compatible is fine. Just be aware that if you redistribute that code, you will have to redistribute under a GPL license.

We cannot provide legal advice for your use of open source software. If you use Composer to install packages that are not compatible with the GPL alongside GPL-licensed projects like Drupal, you may use that software together, but per the terms of the GPL you may not copy, distribute, or modify that software.

"Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted…" GPL 2.0 Section 0.

For more questions about Drupal and the GPL you can read the Licensing FAQ.

The Composer "licenses" command

If you need to check what licenses are in use by your Composer-installed dependencies, you can use the licenses command:

$ composer licenses

This will list the licenses of all the packages your project is using. However, we recommend that all users double check licensing information as some of these packages may be dual-licensed or have other licensing concerns that may not be immediately apparent from this command.

What's next?

At this point, the Drupal.org Composer service is stable and you can use it to manage modules and themes in your production websites. That said, we do have a roadmap of additional features that we'd like to add. And your contributions are welcome!

  • As development on Drupal.org's Composer service continues, we want to focus on the following features:
  • Supporting Composer-based workflows for distributions and install profiles
  • Providing sub-tree splits of Drupal Core
  • Updating project pages to provide information about using Composer with any given Drupal.org hosted project
  • Adding features to the updates service, to collect statistics about projects installed with Composer, and to explore providing update alerts about external dependencies
  • We also hope to work with core maintainers to add the Drupal.org Composer repositories to Drupal Core's composer.json file

If you're interested in learning more about our roadmap for Composer, or contributing to this service on Drupal.org, you can learn more in the Composer plan issue.

How you can help

If you’re interested in helping to improve Drupal.org's support for Composer workflows, please take a look at the issue above, find us on irc in #drupal-infrastructure, or send us a volunteer proposal.

Thanks to our Community Initiative contributors

We'd like to thank the individuals who worked with us as part of this Community Initiative.
In particular, we'd like to thank:

We'd also like to thank Appnovation, who sponsored the initial development of Drupal.org's composer endpoints.

To these volunteers and sponsoring organizations—it is your expertise, your insight, and your affirmation of our work that make these Community Initiatives successful. Thank you!

Dec 08 2016
Dec 08

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

The engineering team at the Drupal Association had much to be thankful for in November. With the support of the wonderful volunteers in our community and the contributions of our Supporting Partners we were able to deliver some great tools for the project. Let's dive and see what's new.

Drupal.org updates

Promoting Drupal by Industry

In November we finished the technical scaffolding for the upcoming industry pages, and began working with the wider Association team on content development for these pages. Because we were ahead of our internal targets for this page and we felt it would add significant value, we've also added the ability to geotarget content on these industry pages.

This is the first instance of geo-targeting on Drupal.org, and we'll be using it to help connect Drupal evaluators with regionally appropriate content and partners on these pages. Work on the industry pages is ongoing, but we're excited to bring them to you soon.

Developer Tools Evaluation

During November the engineering team also had a two day retreat here in Portland, OR with webchick - one of the members of the Technical Advisory Committee. We used this retreat to do a deep dive into the current state of developer tools on Drupal.org, and to evaluate our options to continue evolving the tools we offer to the community.

We gave a summary of our exploration along with some next steps to the Drupal Association Board on November 22nd. You can find the minutes and a recording here.

Core release packaged with --no-dev composer dependencies

Starting with the Drupal 8.2.3 release, we are now packaging full releases of Drupal core with --no-dev composer dependencies. This means that packages downloaded will not include extraneous developer extras that should not be used in production sites, and that the release packages will be smaller. We will continue to package dev releases with the dev dependencies.

Feature branch testing support

Drupal.org allows maintainers to create feature branches for issues by using the name format [issue#]-[short-description]. Any commits made to a branch in this format will appear in the sidebar of the associated issue. To improve the utility of these feature branches, DrupalCI patch file tests now also run on push to these branches.

Feature branch testing UI on issues

To add tests, users can simply click on the 'add test' link beneath the git branch in the issue sidebar, or click on the existing test result bubble to re-test or add a new test. Since this feature was introduced we've run over 200 issue branch tests.

Project maintainers can add Documentation Guides

UI for relating guides to projects

Display of related projects on Guides
We're continuing to support the migration of documentation to the new documentation system, and we've now enabled Project Maintainers to add related documentation guides to their projects. Once added, the related projects will appear on the documentation guides, in the sidebar.

Documentation Maintainers can find their Guides

Many community volunteers have stepped up to become maintainers of the new documentation guides. We want to make sure we're giving them the tools they need to do the work of maintaining those guides and the pages within them.

Your Guides tab on user profile

We've added a 'Your Guides' section to the user profile which will list all of the guides that a user maintains, as well as the pages within those guides. This should allow maintainers to see when pages have been recently changed or added, and to easily keep their guide content curated and up to date.

Infrastructure

Virtualization and Improved Config Management

In November, we completed the majority of two major infrastructure projects. Firstly, we've virtualized the majority of the infrastructure and standardized on Debian 8 images. Secondly we've updated our configuration and user management from Puppet 3 + LDAP to Puppet 4 + Hiera. This is a significant milestone for our infrastructure, and gives us a more portable and maintainable infrastructure to manage moving forwards.

Community Initiatives

Community initiatives are a collaboration; with dedicated community volunteers building improvements to Drupal.org with the architectural guidance and oversight of the Drupal Association engineering team.

Drupal 8 User Guide Launched!

We're very happy to say that the Drupal 8 User Guide is now live on Drupal.org! This documentation guide is carefully curated to provide all the information a new user needs to become skilled at managing a Drupal 8 site. We want to give a special thanks to jhodgdon for all her work on the User Guide project.

Initiatives need your help

Are you a Drupal.org power user who relies on Dreditor? Markcarver, who is currently leading the charge to port Dreditor features to Drupal.org, has invited anyone interested in contributing to join him in #dreditor on freenode IRC or the Dreditor GitHub.

Is the written word your domain? Consider putting your skills to use by becoming a maintainer of Drupal documentation. If you are a developer interested in contributing code to the new documentation system, please contact tvn.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects.

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Nov 11 2016
Nov 11

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

The Drupal Association team has been getting back to work after coming back from DrupalCon Dublin in September. For the engineering team, October has been focused on some back-end services and infrastructure that support the Drupal project, while we continue to move forward on some longer term front facing initiatives.

Drupal.org updates

Promoting Drupal by Industry

Last month we talked about the new homepage we released for Drupal.org, and using those editorial tools to build a membership campaign. We hinted that additional changes will be coming soon. While we're not ready to launch this new content - we can talk about it in some greater detail.

Dries Buytaert, the project founder, has called Drupal the platform for ambitious digital experiences. That phrase expresses the incredible power and flexibility of Drupal, but also encapsulates an aspect of Drupal that can be difficult for newcomers. It can be very hard for newcomers to Drupal to understand how to take a base install of Drupal core, and extend that to achieve that ambitious vision.

We want to help close that gap in understanding—to help evaluators see how Drupal achieves these ambitions. To do this, we'll be creating a series of landing pages that focus granularly on how Drupal creates success stories in particular industries. Look for more on this topic in coming months.

DrupalCon Vienna Site Launched

Vienna 2017 Logo

As is tradition, during the closing session of DrupalCon Dublin we announced that the next DrupalCon in Europe will be held in Vienna! We launched the splash page announcing the event at vienna2017.drupal.org and we have information about sponsorship and hotel reservations already available.

DrupalCon Vienna will happen from the 25th to 29th of September 2017, and we'll hope to see you there!

More flexible project testing

We've made a significant update to how tests are configured on the Automated Testing tab of any project hosted on Drupal.org. Automated testing, using the DrupalCI infrastructure, allows developers to ensure their code will be compatible with core, and with a variety of PHP versions and database environments. In October, we updated the configuration options for module maintainers.

Maintainers can now select a specific branch of core, a specific environment, and select whether to run the test once, daily, on commit, or for issues. Issues are limited to a single test configuration, to ensure that the code works in a single environment before being regression tested against multiple environments on on-commit or daily tests.

Better database replication and reliability

Behind the scenes, we've made some updates to our database cluster - part of our infrastructure standardization on Debian 8 environments managed in Puppet 4. We've made some improvements to replication and reliability - and while these changes are very much behind the scenes they should help maintain a reliable and performant Drupal.org.

Response to Critical Security Vulnerabilities

DirtyCow Public Domain Logo

When it rains, it pours—a maxim we take to heart in Portland, Oregon—and that was especially true in the realm of security in October. The most widely known vulnerability disclosed was the 'DirtyCow' vulnerability in the Linux kernel. A flaw in the copy-on-write system of the Linux kernel made it possible, in principle, for an unprivileged user to elevate their own privileges.

Naturally, responding to this vulnerability was a high priority in October, but DirtyCow was not the only vulnerability disclosed, as security releases were also made for PHP, mariadb, tar, libxslt, and curl. We mitigated each of these vulnerabilities in short order.

Community Initiatives

Community initiatives are a collaboration; with dedicated community volunteers building improvements to Drupal.org with the architectural guidance and oversight of the Drupal Association engineering team.

Drupal 8 User Guide

The Drupal 8 User Guide is getting very close to being available on Drupal.org. We are working closely with contributor jhodgdon to resolve some perplexing inconsistencies between what we're seeing in our development environment and in our initial production deployment.

Dreditor

markcarver who is currently leading the charge to port Dreditor features to Drupal.org, has invited anyone interested in contributing to join him in #dreditor on freenode IRC or the Dreditor GitHub issue queue.

Documentation Maintainership

Finally, we want to continue to encourage the community to become maintainers of Drupal documentation. If you are a developer interested in contributing code to the new documentation system, please contact tvn.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects.

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Oct 21 2016
Oct 21

Republished from buytaert.net

Nasdaq chooses Drupal 8

I wanted to share the exciting news that Nasdaq Corporate Solutions has selected Drupal 8 as the basis for its next generation Investor Relations Website Platform. About 3,000 of the largest companies in the world use Nasdaq's Corporate Solutions for their investor relations websites. This includes 78 of the Nasdaq 100 Index companies and 63% of the Fortune 500 companies.

What is an IR website? It's a website where public companies share their most sensitive and critical news and information with their shareholders, institutional investors, the media and analysts. This includes everything from financial results to regulatory filings, press releases, and other company news. Examples of IR websites include http://investor.starbucks.comhttp://investor.apple.com andhttp://ir.exxonmobil.com -- all three companies are listed on Nasdaq.

All IR websites are subject to strict compliance standards, and security and reliability are very important. Nasdaq's use of Drupal 8 is a fantastic testament for Drupal and Open Source. It will raise awareness about Drupal across financial institutions worldwide.

In their announcement, Nasdaq explained that all the publicly listed companies on Nasdaq are eligible to upgrade their sites to the next-gen model "beginning in 2017 using a variety of redesign options, all of which leverage Acquia and the Drupal 8 open source enterprise web content management (WCM) system."

It's exciting that 3,000 of the largest companies in the world, like Starbucks, Apple, Amazon, Google and ExxonMobil, are now eligible to start using Drupal 8 for some of their most critical websites. 

Oct 20 2016
Oct 20

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

This month's update comes to you a couple weeks late, but only because we were on site at DrupalCon together with the community to move the project forward! DrupalCon Dublin was a great event, with the entire Drupal Association staff engaged to make DrupalCon the best place to develop your Drupal skills, learn what's coming for the project, and sprint on core and contrib. We are tremendously thankful to the community that joins us for DrupalCon, and to the incredible volunteers that help us put on the event. If you couldn't join us in person, you can still review the session recordings.

Now, on to the updates!

Drupal.org updates

New homepage

Screenshot of the new homepage

Certainly the most visible change to Drupal.org in September was the refresh of our home page. As the front door of our community home, the front page needs to be inviting to both existing community members, and people new to Drupal who are just beginning their adoption journey. The changes are more than aesthetic. We also put in place new editorial tools to give us greater flexibility with the front page itself, and with future landing pages that we hope to create in the same highly-designed, attractive style. In addition to these structural and editorial changes we made some content changes as well, cleaning up our news feed, and giving DrupalCon a new, more prominent position on the home page.

And there are more updates to come! Using the same editorial tools we'll soon be rolling out additional content for Drupal evaluators - promoting proven solutions built using Drupal in specific industries. Look forward to this in the coming months.

Membership campaign

We used the same editorial tools that built the new homepage to build a landing page for our fall membership campaign. This campaign showcases how Drupal Association members make community cultivation grants possible - and the stories that those grants create.

"We wanted to bring international speakers and sponsors to the first DrupalCamp in Mexico City. Getting the grant we needed to build a Drupal ecosystem in a metropolis like Mexico City was terrific."

These community stories run to the heart of our mission - enabling our global community build connections on the local level, and extending Drupal's reach across the world.

Case studies on organization profiles

In September we also made a small but significant update to organization profiles. We've moved the often unwieldy index of people associated with an organization to a subpage, in order to make room for listing the case studies that an organization has created. We want to encourage Drupal organizations of all kinds to share their stories of success, especially around Drupal 8.

If your organization has never created a Drupal case study before, we have some materials to teach you how to create a case study on Drupal.org.

Issue Credit Updates

The issue credit system has had a remarkable impact on the community. Being able to quantify the contribution of organizations to Drupal's codebase has lead to an unprecedented level of healthy competition between organizations who support the project—each trying to outdo the other with their contributions. It has been amazing to see how generous these organizations are, sponsoring the work of committed community contributors to advance the project.

To maintain this system in a healthy way, we need to monitor it carefully and make small adjustments to ensure that we're creating the right incentives for true contribution, and not a system to be gained for self-promotion. We've made a few small tweaks in september to reduce spurious re-opening of issues in order to 'reset the clock' on credits, and we have a few more fixes on the plate to keep this ecosystem healthy.

We're also looking to expand the kinds of activities that receive contribution credit - so look forward to further updates on that front in the coming months.

Community Initiatives

Finally, here are some updates on our active community initiatives. Community initiatives are a collaboration; with dedicated community volunteers building improvements to Drupal.org with the architectural guidance and oversight of the Drupal Association engineering team.

Documentation Migration

The migration of Drupal.org documentation to the new documentation content types is well under way. Tremendous thanks to tvn and eojthebrave for spearheading this effort and recruiting additional volunteers to help maintain the new documentation guides and move the community over into the new system.

We still need your help! We need community volunteers to take on small sub-sets of documentation to maintain, and make sure they're cleaned up post-migration.

If you don't want to commit to maintaining a guide, you can still help out by doing some of the pending tasks for any of the documentation pages.

Lastly, if any Drupal developers are interested in contributing code to the new documentation system to clean up a few minor bugs and features, please contact tvn.

Drupal 8 User Guide

As outlined in our previous update, the Drupal 8 User guide is a special subset of documentation that's been produced in a highly curated, editorially controlled way - to create a guide to Drupal 8 that rivals the standards of an industry publication. All of the components needed to publish this guide to Drupal.org are now in place, so our final step will be to coordinate some last tweaks and bug fixes with jhodgdon, and then to begin linking it prominently on Drupal.org.

Dreditor

In the weeks leading up to DrupalCon Dublin there was a small crisis in the contributor community. Because of changes in the browser add-on validation process, the incredibly valuable and popular Dreditor browser extension, first developed by sun, and currently maintained by markcarver, andcottser has reached its end of life—or has it?

After a tremendous outpouring from the community a new plan was made, and now Mark is working on porting the features of Dreditor directly to Drupal.org. Work is still ongoing, but as it proceeds, users will be able to optionally enable these features component by component on their user profiles.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects.

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Oct 18 2016
Oct 18

Workflow iconAt DrupalCon Dublin, I spoke about The Association’s commitment to help Drupal thrive by improving the contribution and adoption journeys through our two main community assets, DrupalCon and Drupal.org. You can see the video here.

One area I touch on was my experience as a new code contributor. Contributing my patch was a challenging, but joyous experience and I want more people to have that feeling—and I want to make it as easy as possible for others to contribute, too. It’s critical for the health of the project.

At the heart of the Drupal contributor community are our custom development tools, including the issue tracker, Git repositories, packaging, updates server, and automated testing. We believe there are many aspects of Drupal’s development workflow that have been essential to our project's success, and our current tooling reflects and reinforces our community values of self-empowerment, collaboration, and respect, which we seek to continue to uphold.

It’s time to modernize these developer tools. To support the Association with this objective The Drupal Association created a Technical Advisory Committee (TAC). The TAC consists of community members Angie Byron, Moshe Weitzman, and Steve Francia, who is also our newest Drupal Association board member. The TAC acts in an advisory role and reports to me.

Building off of the work the community has already done, the TAC is exploring opportunities to improve the tools we use to collaborate on Drupal.org. The crux of this exploration is determining whether we should continue to rely on and invest in our self-built tools, or whether we should partner with an organization that specializes in open source tooling.

Our hope is that we will be able to bring significant improvements to our contribution experience faster by partnering with an organization willing to learn from our community and adapt their tools to those things we do uniquely well. Such a partnership would benefit both the Drupal community—with the support of their ongoing development—and potentially the broader open source community—by allowing our partner to bring other projects those aspects of our code collaboration workflow.

The TAC will use a collaborative process, working with staff and community to make a final recommendation. The TAC has already begun the process and has some very positive exploratory conversations. The TAC and staff will be communicating their progress with the community in upcoming blog posts.  

Sep 21 2016
Sep 21

As you can see we've put a fresh coat of paint on Drupal.org - but the changes run below the surface. This latest iteration of the front page brings the key concepts of our design system to the forefront: Clean, Modern, Technical.

Preview of the new home page design

This change also brings new editorial tools for Drupal.org content editors. The new home page provides us more flexibility with content and presentation, and so you'll see more frequent updates, more information about DrupalCon, and more editorial flexibility on the home page than you've seen in the past. These tools are also helping us to build cleaner, modern landing pages - like you've just seen with our Fall Membership Campaign.

We've previewed this work with several key members of the community and the board, and we want to say thank you to everyone who's given us their feedback on this first step for our new home page. We also want to give an extra special thank you to dyannenova for her contributions to this effort.

This is just the beginning - very soon we'll have a new visual look for the case studies that are featured on the home page, and then shortly after that we'll begin promoting solutions to Drupal evaluators in specific industries, like Higher Education, Media & Publishing, and Government.

If Drupal.org is the home of the community, then the front page is our front door. We want to welcome new users and evaluators of Drupal, highlight the project's strengths, and promote news and happenings from throughout the ecosystem.

We hope you like the changes, and we think you'll like the upcoming iterations even more. We'd love to hear your feedback!

Sep 13 2016
Sep 13

Read our Roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community.

Our latest update about Drupal.org comes as the Drupal Association has moved out of our central office in Portland, OR, and gone to an all-distributed team. A move of that sort always creates some upheaval but amidst the move we've continued to push forward on several initiatives to improve Drupal.org.

At the same time we've been pushing forward towards DrupalCon Dublin at the end of September- and we hope to see you there!

Drupal.org updates

A new home page, coming soon

As we recently previewed on the Drupal.org blog, some changes are coming to the home page. We're building some new editorial tools to allow for more flexibility with the home page content, and to enable an increased focus on the adoption journey for visitors to Drupal.org. You'll see styles reminiscent of the Drupal 8 release announcement pages, and a continued modernization of theme.

The launch of the new home page is coming soon, but as a precursor we've been making some small improvements. The new user menu which we launched in July has been updated for better keyboard accessibility, and to show a user picture as an indicator that a user has logged in. We've also moved the search feature into an icon in the top navigation. This gives us more flexibility with the header, which can be customized per-page type or per-section with the overall site search box still being present. For example, the header in the new documentation section features search box specific to this particular section, so while you are there you can search for other documentation without having to go through the full-site search and then filtering down. Lastly, we've merged the 'Get Started' and 'Download & Extend' pages. 90% of the content on these pages was duplicated with each other - and the new page presents a cleaner experience with the essential details needed for getting started with Drupal.

The new front page is beginning editorial review, with the help of DA staff, a marketing task-force from the Drupal Association board, and a few key community members.

We've also just launched our fall membership campaign, and we've used this opportunity to beta test some of these new editorial tools to build the campaign landing page. Your support makes our work possible. Thank you!

Documentation

There's some news to report on the documentation front as well. Firstly, as mentioned above, we've updated the header of the documentation section to default to a documentation-specific search box. While not so important for other areas of the site,, we want to preserve and improve the highly-visible, in context search for Documentation.

We've also made some updates to the new system for Documentation maintainers. Authors of new documentation guides will now automatically become maintainers of those guides and automatically 'follow' the guide content so that they will receive notifications of activity in that guide. Any user following a guide can modify notifications settings at any time from their user profile. Within the notification settings a user can select their prefered method of receiving updates - via email or via their tracker page.

Tvn has continued to spearhed the migration of documentation from the old book pages, to our new documentation system.

We have completed the migration of the majority of the 'general' documentation. While that is done, there is still a lot of work to do to make the documentation content better using the new tools that are now available.

We need community volunteers to take on small sub-sets of documentation to clean them up post-migration and to maintain going forward.

If you don't want to commit to maintaining a guide, you can still help out by doing some of the pending tasks for any of the documentation pages.

Lastly, if any Drupal developers are interested in contributing code to the new documentation system to clean up a few minor bugs and features, please contact tvn. And if you are going to be at DrupalCon Dublin, consider joining us at the sprints!

Quality of Life Improvements

We also took the time in August to make a few quality of life improvements, both for our end users, and for our own team. Firstly, we've made it easier than ever to download a copy of your invoice for DrupalCon. Any user can now log into events.Drupal.org and any time, go to "My Account" -> "Orders" and download a pdf of their invoice for any past event. If your company is sending you to DrupalCon, this makes the process easier than ever. (And if they're not, here are some tools to convince your boss!)

Behind the scenes, we've made some additional improvements to our sophisticated spam prevention system, which focuses on preventing bad actors from even registering on Drupal.org in the first place. For those few bad actors that do get through, the system is also tuned to allow us to prevent those users from making multiple account registrations, as one of the primary methods for targeting Drupal.org in the past has been to make a large number of 'sleeper' account registrations that can be later updated with spam links. Unfortunately, on rare occasions this tool can make it difficult for legitimate users to register an account, so we've updated the system with a whitelisting system that allows legitimate to register, without opening the floodgates to the bad actors.

Infrastructure

Virtualization and better Drupal.org dev sites

On the infrastructural side we've been focused on improving the maintainability, stability, and portability of our infrastructure with our smaller engineering team. In particular we've been focusing on virtualizing all the components of our infrastructure.

In August in particular we completed the virtualization of pre-production services. We've optimized the snapshotting and whitelisting process that allows us to create staging and development environments to make that process more efficient and easier to manage. We've also replaced our drupal.org dev site architecture with a new architecture that is no longer vulnerable to docker-fs faults which have multiple times resulted in data loss on our development environments. Drupal.org contributors who've been affected by dev site fragility should find dev sites to be much more robust moving forward.

Community Initiative Updates

Finally, here are some updates on our active community initiatives. Community initiatives are a collaboration; with dedicated community volunteers building improvements to Drupal.org with the architectural guidance and oversight of the Drupal Association engineering team.

Drupal 8 User Guide

The Drupal.org user guide is an effort lead by jhodgdon and a number of other contributors to create a highly produced, tightly editorially controlled guide to using Drupal 8. This user guide has been written to the standard of an industry publication, and uses a custom editorial workflow with git + asciidoc. Jhodgdon has been building out functionality to publish the user guide to a Documentation guide on Drupal.org.

Security

A few interrelated initiatives are in progress to improve how information about project security is displayed on Drupal.org. Mlhess has been working on a new security advisory content type for Drupal.org, which will allow security advisory content to be more easily related to project releases, among other things.

With the input and collaboration of quite a few community members, including the security working group, we've also deployed an update to project pages.

This update adds a shield icon next to stable releases. This shield icon indicates which releases are covered by the security advisory policy. This small change is also part of the groundwork for a project application revamp.

Community initiatives are not work that the Drupal Association can tackle on our own. Our mandate requires us to remain focused. That said, whenever the community has arrived at a strong plan and individual volunteers are ready to contribute code, the engineering team can provide architectural advice, code review, and deployment support.

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who made it possible for us to work on these projects.

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web