Technical Advisory Committee Update

In October of last year the Technical Advisory Committee was formed to evaluate options for the developer tools we use on Drupal.org. The TAC consists of Angie Byron, Moshe Weitzman, and Steve Francia, acting as advisors to Megan Sanicki, the Executive Director of the Drupal Association.

The TAC's mandate is to recommend a direction for the future of our tools on Drupal.org. Megan will evaluate this recommendation, make a decision, and prioritize that work in the development roadmap of the Drupal Association engineering team.

What is the motivation behind looking at our developer tools now?

Close followers of the Drupal project will have noticed a trend in the last several months. From Dries' announcement of easy upgrades forever, to the revamp of the project application process, to the discussion about making tools for site builders— there is a unifying theme: broadening the reach of Drupal.

This is the same motivation that underlies this evaluation of our developer tools, and defines the goals and constraints of this initiative:

• Adopt a developer workflow that will be familiar to the millions of developers outside our community
• Preserve those unique elements of how we collaborate that have made the Drupal project so successful
• If possible, leverage an expert partner who will help keeping our tooling up to date as open source collaboration tools continue to evolve

This means looking at a number of elements of the Drupal.org developer tool stack:

• The underlying git service
• How we tag and package releases
• The contribution workflow (patch vs. pull request)
• Project management workflows (the issue queues and tags)
• CI integration
• Maintainership
• Project pages

If this looks like a tremendous undertaking - that's because it is. But there are some things we already know:

• Drupal.org should continue to be the home of project pages
• We should adopt a pull request workflow (and ideally we want to be able continue to accept patches as well, at least in the interim)
• We should move contrib projects to semver, following core's lead
• We want to preserve our familiar understanding of maintainership
• We want to avoid forked code and forked conversation
• We want to ensure the security team still has the tools they need to provide their service to the community

We also know that whatever decision is made, these changes cannot happen all at once. We'll need to take a progressive approach to the implementation, and focus on the parts of the stack that need to change together, so that we don't bite off more than we can chew.

What options are being considered?

At this time, the technical advisory committee is considering three options as they prepare to make their recommendation. The options are: GitLab, which offers both self-hosted and SaaS options; GitHub, which has recently been adding long-requested new features; or continuing to evolve our custom-built tooling, perhaps via issue workspaces.

GitLab

GitLab is the up-and-comer among Git hosts. GitLab can be self hosted using either their community or enterprise editions, or repositories can be hosted at GitLab.com. Though they recently stumbled, they have been notably open and transparent about their efforts to build a leading collaboration platform.

Gitlab is itself open-source, and has just released its 9.0 edition. GitLab has aggressively pursued the latest in development tools and workflow features, including project management tools, a ui for merge conflict resolution with in-line commenting and cherry-picking, docker registries for projects, integration with CI tools, and even Gitter, an IRC alternative for real-time collaboration.

GitHub

For quite some time, GitHub was the only real player in git repository hosting outside of rolling a custom solution (as we did for Drupal.org). Over the years it has become the home of many open source projects, and while most of those don't match the sheer scale of Drupal in terms of codebase size and number of contributors, there are certainly other major projects that have made their home there.

However, for all of its presence and longevity in the open source world, there are very few options for customizing its toolset for our needs, and we could no longer self-host our canonical repositories. The Drupal project would need to adapt to GitHub, rather than the other way around.
 

That said, in recent months, GitHub has been putting a strong focus on feature development, adding a number of new features including: automated licensing information,  protected branches, and review requests.

Custom tooling

We also must consider that the tools we have now are what built Drupal into what it is today. A great deal of work has gone into our existing developer tools over the years, and we have some unique workflows that would have to be given up if we switched over to a tooling partner. An idea like the issue workspaces proposal would allow us to achieve the goal of modernizing our tools, and likely do so in a way that is better tailored to those unique things about the Drupal workflow that we may want to preserve. However, doubling down on building our own tooling would come at a cost of continuing to be unfamiliar to the outside development community, and dependent on an internal team's ability to keep up with the featureset of these larger players.

Each of these three options would be a compromise between reaching outward and creating familiarity, and looking inward to preserve the Drupal specific workflows that have brought the project to where it is today.

What have we learned so far?

The TAC has conducted their own internal evaluation of the options as well as worked with Drupal Association staff in a two day exploratory session at the end of last year. The primary focus was to identify and triage gaps between the different toolsets in the following areas:

• Migration effort
• Project management
• Code workflow
• Project handling
• Testing
• Git Back-end/Packaging
• Integrations beyond tools

This initial study also looked at the impact of each option on Drupal community values, and some key risks associated with each.

What comes next?

The next step for the TAC is to make their formal recommendation to the Executive Director of the Drupal Association. At that point, she will direct staff to validate the recommendation by prototyping the recommended solution. Once the recommendation has been validated, Megan will make a final decision and prioritize the work to fully implement this option, relative to other Drupal Association imperatives.

In the comments below, we would love to hear from the community:

What aspects of the way the Drupal community collaborates are the most important to you? What workflow elements do you feel need to be preserved in the transition to any new tooling option?