Developer Tools Initiative - Part 1: An update, and where we stand

Parent Feed: 

The initiative to improve Drupal.org's developer tools is part of a broader effort to broaden the reach of Drupal, not just to end-users and evaluators, but to a wider audience of developers as well. Improvements to our tools are an opportunity to remove friction to changes, increase the quality of changes, improve velocity of changes, and make contributing to Drupal delightful.

The initiative began in coordination with Drupal Association Staff, the Technical Advisory Committee, and a small group of volunteers from the community. We first announced the initiative in October of 2016, and provided our last update at the end of April of this year. Since that time a great deal of work has been underway to evaluate and prototype our options, collaborate with the vendors who supply these toolsets, and make some decisions about the direction we want to move in, with more data in hand.

Buckle up! Because this is such a large topic we've broken it into multiple posts:

Want a TLDR?

Where do we stand now?

Drupal is one of the longest-running open source projects on the web, and for more than 15 years (and many more to come), Drupal.org has been the home of the project. Over the years, we've built a developer toolset to serve the unique needs of the project. These tools have evolved as the open source environment changed. Right now, we use a combination of best-of-breed third party technologies, such as Git, Jenkins, Fastly, OpsGenie, integrated with our own bespoke tools, including issue queues, project pages, etc.

Right now, some of the third party and bespoke tools that we are using are market leading, whereas others have fallen behind. For example, CGIT (third party) and the patch workflow (bespoke) have both fallen behind compared to toolsets that can be found on other tooling providers. On the other hand, our issue crediting system (bespoke) is market leading, and a model for other projects to follow.

In the end, we will always be negotiating a balance between what we can do uniquely well with bespoke solutions, and integrating the latest and greatest of third party solutions as they coalesce around best practices that we want to adopt as well.

At DrupalCon Vienna it was decided that the Technical Advisory Committee had fulfilled their threefold mandate:

  • Helping the technical leadership transition on the DA engineering team after the downsizing in summer of 2016.
  • Helping manage the prioritization and scope of work.
  • Making recommendations to the board and DA staff on key initiatives, such as the project application process changes, and the direction to take with our developer tools.

We want to thank Angie Byron, Steve Francia, and Moshe Weitzman for serving on the committee. From here, DA engineering staff will carry the torch, though we will continue to rely on the individuals who were part of the TAC and other volunteers for feedback and help from time to time as we move forward.

Our evaluation

The most recent phase of this initiative was an evaluation of the leading contenders for open source tooling providers that Drupal.org could integrate with. These options were GitHub and GitLab, and after setting out to develop prototypes for these options, we added BitBucket to the list as well. For each of these options we built MVP prototypes, either as integrations with Drupal.org development environments, or with private organizations/repositories. Finally, we wanted to compare these options to the bespoke Issue Workspaces solution that we had proposed several years ago.

The Criteria

For the Developer Tools initiative to be successful we have to understand the criteria for improvement. From a user requirements point of view, contrib and core developers are unique stakeholders with unique requirements. Ideally we want a solution that increases velocity for both types of users, without fundamentally sacrificing the needs of one or the other.

  • Adopt a developer workflow that will be familiar to the millions of developers outside our community.
  • Preserve those unique elements of how we collaborate that have made the Drupal project so successful.
    • Many-to-one collaboration: that is to say, many developers collaborating on a single solution to a problem.
    • Issue workflow.
    • Picking up on long standing issues where other collaborators left off.
    • Contribution credit.
  • If possible, leverage an expert partner who will help keeping our tooling up to date as open source collaboration tools continue to evolve.

There are also some technical requirements that came out of our evaluation process.

  • Retention of our data/ability to migrate.
    • Where possible, retain existing Git remote urls for projects.
  • Maintainability for a small staff.
  • Project maintainer management: including abandoned project reassignment, fork control, security release management, etc.

In terms of features, we were looking for:

  • Merge/pull requests.
  • Code review.
  • Inline editing.
  • Branch permissioning to allow collaboration on merge/pull requests.
  • Administrative tools for managing project maintainership.
  • Project management tools that equal or exceed what we have with the issue queues.*
  • Extensibility, so that we can preserve areas where the Drupal project is a market leader, such as with our contribution credit system.

*Our tools for issues are very sophisticated on an individual issue level, however we are sorely lacking in tools for grouping and prioritizing sets of issues, ie: issue boards.

Finally, cost:

  • Given the care with which we must use our funding from the community, any option we consider must be cost neutral with the current cost of maintaining our tools.

It's difficult to condense the scope of our evaluation into a short blog series, but in the following posts we'll talk about how the options we considered measured up to these criteria, what implementing one of these options could look like for Drupal.org, and our suggested implementation roadmap.

Author: 
Original Post: 

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web