Developer Tools Initiative - Part 2: Comparing our options

Parent Feed: 

This is the second post in our series about integrating Drupal.org with a 3rd party developer tooling provider:

In this update we'll talk about the specific options we've been evaluating to improve the tools for developers on Drupal.org. For each of these options we've built prototype workflows, stood up test integrations, and opened a dialogue with the creators of these toolsets about any gaps we identified. You'll see a summary of how each tooling option does or does not live up to the criteria we outlined in our last post.

Issue Workspaces

At DrupalCon Los Angeles in 2015, Mixologic from the DA Engineering team proposed the concept of Issue Workspaces. While we've historically talked about this idea as a single concept, it actually breaks down into two key components:

  1. The definition of a modern, idealized workflow for the Drupal project.
  2. A technical implementation using a bespoke tooling solution built on Git Namespaces to implement this workflow.

Criteria (for a bespoke implementation):

  • Familiar workflow.
    • ❌/ While an implementation of workspaces based on a bespoke git-namespaces implementation has several technical advantages, it will not appear nearly as familiar to outside developers as a more standard 'pull request' implementation. (Though it would be functionally very similar).
  • Preserve the most valuable parts of Drupal's workflow.
    • By implementing a bespoke solution we could exactly tailor the solution to the needs of the Drupal community.
    • ❌ However, the needs of that community are great, and our capacity to build a 100% bespoke solution that meets all the community's needs is not clear.
  • Leverage a partner who will keep evolving their code collaboration featureset.
    • ❌ Building issue workspaces using a bespoke git namespaces implementation would require us to continuously compete on features with major third party tooling providers that have much greater resources.

A purely bespoke implementation misses the mark on two of our three primary criteria. Fortunately, however, two years after the original proposal, it is no longer necessary to build a bespoke solution in order to implement an idealized workflow for the Drupal project. It is something that can be done by integrating the existing Drupal.org project pages and issue queues with third-party tooling solutions. Leveraging third party solutions provides several advantages including: familiarity to a wider audience of developers, a continually evolving featureset, and mitigation of the risks of maintaining a bespoke solution with a small team of engineers.

Here is the idealized 'Drupal-flow' visualized:

Drupal Flow - the interaction between issues and workspaces

  • Every issue should be associated with one or more canonical workspaces (git repos+merge requests) which can be collaborated on by any contributor, and merged into the canonical parent by a project maintainer.
  • Contributors should be able to modify the code in these workspaces by:
    • Checking out the code and committing/pushing changes to the workspace.
    • Inline editing files within a workspace.
    • Legacy: uploading a patch.
    • Any contribution method should trigger the appropriate tests.
  • Workspaces should be rebased (manually or automatically) on any changes to the canonical parent.

This workflow is not dissimilar to the default GitHub flow, GitLab flow, or even the suggested Bitbucket flow. These are all variations of the generic concept of 'Git flow' created by Vincent Driessen. However, the important difference comes in how each of these solutions tries to integrate a gitflow model into their issue/pull-request model.

The dominant model of these large third party tooling providers is not particularly collaborative. It encourages forks-of-forks, separates conversation about potential solutions onto multiple branch comments or merge requests threads, and generally caters to single developers working on individual proposed solutions, with one ultimately selected by the maintainer and the rest thrown away.

This is a key area in which the 'Drupal-flow' differs from the standard workflow that is implicitly enforced by these toolsets. Our ethos encourages collaboration of many developers on a single solution, and a single threaded conversation.

The gitflow behind the scenes may be identical, but the user experience wrapped around that flow makes very different assumptions about how people will collaborate.

GitHubGitHub

GitHub is of course the dominant player in the open-source tooling space. For many years though, their toolset was very much targeted at smaller open-source projects, and their default workflow highly encourages a many-to-many, forks-of-forks workflow, rather than the many-to-one single-threaded collaboration that is part of the Drupal community's ethos. More recently, GitHub has been improving their featureset, providing better issue management tools, more control over organization-level namespaces, and a new app marketplace that allows developers to extend GitHub's core featureset.

We reached out to GitHub and spoke with members of the technical and partnership teams, and while they were excited by the idea of bringing a project like Drupal to GitHub, we did run into some unresolvable blockers - most critically, by policy GitHub does not allow users of GitHub Enterprise to run their own public instances. This means we would have to migrate Drupal projects to GitHub.com, rather than self-hosting our instance.

Criteria:

  • Familiar workflow
    • GitHub is unquestionably the dominant platform for code collaboration on the web today.
  • Preserve the most valuable parts of Drupal's unique workflow
    • ❌ No way to enforce Drupal.org as the home of projects. While we can still keep project pages on Drupal.org there is nothing to encourage visitors who find the project on GitHub to treat the Drupal.org project page as canonical.
    • ❌ ;No way to enforce our workflow - forks of forks of forks of Drupal core into personal namespaces would be one click away.
    • ❌ Issue management tools are not setup for large, crowd-sourced teams.
      • Even for large community projects, it's more typical for only one or two developers to work on a single github issue at a time. Collaboration tends to be divided across issues, rather than within them.
    • Hard blocker Changing issue metadata requires maintainer permissions. (i.e: No way for non-maintainer collaborators to set an issue to RTBC)
    • ❌ Limited ability to manage project maintainers, especially in the case of abandoned or insecure projects.
  • Leverage a partner who will keep evolving their code collaboration featureset.

Other issues:

GitHub is a no-go.

GitLabGitLab

GitLab is the up-and-comer, and is becoming exceptionally popular in the open-source space - even beginning to be adopted by larger open source projects such as GNOME and Debian.

We've worked closely with the GitLab team as well. In contrast to GitHub, GitLab's terms would allow us to run our own instance of the service. GitLab's fundamental workflow is very much like GitHub's - a one-to-many model of forks of forks. After discussing our workflow with their engineers GitLab has begun implementing their new 'fork network' feature, allowing merge requests across forks. However, this doesn't quite achieve the goal of allowing our many-to-one collaboration model just yet.

GitLab is very close to being a good solution for an Open Source project of Drupal's scale, and the rate at which they are improving the service is impressive. GitLab is also already a favorite of many in our community for personal or professional projects. Given some more time, it may ultimately be able to tick all the boxes.

Criteria:

  • Familiar workflow
    • GitLab's workflow is very similar to GitHub's and it is the second most widely used code collaboration toolset.
  • Preserve the most valuable parts of Drupal's unique workflow
    • Hard Blocker - Maintainers cannot yet collaborate on downstream merge requests
    • ❌ Monolithic toolset - GitLab's featureset is designed to be used in an all or nothing way. There is no provision for using specific elements while disabling others.
      • For example, there is no way to disable GitLab project pages in order to keep the canonical project pages on Drupal.org.
    • No plugin architecture - changes must be accepted as merge requests upstream in GitLab, or re-rolled as patches with each release (aka, "hacking core").
      • No easy way to turn on or off elements of the GitLab UI.
      • No way to extend the GitLab featureset.
      • Very limited issue management states, and no ability to customize them.
  • Leverage a partner who will keep evolving their code collaboration featureset.
    • GitLab's roll-out of new features is the fastest paced of the tooling providers that exist today.

Other issues:

  • License/Source availability
    • / GitLab Community Edition is open source, but GitLab Enterprise Edition (which we would need for a project of our scale and with our workflow needs) is not open source, only source-available.
  • API availability
    • GitLab has a fairly robust api, but in our evaluation we found that many api features were undocumented, and others were deprecated without notification between minor versions.
  • Hybrid integration
    • A hybrid integration between Drupal.org and GitLab is possible with SSO, automatic repository syncing, and a choice of using our issue queue or theirs.
    • ❌ However, there would be no portability of issues across a project that uses GitLab to one that does not.
    • ❌ Projects choosing to use the GitLab issues would not have DrupalCI integration, or issue credits.
  • Data portability/vendor lock-in
    • GitLab's API and documented data model should make it possible to migrate our data to another system should it become necessary.
  • Developer support
    • As an open-source project, GitLab has a growing community of contributors, as well as the support of a venture funded mothership organization.
  • Maintainability for an engineering team of limited resources
    • Hard Blocker - GitLab's current file-handling makes full copies of a repository for every fork. Taking the number of open core issues as a representative example, Drupal.org would need a new 200 TB+ redundant storage apparatus, just to accommodate Core. GitLab needs to transition to a shared git-object model, or an alternative like git namespaces.
  • Cost
    • The GitLab team is willing to provide the Enterprise Edition to the Drupal community for free.
  • Unintended consequences

GitLab: not-yet.

BitbucketBitbucket

Bitbucket was the dark horse candidate. Atlassian products are well known in software development circles, but they are often more tolerated than beloved. However, when we sat down at Midwest Drupal Summit in August and looked at the options we explored so far and found them to be not quite baked - Bitbucket began to show some advantages as a solution.

For one thing, Bitbucket has recently put a heavy focus on user experience and collaboration features. It also has the most flexible permissions model for control over branch permissions, forking, and automatic rebasing of any of the toolsets we evaluated. Finally, it can be implemented modularly - allowing us to use the components that serve our workflow and disable the ones that don't.

Criteria:

  • / Familiar workflow
    • Bitbucket implements a very standard pull request workflow. It is not as widely popular as GitHub or GitLab, but should be easily learned by developers familiar with either of those systems.
  • Preserve the most valuable parts of Drupal's unique workflow
    • Of all the options we prototyped, Bitbucket is the only one which provides the flexibility needed to preserve key elements of our workflow - in particular the ability to cleanly use only the parts of Bitbucket that we need (specifically, merge requests, inline editing, and code commenting) without having to use issues or project pages that are less sophisticated than our existing solutions.
  • Leverage a partner who will keep evolving their code collaboration featureset.
    • Though it's flown under the radar, Atlassian has been making some significant improvements to Bitbucket from a feature and user experience point of view over the course of the past year.

Other issues:

  • License/Source availability
    • /❌ Bitbucket is not open-source, only source-available, though they have an open source licensing program.
  • API availability
  • Hybrid integration
    • Bitbucket is much more flexible in terms of choosing which features to use, and which to disable- allowing a fairly tight hybrid integration.
  • Data portability/vendor lock-in
    • / Bitbucket stores repositories as standard git objects on the file system, making it relatively straightforward in case of a future migration to another system.
  • Developer support
    • Bitbucket has the professional support of Atlassian, but does not have a robust open source community driving feature development.
  • Maintainability for an engineering team of limited resources
    • Bitbucket is implemented with an api-first methodology, and is well documented, making it straightforward for a relatively small team to support a selective integration such as what we propose here.
  • Cost
  • Unintended consequences

Bitbucket: the best tool in the current landscape given our community's needs.

To sum up, each of the toolsets we evaluated have advantages and disadvantages. However, if we want to move forward with making improvements for our developers TODAY, we have determined that the best way to do that is to create modular integration points, and use the toolset that currently meets our criteria: Bitbucket. As these toolsets continue their rapid development, we will continue to periodically re-evaluate them.

Author: 
Original Post: 

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web