Feb 26 2015
Feb 26

This is the third in a series of blog posts about the relationship between Drupal and Backdrop CMS, a recently-released fork of Drupal. The goal of the series is to explain how a module (or theme) developer can take a Drupal project they currently maintain and support it for Backdrop as well, while keeping duplicate work to a minimum.

  • In part 1, I introduced the series and showed how for some modules, the exact same code can be used with both Drupal and Backdrop.
  • In part 2, I showed what to do when you want to port a Drupal module to a separate Backdrop version and get it up and running on GitHub.
  • In part 3 (this post), I'll wrap up the series by explaining how to link the Backdrop module to the Drupal.org version and maintain them simultaneously.

Linking the Backdrop Module to the Drupal.org Version and Maintaining Them Simultaneously

In part 2 I took a small Drupal module that I maintain (User Cancel Password Confirm) and ported it to Backdrop. In the end, I wound up with two codebases for the same module, one on Drupal.org for Drupal 7, and one on GitHub for Backdrop.

However, the two codebases are extremely similar. When I fix a bug or add a feature to the Drupal 7 version, it's very likely that I'll want to make the exact same change (or at least an extremely similar one) to the Backdrop version. Wouldn't it be nice if there were a way to pull in changes automatically without having to do everything twice manually?

If you're a fairly experienced Git user, you might already know that the answer is "yes". But if you're not, the process isn't necessarily straightforward, so I'm going to document it step by step here.

Overall, what we're doing is simply taking advantage of the fact that when we imported the Drupal.org repository into GitHub in part 2, we pulled in the entire history of the repository, including all of the Drupal commits. Because our Backdrop repository knows about these existing commits, it can also figure out what's different and pull in the new ones when we ask it to.

In what follows, I'm assuming a workflow where changes are made to the Drupal.org version of the module and pulled into Backdrop later. However, it should be relatively straightforward to reverse these instructions to do it the other way around (or even possible, but perhaps less straightforward, to have a setup where you can do it in either direction).

  1. To start off, we need to make our local clone of the Backdrop repository know about the Drupal.org repository. (A local clone is obtained simply by getting the "clone URL" from the GitHub project page and copying it locally, for example with the command shown below.)
    git clone [email protected]:backdrop-contrib/user_cancel_password_confirm.git
    

    First let's check what remote repositories it knows about already:

    $ git remote -v
    origin    [email protected]:backdrop-contrib/user_cancel_password_confirm.git (fetch)
    origin    [email protected]:backdrop-contrib/user_cancel_password_confirm.git (push)
    

    No surprise there; it knows about the GitHub version of the repository (the "origin" repository that it was cloned from).

    Let's add the Drupal.org repository to this list and check again:

    $ git remote add drupal http://git.drupal.org/project/user_cancel_password_confirm.git
    $ git remote -v
    drupal    http://git.drupal.org/project/user_cancel_password_confirm.git (fetch)
    drupal    http://git.drupal.org/project/user_cancel_password_confirm.git (push)
    origin    [email protected]:backdrop-contrib/user_cancel_password_confirm.git (fetch)
    origin    [email protected]:backdrop-contrib/user_cancel_password_confirm.git (push)
    

    The URL I used here is the same one I used in part 2 to import the repository to GitHub (that is, it's the public-facing Git URL of my project on Drupal.org, available from the "Version control" tab of the drupal.org project page, after unchecking the "Maintainer" checkbox - if it’s present - so that the public URL is displayed). I've also chosen to give this repository the name "drupal". (Usually the convention is to use "upstream" for something like this, but in GitHub-land "upstream" is often used in a slightly different context involving development forks of one GitHub repository to another. So for clarity, I'm using "drupal" here. You can use anything you want to.)

  2. Next let's pull in everything from the remote Drupal repository to our local machine:
    $ git fetch drupal
    remote: Counting objects: 4, done.
    remote: Compressing objects: 100% (2/2), done.
    remote: Total 3 (delta 0), reused 0 (delta 0)
    Unpacking objects: 100% (3/3), done.
    From http://git.drupal.org/project/user_cancel_password_confirm
    * [new branch]          7.x-1.x -> drupal/7.x-1.x
    * [new branch]          master  -> drupal/master
    * [new tag]             7.x-1.0-rc1 -> 7.x-1.0-rc1
    

    You can see it has all the branches and tags that were discussed in part 2 of this series. However, although I pulled the changes in, they are completely separate from my Backdrop code (the Backdrop code lives in "origin" and the Drupal code lives in "drupal").

    If you want to see a record of all changes that were made to port the module to Backdrop at this point, you could run git diff drupal/7.x-1.x..origin/1.x-1.x to examine them.

  3. Now let's fix a bug on the Drupal.org version of the module. I decided to do a simple documentation fix: Fix documentation of form API functions to match coding standards

    I made the code changes on my local checkout of the Drupal version of the module (which I keep in a separate location on my local machine, specifically inside the sites/all/modules directory of a copy of Drupal so I can test any changes there), then committed and pushed them to Drupal.org as normal.

  4. Back in my Backdrop environment, I can pull those changes in to the "drupal" remote and examine them using git log:
    $ git fetch drupal
    remote: Counting objects: 5, done.
    remote: Compressing objects: 100% (3/3), done.
    remote: Total 3 (delta 2), reused 0 (delta 0)
    Unpacking objects: 100% (3/3), done.
    From http://git.drupal.org/project/user_cancel_password_confirm
      7a70138..997d82d  7.x-1.x     -> drupal/7.x-1.x
    
    $ git log origin/1.x-1.x..drupal/7.x-1.x
    commit 997d82dce1a4269a9cee32d3f6b2ec2b90a80b33
    Author: David Rothstein 
    Date:   Tue Jan 27 13:30:00 2015 -0500
    
            Issue #2415223: Fix documentation of form API functions to match coding standards.
    

    Sure enough, this is telling me that there is one commit on the Drupal 7.x-1.x version of the module that is not yet on the Backdrop 1.x-1.x version.

  5. Now it's time to merge those changes to Backdrop. We could just merge the changes directly and push them to GitHub and be completely done, but I'll follow best practice here and do it on a dedicated branch with a pull request. (In reality, I might be doing this for a more complicated change than a simple documentation fix, or perhaps with a series of Drupal changes all at once rather than a single one. So I might want to formally review the Drupal changes before accepting them into Backdrop.)

    By convention I'm going to use a branch name ("drupal-2415223") based on the Drupal.org issue number:

    $ git checkout 1.x-1.x
    Switched to branch '1.x-1.x'
    
    $ git checkout -b drupal-2415223
    Switched to a new branch 'drupal-2415223'
    
    $ git push -u origin drupal-2415223
    Total 0 (delta 0), reused 0 (delta 0)
    To [email protected]:backdrop-contrib/user_cancel_password_confirm.git
    * [new branch]          drupal-2415223 -> drupal-2415223
    Branch drupal-2415223 set up to track remote branch drupal-2415223 from origin.
    
    $ git merge drupal/7.x-1.x
    Auto-merging user_cancel_password_confirm.module
    Merge made by the 'recursive' strategy.
    user_cancel_password_confirm.module |   10 ++++++++--
    1 file changed, 8 insertions(+), 2 deletions(-)
    

    In this case, the merge was simple and worked cleanly. Of course, there might be merge conflicts here or other changes that need to be made. You can do those at this time, and then git push to push the changes up to GitHub.

  6. Once the changes are pushed, I went ahead and created a pull request via the GitHub user interface, with a link to the Drupal.org issue for future reference (I could have created a corresponding issue in the project's GitHub issue tracker also, but didn't bother):
    • Fix documentation of form API functions to match coding standards (pull request) (diff)

    Merging this pull request via the GitHub user interface gets it onto the official 1.x-1.x Backdrop branch, and into the Backdrop version of the module.

    Here's the commit for Drupal, and the same one for Backdrop:

    http://cgit.drupalcode.org/user_cancel_password_confirm/commit/?id=997d8...
    https://github.com/backdrop-contrib/user_cancel_password_confirm/commit/...

Using the above technique, it's possible to have one main issue (in this case on Drupal.org) for any change you want to make to the module, do essentially all the work there, and then easily and quickly merge that change into the Backdrop version without the hassle of repeating lots of manual, error-prone steps.

Hopefully this technique will be useful to developers who want to contribute their work to Backdrop while also continuing their contributions to Drupal, and will help the two communities continue to work together. Thanks for reading!

Further Backdrop Resources

Do you have any thoughts or questions, or experiences of your own trying to port a module to Backdrop? Leave them in the comments.

Feb 17 2015
Feb 17

This is the second in a series of blog posts about the relationship between Drupal and Backdrop CMS, a recently-released fork of Drupal. The goal of the series is to explain how a module (or theme) developer can take a Drupal project they currently maintain and support it for Backdrop as well, while keeping duplicate work to a minimum.

  • In part 1, I introduced the series and showed how for some modules, the exact same code can be used with both Drupal and Backdrop.
  • In part 2 (this post), I'll explain what to do when you want to port a Drupal module to a separate Backdrop version and get it up and running on GitHub.
  • In part 3, I'll explain how to link the Backdrop module to the Drupal.org version and maintain them simultaneously.

Porting a Drupal Module to Backdrop and Getting it Up and Running on GitHub

For this post I’ll be looking at User Cancel Password Confirm, a very small Drupal 7 module I wrote for a client a couple years back to allow users who are canceling their accounts to confirm the cancellation by typing in their password rather than having to go to their email and click on a confirmation link there.

We learned in part 1 that adding a backdrop = 1.x line to a module’s .info file is the first (and sometimes only) step required to get it working with Backdrop. In this case, however, adding this line to the .info file was not enough. When I tried to use the module with Backdrop I got a fatal error about a failure to open the required includes/password.inc file. What's happening here is simply that Backdrop (borrowing a change that's also in Drupal 8) reorganized the core directory structure compared to Drupal 7 to put most core files in a directory called "core". When my module tries to load the includes/password.inc file, it needs to load it from core/includes/password.inc in Backdrop instead.

This is a simple enough change that I could just put a conditional statement into the Drupal code so that it loads the correct file in either case. However, over the long run this would get unwieldy. Furthermore, if I had chosen a more complicated module to port, one which used Drupal 7's variable or block systems (superseded by the configuration management and layout systems in Backdrop) it is likely I'd have more significant changes to make.

So, this seemed like a good opportunity to go through the official process for porting my module to Backdrop.

Backdrop contrib modules, like Backdrop core, are currently hosted on GitHub. Regardless of whether you're already familiar with GitHub from other projects, there are some steps you should follow that might not be familiar, to make sure your Backdrop module's repository is set up properly and ultimately to get it included on the official list of Backdrop contributed projects.

Importing to GitHub

The best way to get a Drupal module into GitHub is to import it; this preserves the pre-Backdrop commit history which becomes important later on.

Before you do this step, if you're planning to port a Drupal module that you don't maintain, it's considered best practice to notify the current maintainer and see if they'd like to participate or lead the Backdrop development themselves (see the "Communicating" section of the Drupal 7 to Backdrop conversion documentation for more information). In my case I'm already the module maintainer, so I went ahead and started the import:

  1. Go to the GitHub import page and provide the public URL of the Drupal project's Git repository (which I got from going to the project page on Drupal.org, clicking the "Version control" tab, and then - assuming you are importing a module that you maintain - making sure to uncheck the "Maintainer" checkbox so that the public URL is displayed). Drupal.org gives me this example code:


    git clone --branch 7.x-1.x http://git.drupal.org/project/user_cancel_password_confirm.git

    So I just grabbed the URL from that.

  2. Where GitHub asks for the project name, use the same short name (in this case "user_cancel_password_confirm") that the Drupal project uses.
  3. Import the project into your own GitHub account for starters (unless you're already a member of the Backdrop Contrib team - more on that later).

Here's what it looks like:
GitHub import

Submitting this form resulted in a new GitHub repository for my project at https://github.com/DavidRothstein/user_cancel_password_confirm.

As a final step, I edited the description of the GitHub project to match the description from the module's .info file ("Allows users to cancel their accounts with password confirmation rather than e-mail confirmation").

Cleaning Up Branches and Tags

Next up is some housekeeping. First, I cloned a copy of the new repository to my local machine and then used git branch -r to take a look around:


$ git clone [email protected]:DavidRothstein/user_cancel_password_confirm.git
$ git branch -r
origin/7.x-1.x
origin/HEAD -> origin/master
origin/master

Like many Drupal 7 contrib projects, this has a 7.x-1.x branch where all the work is done and a master branch that isn't used. When I imported the repository to GitHub it inherited those branches. However, for Backdrop I want to do all work on a 1.x-1.x branch (where the first "1.x" refers to compatibility with Backdrop core 1.x).

  1. So let's rename the 7.x-1.x branch:


    $ git checkout 7.x-1.x
    Branch 7.x-1.x set up to track remote branch 7.x-1.x from origin.
    Switched to a new branch '7.x-1.x'
    $ git branch -m 7.x-1.x 1.x-1.x
    $ git push --set-upstream origin 1.x-1.x
    Total 0 (delta 0), reused 0 (delta 0)
    To [email protected]:DavidRothstein/user_cancel_password_confirm.git
    * [new branch] 1.x-1.x -> 1.x-1.x
    Branch 1.x-1.x set up to track remote branch 1.x-1.x from origin.

  2. And delete the old one from GitHub:


    $ git push origin :7.x-1.x
    To [email protected]:DavidRothstein/user_cancel_password_confirm.git
    - [deleted] 7.x-1.x

  3. We want to delete the master branch also, but can't do it right away since GitHub treats that as the default and doesn't let you delete the default branch.

    So I went to the module's GitHub project page, where (as the repository owner) I have a "Settings" link in the right column; via that link it's possible to change the default branch to 1.x-1.x through the user interface.

    Now back on my own computer I can delete the master branch:


    $ git push origin :master
    To [email protected]:DavidRothstein/user_cancel_password_confirm.git
    - [deleted] master

  4. On Drupal.org, this module has a 7.x-1.0-rc1 release, which was automatically imported to GitHub. This won't be useful to Backdrop users, so I followed the GitHub instructions for deleting it.
  5. Finally, let's get our local working copy somewhat in sync with the changes on the server. The cleanest way to do this is probably just to re-clone the repository, but you could also run git remote set-head origin 1.x-1.x to make sure your local copy is working off the same default branch.

The end result is:


$ git branch -r
origin/1.x-1.x
origin/HEAD -> origin/1.x-1.x

Just what we wanted, a single 1.x-1.x branch which is the default (and which was copied from the 7.x-1.x branch on Drupal.org and therefore contains all its history).

Updating the Code for Backdrop

Now that the code is on GitHub, it's time to make it Backdrop-compatible.

To do this quickly, you can just make commits to your local 1.x-1.x branch and push them straight up to the server. In what follows, though, I'll follow best practices and create a dedicated branch for each change (so I can create a corresponding issue and pull request on GitHub). For example:


$ git checkout -b backdrop-compatibility
$ git push -u origin backdrop-compatibility

Then make commits to that branch, push them to GitHub, and create a pull request to merge it into 1.x-1.x.

  1. To get the module basically working, I'll make the simple changes discussed earlier:
    • Add basic Backdrop compatibility (issue) (diff)

    If you look at the diff, you can see that instead of simply adding the backdrop = 1.x line to the .info file, I replaced the core = 7.x line with it (since the latter is Drupal-specific and does not need to be in the Backdrop version).

    With that change, the module works! Here it is in action on my Backdrop site:

    Cancel account using password

    (Also visible in this screenshot is a nice effect of Backdrop's layout system: Editing pages like this one, even though they are using the default front-end Bartik theme, have a more streamlined, focused layout than normal front-end pages of the site, without the masthead and other standard page elements.)

  2. Other code changes for this small module weren't strictly necessary, but I made them anyway to have a fully-compatible Backdrop codebase:
    • Replace usage of "drupal" with "backdrop" in the code (issue) (diff)
    • Use method on the user account object to determine its ID (issue) (diff)
  3. Next up, I want to get my module listed on the official list of Backdrop contributed projects (currently this list is on GitHub, although it may eventually move to the main Backdrop CMS website).

    I read through the instructions for applying to the Backdrop contributed project group. They're relatively simple, and I've already done almost everything I need above. The one thing I'm missing is that Backdrop requires a README.md file in the project root with some standard information in it (I like that they're enforcing this; it should help developers browsing the module list a lot), and it also requires a LICENSE.txt file. These were both easy to create following the provided templates and copying some information from the module's Drupal.org project page:

    Once that's done, and after reading through the rest of the instructions and making sure I agreed with them, I proceeded to create an issue:

    Application to join contrib team

    In my case it was reviewed and approved within a few hours (perhaps helped by the fact that I was porting a small module), and I was given access to the Backdrop contributed project team on GitHub.

  4. To get the module transferred from my personal GitHub account to the official Backdrop contrib list, I followed GitHub's instructions for transferring a repository.

    They are mostly straightforward. Just make sure to use "backdrop-contrib" as the name of the new owner (who you are transferring the repository to):

    Transfer repository to backdrop-contrib

    And make sure to check the box that gives push access to your repository to the "Authors" team within the Backdrop Contrib group (if you leave it as "Owners", you yourself wouldn't be able to push to it anymore):

    Grant access to the Authors team

    That's all it took, and my module now appears on the official list.

    You'll notice after you do this that all the URLs of your project have changed, although the old ones redirect to the new ones. That's why if you follow many of the links in this post, which point to URLs like https://github.com/DavidRothstein/user_cancel_password_confirm, you'll see that they actually redirect you to https://github.com/backdrop-contrib/user_cancel_password_confirm.

    For the same reason, you can keep your local checkout of the repository pointed to the old URL and it will still work just fine, although to avoid any confusion you might want to either do a fresh clone at this point, or run a command like the following to update the URL:

    git remote set-url origin [email protected]:backdrop-contrib/user_cancel_password_confirm.git
    

With the above steps, we’re all set; the module is on GitHub and can be developed further for Backdrop there.

But what happens later on when I make a change to the Drupal version of the module and want to make the same change to the Backdrop version (certainly a common occurrence)? Do I have to repeat the same changes manually in both places? Luckily the answer is no. In part 3 of this series, I’ll explain how to link the Backdrop module to the Drupal.org version and maintain them simultaneously. Stay tuned!

Further Backdrop Resources

Do you have any thoughts or questions, or experiences of your own trying to port a module to Backdrop? Leave them in the comments.

Feb 03 2015
Feb 03

Part 1 - Reuse the Same Code

In mid-January, the first version of Backdrop CMS was released. Backdrop is a fork of Drupal that adds some highly-anticipated features and API improvements to the core Drupal platform while focusing on performance, usability, and developer experience.

When an open-source fork makes the news, it's often because it was born from a fierce, acrimonious battle (example: Joomla forking from Mambo); the resulting projects compete with each other on the exact same turf and developers are forced to choose sides. Backdrop's goal, however, is not to destroy or replace the original Drupal project. Rather, it aims to be a "friendly fork" that focuses on Drupal's traditional audience of site builders and developers, an audience which the Backdrop founders believe are being slowly left behind by the Drupal project itself.

Because of this, I expect that many existing Drupal developers will not want to choose between the platforms, but instead will continue working with Drupal while also beginning to use Backdrop. In this series of blog posts, I will explain how a module (or theme) developer can take a Drupal project they currently maintain and support it for Backdrop as well, while keeping duplicate work to a minimum.

  • In part 1 (this post), I'll show how for some modules, the exact same code can be used with both Drupal and Backdrop.
  • In part 2, I'll explain what to do when you want to port a Drupal module to a separate Backdrop version and get it up and running on GitHub.
  • In part 3, I'll explain how to link the Backdrop module to the Drupal.org version and maintain them simultaneously.

Sometimes the Same Exact Code can be Used With Both Drupal and Backdrop

To start things off let's look at Field Reference Delete, a Drupal 7 module I maintain which does some behind-the-scenes cleanup in your database when entities such as nodes or taxonomy terms are deleted on the site. It's a moderately-complex module which makes heavy use of Drupal's field and entity systems.

To make this or any Drupal module work with Backdrop, there is one step that is always required: Adding a backdrop = 1.x line to the .info file to inform Backdrop core that the code is Backdrop-compatible.

Easy enough. The big question is what changes are required beyond that?

Checking for Changes

Although Backdrop is not 100% compatible with Drupal 7 (due to the features and API improvements it adds) it aims to be backwards-compatible as much as possible, for example by including a compatibility layer to allow Drupal functionality that is deprecated in Backdrop to still work correctly.

The Backdrop change records and module conversion guide provide technical advice for developers on how and when to upgrade their code. The biggest changes are probably the configuration management system (Backdrop’s replacement for Drupal 7’s variable API and other configuration that was previously stored in the database) and layout system (which removes much of the functionality of the Drupal 7 Block module in favor of a newer, more powerful Layout module).

If your Drupal 7 module makes heavy use of these systems, it’s likely you’ll want to make some changes in order to work with Backdrop. However, the compatibility layer means that you might not actually need to. For example, Backdrop retains Drupal 7’s variable API (although it is marked as deprecated and is not as powerful as the configuration system which replaces it). So your code might still work even if it uses this system. It really depends on the details of how your module works, so the best advice is to test it out and see what (if anything) is broken.

It’s also worth noting that because Backdrop was forked from an early version of Drupal 8 (not from Drupal 7) it inherited a smattering of changes that were added to Drupal 8 early in the release cycle. Not all of these have made it into the list of Backdrop change records yet, although work is ongoing and people are adding them as they are noticed.

In the case of Field Reference Delete, I tested the module on Backdrop and it worked fine. I also skimmed the change records and module conversion guide mentioned above and didn't see anything that obviously needed to change. Even though entities in Backdrop have been converted to classed objects and the field API has been converted to use the configuration management system, Field Reference Delete’s heavy use of the entity and field systems still didn’t require that any changes be made. All I had to do to get the module working was add that one backdrop = 1.x line to the .info file.

Adding the One Line of Code on Drupal.org

Interestingly enough, since Drupal will happily ignore a backdrop = 1.x line in a module's .info file, it's possible to simply add that code to the Drupal.org version of the module and use the same version of the module for either Drupal or Backdrop. I did that in this issue; the resulting diff is simply this:


diff --git a/field_reference_delete.info b/field_reference_delete.info
...
name = Field reference delete
description = Immediately removes references to a deleted entity from fields stored in an SQL database.
core = 7.x
+backdrop = 1.x

Drupal uses the core = 7.x line to determine Drupal compatibility, and Backdrop uses the backdrop = 1.x line to determine Backdrop compatibility. The 7.x-1.0-beta1 release of Field Reference Delete contains the above change and can be used equally well on a Drupal or Backdrop site. Simple!

There are some downsides to doing this, however:

  • Although no changes to the module code may be strictly required, there are usually optional (and non-backwards-compatible) changes that can be made to better integrate with new Backdrop features.
  • It is hard for Backdrop users and developers to find the module and know that it's compatible with Backdrop. I tried to improve its discoverability by adding a "Backdrop compatibility" tag to the above-mentioned issue, and I also put a note on the project page explaining that Backdrop is supported. These aren't ideal, but should help; perhaps a standard will eventually catch on but there isn't a formal one yet.

Despite these disadvantages, for the time being I'd like to just have one copy of the code for this particular module (hosted in one place), and it's nice to know that's possible.

In part 2 of this series, I’ll take a look at a different module I maintain where it isn’t possible to use the same exact code for Drupal and Backdrop, and I’ll show how I went through the official process for porting my module to Backdrop and getting it hosted on GitHub. Stay tuned!

Further Backdrop Resources

Do you have any thoughts or questions, or experiences of your own trying to port a module to Backdrop? Leave them in the comments.

Aug 29 2010
Aug 29

DrupalCon Copenhagen comes to an end, as does my blogging hiatus.

Two of my primary learning objectives here in Copenhagen were configuration management and deployment process. Historically, working with Drupal in these areas has been unpleasant, and I think that's why there is tons of innovation going on in that space right now. It needs to be fixed, and new companies are springing up to say "hey, we fixed it." Often, the people running the companies are the same people running the project that encapsulates the underlying technologies. I'm referring to:

  • The hyper-performant core distro, Pressflow
  • Distros with sophisticated install profiles, like OpenAtrium, ManagingNews and OpenPublish
  • Configuration externalization with Features
  • Development Seed's "for every site, a makefile" workflow using drush make
  • The different-yet-overlapping hosting platforms Pantheon and Aegir

Dries commented in his keynote that as Drupal continues to grow, it also needs to grow up. I think advances like these are part of the community's answer to that. I want to wrap my head around some of these tools, while continuing to watch how they progress. Others, I want to implement right now. What's perfectly clear though is that I have a lot of work to do to keep up with the innovation going on in this hugely powerful community. Which is actually nothing new, but reading a blog post about these technologies doesn't make my jaw drop the way that it does when I'm in the room watching Drupal advance.

Oct 17 2008
Oct 17

Manual upgrades of Drupal core are more time-consuming and error-prone than need be. svn vendor branches can help, but it's still not perfect.

The critical thing to realize is that the basic unit of work in svn is the changeset. When I first started playing with vendor branches, I tried to take a shortcut and just diff two arbitrary branches anywhere in my repository. But without a changeset between two revisions, that will get you nowhere. svn can't automatically diff the branches - it will just delete every element and then re-add it. And that will only bring more chaos to a process where we are trying to restore order.

So we need to create a changeset between two revisions of Drupal. But remember why we want to automate upgrades in the first place. For major releases, this set of changes is going to be so complex that doing it manually would be prohibitively error-prone. It's also going to be a huge number of changes, which makes it time-consuming. What to do?

svn_load_dirs

This script will look at a (source) set of files in an svn repository, compare it to a (destination) set of files, and then automate whatever adds, changes, and deletes are necessary to make the source like the destination. It will check the changes in for you, and it will optionally tag the changeset.

If svn_load_dirs isn't already installed on your system, it can get a little tricky. You need to get svn_load_dirs.pl.in, rename it to svn_load_dirs.pl, then point it to your svn binary, which you can find like this:

$ which svn
/usr/bin/svn

Run svn_load_dirs with an older version of core as the source, and the new version as the destination, and we have a changeset that represents all the differences between those versions of core. Now we can merge that changeset with any other existing installation in the repository, and it will upgrade core without touching any of your contrib modules or themes. If you've hacked core at all, it should prompt you to perform a merge.

This worked beautifully for a little while, but there's a catch. Because I frequently partner with other developers, or work with code-savvy clients who need access, putting all my Drupal projects in the same repository won't work for me. And the catch is: you can't merge across repositories.

It would be nice, perfect actually, to have one repository where I maintain vendor branches, and upgrade core on all my Drupal projects from there. Having to maintain vendor branches in every repository that contains a copy of Drupal core is a hassle, but it works.

Oct 10 2008
Oct 10

First, the result is not necessarily an accurate representation of the new Drupal core, because it doesn't address moved or deleted files, only added and changed ones. Second, it will overwrite any changes you've made to core. Yes, I know this should never ever happen because you should never ever hack core. Still, it overwrites without asking, when a manual merge is what's really called for.

Has anyone else solved this differently? I have a method that has worked almost perfectly that I'll write more about next week.

Update I

For an example of moved/deleted files in Drupal core, and the problems they might cause, please direct your attention to the release announcement for Drupal 5:

"Because the directory structure in Drupal changed, it is recommended not to extract Drupal 5.0 on top of your existing installation. Start with a fresh copy instead, and copy over your custom modules, themes and configuration file."

With version 5 of core, .module files were moved into their own respective folders with .info files. But upgrading from 4.7 to 5 this way would leave old versions of all the core modules sitting in the modules/ directory.

Update II

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web