Upgrade Your Drupal Skills
We trained 1,000+ Drupal Developers over the last decade.
See Advanced Courses NAH, I know EnoughSolutions exist to make working in Drupal 7 more like working in Drupal 8. Use this three-part approach to have fun with Drupal 7 development.
The post Make Drupal 7 Development Fun appeared first on Four Kitchens.
Today, I came across an interesting bug with composer-patches plugin (it really is a git-apply bug/behavior). TL;DR, it is fixed in the latest version of composer-patches plugin as of this writing – 1.6.4. All you have to do is run composer require cweagans/composer-patches:^1.6.4 to get the fix.
The problem is simple: Patches are not applied even though you might see it in the composer log output. In fact, even a PATCHES.txt file is generated with the correct listing of patches. There are no errors and no indication that the patching failed for any reason. The problem is because git apply fails silently. It does not give any output nor sets an error exit code.
The problem was first documented and fixed in cweagans/composer-patches#165; however, the fix used there relies on git-apply outputting log messages saying patches were skipped. Unfortunately, that behavior was only introduced in 2.9.0. The Docker image I was using only contained version 2.1.4. All I needed was either these “Skipped patch” messages or an exit code and the plugin would fall back on patch command, but neither happened.
Also, this is actually supposed to be git behavior: git apply will fail to do anything when used within a local checkout of a git repository (other than the one for the project the patch is made for), such as if you are patching a module that is within a site that is in Git version control. The suggestion is to use patch -p1 < path/file.patch instead. While an obvious solution was to upgrade git, a simple Google search told me that this problem has happened to others and I decided to dig deeper. All variations of git apply simply didn't work, nor would set the exit code. I followed the debugging steps in one of the comments in the pull request, but I still wouldn’t see any messages similar to “Skipped patch” or applying patch, or any error.
In testing, I found a way to get it working with the ‘--directory‘ parameter to ‘git apply‘. The command executed by the plugin is similar to the following:
git -C package-directory apply -p1 /tmp/patchfile.patch
Instead, this command worked (even if there is no git repository involved at all, even in the current directory):
git apply --directory=package-directory apply -p1 /tmp/patchfile.patch
On digging a bit more, I found another workaround that worked for me in cweagans/composer-patches#175. What’s more, this was already merged and tagged. All I had to do was update to the latest version 1.6.4 and it would work. This change checks for the path being patched and if it actually is a git repository. If the package is not a git repository, it does not attempt a git apply at all, but falls back to patch, which works perfectly.
In the modern world of web / application development, using package managers to pull in dependencies has become a de-facto standard. In fact, if you are developing enterprise software and you aren't leveraging package managers I would challenge you to ask yourself why not?
Drupal was very early to adopt this mindset of pulling in dependencies almost a decade ago when Dmitri Gaskin created an extension for Drush (the Drupal Shell) that added the ability to pull contributed modules by listing them in a make file (I think Dmitri was 12 years old when he wrote the Drush extension, pretty amazing!). Since that time, the make extension has been added to Drush core.
Composer is the current standard for putting together PHP applications, which is why Drupal 8 has gone this direction, so why not use Composer to put together Drupal 7 applications?
First off, I want to clarify what I'm not talking about in this post. I am not advocating that we ditch Drush all together, I still find value in other aspects of what Drush can do. I am specifically referring to the Make aspect of Drush. Is Drush Make still necessary?
This post is also not about Drupal Console vs Drush, both CLI tools add tremendous value to development workflow, and there isn't 100% overlap with these tools [yet]. I think we still need both tools.
This post is about how I came to see the benefit of switching to Composer from Drush Make. I recommend making this move for Drupal 7 and Drupal 8. This Drupal Composer workflow is not new, it has been around for a while. I just never saw a good reason to make the jump from Drush Make to this new process, until now. We have been asked in the comments on previous posts, "Why haven't you adopted the Composer process?" I now have a good reason to change our process and fully jump on board with Composer building Drupal 7 applications. We appreciate all the comments we get on our blog, it sharpens everyone involved!
We have blogged about the Composer workflow in a previous post on our Drupal 8 build process in the past, but the main motivation there was to be proactive about where PHP application development is going [already is]. We didn't have a real use case for the switch to Composer until now. This post will review how I came to that revelation.
Dependency Managers
I want to make one more point before I make the case for Composer. There are many reasons to use package managers to pull in dependencies. I'll save the details for another blog post. The main reason developers use package managers is so that your project repository does not include libraries and modules that you do not maintain. That is why tools like Composer, npm, Yarn, Bower, and Bundler exist. Hook up your RSS reader to our blog, I'll explain in more detail in a future post, but for now I'll leave this link to the Composer site explaining why committing dependencies is a bad idea, in your project repo.
Version Numbers
The #1 reason to make the switch to Composer is the ability to manage version numbers. You may be asking "What's the big deal, Drush Make handles version numbers as well?" let me give you a little context of why using Composer version numbers are a better approach.
The Back Story
Recently in a strategy meeting with one of our enterprise clients, we were discussing how to approach launching 100's of sites on one Drupal Core utilizing multiple installation profiles on top of Acquia Site Factory. Our goal was to figure out how we could sanely manage updating potentially dozens of installation profiles without explicitly defining each version number of the profile being updated. This type of Drupal architecture is also a topic for a future blog post, but for now read Acquia's explanation of why architecting with profiles is a good idea.
As a developer, it is common place to lock down versions to a very specific version so that we know exactly what versions we are using / deploying. This is the reason composer.lock, Gemfile.lock, yarn.lock, and npm shrinkwrap exist. We have experienced the pain of unexpected defects in applications due to an obscure dependency changing deep in the dependency tree. Most dependency managers have a very explicit command for updating dependencies, i.e. composer update, bundle update, yarn upgrade respectively, which in turn update the lock file.
A release manager does not need to know explicitly which version of a dependency (installation profile, module, etc), to release next, she simply wants the latest stable release.
Herein lies the problem with Drush Make. There are practices that exist that solve both the developer problem and release manager problem that do not exist in Drush Make, but do exist in Composer and other application development environments. It's a common pattern that has been around for a while, it's called semantic versioning.
Semantic Versioning
If you haven't heard of semantic versioning (semver), go check it out now. Pretty much every package manager I have dealt with has adopted semver. Adopting semver gives the developer, or release manager, the choice of how to update dependencies within their app. There are very distinct numbers in semver for introducing breaking changes, new features, and bug fixes. How does this play into what problem use cases I mentioned above?
A developer has the ability to specify in the composer.json file specific versions, while leaving the version number flexible to pull in new bug fixes and feature improvements (patch and minor releases). Look at the example below:
{
"name": "My Drupal Platform",
...
"require": {
...
"drupal/drupal": "~7.53.0",
"drupal/views": "^3.14.0"
},
...
}The tilde ~ and caret ^ symbols have special meanings when specifying version numbers. The tilde matches the most recent minor version (updates patch release number, the last number), the caret will update you to the most recent major version (updates minor release number, the middle number).
The above example basically says, use the views module at version 3.14, and when version 3.15 comes out, update me to that version when I run composer update.
Breaking changes should only be introduced when you update the first number, the major release. Of course, if you completely trust the developer writing the contributed code this system would be enough, but not all developers follow best practice, which is why the lock file was created and the need to explicitly run composer update.
With this system in place, a release manager now only needs to worry about running one command to get the latest stable release of all dependencies. This command could also be hidden behind a nice UI (a CI Server) so all she has to do is push one button to grab all the latest dependencies and push to a testing site for verification.
Understanding everyones needs
In the past, I didn't have a good reason to move away from Drush Make, because it did the job, and Drush is so much more than Drush Make. The strategy session we had was eye opening. Understanding the needs from an operations perspective, while not jeopardizing the integrity of the application led us down a path to see a problem that the wider development community at large has already solved (not just the PHP community). It's very rewarding to solve problems like this, especially when you come to the conclusion that someone has already solved the problem! "We just had to find the path to the water! (--A.W.)"
What do you think about using Drush Make vs Composer for pulling together a Drupal Application? Leave us your thoughts in the comments.
Tom Friedhof
Senior Software Engineer
Tom has been designing and developing for the web since 2002 and got involved with Drupal in 2006. Previously he worked as a systems administrator for a large mortgage bank, managing servers and workstations, which is where he discovered his passion for automation and scripting. On his free time he enjoys camping with his wife and three kids.
There are thousands of situations in which you do not want to reinvent the wheel. It is a well known principle in Software Engineering, but not always well applied/known into the Drupal world.
Let’s say for example, that you have a url that you want to convert from relative to absolute. It is a typical scenario when you are working with Web (but not just Web) crawlers. Well, you could start building your own library to achieve the functionality you are looking for, packaging all in a Drupal module format. It is an interesting challenge indeed but, unless for training or learning purposes, why wasting your time when someone else has already done it instead of just focussing on the real problem? Especially if your main app purpose is not that secondary problem (the url converter).
What’s more, if you reuse libraries and open source code, you’ll probably find yourself in the situation in which you could need an small improvement in that nice library you are using. Contributing your changes back you are closing the circle of the open source, the reason why the open source is here to stay and conquer the world (diabolical laugh here).
That’s another one of the main reasons why lot’s of projects are moving to the Composer/Symfony binomium, stop working as isolated projects and start working as global projects that can share code and knowledge between many other projects. It’s a pattern followed by Drupal, to name but one, and also by projects like like phpBB, ezPublish, Laravel, Magento,Piwik, …
Composer and friends
Coming back to our crawler and the de-relativizer library that we are going to need, at this point we get to know Composer. Composer is a great tool for using third party libraries and, of course, for contributing back those of your own. In our web crawler example, net_url2 does a the job just beautifully.
Nice, but at this point you must be wondering… What does this have to do with Drupal, if any at all? Well, in fact, as everyone knows, Drupal 8 is being (re)built following this same principle (DRY or don’t repeat yourself) with an strong presence of the great Symfony 2 components in the core. Advantages? Lots of them, as we were pointing out, but that’s the purpose of another discussion
The point here is that you don’t need to wait for Drupal 8, and what’s more, you can start applying some of this principles in your Drupal 7 libraries, making your future transition to Drupal 8 even easier.
Let’s rock and roll
So, using a php library or a Symfony component in Drupal 7 is quite simple. Just:
- Install composer manager
- Create a composer.json file in your custom module folder
- Place the content (which by the way, you’ll find quite familiar if you’ve already worked with Symfony / composer yaml’s):
"require": { "pear/net_url2": "2.0.x-dev" } - enable the custom module
And that’s it basically. At this point we simply need to tell drupal to generate the main composer.json. That’s basically a composer file generated from the composer.json found in each one of the modules that include a composer themselves.
Lets generate that file:
drush composer-rebuild
At this point we have the main composer file, normally in a vendor folder (if will depend on the composer manager settings).
Now, let’s make some composer magic :
drush composer update
At this point, inside the vendors folder we should now have a classmap, containing amongst others our newly included library.
Hopefully all has gone well, and just like magic, the class net_url2 is there to be used in our modules. Something like :
$base = new Net_URL2($absoluteURL);
Just remember to add the library to your class. Something like:
use Net_URL2;
In the next post we’ll be doing some more exciting stuff. We will create some code that will live in a php library, completely decoupled but at the same time fully integrated with Drupal. All using Composer magic to allow the integration.
Why? Again, many reasons like:
- Being ready for Drupal 8 (just lift libraries from D7 or D6 to D8),
- Decoupling things so we code things that are ready to use not just in Drupal, and
- Opening the door to other worlds to colaborate with our Drupal world, …
- Why not use Dependency Injection in Drupal (as it already happens in D8)? What about using the Symfony Service container? Or something more light like Pimple?
- Choose between many other reasons…
See you in my next article about Drupal, Composer and friends, on the meantime, be good :-).
Updated: Clarified that we are talking about PHP Libraries and / or Symfony components instead of bundles. Thanks to @drrotmos and @Ross for your comments.
About Drupal Sun
Drupal Sun is an Evolving Web project. It allows you to:
- Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
- Facet based on tags, author, or feed
- Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
- View the entire article text inline, or in the context of the site where it was created
See the blog post at Evolving Web
