Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Apr 05 2021
hw
Apr 05

This is the fourth post in my DrupalFest series and I am excited to keep it going. I want to write about different tools I am aware of for running quality checks on Drupal code. This will be somewhat similar to my last post where I presented various options but focused on the method I use nowadays.

First of all, what am I talking about? I believe that the code we write is read a lot more times than it is written. It is read by other people than yourselves (2-week older you is another person) and they (you) have to understand what is written. Therefore, code must be optimized for readability. Performant code is a must, of course, but not at the expense of readability. When your best-performing code breaks and you can’t understand it to fix it, that performance is useless.

Types of checks

One of the low hanging fruits here is following a consistent code style. Drupal documents its coding style in detail and the Drupal core and contributed modules (most of them anyway) follow it. True, it’s not like PSR-2 or PSR-12; it was developed long before there were PSR’s, but if you are working with Drupal, it will be a good idea to follow this coding style consistently. Now, you could manually review each and every line of your code to make sure you are following the code style and halt your pull requests, but that is not a good use of your time. Hence, tools.

Apart from the coding style, there are ways to prove “correctness” of your code. Broadly speaking, there are two aspects of correctness–the code is coherent and the business logic is correct. The coherent code aspect can be checked using various static analysis tools; PHPStan and Psalm are few of the popular ones. For verifying the business logic, we write automated tests and PHPUnit is the defacto standard for that.

Tools

Before we get into the Drupal site of things, I’ll list the various tools that we use. I won’t attempt to document them here (in the interest of time) but you shouldn’t have trouble finding their documentation.

  • PHP Code Sniffer – Mainly for code style checks but can be a bit more sophisticated using rules.
  • Drupal coder rules – Code sniffs for checking Drupal coding style.
  • DrupalPractice coder rules – Code sniffs for checking Drupal conventions.
  • PAReview.sh – Automated tool that calls PHPCS with Drupal-related coder sniffs apart from other checks. This is commonly used to check if your module is contribution ready.
  • PHPStan – Static analyzer for PHP code that can perform various levels of type checks and other correctness checks.
  • Psalm – This is another static analyzer with same basic level of checks as PHPStan and then some interesting checks.
  • PHPLint – A simple PHP linter which supports parallel checks and better error reporting than just running php.
  • Other linters such as ESLint, TwigCS, etc – Linters for their specific languages.
  • PHPCPD – PHP Copy Paste Detecter. Like the name says, it checks for repeated lines of code.
  • PHPMD – PHP Mess Detector. Analyze source code for various mathematic measures of code.
  • PHPUnit – Test runner for unit, integration, and functional tests.
  • Behat – Test runner for Behavior tests.

Almost all of these tools can be installed via composer or PHAR files. Composer is an easy way to get these tools but they end up affecting your project dependencies. PHAR files have to be installed on your machine (and everyone in the team should do that too). In both of these methods, you still have to remember to run the tools. That is where the next set of tools come in.

Automatically running checks

One of the ways to make sure everyone on the team adheres to the checks is by running the test in CI. This way, the team would immediately know if their commit is failing checks without even someone having to say so in a pull request. You could again choose to have all of these tools in your composer.json and install them while running your CI, but there is a better way for most cases. DrupalQA is a Docker image which packages almost all of the above tools and you can just run the commands you want within the Docker container. Almost all modern CI tools provide some way of running tests inside a container and you should find documentation to do that for your CI tool. Here is an example of a Gitlab CI job definition which uses the PHP 7.4 version of this image.

drupal_codequality:
  image: hussainweb/drupalqa:php7.4
  stage: test
  script:
    - composer validate
    - phplint --no-cache -v web/modules/custom/
    - phpcs --standard=phpcs.xml.dist --extensions=php,module,inc,install,test,profile,theme --ignore=/node_modules/ web/modules/custom
    - phpmd web/modules/custom/ text phpmd.xml

You could even run the Docker image locally but that’s more trouble than it’s worth. There is a better way in any case.

Have your team run checks automatically

The CI is fine to run these tests but what if you want to save time between commit, push, and CI runs. Wouldn’t it be better for developers to run these checks on their machines before they commit their code. GrumPHP is a tool which allows you to do just that. Vijay CS has written a wrapper on GrumPHP that provides default configuration for checks related to Drupal. While this is great for general use cases, I wanted to make a highly opinionated one for my team at Axelerant. While it is opinionated, it is still available for use and you would just install it using composer this way:

composer require axelerant/drupal-quality-checker

After this, anyone using your project would automatically get git hooks installed (once they run composer install) to run a few Drupal-specific checks every time they commit. You can modify the default configuration and add/remove any tools you wish. Just follow GrumPHP documentation for the same. Of course, more documentation is also available at the Github page for axelerant/drupal-quality-checker.

GrumPHP recently made a lighter package available called GrumPHP Shim. This provides the same set of features but installs a PHAR file instead of a regular composer plugin. This has the benefit of keeping your project dependencies free of GrumPHP’s dependencies reducing the chances of conflict. Axelerant’s version of drupal-quality-checker uses the shim for this reason.

I think I covered all the tools I am aware of and could recollect in the span of couple of hours today. I’m sure there are more tools that are missing here but I am going to call this DrupalFest post done for now. If I am missing something obvious (or even niche), please point it out in the comments so that I can revise or write about it in a separate post.

Jan 30 2019
Jan 30

Why not just use the .gitignore file?

Kaleem ClarksonPhoto by Tim Wright on Unsplash

As many of you know, I am a huge Pantheon hosting fanboy and can still remember the days during the beta launch of being blown away that I have three different environments out of the box, with dev, test and live. Another great service they added recently is that all sites receive SSL certificates automatically and all you have to do is redirect all traffic to use HTTPS. In Drupal 8 they suggest doing this in your settings.php file.

After adding the redirect code everything works great until you fire up your local environment (I am currently using Lando) and you are getting a blank screen. After further investigation, you notice it’s the redirect to HTTPS that is causing the issue. My first thought was to make sure my settings.local.php file was correctly being used but for the life of me, I could not get that file to override the redirect code in my settings.php file. If you are reading this and have a better idea on to how to accomplish this then let me know in the comments :)

My next thought was to simply add the settings.php file to my .gitignore file but when I went to my production website I was prompted to reinstall my Drupal site. When adding a file to .gitignore the repo pretends it doesn’t exist so therefore Drupal was telling me to reinstall. Whoooops, my production site kind of needs this file hahahah. So I thought to myself,

How can I ignore my settings.php file in my local repo but still have the original file on production?

Now that we understand what git update-index does, the real magic happens with the options that you can add to the command. In this case, the option that Ian Gloude suggested is the --skip-worktreeoption. The Git documentation explains that the skip worktree bit tells the git index to assume the file is unchanged from this point on regardless if there is an actual change. So what does this mean for us? It means you can change your file on your local environment while the original file on your production server remains unchanged.

Here is the command I use prior to uncommenting out the pantheon redirect code.

git update-index --skip-worktree /sites/default/settings.php

When I need to make some changes to the production settings.php file I can tell Git to watch the file again with this command.

git update-index —-no-skip-worktree web/sites/default/settings.php

Anyway, I hope this helps you keep your local and production environments running smoothly while maintaining your settings differently.

Nov 06 2018
Nov 06
Jody's desk

Hardware

After a long run on MacBook Pros, I switched to an LG Gram laptop running Debian this year. It’s faster, lighter, and less expensive. 

If your development workflow now depends on Docker containers running Linux, the performance benefits you’ll get with a native Linux OS are huge. I wish I could go back in time and ditch Mac earlier.

Containers

For almost ten years I was doing local development in Linux virtual machines, but in the past year, I’ve moved to containers as these tools have matured. The change has also come with us doing less of our own hosting. My Zivtech engineering team has always held the philosophy that you need your local environment to match the production environment as closely as possible. 

But in order to work on many different projects and accomplish this in a virtual machine, we had to standardize our production environments by doing our own hosting. A project that ran on a different stack or just different versions could require us to run a separate virtual machine, slowing down our work. 

As the Drupal hosting ecosystem has matured (Pantheon, Platform.sh, Acquia, etc.), doing our own hosting began to make less sense. As we diversified our production environments more, container-based local development became more attractive, allowing us to have a more light-weight individualized stack for each project.

I’ve been happy using the Lando project, a Docker-based local web development system. It integrates well with Pantheon hosting, automatically making my local environment very close to the Pantheon environments and making it simple to refresh my local database from a Pantheon environment. 

Once I fully embraced containers and switched to a Linux host machine, I was in Docker paradise. Note: you do not need a new machine to free yourself from OSX. You can run Linux on your Mac hardware, and if you don’t want to cut the cord you could try a double boot.

Philadelphia City Hall outside Jody's office
A cool office view (like mine of Philly’s City Hall) is essential for development mojo

Editor

In terms of editors/IDEs I’m still using Sublime Text and vim, as I have for many years. I like Sublime for its performance, especially its ability to quickly search projects with 100,000 files. I search entire projects constantly. It’s an approach that has always served me well. 

I also recommend using a large font size. I’m at 14px. With a larger font size, I make fewer mistakes and read more easily. I’m not sure why most programmers use dark backgrounds and small fonts when it’s obvious that this decreases readability. I’m guessing it’s an ego thing.

Browser

In browser news, I’m back to Chrome after a time on Firefox, mainly because the LastPass plugin in Firefox didn’t let me copy passwords. But I have plenty of LastPass problems in any browser. When working on multiple projects with multiple people, a password manager is essential, but LastPass’s overall crappiness makes me miserable.

Wired: Linux, git, Docker, Lando
Tired: OSX, Virtual machines, small fonts
Undesired: LastPass, egos

Terminal

I typically only run the browser, the text editor, and the terminal, a few windows of each. In the terminal, I’m up to 16px font size. Recommend! A lot of the work I do in the terminal is running git commands. I also work in the MySQL CLI a good deal. I don’t run a lot of custom configuration in my shell – I like to keep it pretty vanilla so that when I work on various production servers I’m right at home.

Terminal screenshot

Git

I get a lot of value out of my git mastery. If you’re using git but don’t feel like a master, I recommend investing time into that. With basic git skills you can quickly uncover the history of code to better understand it, never lose any work in progress, and safely deploy exactly what you want to.

Once I mastered git I started finding all kinds of other uses for it. For example, I was recently working on a project in which I was scraping a thousand pages in order to migrate them to a new CMS. At the beginning of the project, I scraped the pages and stored them in JSON files, which I added to git.  At the end of the project, I re-scraped the pages and used git to tell me which pages had been updated and to show me which words had changed. 

On another project, I cut a daily import process from hours to seconds by using git to determine what had changed in a large inventory file. On a third, I used multiple remotes with Jenkins jobs to create a network of sites that run a shared codebase while allowing individual variations. Git is a good friend to have.

Hope you found something useful in my setup. Have any suggestions on taking it to the next level?
 

Dec 13 2017
hw
Dec 13

Today, I came across an interesting bug with composer-patches plugin (it really is a git-apply bug/behavior). TL;DR, it is fixed in the latest version of composer-patches plugin as of this writing – 1.6.4. All you have to do is run composer require cweagans/composer-patches:^1.6.4 to get the fix.

The problem is simple: Patches are not applied even though you might see it in the composer log output. In fact, even a PATCHES.txt file is generated with the correct listing of patches. There are no errors and no indication that the patching failed for any reason. The problem is because git apply fails silently. It does not give any output nor sets an error exit code.

The problem was first documented and fixed in cweagans/composer-patches#165; however, the fix used there relies on git-apply outputting log messages saying patches were skipped. Unfortunately, that behavior was only introduced in 2.9.0. The Docker image I was using only contained version 2.1.4. All I needed was either these “Skipped patch” messages or an exit code and the plugin would fall back on patch command, but neither happened.

Also, this is actually supposed to be git behavior: git apply will fail to do anything when used within a local checkout of a git repository (other than the one for the project the patch is made for), such as if you are patching a module that is within a site that is in Git version control. The suggestion is to use patch -p1 < path/file.patch instead. While an obvious solution was to upgrade git, a simple Google search told me that this problem has happened to others and I decided to dig deeper. All variations of git apply simply didn't work, nor would set the exit code. I followed the debugging steps in one of the comments in the pull request, but I still wouldn’t see any messages similar to “Skipped patch” or applying patch, or any error.

In testing, I found a way to get it working with the ‘--directory‘ parameter to ‘git apply‘. The command executed by the plugin is similar to the following:


git -C package-directory apply -p1 /tmp/patchfile.patch

Instead, this command worked (even if there is no git repository involved at all, even in the current directory):


git apply --directory=package-directory apply -p1 /tmp/patchfile.patch

On digging a bit more, I found another workaround that worked for me in cweagans/composer-patches#175. What’s more, this was already merged and tagged. All I had to do was update to the latest version 1.6.4 and it would work. This change checks for the path being patched and if it actually is a git repository. If the package is not a git repository, it does not attempt a git apply at all, but falls back to patch, which works perfectly.

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web