Nov 28 2018
Nov 28

Once I got a task to fix a bug. While bug itself was easy to fix, I had to find the commit where it was introduced. To describe why I had to do it, I have to explain a bit our development process.

Our branching model

The exact branching and deployment workflow may differ from project to project, but we have two mainstream versions. One is for legacy amazee.io hosting and one is for Lagoon.

Here is the common part. The production instance always uses the latest prod branch. When we start to work on a new task, we create a new branch from prod. When the task is tested and demoed, we deploy it. Separately from other tasks.

We do this to speed up the delivery process, and to make our clients happy.

If a project lives on the legacy hosting system, it usually has PROD and DEV environments. For a task to be tested and demoed we have to deploy it to DEV first.

With Lagoon, we have a separate environment for each task, and this is awesome!

The bug I had to fix was on a project hosted on the legacy system. Also the bug was found on the DEV environment, and it was not present on PROD. So one of the active tasks introduced it (and at that time we had lots of active tasks). I had to find which one.

The bug

An element was appearing on a page, that it should not have appeared on.

The project

The backend is built with Drupal. The frontend is also Drupal, but we used progressive decoupling to embed dynamic Vue.js elements. In between - our beloved GraphQL. No test coverage (nooooooooooooooo.com) yet, but we have a plan to add it with some end-to-end testing framework. Most probably it will be Cypress.

Cypress

It's a modern e2e testing framework. It has lots of cool features, and some of them, like time traveling, help you not only to write tests but to develop in general. Just watch the 1-minute video on the Cypress website and you'll love it.

Git bisect

This is a very easy and very powerful Git tool. To make it work, you just need to give it three things:

  • a commit where things are good
  • a commit where things are bad
  • a command to test if things are good or bad

The result would be the first bad commit.

Docs: https://git-scm.com/docs/git-bisect

The search

Finally, I can share my experience in combining these two tools.

Since we don't yet use Cypress on the project, I installed it globally on my machine with npm i -g cypress and created cypress.json in project root with {} contents. That's all Cypress needed.

To run Git bisect, I used the following commands:

The my_test.sh was looking like this:

(I actually was lucky that for Drupal I only had to run cache clear after each Git jump. If, for example, there would be Drupal core updates in between bad and good commits, then running drush cr would not work. But in this case I could install Drupal every time from an existing configuration. It would have been a bit slower.)

And here is the Cypress test which I put into the path/to/vue/cypress/integration/test.js file:

It took a little time to set this all up. The result was good - I was able to identify the commit in which the bug was introduced.

Sum up

Modern e2e testing frameworks are easy to set up and use. They can do more than just automated testing. All it takes is some your imagination.

For example, once a colleague of mine had a task to do a content update on a project using an Excel file as a source. One way to do it was to do everything by hand, copy-pasting the data. The other way would be to write a one time importer. But instead, he turned the Excel file into JSON data and used TestCafe to do the click-and-paste job. This was faster than the first two options. And it was quite cool to see the visualization of the automated task - it's so nice when you can see the result of your work.
 

Oct 17 2016
Oct 17

In this post we are going to share some Composer recipes we collected while working with Drupal projects created from the Drupal Composer template. Also, we will take a look on how to convert an existing Drupal project to a Composer project.

If you still don't use Composer for managing Drupal projects, you should start doing this right now! The Drupal Composer template will help you to set things up. It's really easy to setup a new project.

If you’re still not convinced, check out the benefits of the Drupal Composer workflow:

  • No need to store contrib code (and the core!) in your version control system.
  • A single package management tool for everything: Drupal core, contrib modules, JS libraries, your own shared modules, etc.
  • Patching of core and modules is easier than ever.
  • It's way simpler than git submodules.

(All recipes consider Drupal 8, but they should work for Drupal 7 as well)

Installing contrib modules

  • composer require drupal/<MODULE_NAME>:~8.0 to get the latest stable version (or latest dev, if there is no “stable release")
  • composer require drupal/<MODULE_NAME>:dev-<BRANCH_NAME> to get the latest dev version
  • composer require drupal/<MODULE_NAME>:dev-<BRANCH_NAME>#<COMMIT_HASH> to get the exact version

Updating Drupal core/modules

  • composer update to update everything
  • composer update --dry-run to check for updates
  • composer update drupal/<MODULE_NAME> to update a single module

Patching packages

The cweagans/composer-patches plugin (which comes with the Drupal Composer template) will take patches from the "extra" section of the composer.json file:

   "extra": {
       "patches": {
           "<PACKAGE/NAME>": {
               "<PATCH DESCRIPTION>": "<PATH/TO/PATCH/OR/URL>",
               ...
           },
           ...
       }
   }

Example:

   "extra": {
       "patches": {
           "drupal/core": {
               "Fix language detection": "patches/2189267-24.patch"
           }
       }
   }

After a new patch is added run:

  • composer install to apply patch
  • composer update nothing (or composer update --lock) to make the composer-patches plugin write necessary changes to the composer.lock file

Installing custom/forked modules from Github

For the case when a module repository contains its own composer.json

Register the repository in the "repositories" section of the composer.json file:

   "repositories": [
       {
           "type": "vcs",
           "url": "https://github.com/<REPOSITORY/NAME>"
       },
       ...
   ],

Use composer require drupal/<MODULE_NAME>:dev-<BRANCH_NAME>#<COMMIT_HASH> to install the module.

For the case when the composer.json file is missing from the module repository

You'll need to use a bit more verbose variant:

   "repositories": [
       {
           "type": "package",
           "package": {
               "name": "drupal/<MODULE_NAME>",
               "version": "dev-custom",
               "type": "drupal-module",
               "source": {
                   "type": "git",
                   "url": "[email protected]:<REPOSITORY/NAME>.git",
                   "reference": "<BRANCH-NAME>"
               }
           }
       },
       ...
   ],

Use composer require drupal/<MODULE_NAME>:dev-custom#<COMMIT_HASH> to install the module.

For the case when the destination path should be different than modules/contrib

In addition to the above recipes, use the composer/installers plugin:

   "extra": {
       "installer-paths": {
           "web/modules/custom/<MODULE_NAME>": ["drupal/<MODULE_NAME>"],
           ...
       }
   }

Adding a JS library

Most popular libraries can be added easily with composer as they exist on Packagist. The tricky part is that most Drupal modules require that libraries are saved under the "libraries" directory while Composer installs them to "vendor". The composer/installers plugin can override package paths, but only for packages that depend on it. So, you'll need to override the composer.json file of the library stating that it has the composer/installers dependency.

Let's take a look at an example:

   "repositories": [
       {
           "type": "package",
           "package": {
               "name": "enyo/dropzone",
               "version": "4.3",
               "type": "drupal-library",
               "source": {
                   "url": "https://github.com/enyo/dropzone.git",
                   "type": "git",
                   "reference": "master"
               },
               "dist": {
                   "url": "https://github.com/enyo/dropzone/archive/v4.3.0.zip",
                   "type": "zip"
               },
               "require": {
                   "composer/installers": "~1.0"
               }
           }
       },
       ...
   ],  
   ...   
   "extra": {
       "installer-paths": {
           "web/libraries/{$name}" : ["type:drupal-library"],
           ...
       }
   }

After the above is added to the composer.json, run composer require enyo/dropzone:4.3 to download the library. Also, here we have used the exact version and added the "dist" section to make it possible for Composer to download a zip archive instead of cloning the Git repository.

Switch a dependency package to a forked version

Add the forked repository to the composer.json:

   "repositories": [
       {
           "type": "vcs",
           "url": "https://github.com/<REPOSITORY/NAME>"
       },
       ...
   ],

Run composer require <PACKAGE/NAME>:dev-<BRANCH_NAME>#<COMMIT_HASH>

Update an existing Drupal 8 project to use Composer

  • Make a backup ;)
  • Delete everything that will be managed by Composer: Drupal's "core" folder, contrib modules, etc.
  • Delete all Drupal "root" files, such as index.php, update.php, README.txt... All of them.
  • Create "web" directory in the project root and move all remaining Drupal folders (sites, modules, themes, libraries, profiles, etc.) into it.
  • Copy the Drupal Composer template files to the project root.
  • Prepare a list of Drupal core and contrib module versions (and everything else that will be managed by Composer) that are used currently on the project. Then run composer require specifying every dependency with the exact version. You'll need to convert Drupal versions into Composer versions, here are some examples:
    • drupal/core:8.1.8 is obvious
    • drupal/admin_toolbar:8.1.15 refers to admin_toolbar 8.x-1.15
    • drupal/ctools:8.3.0-alpha26 refers to ctools 8.x-3.0-alpha26
    • drupal/config_installer:dev-8.x-1.x#a16cc9acf84dd12b9714def53be0ce280a5b0c1a refers to config_installer dev snapshot made from the a16cc9a commit of the 8.x-1.x branch
  • Manually update versions of Drupal core and contrib modules to "~8.0" in the "require" section of the composer.json file. That will make updates possible.
  • Run composer drupal-scaffold which will create the Drupal "root" files.
  • Make your webserver use "web" directory as the web root.
Apr 28 2015
Apr 28

Reasons for language fallback

Let's assume you have a website divided by countries. The site structure is:

  • Global (en, de, fr)
  • Germany (de)
  • France (fr)
  • Switzerland (de, fr)

You have your content translated to three languages. Normally, this works, but there could be cases when you need languages per country. Words might have a slightly different meaning from country to country (examples) or spelling might be different (en-US vs en-GB, or using "ß" in de-DE vs "ss" in de-CH). Or, for example, the "Contact us" page can contain a country specific information - locations.

So, the site structure can be turned to:

  • Global (en, de, fr)
  • Germany (de-DE)
  • France (fr-FR)
  • Switzerland (de-CH, fr-CH)

This can bring a translation nightmare until you have a language fallback ;)

Languages

Having the language fallback, you would only translate strings/content to the "base" languages and, in special cases, you may also translate to "country" languages.

Fortunately, there is a module for that. The Language fallback.

Language fallback 7.x-1.x

The first version of the module provides language fallback only for locale strings (strings that are passed through the t() function). After the module installation, you can find the "Language fallback" option at the language edit form.

Edit language

If a string translation is missing for the de-CH language, the translation from the fallback language (de) will be used in this case.

How the locale fallback works

There is a strings override feature in Drupal 7 core. Basically, you can define translations for some strings via variables. For example, you can put something like that into your settings.php file:

$conf['locale_custom_strings_de-CH']['']['Home'] = 'Startseite';

While this possibility is used really rarely, the Language fallback module transforms it into the real power. It saves instances of the localeWithFallback class to locale_custom_strings_* variables. The localeWithFallback implements ArrayAccess, thus it can return string translations automagically.

Language fallback 7.x-2.x

We also want to have the language fallback for our content translated via the Entity Translation module. The second version of the Language fallback module provides this feature.

Language fallback settings

This option enables the translation fallback for entities on the field level. For example, if an entity does not have a certain field translated to the de-CH language, the de translation of this field will be used for the entity rendering.

How the entity translation fallback works

Actually, this ability is provided by the core Locale module. The Language fallback module just hooks into the process.

Here is the call stack describing how it works:

  • field_language()
  • hook_field_language_alter()
  • locale_field_language_alter()
  • locale_field_language_fallback()
  • language_fallback_get_candidates()
  • hook_language_fallback_candidates_alter()
  • language_fallback_language_fallback_candidates_alter()

The last function does the job.

Language fallback chains

That's another feature only available from language fallback 7.x-2.x.

Edit language

Now, on the language edit form, you can define several fallback languages in a particular order.

Language fallback 7.x-2.x-amazee

If you try to use the language-per-country workflow with the Language fallback module, the first thing you'll miss is the i18n support. The Internationalization module helps us to deal with the translation of menu links, field labels, panels, etc. So, we started work in this direction and have prepared a patch.

After the patch was tested, we found that it would be quite useful to have the fallback information displayed right on the translation overview pages. We implemented this feature, too.

Before:

I18n overview, no patch

After:

I18n overview with patch

We've submitted a patch containing both improvements to #2322883: Add support for i18n strings.

After using the new overview pages for some time, we decided to also improve the entity translation overview pages, since it improves the usability for content editors.

Entity translations overview

This developed into another patch #2444203: Show fallback information on the translation overview. This time, for the Entity translation module. (Be sure to enable the "Show fallback statuses on overview pages" checkbox on the admin/config/regional/entity_translation page after you have applied the patch.)

Language fallback for path aliases

There was an issue we have met implementing language fallback on our projects. The Pathauto module only generates aliases for the existing node translations. To fill that gap we have created the Path alias force module. It forces creation of aliases for all languages. Furthermore, it supports language_fallback, so aliases respect the fallback rules which is nice.

Total language fallback

Having all these patches applied, we have the complete solution for the language fallback on Drupal 7 websites. Now the language-per-country workflow can be used for real projects.

BTW. If you don't want to apply patches, you can use the Amazee Labs repositories which contain all required changes:

Let us know if you have any issues!

Dec 02 2014
Dec 02

It must be weird, but living in Russia I have never attended a russian Drupal event. I was at DrupalCon Prague 2013 and have attended ukrainian DrupalCamps several times before. The Ukraine is located much closer to the town where I live than to the russian capital, so for me it’s faster to get to the neighbour country rather than to visit Moscow.

Amazee Labs travels back to the USSR

This time I decided to visit a russian Drupal event, DrupalCamp Moscow 2014. I didn’t know what to expect. Another country means another people. But with the Drupal community, this rule never works. Drupal folks are pretty much the same all around the world: sociable, nice, friendly, and always ready to help! If you are a professional drupalist, or a newbie, or even if you know nothing about Drupal… You are always welcome!

And this is exactly what Boris and Corina were talking about in their blog posts “Be a part of the community” and “Being part of the community - a non-techie perspective”.

@duozersk ???????????? ??? #angularJS ? #drupal http://t.co/KbMheeFSY6 #dcmsk pic.twitter.com/Cqrl6yLifb

— Nikolay Shapovalov (@RuZniki) 29. November 2014

The sessions I attended were good. I learned how russian drupalists work with Solr, which we use a lot at Amazee Labs, learned some new techniques for high-performance sites, went to Drupal 8 theming, and even learned some cases of using AngularJS with Drupal.

Speaking at Moscow State University feeling like a professor :)Speaking at Moscow State University, feeling like a professor

For us at Amazee Labs it is an essential part of our company culture to contribute back to the community. We are a proud sponsor of the event, and I shared our knowledge and know-how by giving a presentation about Drupal 8 configuration management. My next post will be about it. Subscribe to our RSS FeedTwitter, or Facebook page to not miss it ;)

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web