Oct 28 2018
Oct 28

In 2017, Drupal Association decided not to host a DrupalCon Europe 2018 due to waning attendance and financial losses. They took some time to make the European event more sustainable. After this, the Drupal community decided to organise a Drupal Europe event in Darmstadt, Germany in 2018. My colleagues and I joined the biggest European Drupal event in October and here is my summary of few talks I really enjoyed!

Driesnote

By Dries Buytaert
Track: Drupal + Technology
Recording and slides

This year, Dries Buytaert focuses on improvements made for Drupal users such as content creators, evaluators and developers.

Compared to last year, Drupal 8 contributions increased by 10% and stable modules released by 46%. Moreover, a steady progress is noticeable. Especially in many core initiatives like the last version of Drupal 8 which is shipped with features and improvements created from 4 core initiatives.

Content creators are the key-decision makers in the selection of a CMS now. Their expectations have changed: they need flexibility but also simpler tools to edit contents. The layout_builder core module gives some solutions by enabling to edit a content inline and drag-and-dropping elements in different sections. The management of medias has been improved too and there is a possibility to prepare different “states” of contents using workspaces module. But the progress doesn’t stop here. The next step is to modernize the administrative UI with a refresh of the Seven administration theme based on React. Using this modern framework makes it familiar to Javascript (JS) developers and is building a bridge with the JS community.

Drupal took a big step forward for evaluators as it provides a demo profile called “Umami” now. Evaluators have a clear understanding of what kind of websites can be produced by Drupal and how it works by navigating through the demo website.
The online documentation on drupal.org has also been reorganized with a clear separation of Drupal 7 and Drupal 8. It provides some getting-started guides too. Finally, a quick-install link is available to have a website running within 3 clicks and 1 minute 27 seconds!

Developers experience has been improved as well: minor releases are now supported for 12 months instead of the former 4 weeks. Teams will have more time to plan their updates efficiently. Moreover, Gitlab will be adopted within the next months to manage the code contributions. This modern collaborative tool will encourage more people to participate to projects.

Regarding the support of the current Drupal versions, Dries shares that Symfony 3, the base component of Drupal 8 will be end-of-life by 2021. To keep the CMS secure, it implies to be end-of-life by November 2021 and Drupal 9 should be released in 2020. The upgrade from Drupal 8 to Drupal 9 should be smooth as long as you stay current with the minor releases and don’t use modules with deprecated APIs.
The support of Drupal 7 has been extended to November 2021 as the migration path from Drupal 7 to Drupal 8 is not stable with multilingualism yet.

This is a slide from Driesnote presentation showing a mountain with many tooltips: "Drupal 8 will be end-of-life by November 2021", "Drupal 7 will be supported until November 2021", "Drupal 9 will be released in 2020", "Drupal 8 became a better tool for developers", "You now have up to 12 months to upgrade your sites", "Drupal 8 became much easier to evaluate", "We've begun to coordinate the marketing of Drupal", "Drupal 8 became easier to use for content creators", "Drupal.org is moving to GitLab very soon".Slide from Driesnote showing current state of Drupal.

Last but not least, DrupalCon is coming back next year and will be held in Amsterdam!

JavaScript modernisation initiative

By Cristina Chumillas, Lauri Eskola, Matthew Grill, Daniel Wehner and Sally Young
Track: Drupal + Technology
Recording and slides

After a lot of discussions on which JS framework will be used to build the new Drupal administrative experience, React was finally chosen for its popularity.

The initiative members wanted to focus on the content editing experience. This affects a big group of Drupal users. The goal was to simplify and modernize the current interface. Furthermore, embracing practices that are familiar to JS developers so they can easier join the Drupal community.
On one hand, a UX team ran some user tests. Those showed that users like the flexibility they have with Drupal interface but dislike its complexity usually. A comparative study was ran to know what has been used in other tools or CMSs too. On the other hand, the User Interface (UI) team worked on the redesign of the administrative interface and built a design system based on components. The refreshment of the Seven administration theme is ongoing.
Another group worked on prototyping the User Experience (UX) and User Interface (UI) changes with React. For instance, if an editor quits a page without saving they's last changes, a popup appears to restore the last changes. This is possible due to contents stored to the state of the application.

You can see a demo of the new administrative UI in the video (go to 20 minutes 48 seconds):

[embedded content]Demo of the new administrative UI in Drupal 8

If you are interested, you can install the demo and of course join the initiative!

Drupal Diversity & Inclusion: Building a stronger community

By Tara King and Elli Ludwigson
Track: Drupal Community
Recording

Diversity in gender, race, ethnicity, immigration status, disability, religion etc. helps a lot. Proven it makes a team more creative, collaborative and effective.

Tara King and Elli Ludwigson who are part of the Drupal Diversity and Inclusion team presented how Drupal is building a stronger and smarter community. The initial need was to make Drupal a safer place for all. Especially for the less visible ones at community events such as women, minorities and people with disabilities.
The group addressed several issues, such as racism, sexism, homophobia, language barriers etc. with different efforts and initiatives. For example, diversity is highlighted and supported in Drupal events: pronoun stickers are distributed, #WeAreDrupal hashtag is used on Twitter and social events are organized for underrepresented people as well. Moreover, the group has released an online resource library, which collects articles about diversity. All of this is ongoing and new initiatives were created. Helping people finding jobs or attracting more diverse people as recruiters are only two to name.

Flyer put on a table with the text "Make eye Contact. Invite someone to join the conversation. Consider new perspectives. Call out exclusionary behavior. Be an ally at Drupal events."Diversity and Inclusion flyer, photo by Paul Johnson, license CC BY-NC 2.0 Sign mentionning "All-gender restrooms" at Drupal Europe venue.All-gender restrooms sign, photo by Gábor Hojtsy, license CC BY-SA 2.0

If you are interested in the subject and would like to be involved, there are weekly meetings in #diversity-inclusion Drupal Slack channel. You can join the contrib team or work on the issue queue too.

Willy Wonka and the Secure Container Factory

By Dave Hall
Track: DevOps + Infrastructure
Recording

Docker is a tool that is designed to create, deploy and run applications easily by using containers. It is also about “running random code downloaded from the internet and running it as root”. This quote points out how it is important to maintain secure containers. David Hall illustrates this with practical advice and images from the “Willy Wonka and the chocolate factory” movie. Here is a little recap:

  • Have a light image: big images will slow down deployments and also increase the attack surface. Install an Alpine distribution rather than a Debian which is about 20 times lighter;
  • Check downloaded sources very carefully: for instance, you can use wget command and validate checksum for a file. Plus you can scan your images to check vulnerabilities using tools like Microscanner or Clair;
  • Use continuous development workflows: build a plan to maintain your Docker images, using a good Continous Integration / Continous Delivery (CI/CD) system and document it;
  • Specify a user in your dockerfile: running root on a container is the same as running root on the host. You need to reduce the actions of a potential attacker;
  • Measure your uptime in hours/days: it is important to rebuild and redeploy often to potentially avoid having a compromised system for a long time.

Now you are able to incorporate these advice into your dockerfiles in order to build a safer factory than Willy Wonka’s.

Decoupled Drupal: Implications, risks and changes from a business perspective

By Michael Schmid
Track: Agency + Business
Recording

Before 2016, Michael Schmid and his team worked on fully Drupal projects. Ever since they are working on progressive and fully decoupled projects.
A fully decoupled website means that frontend is not handled with Drupal but with a JS framework such as React. This framework is “talking” to Drupal via an API such as GraphQL. It also means, that all interactions from Drupal are gone: views with filters, webforms, comments etc. If a module provides frontend, it is not useable anymore and needs to be somehow re-implemented.
When it comes to progressive decoupled websites, frontend stack is still built with Drupal. But some parts are implemented with a JS framework. You can have data provided by APIs or injected from Drupal too. The advantage is that you can benefit from Drupal components and don’t need to re-implement everything. A downside of it are conflicts with CSS styling and build systems handled on both sides. Therefore you need to have a clear understanding of what does what.

To be able to run such projects successfully, it is important to train every developer in new technologies: JS has evolved and parts of the logic can be built with it. We can say that backenders can do frontend now. In terms of hiring it means, you can hire full stack developers but also JS engineers. Attracting more developers as they love working with JS frameworks such as React on a global level.

Projects are investments which continue over time and expect failures at the beginning. These kinds of projects are more complex than regular Drupal ones, they can fail or go over budget. Learn from your mistakes and share them with your team in retrospectives. It is also very important to celebrate successes!
Clients request decoupled projects to have a faster and cooler experience for users. They need to understand that this is an investment that will pay off in the future.

Finally, fully decoupled Drupal is a trend for big projects and other CMSs are already using decoupled out of the box. Drupal needs to focus on a better editor experience and a better API. There might also be projects that require simple backend edition instead of Drupal.

Hackers automate but the Drupal Community still downloads updates on drupal.org or: Why we need to talk about Auto Updates

By Joe Noll and Hernani Borges de Freitas
Track: Drupal + Technology
Recording and slides

In 2017, 59% of Drupal users were still downloading modules from drupal.org. In other words, more than half of the users didn’t have any automatisation processes to install modules. Knowing that critical security updates were released in the past months and it is only a matter of hours until a website gets potentially hacked, it comes crucial to have a process to automate these updates.
The update can be quite complex and may take time: installing the update, reviewing the changes, deploying on a test environment, testing either automatically or manually and deploying on production. However this process can be simplify with automation in place.

There is a core initiative to support small-to-medium sites owners that usually are not taking care of security updates. The idea is a process to download the code and update sources in the Drupal directory.
For more complex websites, automating the composer workflow with a CI pipeline is recommended. Everytime a security update is released, the developer pushes it manually in the pipeline. The CI system builds an installation containing the security fix within a new branch. This will be deployed automatically to a non-productive environment where tests can be done and build approved. Changes can be merged and deployed on production afterwards.

A schema showing the update strategy through all steps from a CI pipelineUpdate strategy slide by Joe Noll and Hernani Borges de Freitas

To go further, the update_runner module focuses on automatizing the first part by detecting an update and firing up a push for an update job.

Conclusion

Swiss Drupal community members cheering at a restaurantMeeting the Swiss Drupal community, photo by Josef Dabernig, license CC BY-NC-SA 2.0

We are back with fresh ideas, things we are curious to try and learnings from great talks! We joined social events in the evenings too. Therefore we exchanged with other drupalists, in particular with the Swiss Drupal community! This week went so fast. Thank you Drupal Europe organizers for making this event possible!

Header image credits: Official Group Photo Drupal Europe Darmstadt 2018 by Josef Dabernig, license CC BY-NC-SA 2.0.

Apr 12 2017
Apr 12

As a Swiss-based Drupal Agency, we have to create a lot of multilingual sites. Since Switzerland has three official languages (German, French, Italian) and even one more national language (Rumantsch), we are used to this requirement and we found our way with Drupal to make this an easy task (usually). We mainly used node translations in Drupal 7 for maximum flexibility. We used to separate languages from each other using the various i18n modules, language specific menus, blocks, URL-patterns, terms and so on.

With Drupal 8, things changed.
I struggled a little doing multilingual sites in Drupal 8 the same way I was used to in Drupal 7 because node translation is not available anymore (which is good) so I had to find another way to achieve the same easy to handle translations system. For us and for our clients. Let me explain, what I have learned.

Drupal 8 multilanguage

Image: drupal8multilingual.org

Drupal 8 issues multilanguage challenges

Challenge 1: Node add / edit menu handling

The main challenge I had using Drupal 8, was the ease to build your menus directly from the node creation page. You can do it, but only for the initial language. If you try to add a translated node to another menu or rename the item, it always ends up moving / renaming the source node instead of adding a link to the translation. So it can become quite confusing building a navigation directly from the node creation page or to add translations to the menu. A workaround was to add all navigation items manually in the menu administration if you are using a menu per language. With lots of languages and menus / items, this is not really a convenient task. Fortunately, translations from the node creation page have been implemented with a later release of Drupal 8.

Challenge 2: Untranslated Nodes show up in Menu

Another thing which bothered me was that untranslated nodes show up in the navigation (if you use only one menu). This can be quite confusing since most of the times not every page is translated in every language. Or in some languages, you need a little more than in others. You can read a lot about this topic and the reasons behind (e.g. here and here). However you do it, it’s always wrong in some situations and perfectly fine in others. But to be “limited” and “locked in” to a certain way is not nice and you have to deal with it. To sum up, once a node is put into a menu, it will show up everywhere. Regardless if there are translations or not.

Challenge 3: Language Switcher shows all languages – always.

Somewhat confusing is the Language Switcher. In Drupal 7, a language link was not available or strikethrough if there was no translation available. In Drupal 8, every language is always visible and linked. So if you look on a German page which is only available in German, the language switcher will present you all language links to the same node. A click on those language links mainly changes the interface language but the node content remains the same (since not translated). Usually also with a drupalish URL (node/xxxx) because there is no translation for the node and therefore also no URL alias available. This behavior is confusing and wrong in my point of view

An example to illustrate the above-written challenges.

multilanguage issues with Drupal 8

English Front-Page with mixed navigation items.

The screen above shows an installation with 2 languages (English and German). The English Page is a basic page which has a translation. English is selected. If you choose Deutsch on the language switcher, the English Page becomes Deutsche Seite (see image below) and shows the German content. So far so good. But the second menu item you see with the title Über uns (nur Deutsch) should not appear here since it’s only available in German. But it does. And if you actually go on this page, you will see the German text with everything English around it and no URL-Alias (/node/2 in this example). This is usually not very useful for us.

multilanguage issues with Drupal 8

German only Page – Language Switcher visible.

Also, the language switcher shown in the image above is from my point of view wrong or not very useful. It shows a link to the English version, but there is no English translation for this node. So why is it there? To see a German page with English decoration? Not sure. But I want to get rid of this link or at least modify it to be stroked through if the language is not available.

How to fix improve this?

Luckily, the Drupal community is always good for help. After some “research” on the web, I finally found (besides lots of discussions and comments in the issue queues) a way to achieve the desired setup.

To sum up again: I want to see only menu items which are available in my language and only see a link to another language, if a translation is available.

Since there is no patch and still some ongoing discussions on drupal.org you need to implement it on your own. Implement the following two modules.

Hide untranslated menu items

Code from https://www.drupal.org/node/2466553#comment-11991690. Credits go to michaelkoehne.

<?php use Drupal\Core\Menu\MenuLinkInterface; use Drupal\menu_link_content\Plugin\Menu\MenuLinkContent; use Drupal\Core\Language\LanguageInterface; /** * Implements hook_preprocess_menu(). */ function MYMODULE_preprocess_menu(&$variables) { if ($variables['menu_name'] == 'main') { $language = Drupal::languageManager() ->getCurrentLanguage(LanguageInterface::TYPE_CONTENT) ->getId(); foreach ($variables['items'] as $key => $item) { if (!$variables['items'][$key] = MYMODULE_checkForMenuItemTranslation($item, $language)) { unset($variables['items'][$key]); } } } } function MYMODULE_checkForMenuItemTranslation($item, $language) { $menuLinkEntity = MYMODULE_load_link_entity_by_link($item['original_link']); if ($menuLinkEntity != NULL) { $languages = $menuLinkEntity->getTranslationLanguages(); // Remove links which are not translated to the current language. if (!array_key_exists($language, $languages)) { return FALSE; } else { if (count($item['below']) > 0) { foreach ($item['below'] as $subkey => $subitem) { if (!$item['below'][$subkey] = MYMODULE_checkForMenuItemTranslation($subitem, $language)) { unset($item['below'][$subkey]); } } } return $item; } } } function MYMODULE_load_link_entity_by_link(MenuLinkInterface $menuLinkContentPlugin) { $entity = NULL; if ($menuLinkContentPlugin instanceof MenuLinkContent) { $menu_link = explode(':', $menuLinkContentPlugin->getPluginId(), 2); $uuid = $menu_link[1]; $entity = \Drupal::service('entity.repository') ->loadEntityByUuid('menu_link_content', $uuid); } return $entity; }

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

<?php

use Drupal\Core\Menu\MenuLinkInterface;

use Drupal\menu_link_content\Plugin\Menu\MenuLinkContent;

use Drupal\Core\Language\LanguageInterface;

/**

* Implements hook_preprocess_menu().

*/

function MYMODULE_preprocess_menu(&$variables) {

  if ($variables['menu_name'] == 'main') {

    $language = Drupal::languageManager()

      ->getCurrentLanguage(LanguageInterface::TYPE_CONTENT)

      ->getId();

    foreach ($variables['items'] as $key => $item) {

      if (!$variables['items'][$key] = MYMODULE_checkForMenuItemTranslation($item, $language)) {

        unset($variables['items'][$key]);

      }

    }

  }

}

function MYMODULE_checkForMenuItemTranslation($item, $language) {

  $menuLinkEntity = MYMODULE_load_link_entity_by_link($item['original_link']);

  if ($menuLinkEntity != NULL) {

    $languages = $menuLinkEntity->getTranslationLanguages();

    // Remove links which are not translated to the current language.

    if (!array_key_exists($language, $languages)) {

      return FALSE;

    }

    else {

      if (count($item['below']) > 0) {

        foreach ($item['below'] as $subkey => $subitem) {

          if (!$item['below'][$subkey] = MYMODULE_checkForMenuItemTranslation($subitem, $language)) {

            unset($item['below'][$subkey]);

          }

        }

      }

      return $item;

    }

  }

}

function MYMODULE_load_link_entity_by_link(MenuLinkInterface $menuLinkContentPlugin) {

  $entity = NULL;

  if ($menuLinkContentPlugin instanceof MenuLinkContent) {

    $menu_link = explode(':', $menuLinkContentPlugin->getPluginId(), 2);

    $uuid = $menu_link[1];

    $entity = \Drupal::service('entity.repository')

      ->loadEntityByUuid('menu_link_content', $uuid);

  }

  return $entity;

}

Hide untranslated languages in language switcher

Code from https://www.drupal.org/node/2791231#comment-12004615 (slightly adapted. Links get a class, not removed by default). Credits to Leon Kessler.

<?php /** * @file * Hide language switcher links for untranslated languages on an entity. */ use Drupal\Core\Entity\ContentEntityInterface; /** * Implements hook_language_switch_links_alter(). */ function MYOTHERMODULE_language_switch_links_alter(array &$links, $type, $path) { if ($entity = MYOTHERMODULE_get_page_entity()) { $new_links = array(); foreach ($links as $lang_code => $link) { try { if ($entity->getTranslation($lang_code)->access('view')) { $new_links[$lang_code] = $link; } } catch (\InvalidArgumentException $e) { // This language is untranslated so do not add it to the links. $link['attributes']['class'][] = 'not-translated'; $new_links[$lang_code] = $link; } } $links = $new_links; // If we're left with less than 2 links, then there's nothing to switch. // Hide the language switcher. if (count($links) < 2) { $links = array(); } } } /** * Retrieve the current page entity. * * @return Drupal\Core\Entity\ContentEntityInterface * The retrieved entity, or FALSE if none found. */ function MYOTHERMODULE_get_page_entity() { $params = \Drupal::routeMatch()->getParameters()->all(); $entity = reset($params); if ($entity instanceof ContentEntityInterface) { return $entity; } return FALSE; }

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

<?php

/**

* @file

* Hide language switcher links for untranslated languages on an entity.

*/

use Drupal\Core\Entity\ContentEntityInterface;

/**

* Implements hook_language_switch_links_alter().

*/

function MYOTHERMODULE_language_switch_links_alter(array &$links, $type, $path) {

  if ($entity = MYOTHERMODULE_get_page_entity()) {

    $new_links = array();

    foreach ($links as $lang_code => $link) {

      try {

        if ($entity->getTranslation($lang_code)->access('view')) {

          $new_links[$lang_code] = $link;

        }

      }

      catch (\InvalidArgumentException $e) {

        // This language is untranslated so do not add it to the links.

        $link['attributes']['class'][] = 'not-translated';

        $new_links[$lang_code] = $link;

      }

    }

    $links = $new_links;

    // If we're left with less than 2 links, then there's nothing to switch.

    // Hide the language switcher.

    if (count($links) < 2) {

      $links = array();

    }

  }

}

/**

* Retrieve the current page entity.

*

* @return Drupal\Core\Entity\ContentEntityInterface

*   The retrieved entity, or FALSE if none found.

*/

function MYOTHERMODULE_get_page_entity() {

  $params = \Drupal::routeMatch()->getParameters()->all();

  $entity = reset($params);

  if ($entity instanceof ContentEntityInterface) {

    return $entity;

  }

  return FALSE;

}

Please note: The code above is from Drupal.org and therefore thanks to the original authors linked above.

Enable those two modules and you’re all set!

I did not encounter any issues yet using those two modules. If ever something changes in the way Drupal handles those cases, you just need to switch off the modules and everything should be back to normal. So nothing to lose right?

There are other attempts to this by altering the menu block. One of them is Menu Block Current Language but I had no luck with this one. On my most recent project, it worked with one menu but not if you separate your menu by two blocks (different starting levels).

I would love to hear how you guys handle those cases or how you deal with I18N in general. I’m sure there are a gazillion other ways to do it.

Apr 11 2017
Apr 11

As a Swiss-based Drupal Agency, we have to create a lot of multilingual sites. Since Switzerland has three official languages (German, French, Italian) and even one more national language (Rumantsch), we are used to this requirement and we found our way with Drupal to make this an easy task (usually). We mainly used node translations in Drupal 7 for maximum flexibility. We used to separate languages from each other using the various i18n modules, language specific menus, blocks, URL-patterns, terms and so on.

With Drupal 8, things changed.

I struggled a little doing multilingual sites in Drupal 8 the same way I was used to in Drupal 7 because node translation is not available anymore (which is good) so I had to find another way to achieve the same easy to handle translations system. For us and for our clients. Let me explain, what I have learned.

Drupal 8 multilanguage

Image: drupal8multilingual.org

Drupal 8 issues multilanguage challenges

Challenge 1: Node add / edit menu handling

The main challenge I had using Drupal 8, was the ease to build your menus directly from the node creation page. You can do it, but only for the initial language. If you try to add a translated node to another menu or rename the item, it always ends up moving / renaming the source node instead of adding a link to the translation. So it can become quite confusing building a navigation directly from the node creation page or to add translations to the menu. A workaround was to add all navigation items manually in the menu administration if you are using a menu per language. With lots of languages and menus / items, this is not really a convenient task. Fortunately, translations from the node creation page have been implemented with a later release of Drupal 8.

Challenge 2: Untranslated Nodes show up in Menu

Another thing which bothered me was that untranslated nodes show up in the navigation (if you use only one menu). This can be quite confusing since most of the times not every page is translated in every language. Or in some languages, you need a little more than in others. You can read a lot about this topic and the reasons behind (e.g. here and here). However you do it, it's always wrong in some situations and perfectly fine in others. But to be “limited” and “locked in” to a certain way is not nice and you have to deal with it. To sum up, once a node is put into a menu, it will show up everywhere. Regardless if there are translations or not.

Challenge 3: Language Switcher shows all languages – always.

Somewhat confusing is the Language Switcher. In Drupal 7, a language link was not available or strikethrough if there was no translation available. In Drupal 8, every language is always visible and linked. So if you look on a German page which is only available in German, the language switcher will present you all language links to the same node. A click on those language links mainly changes the interface language but the node content remains the same (since not translated). Usually also with a drupalish URL (node/xxxx) because there is no translation for the node and therefore also no URL alias available. This behavior is confusing and wrong in my point of view

An example to illustrate the above-written challenges.

multilanguage issues with Drupal 8

English Front-Page with mixed navigation items.

The screen above shows an installation with 2 languages (English and German). The English Page is a basic page which has a translation. English is selected. If you choose Deutsch on the language switcher, the English Page becomes Deutsche Seite (see image below) and shows the German content. So far so good. But the second menu item you see with the title Über uns (nur Deutsch) should not appear here since it's only available in German. But it does. And if you actually go on this page, you will see the German text with everything English around it and no URL-Alias (/node/2 in this example). This is usually not very useful for us.

German only Page – Language Switcher visible.

Also, the language switcher shown in the image above is from my point of view wrong or not very useful. It shows a link to the English version, but there is no English translation for this node. So why is it there? To see a German page with English decoration? Not sure. But I want to get rid of this link or at least modify it to be stroked through if the language is not available.

How to fix improve this?

Luckily, the Drupal community is always good for help. After some “research” on the web, I finally found (besides lots of discussions and comments in the issue queues) a way to achieve the desired setup.

To sum up again: I want to see only menu items which are available in my language and only see a link to another language, if a translation is available.

Since there is no patch and still some ongoing discussions on drupal.org you need to implement it on your own. Implement the following two modules.

Hide untranslated menu items

Code from drupal.org/node/2466553#comment-11991690. Credits go to michaelkoehne.

<?php

use Drupal\Core\Menu\MenuLinkInterface;
use Drupal\menu_link_content\Plugin\Menu\MenuLinkContent;
use Drupal\Core\Language\LanguageInterface;

/**
 * Implements hook_preprocess_menu().
 */
function MYMODULE_preprocess_menu(&$variables) {
  if ($variables['menu_name'] == 'main') {
    $language = Drupal::languageManager()
      ->getCurrentLanguage(LanguageInterface::TYPE_CONTENT)
      ->getId();
    foreach ($variables['items'] as $key => $item) {
      if (!$variables['items'][$key] = MYMODULE_checkForMenuItemTranslation($item, $language)) {
        unset($variables['items'][$key]);
      }
    }
  }
}

function MYMODULE_checkForMenuItemTranslation($item, $language) {
  $menuLinkEntity = MYMODULE_load_link_entity_by_link($item['original_link']);

  if ($menuLinkEntity != NULL) {
    $languages = $menuLinkEntity->getTranslationLanguages();

    // Remove links which are not translated to the current language.
    if (!array_key_exists($language, $languages)) {
      return FALSE;
    }
    else {
      if (count($item['below']) > 0) {
        foreach ($item['below'] as $subkey => $subitem) {
          if (!$item['below'][$subkey] = MYMODULE_checkForMenuItemTranslation($subitem, $language)) {
            unset($item['below'][$subkey]);
          }
        }
      }
      return $item;
    }

  }
}

function MYMODULE_load_link_entity_by_link(MenuLinkInterface $menuLinkContentPlugin) {
  $entity = NULL;
  if ($menuLinkContentPlugin instanceof MenuLinkContent) {
    $menu_link = explode(':', $menuLinkContentPlugin->getPluginId(), 2);
    $uuid = $menu_link[1];
    $entity = \Drupal::service('entity.repository')
      ->loadEntityByUuid('menu_link_content', $uuid);
  }
  return $entity;
}

Hide untranslated languages in language switcher

Code from drupal.org/node/2791231#comment-12004615 (slightly adapted. Links get a class, not removed by default). Credits to Leon Kessler.

<?php

/**
 * @file
 * Hide language switcher links for untranslated languages on an entity.
 */
use Drupal\Core\Entity\ContentEntityInterface;

/**
 * Implements hook_language_switch_links_alter().
 */
function MYOTHERMODULE_language_switch_links_alter(array &$links, $type, $path) {
  if ($entity = MYOTHERMODULE_get_page_entity()) {
    $new_links = array();
    foreach ($links as $lang_code => $link) {
      try {
        if ($entity->getTranslation($lang_code)->access('view')) {
          $new_links[$lang_code] = $link;
        }
      }
      catch (\InvalidArgumentException $e) {
        // This language is untranslated so do not add it to the links.
        $link['attributes']['class'][] = 'not-translated';
        $new_links[$lang_code] = $link;
      }

    }
    $links = $new_links;

    // If we're left with less than 2 links, then there's nothing to switch.
    // Hide the language switcher.
    if (count($links) < 2) {
      $links = array();
    }
  }
}

/**
 * Retrieve the current page entity.
 *
 * @return Drupal\Core\Entity\ContentEntityInterface
 *   The retrieved entity, or FALSE if none found.
 */
function MYOTHERMODULE_get_page_entity() {
  $params = \Drupal::routeMatch()->getParameters()->all();
  $entity = reset($params);
  if ($entity instanceof ContentEntityInterface) {
    return $entity;
  }
  return FALSE;
}

Please note: The code above is from Drupal.org and therefore thanks to the original authors linked above.

Enable those two modules and you're all set!

I did not encounter any issues yet using those two modules. If ever something changes in the way Drupal handles those cases, you just need to switch off the modules and everything should be back to normal. So nothing to lose right?

There are other attempts to this by altering the menu block. One of them is Menu Block Current Language but I had no luck with this one. On my most recent project, it worked with one menu but not if you separate your menu by two blocks (different starting levels).

I would love to hear how you guys handle those cases or how you deal with I18N in general. I'm sure there are a gazillion other ways to do it.

Apr 07 2017
Apr 07

After implementing some larger enterprise Drupal 8 websites, I would like to share some insights, how to solve common issues in the deployment workflow with Drupal 8 CMI.

Introduction to Drupal CMI

First of all, you need to understand, how the configuration management in Drupal 8 works. CMI allows you to export all configurations and its dependencies from the database into yml text files. To make sure, you never end up in an inconsistent state, CMI always exports everything. By default, you cannot exclude certain configurations.

Example:

If you change some configuration on the live database, these configurations will be reverted in the next deployment when you use

drush config-import

1

drush config-import

This is helpful and will make sure, you have the same configuration on all your systems.

How can I have different configurations on local / stage / live environments?

Sometimes, you want to have different configurations on your environments. For example, we have installed a “devel” module only on our local environment but we want to have it disabled on the live environment.

This can be achieved by using the configuration split module: https://www.drupal.org/project/config_split

What does Configuration Split?

This module slightly modifies the CMI by implementing a Config Filter (https://www.drupal.org/project/config_filter). Importing and exporting works the same way as before, except some configuration is read from and written to different directories. Importing configuration still removes configuration not present in the files. Thus, the robustness and predictability of the configuration management remains. And the best thing is: You still can use the same drush commands if you have at least Drush 8.1.10 installed.

Configuration Split Example / Installation Guide

Install config_split using composer. You need need at least “8.x-1.0-beta4” and > drush 8.1.10 for this guide.

composer require drupal/config_split "^1.0"

1

composer require drupal/config_split "^1.0"

Enable config_split and navigate to “admin/config/development/configuration/config-split”

drush en config_split -y

1

drush en config_split -y

Optional: Installing the chosen module will make the selection of blacklists / greylists way more easier. You can enable chosen only on admin pages.

composer require drupal/chosen "^1.0"

1

composer require drupal/chosen "^1.0"

I recommend you to create an “environments” subfolder in your config folder. Inside this folder you will have a separate directory for every environment:

Drupal 8 Configuration Management Folders

Now you can configure your environments:

Config Split in Drupal 8 Configuration Management

The most important thing is, that you set every environment to “Inactive”. We will later activate them according to the environment via settings.php

Config Split settings with the Drupal 8 Configuration Management

Here is my example where I enable the devel module on local:

Dev Environment Example

Activate the environments via settings.php

This is the most important part of the whole setup up. Normally, we never commit the settings.php into git. But we have a [environment]-settings.php in git for every environment:

settings.php (not in git) variables-dev.php (in git and included in the settings.php of dev) variables-live.php (in git and included in the settings.php of live) settings.local.php (in git and included locally)

settings.php (not in git)

variables-dev.php (in git and included in the settings.php of dev)

variables-live.php (in git and included in the settings.php of live)

settings.local.php (in git and included locally)

You need to add the following line to the variables-[environment].php. Please change the variable name according to your environment machine name:

// This enables the config_split module $config['config_split.config_split.dev']['status'] = TRUE;

// This enables the config_split module

$config['config_split.config_split.dev']['status'] = TRUE;

If you have done everything correctly and cleared the cache you will see “active (overriden)” in the config_split overview next to the current environment.

Now you can continue using

drush config-import -y drush config-export -y

drush config-import -y

drush config-export -y

and config_split will do the magic.

How can I exclude certain Config Files and prevent them to be overridden / deleted on my live environment?

The most prominent candidates for this workflow are webforms and contact forms. In Drupal 7, webforms are nodes and you were able to give your CMS administrator the opportunity to create their own forms.

In Drupal 8 webforms are config entities, which means that they will be deleted while deploying if the yml files are not in git.

After testing a lot of different modules / drush scripts, I finally came up with an easy to use workflow to solve this issue and give CMS administrators the possibility to create webforms without git knowledge:

Set up an “Excluded” environment

First of all, we need an “excluded” environment. I created a subfolder in my config-folder and added a .htaccess file to protect the content. You can copy the .htaccess from an existing environment, if you are lazy. Don’t forget to deploy this folder to your live system before you do the next steps.

Folders

Excluded

Now you can exclude some config files to be excluded / grey-listed on your live environment:

webform.webform.* contact.form.*

webform.webform.*

contact.form.*

Greylist Webform in Config Split

Set the excluded environment to “Inactive”. We will later enable it on the live / dev environment via settings.php.

Enable “excluded” environment and adapt deployment workflow

We enable the “excluded” environment on the live system via variables-live.php (see above):

// This will allow module config per environment and exclude webforms from being overridden $config['config_split.config_split.excluded']['status'] = TRUE;

// This will allow module config per environment and exclude webforms from being overridden

$config['config_split.config_split.excluded']['status'] = TRUE;

In your deployment workflow / script you need to add the following line before you do a drush config-import:

#execute some drush commands echo "-----------------------------------------------------------" echo "Exporting excluded config" drush @live config-split-export -y excluded echo "-----------------------------------------------------------" echo "Importing configuration" drush @live config-import -y

1

2

3

4

5

6

7

8

#execute some drush commands

echo "-----------------------------------------------------------"

echo "Exporting excluded config"

drush @live config-split-export -y excluded

echo "-----------------------------------------------------------"

echo "Importing configuration"

drush @live config-import -y

The drush command “drush @live config-split-export -y excluded” will export all webforms and contact forms created by your CMS administrators into the folder “excluded”. The “drush config-import” command will therefore not delete them and your administrators can happily create their custom forms.

Benefit of disable “excluded” on local environment

We usually disable the “excluded” environment on our local environment. This allows us to create complex webforms on our local machine for our clients and deploy them as usual. In the end you can have a mix of customer created webforms and your own webforms which is quite helpful.

Final note

The CMI is a great tool and I would like to thank the maintainers of the config_split module for their great extension. This is a huge step forward making Drupal 8 a real Enterprise CMS Tool.

If you have any questions, don’t hesitate to post a comment.

Apr 06 2017
Apr 06

After implementing some larger enterprise Drupal 8 websites, I would like to share some insights, how to solve common issues in the deployment workflow with Drupal 8 CMI.

Introduction to Drupal CMI

First of all, you need to understand, how the configuration management in Drupal 8 works. CMI allows you to export all configurations and its dependencies from the database into yml text files. To make sure, you never end up in an inconsistent state, CMI always exports everything. By default, you cannot exclude certain configurations.

Example:

If you change some configuration on the live database, these configurations will be reverted in the next deployment when you use

drush config-import

This is helpful and will make sure, you have the same configuration on all your systems.

How can I have different configurations on local / stage / live environments?

Sometimes, you want to have different configurations on your environments. For example, we have installed a “devel” module only on our local environment but we want to have it disabled on the live environment.

This can be achieved by using the configuration split module.

What does Configuration Split?

This module slightly modifies the CMI by implementing a Config Filter. Importing and exporting works the same way as before, except some configuration is read from and written to different directories. Importing configuration still removes configuration not present in the files. Thus, the robustness and predictability of the configuration management remains. And the best thing is: You still can use the same drush commands if you have at least Drush 8.1.10 installed .

Configuration Split Example / Installation Guide

Install config_split using composer. You need need at least “ 8.x-1.0-beta4” and > drush 8.1.10 for this guide.

composer require drupal/config_split "^1.0"

Enable config_split and navigate to “admin/config/development/configuration/config-split”

drush en config_split -y

Optional: Installing the chosen module will make the selection of blacklists / greylists way more easier. You can enable chosen only on admin pages.

composer require drupal/chosen "^1.0"

I recommend you to create an “environments” subfolder in your config folder. Inside this folder you will have a separate directory for every environment:

Drupal 8 Configuration Management Folders

Now you can configure your environments:

Config Split in Drupal 8 Configuration Management

The most important thing is, that you set every environment to “Inactive”. We will later activate them according to the environment via settings.php

Config Split settings with the Drupal 8 Configuration Management

Here is my example where I enable the devel module on local:

Dev Environment Example

Activate the environments via settings.php

This is the most important part of the whole setup up. Normally, we never commit the settings.php into git. But we have a [environment]-settings.php in git for every environment:

settings.php (not in git)

variables-dev.php (in git and included in the settings.php of dev)
variables-live.php (in git and included in the settings.php of live)
settings.local.php (in git and included locally)

You need to add the following line to the variables-[environment].php. Please change the variable name according to your environment machine name :

// This enables the config_split module
$config['config_split.config_split.dev']['status'] = TRUE;

If you have done everything correctly and cleared the cache you will see  “active (overriden)” in the config_split overview next to the current environment.

Now you can continue using

drush config-import -y
drush config-export -y

and config_split will do the magic.

How can I exclude certain Config Files and prevent them to be overridden / deleted on my live environment?

The most prominent candidates for this workflow are webforms and contact forms . In Drupal 7, webforms are nodes and you were able to give your CMS administrator the opportunity to create their own forms.

In Drupal 8 webforms are config entities , which means that they will be deleted while deploying if the yml files are not in git.

After testing a lot of different modules / drush scripts, I finally came up with an easy to use workflow to solve this issue and give CMS administrators the possibility to create webforms without git knowledge:

Set up an “Excluded” environment

First of all, we need an “excluded” environment. I created a subfolder in my config-folder and added a .htaccess file to protect the content. You can copy the .htaccess from an existing environment, if you are lazy. Don't forget to deploy this folder to your live system before you do the next steps.

FoldersExcluded

Now you can exclude some config files to be excluded / grey-listed on your live environment:

webform.webform.*
contact.form.*
Greylist Webform in Config Split

Set the excluded environment to “Inactive” . We will later enable it on the live / dev environment via settings.php.

Enable “excluded” environment and adapt deployment workflow

We enable the “excluded” environment on the live system via variables-live.php (see above):

// This will allow module config per environment and exclude webforms from being overridden
$config['config_split.config_split.excluded']['status'] = TRUE;

In your deployment workflow / script you need to add the following line before you do a drush config-import:

#execute some drush commands
echo "-----------------------------------------------------------"
echo "Exporting excluded config"
drush @live config-split-export -y excluded

echo "-----------------------------------------------------------"
echo "Importing configuration"
drush @live config-import -y

The drush command “ drush @live config-split-export -y excluded ” will export all webforms and contact forms created by your CMS administrators into the folder “excluded”. The “drush config-import” command will therefore not delete them and your administrators can happily create their custom forms.

Benefit of disable “excluded” on local environment

We usually disable the “excluded” environment on our local environment. This allows us to create complex webforms on our local machine for our clients and deploy them as usual. In the end you can have a mix of customer created webforms and your own webforms which is quite helpful.

Final note

The CMI is a great tool and I would like to thank the maintainers of the config_split module for their great extension. This is a huge step forward making Drupal 8 a real Enterprise CMS Tool.

If you have any questions, don't hesitate to post a comment.

Jan 10 2017
Jan 10

Drupal is an open source project and really depends on its community to move forward. It is all about getting to know the CMS, spreading the knowledge and contribute to projects.
I will give you some ways to get involved, even if you are not a developer there is a task for you!

A group of Drupal mentors at DrupalCon 2016 in Dublin

Drupal Mentors – DrupalCon Dublin 2016 by Michael Cannon is licenced under CC BY-SA 2.0

Participating in user support

Sharing your knowledge with others is very important to the community: it is a nice thing to do and you might also learn some things by doing so. Whatever your skill level, you can give back to the community with online support. There are many places where you can give support starting with the Support Forums. You can also go to Drupal Answers which is more active than the forums or subscribe to the Support Mailing list. If you prefer real-time chat, you can also join #drupal-support channel on IRC or the Slack channels.

Helping out on documentation

Community members can write, review and improve different sorts of documentation for the project: community documentation on drupal.org, programming API reference, help pages inside the core software, documentation embedded in contributed modules and themes etc.
Contributing is a good way to learn more about Drupal and share your knowledge with others. Beginners are particularly encouraged to participate as they are more likely to know where documentation is lacking.
If you are interested, check out the new contributor tasks for anyone and writers.

Translating Drupal interface in your own language

The default language for the administration interface is English but there are about 100 available languages for translations. There is always a need for translations as many of these translation sets are incomplete or can be improved for core and contributed modules.
All translations are now managed by the translation server. If you are willing to help, all you have to do is logging into drupal.org and join a language team. There is even a video to learn how the translation system works and a documentation.

You can also help to translate documentation into your language. Most language-specific communities have their own documentation so you should get in touch with them directly. To learn more, see the dedicated page.

Improving design and usability

The idea is to improve the usability especially in Drupal 8 regarding the administration interface. The focus is mainly on content creation and site building. The community has done many research to understand the problems that users run into and how the new improvements performs. The purpose is also to educate developers and engage designers in order to grow the UX-team. You can visit the Drupal 8 UX page for more details and join the usability group.

Writing a blog post about Drupal

Writing a blog post about Drupal is a good way to share your knowledge and expertise. There are many subjects to explore, technical or not: talking about a former project you developed or writing a tutorial, telling about the state of a version or sharing about an event you attended… And if you are lucky enough your post can be published on the Weekly Drop, the official Drupal newsletter!

Don’t forget to reference your blog post on Planet Drupal, this platform is an aggregated list of feeds from around the web which shares relevant Drupal-related knowledge and information.

You can also find our Drupal related blog posts on the Liip blog.

Testing core and modules

Testing Drupal projects is necessary to make the platform stable and there are many things to test! If you have a technical background, you can help to review patches or to write unit tests.
For non-technical people, you can provide some feedback about usability of the administration interface that will help to improve the user experience. Follow the process to give a proper feedback.

Contributing to development

There are many ways to contribute code in core and “contrib” projects such as modules or themes.
You can first help to improve existing projects by submitted patches. This would be the natural thing to do when you work with a module and you notice a bug or a missing feature: search in the corresponding issue queue if the problem have been noticed before. If not, post a message explaining the issue and add a snippet of code if you found a potential fix. Then you can create a patch and submit it into the issue queue.
You can also contribute to new projects by creating your very own module or theme or create a sandbox for more experimental projects.

Attending events

The Drupal association organizes many events all around the world to promote the CMS and gather the community.

One of the biggest events are the Drupalcons. A Drupalcon gathers thousands of people and lasts about one week including 3 full days of conferences. These conferences cover many topics: site building, user experience, security, content authoring etc. You can also join sprints to contribute to Drupal projects and social events to meet the members of the community. Check out our report about DrupalCon Barcelona 2015!

“Drupal Dev Days” conferences occur once a year and gather developers to discuss and present topics technically relevant to the community. You can join sprints, intensive coding sessions and technical conferences.

You can also join DrupalCamps to meet your local community. These events last one or two days and focus on sharing knowledge amongst the community. You can attend conferences and sprints.

There are also many Drupal meetups which are free events happening in many cities in the world. Presentations and discussions finish around nice drinks and appetizers.

Sponsoring events

The community holds conventions and meetups in many countries and being a sponsor will not only help Drupal development but it will also enable you to be noticeable within the community. There are different levels of sponsorings that will offer from mentions on social media to advertising online and at the exhibition space of the event. All you have to do is getting in touch with the event organizers. By the way, Liip will sponsor the Drupal Mountain Camp in Davos this year!

Offering a donation

You can give donations to the Drupal association through the website in order to support drupal.org infrastructure and maintenance, worldwide events such as Drupalcons. The donations are either in Euros or Dollars.

You can also become a member of the Drupal Association for the same purpose, as an individual member or an organization member. The minimal fees are 15 Euros. Find more information about membership on drupal.org.

Conclusion

Drupal projects are constantly improving thanks to passionate volunteers who work on many subjects: development, documentation, marketing, events organization, supports… There is for sure a task that will suit you and it only takes small time commitment to make changes.
So join the great Drupal community and start getting involved!

Jan 09 2017
Jan 09

Drupal is an open source project and really depends on its community to move forward. It is all about getting to know the CMS, spreading the knowledge and contribute to projects.

I will give you some ways to get involved, even if you are not a developer there is a task for you!

A group of Drupal mentors at DrupalCon 2016 in Dublin

Drupal Mentors – DrupalCon Dublin 2016 by Michael Cannon is licenced under CC BY-SA 2.0

Participating in user support

Sharing your knowledge with others is very important to the community: it is a nice thing to do and you might also learn some things by doing so. Whatever your skill level, you can give back to the community with online support. There are many places where you can give support starting with the Support Forums. You can also go to Drupal Answers which is more active than the forums or subscribe to the Support Mailing list. If you prefer real-time chat, you can also join #drupal-support channel on IRC or the Slack channels.

Helping out on documentation

Community members can write, review and improve different sorts of documentation for the project: community documentation on drupal.org, programming API reference, help pages inside the core software, documentation embedded in contributed modules and themes etc.

Contributing is a good way to learn more about Drupal and share your knowledge with others. Beginners are particularly encouraged to participate as they are more likely to know where documentation is lacking.

If you are interested, check out the new contributor tasks for anyone and writers.

Translating Drupal interface in your own language

The default language for the administration interface is English but there are about 100 available languages for translations. There is always a need for translations as many of these translation sets are incomplete or can be improved for core and contributed modules.

All translations are now managed by the translation server. If you are willing to help, all you have to do is logging into drupal.org and join a language team. There is even a video to learn how the translation system works and a documentation.

You can also help to translate documentation into your language. Most language-specific communities have their own documentation so you should get in touch with them directly. To learn more, see the dedicated page.

Improving design and usability

The idea is to improve the usability especially in Drupal 8 regarding the administration interface. The focus is mainly on content creation and site building. The community has done many research to understand the problems that users run into and how the new improvements performs. The purpose is also to educate developers and engage designers in order to grow the UX-team. You can visit the Drupal 8 UX page for more details and join the usability group.

Writing a blog post about Drupal

Writing a blog post about Drupal is a good way to share your knowledge and expertise. There are many subjects to explore, technical or not: talking about a former project you developed or writing a tutorial, telling about the state of a version or sharing about an event you attended… And if you are lucky enough your post can be published on the Weekly Drop, the official Drupal newsletter!

Don't forget to reference your blog post on Planet Drupal, this platform is an aggregated list of feeds from around the web which shares relevant Drupal-related knowledge and information.

You can also find our Drupal related blog posts on the Liip blog.

Testing core and modules

Testing Drupal projects is necessary to make the platform stable and there are many things to test! If you have a technical background, you can help to review patches or to write unit tests.

For non-technical people, you can provide some feedback about usability of the administration interface that will help to improve the user experience. Follow the process to give a proper feedback.

Contributing to development

There are many ways to contribute code in core and “contrib” projects such as modules or themes.

You can first help to improve existing projects by submitted patches. This would be the natural thing to do when you work with a module and you notice a bug or a missing feature: search in the corresponding issue queue if the problem have been noticed before. If not, post a message explaining the issue and add a snippet of code if you found a potential fix. Then you can create a patch and submit it into the issue queue.

You can also contribute to new projects by creating your very own module or theme or create a sandbox for more experimental projects.

Attending events

The Drupal association organizes many events all around the world to promote the CMS and gather the community.

One of the biggest events are the Drupalcons. A Drupalcon gathers thousands of people and lasts about one week including 3 full days of conferences. These conferences cover many topics: site building, user experience, security, content authoring etc. You can also join sprints to contribute to Drupal projects and social events to meet the members of the community. Check out our report about DrupalCon Barcelona 2015!

“Drupal Dev Days” conferences occur once a year and gather developers to discuss and present topics technically relevant to the community. You can join sprints, intensive coding sessions and technical conferences.

You can also join DrupalCamps to meet your local community. These events last one or two days and focus on sharing knowledge amongst the community. You can attend conferences and sprints.

There are also many Drupal meetups which are free events happening in many cities in the world. Presentations and discussions finish around nice drinks and appetizers.

Sponsoring events

The community holds conventions and meetups in many countries and being a sponsor will not only help Drupal development but it will also enable you to be noticeable within the community. There are different levels of sponsorings that will offer from mentions on social media to advertising online and at the exhibition space of the event. All you have to do is getting in touch with the event organizers. By the way, Liip will sponsor the Drupal Mountain Camp in Davos this year!

Offering a donation

You can give donations to the Drupal association through the website in order to support drupal.org infrastructure and maintenance, worldwide events such as Drupalcons. The donations are either in Euros or Dollars.

You can also become a member of the Drupal Association for the same purpose, as an individual member or an organization member. The minimal fees are 15 Euros. Find more information about membership on drupal.org.

Conclusion

Drupal projects are constantly improving thanks to passionate volunteers who work on many subjects: development, documentation, marketing, events organization, supports… There is for sure a task that will suit you and it only takes small time commitment to make changes.

So join the great Drupal community and start getting involved!

Jan 05 2017
Jan 05

As a follow-up to my previous blog post about the usage of Migrate API in Drupal 8, I would like to give an example, how to import multilingual content and translations in Drupal 8.

Prepare and enable translation for your content type

Before you can start, you need to install the “Language” and “Content Translation” Module. Then head over to “admin/config/regional/content-language” and enable Entity Translation for the node type or the taxonomy you want to be able to translate.

As a starting point for setting up the migrate module, I recommend you my blog post mentioned above. To import data from a CVS file, you also need to install the migrate_source_csv module.

Prerequisites for migrating multilingual entities

Before you start, please check the requirements. You need at least Drupal 8.2 to import multilingual content. We need the destination option “translations”, which was added in a patch in Drupal 8.2. See the corresponding drupal.org issue here.

Example: Import multilingual taxonomy terms

Let’s do a simple example with taxonomy terms. First, create a vocabulary called “Event Types” (machine name: event_type).

Here is a simplified dataset:

Id Name Name_en 1 Kurs Course 2 Turnier Tournament

You may save this a csv file.

Id;Name;Name_en 1;Kurs;Course 2;Turnier;Tournament

Id;Name;Name_en

1;Kurs;Course

2;Turnier;Tournament

The recipe to import multilingual content

As you can see in the example data,  it contains the base language (“German”) and also the translations (“English”) in the same file.

But here comes a word of warning:

Don’t try to import the term and its translation in one migration run. I am aware, that there are some workarounds with post import events, but these are hacks and you will run into troubles later.

The correct way of importing multilingual content, is to

  1. create a migration for the base language and import the terms / nodes. This will create the entities and its fields.
  2. Then, with an additional dependent migration for each translated language, you can then add the translations for the fields you want.

In short: You need a base migration and a migration for every language. Let’s try this out.

Taxonomy term base language config file

In my example, the base language is “German”. Therefore, we first create a migration configuration file for the base language:

This is a basic example in migrating a taxonomy term in my base language ‘de’.

Put the file into <yourmodule>/config/install/migrate.migration.event_type.yml and import the configuration using the drush commands explained in my previous blog post about Migration API.

id: event_type label: Event Types source: plugin: csv # Full path to the file. Is overriden in my plugin path: public://csv/data.csv # The number of rows at the beginning which are not data. header_row_count: 1 # These are the field names from the source file representing the key # uniquely identifying each node - they will be stored in the migration # map table as columns sourceid1, sourceid2, and sourceid3. keys: - Id ids: id: type: string destination: plugin: entity:taxonomy_term process: vid: plugin: default_value default_value: event_type name: source: Name language: 'de' langcode: plugin: default_value default_value: 'de' #Absolutely necessary if you don't want an error migration_dependencies: {}

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

id: event_type

label: Event Types

source:

  plugin: csv

  # Full path to the file. Is overriden in my plugin

  path: public://csv/data.csv

  # The number of rows at the beginning which are not data.

  header_row_count: 1

  # These are the field names from the source file representing the key

  # uniquely identifying each node - they will be stored in the migration

  # map table as columns sourceid1, sourceid2, and sourceid3.

  keys:

    - Id

ids:

  id:

    type: string

destination:

  plugin: entity:taxonomy_term

process:

  vid:

   plugin: default_value

   default_value: event_type

  name:

    source: Name

    language: 'de'

  langcode:

    plugin: default_value

    default_value: 'de'

#Absolutely necessary if you don't want an error

migration_dependencies: {}

Taxonomy term translation migration configuration file:

This is the example file for the English translation of the name field of the term.

Put the file into <yourmodule>/config/install/migrate.migration.event_type_en.yml and import the configuration using the drush commands explained in my previous blog post about Migration API.

id: event_type_en label: Event Types english source: plugin: csv # Full path to the file. Is overriden in my plugin path: public://csv/data.csv # The number of rows at the beginning which are not data. header_row_count: 1 keys: - Id ids: id: type: string destination: plugin: entity:taxonomy_term translations: true process: vid: plugin: default_value default_value: event_type tid: plugin: migration source: id migration: event_type name: source: Name_en language: 'en' langcode: plugin: default_value default_value: 'en' #Absolutely necessary if you don't want an error migration_dependencies: required: - event_type

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

id: event_type_en

label: Event Types english

source:

  plugin: csv

  # Full path to the file. Is overriden in my plugin

  path: public://csv/data.csv

  # The number of rows at the beginning which are not data.

  header_row_count: 1

  keys:

    - Id

ids:

  id:

    type: string

destination:

  plugin: entity:taxonomy_term

  translations: true

process:

  vid:

    plugin: default_value

    default_value: event_type

  tid:

    plugin: migration

    source: id

    migration: event_type

  name:

    source: Name_en

    language: 'en'

  langcode:

     plugin: default_value

     default_value: 'en'

#Absolutely necessary if you don't want an error

migration_dependencies:

  required:

    - event_type

Explanation and sum up of the learnings

The key in the migrate configuration to import multilingual content are the following lines:

destination: plugin: entity:taxonomy_term translations: true

destination:

  plugin: entity:taxonomy_term

  translations: true

These configuration lines instruct the migrate module, that a translation should be created.

tid: plugin: migration source: id migration: event_type

tid:

  plugin: migration

  source: id

  migration: event_type

This is the real secret. Using the process plugin migration,  we maintain the relationship between the node and its translation.The wiring via the tid field make sure, that Migrate API will not create a new term with a new term id. Instead, the existing term will be loaded and the translation of the migrated field will be added. And thats exactly what we need!

Now go ahead and try to create a working example based on my explanation. Happy Drupal migrations!

Jan 04 2017
Jan 04

As a follow-up to my previous blog post about the usage of Migrate API in Drupal 8, I would like to give an example, how to import multilingual content and translations in Drupal 8.

Prepare and enable translation for your content type

Before you can start, you need to install the “Language” and “Content Translation” Module. Then head over to “admin/config/regional/content-language” and enable Entity Translation for the node type or the taxonomy you want to be able to translate.

As a starting point for setting up the migrate module, I recommend you my blog post mentioned above. To import data from a CSV file, you also need to install the migrate_source_csv module.

Prerequisites for migrating multilingual entities

Before you start, please check the requirements. You need at least Drupal 8.2 to import multilingual content. We need the destination option “translations”, which was added in a patch in Drupal 8.2. See the corresponding drupal.org issue here.

Example: Import multilingual taxonomy terms

Let's do a simple example with taxonomy terms. First, create a vocabulary called “Event Types” (machine name: event_type).

Here is a simplified dataset:

Id Name Name_en 1 Kurs Course 2 Turnier Tournament

You may save this a csv file.

Id;Name;Name_en
1;Kurs;Course
2;Turnier;Tournament

The recipe to import multilingual content

As you can see in the example data,  it contains the base language (“German”) and also the translations (“English”) in the same file.

But here comes a word of warning:

Don't try to import the term and its translation in one migration run. I am aware, that there are some workarounds with post import events, but these are hacks and you will run into troubles later.

The correct way of importing multilingual content, is to

  1. create a migration for the base language and import the terms / nodes. This will create the entities and its fields.
  2. Then, with an additional dependent migration for each translated language, you can then add the translations for the fields you want.

In short: You need a base migration and a migration for every language. Let's try this out.

Taxonomy term base language config file

In my example, the base language is “German”. Therefore, we first create a migration configuration file for the base language:

This is a basic example in migrating a taxonomy term in my base language ‘de'.

Put the file into <yourmodule>/config/install/migrate.migration.event_type.yml and import the configuration using the drush commands explained in my previous blog post about Migration API.

id: event_type
label: Event Types
source:
  plugin: csv
  # Full path to the file. Is overriden in my plugin
  path: public://csv/data.csv
  # The number of rows at the beginning which are not data.
  header_row_count: 1
  # These are the field names from the source file representing the key
  # uniquely identifying each node - they will be stored in the migration
  # map table as columns sourceid1, sourceid2, and sourceid3.
  keys:
    - Id
ids:
  id:
    type: string
destination:
  plugin: entity:taxonomy_term
process:
  vid:
   plugin: default_value
   default_value: event_type
  name:
    source: Name
    language: 'de'
  langcode:
    plugin: default_value
    default_value: 'de'

#Absolutely necessary if you don't want an error
migration_dependencies: {}

Taxonomy term translation migration configuration file:

This is the example file for the English translation of the name field of the term.

Put the file into <yourmodule>/config/install/migrate.migration.event_type_en.yml and import the configuration using the drush commands explained in my previous blog post about Migration API.

id: event_type_en
label: Event Types english
source:
  plugin: csv
  # Full path to the file. Is overriden in my plugin
  path: public://csv/data.csv
  # The number of rows at the beginning which are not data.
  header_row_count: 1
  keys:
    - Id
ids:
  id:
    type: string
destination:
  plugin: entity:taxonomy_term
  translations: true
process:
  vid:
    plugin: default_value
    default_value: event_type
  tid:
    plugin: migration
    source: id
    migration: event_type
  name:
    source: Name_en
    language: 'en'
  langcode:
     plugin: default_value
     default_value: 'en'

#Absolutely necessary if you don't want an error
migration_dependencies:
  required:
    - event_type

Explanation and sum up of the learnings

The key in the migrate configuration to import multilingual content are the following lines:

destination:
  plugin: entity:taxonomy_term
  translations: true

These configuration lines instruct the migrate module, that a translation should be created.

tid:
  plugin: migration
  source: id
  migration: event_type

This is the real secret. Using the process plugin migration,  we maintain the relationship between the node and its translation.The wiring via the tid field make sure, that Migrate API will not create a new term with a new term id. Instead, the existing term will be loaded and the translation of the migrated field will be added. And thats exactly what we need!

Now go ahead and try to create a working example based on my explanation. Happy Drupal migrations!

Nov 30 2016
Nov 30

Freitag logoOur latest site with Drupal Commerce 1.x went live in July 2016. It is Freitag. Since then we’ve been adding several new commerce related features. I feel it’s time to write a wrap-up. The site has several interesting solutions, this article will focus on commerce.

First a few words about the architecture. platform.sh hosts the site. The stack is Linux + nginx +  MySQL + PHP, the CMS is Drupal 7. Fastly caches http responses for anonymous users and also for authenticated users having no additional role (that is, logged-in customers). Authcache module takes care of lazy-loading the personalized parts (like the user menu and the shopping cart). Freitag has an ERP system to which we connect using the OCI8 PHP library. We write Behat and simpletest tests for QA.

We use the highly flexible Drupal Commerce suite. 23 of the enabled Freitag contrib modules have a name starting with ‘commerce’. We applied around 45 patches on them. Most of the patches are authored by us and 15 of them have already been committed. Even with this commitment to solve everything we could in an open-source way we wrote 30.000+ lines of commerce-related custom code. Still, in March 2016 Freitag was the 3rd largest Drupal customer contributor.

The words ‘product’ and ‘product variation’  I’ll be using throughout the article correspond to ‘product display node’ and ‘product’ in Drupal Commerce lingo.

ERP

ERP is the source of all products and product variations. We import this data into Drupal on a regular basis using Feeds. (Now I would use Migrate instead, it’s better supported and easier to maintain.) ERP also lets Drupal know about order status changes, sends the shipping tracking information and informs Drupal about products sent back to Freitag by customers.

There is data flowing in the opposite direction as well. ERP needs to know about all Drupal orders. Also, we create coupons in Drupal and send them to ERP too for accounting and other reasons.

Emails

We send commerce-related emails using the Message stack. This way we can have order-related tokens in our mails and we can manage and translate them outside the Rules UI. Mandrill takes care of the mail delivery.

Payment gateway

It was a client requirement to use the Swiss Datatrans payment gateway. However, at the time of starting  the project, Commerce Datatrans (the connector module on drupal.org) was in dev state and lacked several features we needed. Pressed for time we opted for buying a Datatrans Drupal module from a company offering this solution. It turned out to be a bad choice. When we discovered that the purchased module still does not cover all our needs and looked at the code we found that it was obfuscated and pretty much impossible to change. Also, the module could be used only on one site instance which made it impossible to use it on our staging sites.

We ended up submitting patches to the Commerce Datatrans module hosted on drupal.org. The module maintainer, Sascha Grossenbacher (the well-known Drupal 8 core contributor) helped us solving several issues and feature requests by reviewing our patches. This process has lead to a stable release of Commerce Datatrans with a dozen of feature improvements and bugfixes.

Additional to Datatrans we use Commerce Custom Offline Payments to enable offline store purchases by store staff and bank transfer payments.

Currencies

The site works with 7 different currencies, some of them having two different prices depending on the shipping country. Prices come from ERP and we store them in a field collection field on the product. We do not use the commerce_price field on the product variation.

Tax

Freitag ships to countries all around the world. VAT calculations are performed for EU, Switzerland, UK, Japan, South Korea and Singapore. To implement this functionality our choice fell on the commerce_vat module. Adding commerce_eu_vat and commerce_ch_vat released us from having to maintain VAT rates for EU and Switzerland ourselves. For the 3 Asian countries we implemented our own hook_commerce_vat_rate_info().

We have two different VAT rates for most of the countries. This is because usually a lower VAT rate applies to books. Drupal imports the appropriate VAT rate from the ERP with the product variation data. This information is handled by price calculation rules in Drupal.

Shipping

Freitag delivers its products using several shipping providers (like UPS, Swiss Post) all around the world. Most shipping providers have many shipping rates depending on the destination country, speed and shipped quantity (weight or volume). On checkout the customer can choose from a list of shipping services. This list needs to be compatible with the order.

We used rules to implement the shipping services in Drupal based on the Commerce Flat Rate module. For this end we trained our client to set up and maintain these rules themselves. It was not easy: shipping rules are daunting even for experienced commerce developers. First we needed to set up the “Profile Address” rules components. Then we configured the “Place of Supply” components. We applied these in turn in the condition part of the shipping rules components themselves.

The weakest point of any  implementation based on Rules is the maintenance. It’s not easy to find a specific rule after you created it. Having 250 rules components for only shipping made this feeling stronger.

The shipping line item receives the VAT rate of the product with the highest VAT rate in the order.

Coupons

Freitag has 6 different coupon types. They differ in who can create them, who and where (online/offline) can redeem them, whether partial redemption is possible, whether it’s a fixed amount or percentage discount and whether Freitag accounting needs to know about them or not.

Based on these criteria we came up with a solution featuring Commerce Coupon. Coupons can be discount coupons or giftcards. Giftcard coupons can only have a fixed value. Discount based coupons can also apply a percentage discount. The main difference between them is that customers can partially redeem giftcards, while discount-based coupons are for one-time use.

To make coupons work with VAT was quite tricky. (To make things simpler we only allowed one coupon per order.) Some coupon types work as money which means that from an accounting point of view they do not actually decrease the order total (and thus the VAT) but work as a payment method. Other coupon types however do decrease the order total (and thus the VAT). At the same time Drupal handles all coupons as line items with a negative price and the Drupal order total does decrease in either case.

The solution we found was to use Commerce proportional VATAxel Rutz maintains this brilliant little module and he does this in a very helpful and responsive manner. All the module does is adding negative VAT price components to coupon line items to account for VAT decrease. It decreases the order total VAT amounts correctly even if we have several different VAT rates inside the order.

Conclusion

Although there’s always room for increasing the complexity of the commerce part of the site (let’s find some use case for recurring payments!), it’s already the most complicated commerce site I’ve worked on.  For this Drupal Commerce provided a solid foundation that is pleasant to work with. In the end, Drupal enabled us to deliver a system that tightly integrates content and commerce.

I also would like to say thanks to Bojan Živanović from Commerce Guys who provided me with valuable insights on the legal aspects of tax calculation.

Nov 29 2016
Nov 29
A fairly complex Drupal Commerce site: freitag.ch · Blog · Liip f-square-512x512.png
  • 30 November 2016
  • András Czövek
Freitag logo

András Czövek

Related services
  • Topics
  • Tags

Tell us what you think

Please enable JavaScript to view the comments.
Oct 24 2016
Oct 24

In this blog post I will present how, in a recent e-Commerce project built on top of Drupal7 (the former version of the Drupal CMS), we make Drupal7, SearchAPI and Commerce play together to efficiently retrieve grouped results from Solr in SearchAPI, with no indexed data duplication.

We used the SearchAPI and the FacetAPI modules to build a search index for products, so far so good: available products and product-variations can be searched and filtered also by using a set of pre-defined facets. In a subsequent request, a new need arose from our project owner: provide a list of products where the results should include, in addition to the product details, a picture of one of the available product variations, while keep the ability to apply facets on products for the listing. Furthermore, the product variation picture displayed in the list must also match the filter applied by the user: this with the aim of not confusing users, and to provide a better user experience.

An example use case here is simple: allow users to get the list of available products and be able to filter them by the color/size/etc field of the available product variations, while displaying a picture of the available variations, and not a sample picture.

For the sake of simplicity and consistency with Drupal’s Commerce module terminology, I will use the term “Product” to refer to any product-variation, while the term “Model” will be used to refer to a product.

Solr Result Grouping

We decided to use Solr (the well-known, fast and efficient search engine built on top of the Apache Lucene library) as the backend of the eCommerce platform: the reason lies not only in its full-text search features, but also in the possibility to build a fast retrieval system for the huge number of products we were expecting to be available online.

To solve the request about the display of product models, facets and available products, I intended to use the feature offered by Solr called Result-Grouping as it seemed to be suitable for our case: Solr is able to return just a subset of results by grouping them given an “single value” field (previously indexed, of course). The Facets can then be configured to be computed from: the grouped set of results, the ungrouped items or just from the first result of each group.

Such handy feature of Solr can be used in combination with the SearchAPI module by installing the SearchAPI Grouping module. The module allows to return results grouped by a single-valued field, while keeping the building process of the facets on all the results matched by the query, this behavior is configurable.

That allowed us to:

  • group the available products by the referenced model and return just one model;
  • compute the attribute’s facets on the entire collection of available products;
  • reuse the data in the product index for multiple views based on different grouping settings.

Result Grouping in SearchAPI

Due to some limitations of the SearchAPI module and its query building components, such plan was not doable with the current configuration as it would require us to create a copy of the product index just to apply the specific Result Grouping feature for each view.

The reason is that the features implemented by the SearchAPI Grouping are implemented on top of the “Alterations and Processors” functions of SearchAPI. Those are a set of specific functions that can be configured and invoked both at indexing-time and at querying-time by the SearchAPI module. In particular Alterations allows to programmatically alter the contents sent to the underlying index, while the Processors code is executed when a search query is built, executed and the results returned.
Those functions can be defined and configured only per-index.

As visible in the following picture, the SearchAPI Grouping module configuration could be done solely in the Index configuration, but not per-query.

SearchAPI: processor settings

Image 1: SearchAPI configuration for the Grouping Processor.

As the SearchAPI Grouping module is implemented as a SearchAPI Processor (as it needs to be able to alter the query sent to Solr and to handle the returned results), it would force us to create a new index for each different configuration of the result grouping.

Such limitation requires to introduce a lot of (useless) data duplication in the index, with a consequent decrease of performance when products are saved and later indexed in multiple indexes.
In particular, the duplication is more evident as the changes performed by the Processor are merely an alteration of:

  1. the query sent to Solr;
  2. the handling of the raw data returned by Solr.

This shows that there would be no need to index multiple times the same data.

Since the the possibility to define per-query processor sounded really promising and such feature could be used extensively in the same project, a new module has been implemented and published on Drupal.org: the SearchAPI Extended Processors module. (thanks to SearchAPI’s maintainer, DrunkenMonkey, for the help and review :) ).

The Drupal SearchAPI Extended Processor

The new module allows to extend the standard SearchAPI behavior for Processors and lets admins configure the execution of SearchAPI Processors per query and not only per-index.

By using the new module, any index can now be used with multiple and different Processors configurations, no new indexes are needed, thus avoiding data duplication.

The new configuration is exposed, as visible in the following picture, while editing a SearchAPI view under “Advanced > Query options”.
The SearchAPI processors can be altered and re-defined for the given view, a checkbox allows to completely override the current index setting rather than providing additional processors.

Drupal SearchAPI: view's extended processor settings

Image 2: View’s “Query options” with the SearchAPI Extended Processors module.

Conclusion: the new SearchAPI Extended Processors module has now been used for a few months in a complex eCommerce project at Liip and allowed us to easily implement new search features without the need to create multiple and separated indexes.
We are able to index Products data in one single (and compact) Solr index, and use it with different grouping strategies to build both product listings, model listings and model-category navigation pages without duplicating any data.
Since all those listings leverages the Solr FilterQuery query parameter to filter the correct set of products to be displayed, Solr can make use of its internal set of caches and specifically the filterCache to speed up subsequent searches and facets. This aspect, in addition to the usage of only one index, allows caches to be shared among multiple listings, and that would not be possible if separate indexes were used.

For further information, questions or curiosity drop me a line, I will be happy to help you configuring Drupal SearchAPI and Solr for your needs.

Oct 23 2016
Oct 23

In this blog post I will present how, in a recent e-Commerce project built on top of Drupal7 (the former version of the Drupal CMS), we make Drupal7, SearchAPI and Commerce play together to efficiently retrieve grouped results from Solr in SearchAPI, with no indexed data duplication.

We used the SearchAPI and the FacetAPI modules to build a search index for products, so far so good: available products and product-variations can be searched and filtered also by using a set of pre-defined facets. In a subsequent request, a new need arose from our project owner: provide a list of products where the results should include, in addition to the product details, a picture of one of the available product variations, while keep the ability to apply facets on products for the listing. Furthermore, the product variation picture displayed in the list must also match the filter applied by the user: this with the aim of not confusing users, and to provide a better user experience.

An example use case here is simple: allow users to get the list of available products and be able to filter them by the color/size/etc field of the available product variations, while displaying a picture of the available variations, and not a sample picture.

For the sake of simplicity and consistency with Drupal's Commerce module terminology, I will use the term “Product” to refer to any product-variation, while the term “Model” will be used to refer to a product.

Solr Result Grouping

We decided to use Solr (the well-known, fast and efficient search engine built on top of the Apache Lucene library) as the backend of the eCommerce platform: the reason lies not only in its full-text search features, but also in the possibility to build a fast retrieval system for the huge number of products we were expecting to be available online.

To solve the request about the display of product models, facets and available products, I intended to use the feature offered by Solr called Result-Grouping as it seemed to be suitable for our case: Solr is able to return just a subset of results by grouping them given an “single value” field (previously indexed, of course). The Facets can then be configured to be computed from: the grouped set of results, the ungrouped items or just from the first result of each group.

Such handy feature of Solr can be used in combination with the SearchAPI module by installing the SearchAPI Grouping module. The module allows to return results grouped by a single-valued field, while keeping the building process of the facets on all the results matched by the query, this behavior is configurable.

That allowed us to:

  • group the available products by the referenced model and return just one model;
  • compute the attribute's facets on the entire collection of available products;
  • reuse the data in the product index for multiple views based on different grouping settings.

Result Grouping in SearchAPI

Due to some limitations of the SearchAPI module and its query building components, such plan was not doable with the current configuration as it would require us to create a copy of the product index just to apply the specific Result Grouping feature for each view.

The reason is that the features implemented by the SearchAPI Grouping are implemented on top of the “ Alterations and Processors” functions of SearchAPI. Those are a set of specific functions that can be configured and invoked both at indexing-time and at querying-time by the SearchAPI module. In particular Alterations allows to programmatically alter the contents sent to the underlying index, while the Processors code is executed when a search query is built, executed and the results returned.

Those functions can be defined and configured only per-index.

As visible in the following picture, the SearchAPI Grouping module configuration could be done solely in the Index configuration, but not per-query.

SearchAPI: processor settings

Image 1: SearchAPI configuration for the Grouping Processor.

As the SearchAPI Grouping module is implemented as a SearchAPI Processor (as it needs to be able to alter the query sent to Solr and to handle the returned results), it would force us to create a new index for each different configuration of the result grouping.

Such limitation requires to introduce a lot of (useless) data duplication in the index, with a consequent decrease of performance when products are saved and later indexed in multiple indexes.

In particular, the duplication is more evident as the changes performed by the Processor are merely an alteration of:

  1. the query sent to Solr;
  2. the handling of the raw data returned by Solr.

This shows that there would be no need to index multiple times the same data.

Since the the possibility to define per-query processor sounded really promising and such feature could be used extensively in the same project, a new module has been implemented and published on Drupal.org: the SearchAPI Extended Processors module. (thanks to SearchAPI's maintainer, DrunkenMonkey, for the help and review :) ).

The Drupal SearchAPI Extended Processor

The new module allows to extend the standard SearchAPI behavior for Processors and lets admins configure the execution of SearchAPI Processors per query and not only per-index.

By using the new module, any index can now be used with multiple and different Processors configurations, no new indexes are needed, thus avoiding data duplication.

The new configuration is exposed, as visible in the following picture, while editing a SearchAPI view under “Advanced > Query options”.

The SearchAPI processors can be altered and re-defined for the given view, a checkbox allows to completely override the current index setting rather than providing additional processors.

Drupal SearchAPI: view's extended processor settings

Image 2: View's “Query options” with the SearchAPI Extended Processors module.

Conclusion: the new SearchAPI Extended Processors module has now been used for a few months in a complex eCommerce project at Liip and allowed us to easily implement new search features without the need to create multiple and separated indexes.

We are able to index Products data in one single (and compact) Solr index, and use it with different grouping strategies to build both product listings, model listings and model-category navigation pages without duplicating any data.

Since all those listings leverages the Solr cwiki.apache.org/confluence/display/solr/Common+Query+Parameters#CommonQueryParameters-Thefq(FilterQueryParameter text: FilterQuery query parameter) to filter the correct set of products to be displayed, Solr can make use of its internal set of caches and specifically the filterCache to speed up subsequent searches and facets. This aspect, in addition to the usage of only one index, allows caches to be shared among multiple listings, and that would not be possible if separate indexes were used.

For further information, questions or curiosity drop me a line, I will be happy to help you configuring Drupal SearchAPI and Solr for your needs.

Aug 22 2016
Aug 22

The Drupal accessibility initiative started with some advancements in Drupal 7 to ensure that Drupal core followed the World Wide Web Consortium (W3C) guidelines: WCAG 2.0 (Web Content Accessibility Guidelines) and ATAG 2.0 (Authoring Tool Accessibility Guidelines).
Many elements introduced in Drupal 7 were improved and bugs discovered through intensive testing were addressed and integrated to Drupal 8 core as well. Let’s take a tour of the accessibility in Drupal 8 !

Contrasts improved

Drupal’s accessibility maintainers improved contrasts in core themes so people that suffer from colorblindness are able to visit websites clearly. It is also good when visiting the website under bright sunlight, on mobile for instance.

A screenshot that compares Bartik headers in Drupal 7.43 and Drupal 8

Color contrasts in Bartik theme in Drupal 7.43 and Drupal 8.

See the related WCAG 2.0 section about contrasts.

Alternative texts for images

The alternative text for images is really useful for blind people who use screen readers. They can understand the meaning of an image through short descriptive phrases. This alternative text is now by default a required field in Drupal 8.

A screenshot showing that the alternative text is required when uploading an image in Drupal 8.

The alternative text for an image is required by default in Drupal 8 content edition.

See the related WCAG 2.0 section about alternative texts.

More semantics

Many accessibility improvements are hard to see as it involves semantics. Drupal 8 uses HTML5 elements in its templates which add more meaning into the code. For instance, assistive technology such as screen readers can now interpret elements like <header>, <footer> or <form>.

Moreover, WAI-ARIA (Web Accessibility Initiative – Accessible Rich Internet Applications) additional markup really improved semantics using:

  • landmarks to identify regions in a page, for instance: role="banner" ;
  • live regions to indicate that an element will be updated, for instance: aria-live="polite";
  • roles to describe the type of widgets presented, for instance: role="alert";
  • properties: attributes that represent a data value associated with the element.

See the related WCAG 2.0 sections about semantics.

Tabbing order

Drupal 8 introduces the TabbingManager javascript feature. It enables to constrain tabbing order on the page and facilitates navigation with keyboards. It is really helpful to guide a non-visual user to the most important elements on the page and minimize confusion with screen readers.

See the related WCAG 2.0 section about keyboard operations.

Forms

Drupal 8 accessibility involves many improvements regarding forms in Drupal 8.

In Drupal 7, all errors were displayed by default on top of the form and fields were highlighted in red. It was not right for colorblind people to understand where the errors were.
In Drupal 8, there is an experimental option to enable form inline errors and an error icon is displayed next to the field. Nevertheless, note that this feature is not enabled by default as there are still some pending issues.

Screenshot of a password field highlighted in red with an error message below "The specified passwords do not match"

The error message is displayed below the field when the Form Inline Error module is enabled.

See the related WCAG 2.0 sections about error identification.

Regarding the form API, radios and checkboxes are now embedded in fieldsets to meet WCAG compliance. Indeed, grouping related elements will help screen readers to navigate in complex forms. Plus, all fields have a label associated with the right element using the “for” attribute.

Here is an example of HTML code for radio buttons:

Poll status

See the related WCAG technical section about fieldsets

Tables and views

As Views UI module is in core now, it became accessible.
The views tables markup is more semantic. Data cells are associated with header cells through “id” and “headers” attributes. It is also possible to add a <caption> element to explain the purpose of the table and a <summary> element to give an overview on how the data is organized and how to navigate the table.
Plus, the “scope” attribute enables to explicitly mark row and column headings.

Here is an example of a table HTML code generated by a view:

Content type <a href="https://blog.liip.ch/admin/structure/types/manage/article">Article</a> <a href="https://blog.liip.ch/admin/structure/types/manage/article">Article</a>

Content type

<a href="/admin/structure/types/manage/article">Article</a>

<a href="/admin/structure/types/manage/article">Article</a>

Details for the table

Description for details

See the related WCAG section about tabular information.

Hidden elements

Using "display:none;" CSS styling can be problematic as it will hide elements for both visual and non-visual users and consequently, screen readers will not be able to read them.
Drupal 8 accessibility maintainers decided to standardize in the naming convention of HTML5 Boilerplate using different classes to hide elements:

  • hidden“: hide an element visually and from screen readers;
  • visually-hidden“: hide an element only visually but available for screen readers;
  • invisible“: hide an element visually and from screen readers without affecting the layout.

Aural alerts

Users with visual impairment will not be able to see all visual updates of the page such as color changes, animations or texts appended to the content. In order to make these changes apparent in a non-visual way, Drupal provides the Drupal.announce() JavaScript method which creates an “aria-live” element on the page. This way, text appended to the node can then be read by a screen reading user agent.

Here is an example of a code using the aural alert:

Drupal.announce('Please fill in your user name', 'assertive');

1

Drupal.announce('Please fill in your user name', 'assertive');

The first parameter is a string for the statement, the second is the priority:

  • polite“: this is the default, polite statements will not interrupt the user agent;
  • assertive“: assertive statements will interrupt any current speech.

See the related WCAG technical section about how to use live regions to identify errors.

CKEditor WYSIWYG accessibility

Drupal community helped improving CKEditor accessibility.
First of all, the WYSIWYG editor now comes with keyboard shortcuts which are beneficial for both power users and keyboard-only users.
Drupal 8 implements more semantic elements. For instance, the user can create HTML tables with headers, caption and summary elements. <figure> and <figcaption> HTML5 tags are also available to add captions to images.
Moreover, every image added through CKEditor are required by default, as it is on image fields.

CKEditor module also introduces a language toolbar button so that users can select a part of text and specify the language used. Screen readers will be able then to choose the appropriate language for each content.

See the related WCAG technical section about language attributes.

Finally, there is an accessibility checker plugin for CKEditor. It is not in core yet as a CKEditor issue blocks its integration, you can find more information on the related Drupal issue queue. However, you will find a module that implements it currently: ckeditor_a11checker.

All these options will definitely help users to generate accessible contents.

Conclusion

Drupal core maintainers accomplished great enhancements regarding accessibility in Drupal 8. These accessibility features will definitively be beneficial to keyboard-only users, low-vision users and colorblind people but will also be good for the usability and the SEO of your website.
Nevertheless, there is still work to be done to make Drupal 8 core fully accessible and Drupal needs contributors to tackle the remaining issues.

If you want to learn more about Drupal 8 accessibility, you can watch the presentation about “How Drupal 8 makes your website more easily accessible” given by Mike Gifford, one of the main accessibility core maintainer for Drupal.

Aug 21 2016
Aug 21

The Drupal accessibility initiative started with some advancements in Drupal 7 to ensure that Drupal core followed the World Wide Web Consortium (W3C) guidelines: WCAG 2.0 (Web Content Accessibility Guidelines) and ATAG 2.0 (Authoring Tool Accessibility Guidelines).

Many elements introduced in Drupal 7 were improved and bugs discovered through intensive testing were addressed and integrated to Drupal 8 core as well. Let's take a tour of the accessibility in Drupal 8 !

Contrasts improved

Drupal's accessibility maintainers improved contrasts in core themes so people that suffer from colorblindness are able to visit websites clearly. It is also good when visiting the website under bright sunlight, on mobile for instance.

A screenshot that compares Bartik headers in Drupal 7.43 and Drupal 8

Color contrasts in Bartik theme in Drupal 7.43 and Drupal 8.

See the related WCAG 2.0 section about contrasts.

Alternative texts for images

The alternative text for images is really useful for blind people who use screen readers. They can understand the meaning of an image through short descriptive phrases. This alternative text is now by default a required field in Drupal 8.

A screenshot showing that the alternative text is required when uploading an image in Drupal 8.

The alternative text for an image is required by default in Drupal 8 content edition.

See the related WCAG 2.0 section about alternative texts.

More semantics

Many accessibility improvements are hard to see as it involves semantics. Drupal 8 uses HTML5 elements in its templates which add more meaning into the code. For instance, assistive technology such as screen readers can now interpret elements like <header>, <footer> or <form>.

Moreover, WAI-ARIA (Web Accessibility Initiative – Accessible Rich Internet Applications) additional markup really improved semantics using:

  • landmarks to identify regions in a page, for instance: role="banner" ;
  • live regions to indicate that an element will be updated, for instance: aria-live="polite";
  • roles to describe the type of widgets presented, for instance: role="alert";
  • properties: attributes that represent a data value associated with the element.

See the related WCAG 2.0 sections about semantics.

Tabbing order

Drupal 8 introduces the TabbingManager javascript feature. It enables to constrain tabbing order on the page and facilitates navigation with keyboards. It is really helpful to guide a non-visual user to the most important elements on the page and minimize confusion with screen readers.

See the related WCAG 2.0 section about keyboard operations.

Forms

Drupal 8 accessibility involves many improvements regarding forms in Drupal 8.

In Drupal 7, all errors were displayed by default on top of the form and fields were highlighted in red. It was not right for colorblind people to understand where the errors were.

In Drupal 8, there is an experimental option to enable form inline errors and an error icon is displayed next to the field. Nevertheless, note that this feature is not enabled by default as there are still some pending issues.

Screenshot of a password field highlighted in red with an error message below "The specified passwords do not match"

The error message is displayed below the field when the Form Inline Error module is enabled.

See the related WCAG 2.0 sections about error identification.

Regarding the form API, radios and checkboxes are now embedded in fieldsets to meet WCAG compliance. Indeed, grouping related elements will help screen readers to navigate in complex forms. Plus, all fields have a label associated with the right element using the “for” attribute.

Here is an example of HTML code for radio buttons:

Poll statusClosedActive

See the related WCAG technical section about fieldsets

Tables and views

As Views UI module is in core now, it became accessible.

The views tables markup is more semantic. Data cells are associated with header cells through “id” and “headers” attributes. It is also possible to add a <caption> element to explain the purpose of the table and a <summary> element to give an overview on how the data is organized and how to navigate the table.

Plus, the “scope” attribute enables to explicitly mark row and column headings.

Here is an example of a table HTML code generated by a view:

Content type

<a href="https://www.liip.ch/admin/structure/types/manage/article">Article</a> <a href="https://www.liip.ch/admin/structure/types/manage/article">Article</a>

<table class="views-table views-view-table cols-2">Caption for the table Details for the tableDescription for details
  <tbody>
    <tr>
      <th id="view-title-table-column" class="views-field views-field-title" scope="col">Title</th>
    </tr>
    <tr>
      <td class="views-field views-field-title" headers="view-title-table-column">Premo Quae Vero</td>
      <td class="views-field views-field-title" headers="view-title-table-column">Capto Dolor</td>
    </tr>
  </tbody>
</table>

See the related WCAG section about tabular information.

Hidden elements

Using "display:none;" CSS styling can be problematic as it will hide elements for both visual and non-visual users and consequently, screen readers will not be able to read them.

Drupal 8 accessibility maintainers decided to standardize in the naming convention of HTML5 Boilerplate using different classes to hide elements:

  • hidden“: hide an element visually and from screen readers;
  • visually-hidden“: hide an element only visually but available for screen readers;
  • invisible“: hide an element visually and from screen readers without affecting the layout.

Aural alerts

Users with visual impairment will not be able to see all visual updates of the page such as color changes, animations or texts appended to the content. In order to make these changes apparent in a non-visual way, Drupal provides the Drupal.announce() JavaScript method which creates an “aria-live” element on the page. This way, text appended to the node can then be read by a screen reading user agent.

Here is an example of a code using the aural alert:

Drupal.announce('Please fill in your user name', 'assertive');

The first parameter is a string for the statement, the second is the priority:

  • polite“: this is the default, polite statements will not interrupt the user agent;
  • assertive“: assertive statements will interrupt any current speech.

See the related WCAG technical section about how to use live regions to identify errors.

CKEditor WYSIWYG accessibility

Drupal community helped improving CKEditor accessibility.

First of all, the WYSIWYG editor now comes with keyboard shortcuts which are beneficial for both power users and keyboard-only users.

Drupal 8 implements more semantic elements. For instance, the user can create HTML tables with headers, caption and summary elements. <figure> and <figcaption> HTML5 tags are also available to add captions to images.

Moreover, every image added through CKEditor are required by default, as it is on image fields.

CKEditor module also introduces a language toolbar button so that users can select a part of text and specify the language used. Screen readers will be able then to choose the appropriate language for each content.

See the related WCAG technical section about language attributes.

Finally, there is an accessibility checker plugin for CKEditor. It is not in core yet as a CKEditor issue blocks its integration, you can find more information on the related Drupal issue queue. However, you will find a module that implements it currently: ckeditor_a11checker.

All these options will definitely help users to generate accessible contents.

Conclusion

Drupal core maintainers accomplished great enhancements regarding accessibility in Drupal 8. These accessibility features will definitively be beneficial to keyboard-only users, low-vision users and colorblind people but will also be good for the usability and the SEO of your website.

Nevertheless, there is still work to be done to make Drupal 8 core fully accessible and Drupal needs contributors to tackle the remaining issues.

If you want to learn more about Drupal 8 accessibility, you can watch the presentation about “ How Drupal 8 makes your website more easily accessible” given by Mike Gifford, one of the main accessibility core maintainer for Drupal.

Aug 17 2016
Aug 17

Last week Drupalaton 2016 took place. With about 150 registrations this was the largest Drupalaton so far. The organizers did an amazing job in coping with this mass. There were two session threads and a sprint room. Of the many interesting presentations I would like to mention Fabian Bircher’s “Configuration Management: theory and practice” (a must for everyone who gets lost while trying to work in a team on a Drupal8 project) , Pieter Frenssen’s “Working with REST APIs”  (it was good to see how simple it is in Drupal8) and “Drupal 8 Media” from Pónya Péter, Szanyi Tamás and Rubén Teijeiro (seems we have a huge step forward in media handling since Drupal7!). I held a session on caching in Drupal 8 which was the shortened version the one I did on Drupal Developer Days in Milan.

Liip was a silver sponsor of the event.

Finally, some pictures on the Friday ship cruise. Thanks to Brainsum for sponsoring it!

Aug 16 2016
Aug 16

Last week Drupalaton 2016 took place. With about 150 registrations this was the largest Drupalaton so far. The organizers did an amazing job in coping with this mass. There were two session threads and a sprint room. Of the many interesting presentations I would like to mention Fabian Bircher's “Configuration Management: theory and practice” (a must for everyone who gets lost while trying to work in a team on a Drupal8 project) , Pieter Frenssen's “Working with REST APIs”  (it was good to see how simple it is in Drupal8) and “Drupal 8 Media” from Pónya Péter, Szanyi Tamás and Rubén Teijeiro (seems we have a huge step forward in media handling since Drupal7!). I held a session on caching in Drupal 8 which was the shortened version the one I did on Drupal Developer Days in Milan.

Liip was a silver sponsor of the event.

Finally, some pictures on the Friday ship cruise. Thanks to Brainsum for sponsoring it!

Jul 11 2016
Jul 11

After having contributed to the official styleguide of the Swiss Federal Government and having implemented it on a couple of websites, we decided to go further and bring these styleguide into a theme for Drupal, a well-known, pluripotent and robust CMS we implement regularly at Liip.

Screenshot of Drupal theme for the Swiss Confederation

The current result is a starterkit providing the essential bricks to start in a snap a website project for the federal government running with Drupal 8, based on the version 3 of the official styleguide.

Navigation modules, multilingual environnement per default (German, French, Italian, Rumantch and English), responsive layout following the Web Content Accessibility Guidelines, we threw the fundamental stones for bootstraping a web platform for the Confederation.

con~foederatio : to build a league, together.

In other words, joining forces, to support a common cause. From the very start of the project we decided to opensource the code, as a participatory initiative.
Learn more about this intent.

Any developer working on a new website for the swiss government can now quickly start developing with this Drupal starterkit, then modify, contribute and improve it collegially. Pulling requests and opening issues on GitHub is the recommended way to help us extend further the project.

What’s inside the box

The Bund-Starterkit provides theme and elements based on the official styleguide (version 3.0.0) of the Swiss Federal Administration.

This starterkit also contains a base to quickly implement a website running on Drupal 8 for the Swiss Federal Administration. Currently, it provides the following Drupal and frontend elements:

  • Multilingual main navigation blocks
  • Multilingual service navigation blocks
  • Multilingual footer service navigation blocks
  • Logo block
  • Language switcher block with German, French, Italian, Rumantsch enabled
  • All the assets (CSS, SASS. JS files) provided by the official styleguide
  • A ready-to-use SASS workflow

Installation process, an overview

Please check the Readme file to quickly start your project. But let’s have a look at the details of the installation process. First of all, Composer (a PHP dependencies manager) is binding together for us the following repositories:

After downloading the sources with Composer and setting your vhost and hosts files, you have two options. Continuing with a few drush commands to run the Drupal installation process, or following the installation wizard in the browser. If you choose this last option, don’t forget to select the «Bund profile» option when the wizard ask you to choose a profile:

Chose a profile for the Drupal theme for the Swiss Confederation

Continue with the last steps of the wizard and that’s it. you should be able to see an empty Drupal 8 website, painted with the swiss administration’s corporate sauce.

Inserting menus content

With the help of a .CSV file and some drush commands, you can quickly import your menu structure. Once done, create and assign your content the the freshly created menu items through the Drupal administration interface.

Theming

Don’t forget to create a personal Drupal sub-theme from the bund_drupal_starterkit_theme, as a Drupal best practice.  Don’t edit the existing theme directly or you could loose your changes after a future update.

Frontend

This starterkit use the official styleguide (version 3.0.0) as a submodule. All existing CSS/JS files and assets are imported and available per default, but not necessary integrated as a drupal module at the moment. We highly encourage you to check the official styleguide before adding any new CSS style or JS files to your project. Based on the existing styles, it should be possible to create a lot of Drupal templates without modifying or extending any CSS. And as already said, we invite you to share any Drupal template matching the styleguide you would develop for your project.

Further reading

Jul 10 2016
Jul 10

After having contributed to the official styleguide of the Swiss Federal Government and having implemented it on a couple of websites, we decided to go further and bring these styleguide into a theme for Drupal, a well-known, pluripotent and robust CMS we implement regularly at Liip.

After having contributed to the official styleguide of the Swiss Federal Government and having implemented it on a couple of websites, we decided to go further and bring these styleguide into a theme for Drupal, a well-known, pluripotent and robust CMS we implement regularly at Liip.

Screenshot of Drupal theme for the Swiss Confederation

The current result is a starterkit providing the essential bricks to start in a snap a website project for the federal government running with Drupal 8, based on the version 3 of the official styleguide.

Navigation modules, multilingual environnement per default (German, French, Italian, Rumantch and English), responsive layout following the Web Content Accessibility Guidelines, we threw the fundamental stones for bootstraping a web platform for the Confederation.

con~foederatio : to build a league, together.

In other words, joining forces, to support a common cause. From the very start of the project we decided to opensource the code, as a participatory initiative.

Learn more about this intent.

Any developer working on a new website for the swiss government can now quickly start developing with this Drupal starterkit, then modify, contribute and improve it collegially. Pulling requests and opening issues on GitHub is the recommended way to help us extend further the project.

What's inside the box

The Bund-Starterkit provides theme and elements based on the official styleguide (version 3.0.0) of the Swiss Federal Administration.

This starterkit also contains a base to quickly implement a website running on Drupal 8 for the Swiss Federal Administration. Currently, it provides the following Drupal and frontend elements:

  • Multilingual main navigation blocks
  • Multilingual service navigation blocks
  • Multilingual footer service navigation blocks
  • Logo block
  • Language switcher block with German, French, Italian, Rumantsch enabled
  • All the assets (CSS, SASS. JS files) provided by the official styleguide
  • A ready-to-use SASS workflow

Installation process, an overview

Please check the Readme file to quickly start your project. But let's have a look at the details of the installation process. First of all, Composer (a PHP dependencies manager) is binding together for us the following repositories:

After downloading the sources with Composer and setting your vhost and hosts files, you have two options. Continuing with a few drush commands to run the Drupal installation process, or following the installation wizard in the browser. If you choose this last option, don't forget to select the «Bund profile» option when the wizard ask you to choose a profile:

Chose a profile for the Drupal theme for the Swiss Confederation

Continue with the last steps of the wizard and that's it. you should be able to see an empty Drupal 8 website, painted with the swiss administration's corporate sauce.

Inserting menus content

With the help of a .CSV file and some drush commands, you can quickly import your menu structure. Once done, create and assign your content the the freshly created menu items through the Drupal administration interface.

Theming

Don't forget to create a personal Drupal sub-theme from the bund_drupal_starterkit_theme, as a Drupal best practice.  Don't edit the existing theme directly or you could loose your changes after a future update.

Frontend

This starterkit use the official styleguide (version 3.0.0) as a submodule. All existing CSS/JS files and assets are imported and available per default, but not necessary integrated as a drupal module at the moment. We highly encourage you to check the official styleguide before adding any new CSS style or JS files to your project. Based on the existing styles, it should be possible to create a lot of Drupal templates without modifying or extending any CSS. And as already said, we invite you to share any Drupal template matching the styleguide you would develop for your project.

Further reading

Jul 07 2016
Jul 07

Eight months ago Drupal 8.0.0 was released. Exciting news for drupalists. Since then comparing D8’s features to its predecessor is a topic in daily business. «Can drupal 8 do what we can do now with 7 today?”. After playing around with D8 i get the feeling some crucial features are missing. Dries invited people to tell ”why we not use or migrate to drupal 8” – and got a clear answer: A majority of drupalist (60%) are waiting for certain modules. So the follow up question would be what are these modules.

On the fly my top 10 wishlist would be:

  • pathauto
  • token
  • webform
  • metadata
  • views_bulk_operations
  • flag
  • rules
  • xmlsitemap
  • redirect
  • search_api

Today it seems quite difficult to get a good overview of D8 ecosystem. Also because some module development moved to github to have a better collaboration tool. I was irritated to see no D8 version of the webform module in the download section on drupal.org  – That’s a module with 1/2 million downloads for D7. Comments on this issue gives some answers. Without committed maintainers from the beginning the porting takes much longer. A highly complex module like webform probably needs almost complete rewrite to fit into the new core of D8. Porting module from D7 to D6 was much easier. For forms we could use in some cases the core Form API, core contact forms or the eform module. But our clients would most likely miss out on the experience of D7s webform module.

Under the hood Drupal 8 core changed significantly. Symfony2 for example is now playing its music to give us new possibilities. I guess in some cases there are new solutions we have yet to discover. From a suitebuilder point of view, D8 is delightfully similar to what we know from D7. However, it will take some getting used to not trying to add the old modules we know to this new architecture.

In the end the importance of a variety of mature modules that play together nicely is crucial when it comes to efficiency, maintainability and stability of a project…

“I am confident that Drupal 8 will be adopted at “full-force” by the end of 2016.”
Dries Buytaert

Drupal is a registered trademark of Dries Buytaert.

Jul 06 2016
Jul 06
  • 07 July 2016
  • Lennart Jegge

Eight months ago Drupal 8.0.0 was released. Exciting news for drupalists. Since then comparing D8's features to its predecessor is a topic in daily business. «Can drupal 8 do what we can do now with 7 today?”. After playing around with D8 i get the feeling some crucial features are missing. Dries invited people to tell ” why we not use or migrate to drupal 8” – and got a clear answer: A majority of drupalist (60%) are waiting for certain modules. So the follow up question would be what are these modules.

On the fly my top 10 wishlist would be:

  • pathauto
  • token
  • webform
  • metadata
  • views_bulk_operations
  • flag
  • rules
  • xmlsitemap
  • redirect
  • search_api

Today it seems quite difficult to get a good overview of D8 ecosystem. Also because some module development moved to github to have a better collaboration tool. I was irritated to see no D8 version of the webform module in the download section on drupal.org  – That's a module with 1/2 million downloads for D7. Comments on this issue gives some answers. Without committed maintainers from the beginning the porting takes much longer. A highly complex module like webform probably needs almost complete rewrite to fit into the new core of D8. Porting module from D7 to D6 was much easier. For forms we could use in some cases the core Form API, core contact forms or the eform module. But our clients would most likely miss out on the experience of D7s webform module.

Under the hood Drupal 8 core changed significantly. Symfony2 for example is now playing its music to give us new possibilities. I guess in some cases there are new solutions we have yet to discover. From a suitebuilder point of view, D8 is delightfully similar to what we know from D7. However, it will take some getting used to not trying to add the old modules we know to this new architecture.

In the end the importance of a variety of mature modules that play together nicely is crucial when it comes to efficiency, maintainability and stability of a project…

“I am confident that Drupal 8 will be adopted at “full-force” by the end of 2016.”

Dries Buytaert

Drupal is a registered trademark of Dries Buytaert.

Lennart Jegge

Software Developer, Drupalista

Related services
  • Topics
  • Tags
Jun 27 2016
Jun 27

DDD is mostly for – surprise! – Drupal developers. This year it took place between 21 and 26 of June in Milan. People were on code sprints all week long and on Thursday, Friday and Saturday there were sessions and workshops as well.

I went to 2 sessions. The keynote of Bojan Živanović was about building reusable php libraries. Bojan is the architect behind Drupal Commerce 2 which is a prominent example of adopting the “leave the Drupal island” principle. They are not only advocating the usage of external solutions in Drupal but also creating libraries that are usable outside Drupal.

The session of Major Zsófi about organizing Drupal events could not have come from a more authentic source. She shared her experience about the practical aspects of building a community and the importance of providing coffee.

All session recordings are or will be available online.

I attended three workshops. A really excellent one by Florian Loretan was about the trending search solution, elasticsearch. Pieter Frenssen had a workshop about Automated testing in Drupal 8. For me this proved to be the most valuable one since I could not keep up with the changes in this field since Drupal 7 and I need it in my contrib work. All my respects to Pieter who was able to present for 3.5 hours in a way that noone fell asleep even though we were just after lunch.

The third workshop I attended was my own 2 hours workshop about Caching in Drupal 8. I learnt a lot about this important topic during preparation and since only around one person left the room it might have been useful for the audience as well.

In the sprint room I joined the Commerce team. The team seemed to have been cursed. A laptop was stolen from the sprint site on Wednesday. Then on Thrusday night Bojan’s MacBook got also stolen from a restaurant with days of uncommitted work. Fortunately, with the effective help of the organizers (which among others included providing the victims a spare laptop and taking them to the police to file a report) they could participate in the sprints only with a minimal amount of delay. As a result we could finish several issues in the Commerce, Commerce Migrate, Token and Address modules.

Sightseeing with drupalists

Sightseeing with drupalists

But the most important part of DDD was the social aspects. I met old friends and got to know new interesting people. Wednesday evening there was a quantitywise challenging dinner for speakers. On other nights we visited several parts of the beautiful city of Milano. Huge thanks to all the organisers, you did an amazing job! Hope to see you next year!

Jun 20 2016
Jun 20

It has been nearly 7 months since Drupal 8 first release and as a developer, I am still in the learning process. It can be hard sometimes to know what is going wrong with your code and how to solve it. I will tell you about few things to know on how to develop and debug Drupal 8 projects and continue learning, learning and learning !

Disabling cache

First of all, to avoid having a crazy terminal with thousands of drupal cr hits, you can disable Drupal caching during development. You need to copy and rename sites/example.settings.local.php file to sites/default/settings.local.php. Then uncomment/update some values:

  • uncomment this line to enable the “null cache service”:

    $settings['container_yamls'][] = DRUPAL_ROOT . '/sites/development.services.yml';

    1

    $settings['container_yamls'][] = DRUPAL_ROOT . '/sites/development.services.yml';

  • uncomment these lines to disable CSS/JS aggregation:

    $config['system.performance']['css']['preprocess'] = FALSE; $config['system.performance']['js']['preprocess'] = FALSE;

    $config['system.performance']['css']['preprocess'] = FALSE;

    $config['system.performance']['js']['preprocess'] = FALSE;

  • uncomment these lines to disable the render cache and the dynamic page cache:

    $settings['cache']['bins']['render'] = 'cache.backend.null'; $settings['cache']['bins']['dynamic_page_cache'] = 'cache.backend.null';

    $settings['cache']['bins']['render'] = 'cache.backend.null';

    $settings['cache']['bins']['dynamic_page_cache'] = 'cache.backend.null';

  • you can allow test modules and themes to be installed if needed with:

    $settings['extension_discovery_scan_tests'] = TRUE;

    1

    $settings['extension_discovery_scan_tests'] = TRUE;

To include this file as part of Drupal’s settings file, open sites/default/settings.php file and uncomment these lines:

if (file_exists(__DIR__ . '/settings.local.php')) { include __DIR__ . '/settings.local.php'; }

if (file_exists(__DIR__ . '/settings.local.php')) {

    include __DIR__ . '/settings.local.php';

}

Then, to disable Twig caching, open sites/development.services.yml file and add the following settings:

parameters: twig.config: debug: true auto_reload: true cache: false

parameters:

    twig.config:

        debug: true

        auto_reload: true

        cache: false

Finally, rebuild the Drupal cache and it is done !

Displaying errors

As in Drupal 7, you can set different levels of errors display (by visiting this page: /admin/config/development/logging in the administration interface):

  • None
  • Errors and warnings
  • All

In Drupal 8, there is a fourth level called “All messages, with backtrace information”. This is native to Drupal core and it allows to display the error backtrace in the message area.

You can also adjust the level of errors in your local setting file:

$config['system.logging']['error_level'] = 'verbose';

1

$config['system.logging']['error_level'] = 'verbose';

Creating log messages

Developers from Drupal 7 know the Database Logging module that allows to log messages in the database using the famous watchdog() function.
Well, this module still exists in Drupal 8 but the function has a replacement : the Drupal 8 logger class.
It looks like this:

// Logs an error \Drupal::logger('my_module')->error($message);

// Logs an error

\Drupal::logger('my_module')->error($message);

Let’s have a look at the different parts of the code:

  • \Drupal::logger(‘my_module’) is the helper method that quickly creates a copy of the logger service. As a parameter, it takes the module name from where we log the information;
  • ->error: this is the severity-level method (it can be debug, info, notice, warning, error, critical, alert, emergency);
  • $message is the log message. It can be a simple string or it can contain some placeholders. In this case, you can pass the associative array (placeholders as keys) as a second parameter.

All the messages created with the logger service can be viewed in the reports page of the administration interface in /admin/reports/dblog as it used to be in Drupal 7.

Debugging Twig templates

Drupal core comes with a theming debug mode that is really helpful for local environments. To enable it, copy-and-paste the following code into sites/default/services.yml file (if you haven’t already added these lines in development.services.yml):

parameters: twig.config: debug: true

parameters:

    twig.config:

        debug: true

Thanks to this mode, it will be easier to find out which portion of HTML code has been written in which templates: in the source code, you will see each part of Twig templates (pages, nodes, blocks, menus etc.) surrounded by HTML comments that contain the matching suggestion templates. The one in current use will be checked.
Make sure to enable comments in your web browser debug tab and also note that this feature has been backported to Drupal 7.

Twig debug in comments

Used templates mentioned in HTML comments

Inside a Twig template, you will also be able to use {{ dump(my_variable) }} syntax to print a variable content.
If you have Devel Kint module installed, you can use {{ kint(my_variable) }} to dump the variable in a nice formatted structure: you can hide/show levels of arrays/objects which is very helpful as Drupal variables can have many levels inside. Kint is indeed the successor of the krumo() function from Drupal 7.

content_attributes printed with kint function

Print content_attributes variable with kint from node.html.twig template

Profiling pages

When dealing with performance issues, profiling a website will help finding what is the root cause.
The well-known debugging module Devel offers in its latest version a Webprofiler. It is actually a (partial) port of the Symfony profiler and it displays a footer bar on every page with useful data collectors such as:

  • Drupal current version;
  • PHP configuration (current version, loaded modules);
  • route and controller name;
  • page load timeline and memory use;
  • query time and number of queries;
  • number of blocks loaded and rendered;
  • number of views;
  • number of modules and themes available;
  • cache statistics.
Webprofiler bar in Drupal 8

Webprofiler bar in Drupal 8

By clicking on each section, you will be redirected to a specific page with more details about the collected data. For instance, if you look at the page request details, you will see which was the matching route, the route object with the passed parameters, response headers etc. Well, all the information needed to debug requests.

Webprofiler - details of a page request

Details of a page request provided by Webprofiler

Using command line tools

The Drupal Console project is a powerful command line tool that makes use of the Symfony Console and other third-party components. It is complementary to Drush and allows you to generate code to build modules and themes (code scaffolding), to interact with your Drupal installation and help to debug your code.

Once the Drupal Console installed, you can run the drupal list command to show all available commands.

Here are some useful commands for debug:

  • drupal check: check system requirements;
  • drupal site:status: show current Drupal installation status (Versions of Drupal, PHP, MySQL and libraries, updates status, cron last run, database connection etc.);
  • drupal database:table:debug: show all tables of the database;
  • drupal database:table:debug my_table: show columns of my_table table;
  • drupal config:debug: list all configurations;
  • drupal config:debug image.settings: show configuration for image.settings;
  • drupal config:settings:debug: display current key:value from the settings file;
  • drupal container:debug: display all services ID with the matching class name;
  • drupal router:debug: display all route names with the matching paths;
  • drupal router:debug dblog.overview: display route information about dblog overview page;
  • drupal database:log:debug: display current log events;
  • drupal database:log:debug 107: display one log event in details;
  • drupal site:statistics: show some statistics about the website (number of modules enabled/disabled, number of users and comments etc.).

The Drupal Console is already used by many companies well known in the Drupal community such as Acquia, Amazee Labs, Phase2 and Commerce Guys and is becoming the standard command line tool for Drupal 8.
But note that the current version is not yet fully compatible with Drupal 8.1.x, for instance, there are still some issues with migration commands. The team of the project is currently looking for some financial support and more contributors to get a full stable release.

For more information about the project, check out the official website and the documentation.

May 25 2016
May 25

Today I will describe a way to handle multiple teams with their own private file folders using the IMCE module.

Let’s pretend that we have to develop a website called awesome-website.com which consists of three (or more) different teams. The team structure could look as followed:

Website Groups

Every team should only be allowed to edit their own pages but no page from any other team. Therefore it would also make sense to separate the team’s file folders so that the files can be stored separately to secure its privacy.
Of course, we could simply add three IMCE profiles and define their folder access rights individually there. But what about when working with 10 teams? Or 50? Or even more? Then we definitely would prefer a more flexible solution.
Thankfully, IMCE ships with the ability to define user folders by PHP execution, how awesome! But in order to achieve this, we’ll have to set up teams as taxonomy terms first and reference them from our user entities.

Setting up the “Teams” taxonomy vocabulary

First things first: Let’s create a new taxonomy vocabulary called “Teams”. For every team that we will have on our website, we have to create a new taxonomy term in this vocabulary.
Before adding any teams as taxonomy terms though, we’ll have to add a new field called “FTP Folder” to the taxonomy vocabulary.
This field will specify the name of every team’s root folder. So, naturally it shouldn’t contain any spaces or other wicked special characters and it should be URL readable.
In order not to face any unusual results later, it is recommended to configure this field as required.

Afterwards, we can add our three terms, “Team Alpha”, “Team Beta” and “Team Gamma”.
As value for their FTP Folders, we use “team-alpha”, “team-beta” and so on.

That’s it for the taxonomy part! Now let’s link this information to the team’s users.

Adding a taxonomy term reference field to the user entity

In my case, I didn’t have multiple roles for the teams. I only had one, called “Team member”. Because every team has exactly the same rights as the others, maintaining only one role suited me best.
For really special cases, I could always just create a new role with the special permissions.

So, how do we link users to their teams the easiest? Exactly, by just adding a taxonomy term reference field to the user entity!
Let’s call this field “Team” and reference our previously created taxonomy vocabulary “Teams” with it.

Now, when adding a new user, we can select it’s team belonging and IMCE will be able to grab the needed information from there.
Yes, IMCE will be able to do that but it’s not doing it yet.
Getting the teams ftp folder for the current user is still something we have to code, so let’s proceed to the next step.

Writing a custom function to provide the accessible directories for an user

Now we need to provide IMCE the information that we’ve set up before.
We’ve created users belonging to teams, which hold the FTP root folder name for the teams.
What’s left to do, is to write a function (ideally in a custom module, in my example the module is called “awesome_teams”), that combines all information and returns it to IMCE.
Following function would do that for us:

function awesome_teams_imce($user) { $user_folders = array('cms/teams/all'); $user_wrapper = entity_metadata_wrapper('user', $user); $user_teams = $user_wrapper->field_team->value(); foreach ($user_teams as $user_team) { $user_team_wrapper = entity_metadata_wrapper('taxonomy_term', $user_team); array_push($user_folders, 'cms/teams/' . $user_team_wrapper->field_ftp_folder->value()); } return $user_folders; }

1

2

3

4

5

6

7

8

9

10

11

12

13

function awesome_teams_imce($user) {

  $user_folders = array('cms/teams/all');

  $user_wrapper = entity_metadata_wrapper('user', $user);

  $user_teams = $user_wrapper->field_team->value();

  foreach ($user_teams as $user_team) {

    $user_team_wrapper = entity_metadata_wrapper('taxonomy_term', $user_team);

    array_push($user_folders, 'cms/teams/' . $user_team_wrapper->field_ftp_folder->value());

  }

  return $user_folders;

}

The function expects an user object as argument and will return an array of strings containing all the folder names an user is allowed to access.
Our folder structure would look like this:

  • sites/default/files/cms
  • sites/default/files/cms/teams
  • sites/default/files/cms/teams/all
  • sites/default/files/cms/teams/team-alpha
  • sites/default/files/cms/teams/team-beta
  • sites/default/files/cms/teams/team-gamma

Note: The folder “cms/teams/all” is a special folder and every user is allowed to access it.
It will be used to save files which are used globally over multiple or even all teams.

What our code does, is actually looping over all assigned teams for the given user (yes, an user can be in multiple teams!), and adding the teams ftp folder names to the array of accessible folders.

There is no “hook_imce” hook, the “_imce” in the function name does nothing till now. You can also name your function differently. The link from IMCE to our function is something we have to set up in an IMCE profile.
Let’s proceed to the last step then, shall we?

Creating the IMCE profile “Team member”

Now, as the last step, let’s create an IMCE profile called “Team member”. You’re free to define any settings as you like, there’s only one thing that will be special about this profile: The accessible directories path.

Instead of writing something constant as “cms/teams/team-alpha”, we’ll write “php: return awesome_teams_imce($user);” here.
So, the setting should look like this:

imce-profile-settings

Now save the profile and you are done!

As soon as one team member now accesses the IMCE page (either via /imce or by the configured file/image fields), he will only see his team’s directories and the special directory “all” which is meant for exchange.

This wasn’t that difficult, was it?

I hope I was able to give you an insight on how to solve more complicated file permission issues with IMCE.
Don’t forget to give feedback, ask questions and follow our blog if you want to read more about our Drupal experiences at Liip!

May 11 2016
May 11

tl;dr Keep your site modules up-to-date.

Drupal is famous for its security and it also does not miss a chance to boast about it. However, security does not come automatically, steps need to be taken to ensure it. One of the most important of these steps is keeping site modules and core updated. Failing to do so can lead to incidents like the recent Panama papers incident where an outdated WordPress and Drupal site might have played a role in the data leak.

So what does Drupal do for you to make security easier?

The Drupal Security Team was set up in 2005. It has around 40 security experts from all around the globe who communicate through private channels.

When a security issue is discovered in Drupal (let it be a contributed module, a theme or core itself) an issue is created in the security issue tracker. The issue is visible only for a small group of people (usually the security team, the maintainer of the affected module and the reporter of the issue) to prevent the vulnerability to be exploited before a fix is created. When a fix is ready, the security team issues a public Security Advisory that has informations on the affected module, the security risk level and the solution for the issue (which is usually updating the module).

Security issues are reported almost daily to the security team but some of these are non valid. For example, only modules with a stable release (i.e. non-dev/alpha/beta/rc) are considered by the Security Team. Still, 2015 saw 160 security advisories. The most frequent issues are related to XSS.

Security updates are released on Wednesdays. For core that’s usually the third Wednesday of the month, for contrib it can be any Wednesday. This does not mean that a security release appears on every Wednesday, only that site administrators should look out for them.

In Drupal 8 there are several security improvements. One of them is Twig autoreplacing which drastically decreases the chances for a piece of code to have a XSS vulnerability. Another source of insecurities, the PHP filter module has been removed from core. Also, the routing system now has support for protection against CSRF attacks by providing tokens to urls.

After learning what Drupal does for security, it’s time to see what site administrators should make sure of. Keeping the following 3 things in mind you as site admin should be fine for 95% of the cases.   (These are only the Drupal-specific aspects, we won’t go into general security principles.)

  1. To make sure you have an up-to-date site follow at least one of the security news channels. There are some RSS feeds, a twitter account and also a newsletter. Update your site as soon as a security update is released.
  2. There are several modules improving security or helping in finding security issues. A few of these are Security review, Paranoia and Two factor authentication.
  3. A Drupal-specific hosting provider can also have its benefits. For example, in the case of the infamous 2014 Drupalgeddon security advisory Pantheon and Acquia Cloud sites were protected against attacks without any action taken by the site administrator.

If you have not done it yet go and check your module update status page right now.

This blog post is heavily based on the Lullabot podcast on Drupal Security.

For further links we recommend the Barcelona presentation of scor and klausi.

May 04 2016
May 04

We at Liip AG believe, that the migration API is the best and most efficient way to import data into Drupal. Here are some reasons, why you should use migrate instead of the feeds module or any other custom importer modules:

  • Since Drupal 8, Migrate API is part of Drupal core
  • Migrate will be maintained and supported as long as Drupal 8 exists as it provides the upgrade path for older Drupal versions to Drupal 8
  • Migrate is sponsored by Acquia and mainly supported by Mike Ryan, a well-known and skilled Drupal developer.
  • Migrate has out of the box support for all important Drupal objects such as nodes, users, taxonomy terms, users, files, entities and comments.
  • Migrate has a Drush integration, that allows you, to run import tasks from command-line or via cron job
  • Migrate maintains a mapping-table, has rollback functionality and even supporting a highwater field, that allows to import only new or changed datasets.
  • Migrate is well documented and there is an example module.

Getting started with Drupal 8 Migrate Module

The Migrate 8 module in core is only an API. There is no user interface. This makes it difficult for new developer to start with Migrate 8.

I suggest you, to install the below listed extension modules right away before you start developing if you want to realize the full potential of migrate:

Migrate Plus (https://www.drupal.org/project/migrate_plus)

  • Extends the migration framework with groups
  • Delivers a well documented example module

Migrate Tools (https://www.drupal.org/project/migrate_tools)

  • Provides Drush commands for running and managing migrations in Drupal 8

Migration Source Plugins

Installing a fully working Drupal 8 Migrate setup using composer

Instead of starting now to download in install all the module mentioned above, you can use my installation profile based on a composer.json file. Because a lot of modules with specific version are involved, I have prepared a fully working migrate example environment for a Liip hackday.

If you want to quickly start with the migrate module, head over to my github repository and install Drupal 8 Migrate using composer and drush. You just need to follow the instruction in the README.md

https://github.com/ayalon/drupal8-migrate

Comparing Drupal Migrate 7 with Drupal Migrate 8

Some of you might already have used Migrate 7. A traditional mapping was done in the constructor of a Migration class:

public function __construct($arguments) { parent::__construct($arguments); $this->description = t('Page Placeholder import'); // Set up our destination - nodes of type migrate_example_beer. $this->destination = new MigrateDestinationNode('page'); $this->csvFile = DRUPAL_ROOT . '/docs/navigation.csv'; $this->map = new MigrateSQLMap($this->machineName, array( 'ID' => array( 'type' => 'int', 'unsigned' => TRUE, 'not null' => TRUE, ), ), MigrateDestinationNode::getKeySchema() ); $this->source = new MigrateSourceCSV($this->csvFile, array(), array('header_rows' => 1, 'delimiter' => ';')); // Force update. $this->highwaterField = array(); // Mapped fields. $this->addFieldMapping('title', 'name'); $this->addFieldMapping('uid')->defaultValue(1); $this->addFieldMapping('status')->defaultValue(1); $this->addFieldMapping('promote')->defaultValue(0); $this->addFieldMapping('sticky')->defaultValue(0); $this->addFieldMapping('language')->defaultValue('de'); // Unmapped destination fields. $this->addUnmigratedDestinations(array( 'body:format', 'changed', 'comment', 'created', 'is_new', 'log', 'revision', 'revision_uid', 'tnid', 'translate', )); }

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

public function __construct($arguments) {

  parent::__construct($arguments);

  $this->description = t('Page Placeholder import');

  // Set up our destination - nodes of type migrate_example_beer.

  $this->destination = new MigrateDestinationNode('page');

  $this->csvFile = DRUPAL_ROOT . '/docs/navigation.csv';

  $this->map = new MigrateSQLMap($this->machineName,

    array(

      'ID' => array(

        'type' => 'int',

        'unsigned' => TRUE,

        'not null' => TRUE,

      ),

    ),

    MigrateDestinationNode::getKeySchema()

  );

  $this->source = new MigrateSourceCSV($this->csvFile, array(), array('header_rows' => 1, 'delimiter' => ';'));

  // Force update.

  $this->highwaterField = array();

  // Mapped fields.

  $this->addFieldMapping('title', 'name');

  $this->addFieldMapping('uid')->defaultValue(1);

  $this->addFieldMapping('status')->defaultValue(1);

  $this->addFieldMapping('promote')->defaultValue(0);

  $this->addFieldMapping('sticky')->defaultValue(0);

  $this->addFieldMapping('language')->defaultValue('de');

  // Unmapped destination fields.

  $this->addUnmigratedDestinations(array(

    'body:format',

    'changed',

    'comment',

    'created',

    'is_new',

    'log',

    'revision',

    'revision_uid',

    'tnid',

    'translate',

  ));

}

In Migrate 8 this format has been replaced with yaml files. The same mapping as above looks like that in Drupal 8:

# Migration configuration id: page_node label: Dummy pages migration_group: liip source: plugin: page_node # Full path to the file. Is overridden in my plugin path: public://csv/navigation_small.csv # The number of rows at the beginning which are not data. header_row_count: 1 # These are the field names from the source file representing the key # uniquely identifying each node - they will be stored in the migration # map table as columns sourceid1, sourceid2, and sourceid3. keys: - ID destination: plugin: entity:node process: type: plugin: default_value default_value: page title: name uid: plugin: default_value default_value: 1 sticky: plugin: default_value default_value: 0 status: plugin: default_value default_value: 1 promote: plugin: default_value default_value: 0 'body/value': body 'body/summary': excerpt #Absolutely necessary if you don't want an error migration_dependencies: {}

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

# Migration configuration

id: page_node

label: Dummy pages

migration_group: liip

source:

plugin: page_node

# Full path to the file. Is overridden in my plugin

path: public://csv/navigation_small.csv

# The number of rows at the beginning which are not data.

header_row_count: 1

# These are the field names from the source file representing the key

# uniquely identifying each node - they will be stored in the migration

# map table as columns sourceid1, sourceid2, and sourceid3.

keys:

   - ID

destination:

plugin: entity:node

process:

type:

   plugin: default_value

   default_value: page

title: name

uid:

   plugin: default_value

   default_value: 1

sticky:

   plugin: default_value

   default_value: 0

status:

   plugin: default_value

   default_value: 1

promote:

   plugin: default_value

   default_value: 0

'body/value': body

'body/summary': excerpt

#Absolutely necessary if you don't want an error

migration_dependencies: {}

Understanding the new mapping with yaml files in Migrate 8

The mapping is quite straightforward.

  • First you have to define your Migrate source. In the example we have used a CSV source plugin. (https://www.drupal.org/node/2129649)
  • Then you can map the source fields to a Migrate destination. In our case, we use a node destination (https://www.drupal.org/node/2174881)
  • You can map now all source fields to destination fields, for example you map a column of the CSV file to the node title field
  • Every field can be processed and modified before it is passed to the final node field. There are a lot of useful process plugins like “default_value”, “callback” or “skip_if_empty”
  • You can find a list of all process plugins here: https://www.drupal.org/node/2129651
  • Of course you can easily write your own plugins and use them while migrating data

Example: Importing a menu tree and create dummy nodes using Drupal Migrate 8

For demonstration purpose I created a small module, that reads a menu tree from a CSV file and imports into Drupal.

The module is split into 2 tasks:

  1. Creating a page node for every row in the csv file
  2. Create a menu item for every row and attach it to the correct node

Migrate handles these dependencies in the yaml file:

config/install/migrate_plus.migration.menu_item.yml

1

config/install/migrate_plus.migration.menu_item.yml

Dependancies

migration_dependencies: required: - page_node

migration_dependencies:

required:

   - page_node

See the full module code here:

https://github.com/ayalon/drupal8-migrate/tree/master/web/modules/custom/migrate_menu

Developing your own Drupal 8 migrate modules and fighting caching issues

You have learned, that the whole migration mapping is now done in yaml files. But how about writing your own migration yaml files?

Unfortunately, there are some pitfalls for new Drupal 8 developers. Because of the Configuration Management Interface (https://www.drupal.org/documentation/administer/config) of Drupal 8, all yml files in the “config/install” directory are only imported when installing the module.

This is very impractical if you want to develop new configuration files. To address this, a module “Configuration Development” (https://www.drupal.org/project/config_devel) which resolves the caching issues can be installed. It is possible to import certain yml files on every request. But unfortunately drush commands are not supported yet. So we need to add all yaml files we want to import into a new section in our module.info.yml.

config_devel: install: - migrate_plus.migration.page_node - migrate_plus.migration.menu_item - migrate_plus.migration_group.liip

config_devel:

install:

   - migrate_plus.migration.page_node

   - migrate_plus.migration.menu_item

   - migrate_plus.migration_group.liip

Then we can run the following commands after updating the yml file. This will import the new configuration file into CMI.

drush cdi <module_name> drush cr

drush cdi <module_name>

drush cr

In short:

You always have to remember, that you have to import the yaml files and clear the cache after changing the mapping before executing the migration again.

Testing and running your migration

If your module is set up correctly, you can run “drush ms” to see if the migration is ready:

drush-1

Now you can run the migration using

drush mi <migration_name>

1

drush mi <migration_name>

drush-2

If you want to update already imported items you can use the –update option:

drush-3

Advanced example importing a JSON source into Drupal nodes

During the last hackday at Liip I wrote a small module that is consuming a JSON source from

http://jsonplaceholder.typicode.com importing some postings and comments to a Drupal 8 website.

The module is split into 3 tasks:

  1. A feature module that installs the node type and fields called “migrate_posts”
  2. A migration importing post nodes
  3. A migration importing comments and attaching them to the already imported nodes

You can find the feature module here:

https://github.com/ayalon/drupal8-migrate/tree/master/web/modules/custom/migrate_posts

The migration module itself is here:

https://github.com/ayalon/drupal8-migrate/tree/master/web/modules/custom/migrate_json_hackday

Migration Process Plugin

Inside the migration module from above you will find a simple process plugin. The subject field of a comment in drupal only accepts a certain number of chars by default. Therefore I wrote a small process plugin, that truncates the incoming subject string:

subject: plugin: truncate source: name

subject:

plugin: truncate

source: name

The process plugin needs an annotation (https://api.drupal.org/api/drupal/core%21core.api.php/group/annotation/8.2.x) to be picked up by the migration API. You can later refer to the id in the yaml file:

<?php namespace Drupal\migrate_json_hackday\Plugin\migrate\process; use Drupal\migrate\MigrateExecutableInterface; use Drupal\migrate\ProcessPluginBase; use Drupal\migrate\Row; use Drupal\Component\Utility\Unicode; /** * * @MigrateProcessPlugin( * id = "truncate" * ) */ class Truncate extends ProcessPluginBase { /** * {@inheritdoc} */ public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) { return Unicode::truncate($value, 64); } }

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

<?php

namespace Drupal\migrate_json_hackday\Plugin\migrate\process;

use Drupal\migrate\MigrateExecutableInterface;

use Drupal\migrate\ProcessPluginBase;

use Drupal\migrate\Row;

use Drupal\Component\Utility\Unicode;

/**

*

* @MigrateProcessPlugin(

*   id = "truncate"

* )

*/

class Truncate extends ProcessPluginBase {

  /**

   * {@inheritdoc}

   */

  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {

    return Unicode::truncate($value, 64);

  }

}

You can see the whole module code under:

https://github.com/ayalon/drupal8-migrate/tree/master/web/modules/custom/migrate_json_hackday

Final Words

At the moment, there migration is under heavy development. For Drupal 8.1 a lot of changes have been made and these changes were breaking a lot of helper modules. Especially the Migrate UI is not working since several months now under Drupal 8.1. You can see more information under https://www.drupal.org/node/2677198.

But nevertheless, Migrate 8 is getting more and more mature and stable. And it’s time to learn the new yaml syntax now! If you have any suggestions, just drop a comment.

Apr 21 2016
Apr 21

The two biggest players in the Drupal 7 webshop field are Drupal Commerce (also known as DC1) and Übercart. DC1 actually started as an Übercart rewrite to make use of Drupal 7 APIs. After the split Übercart was ported to Drupal 7 too but it was still using Drupal 6 technologies.

Although still very much in development, it seems something similar will be true for Drupal 8 as well. The developers of DC2 (the Drupal 8 version of Drupal Commerce), lead by Bojan Živanović rewrote the whole system from scratch to make use of the huge changes in Drupal 8. They are active members of the Drupal developer community so they not only know but also form the actual best practices. While working on DC2 they have fixed many dozens of Drupal 8 core issues and much more in other contributed modules (such as Entity, Inline Entity Form, Profile).

A great realisation when rewriting Commerce was that several components of a webshop could be reused by other (not even necessarily webshop or Drupal) systems. Some typical examples are address formats, currencies and taxes. These components are usually a huge pain to maintain because of the small differences from country to country. So they have created standalone PHP libraries usually using authorative third party datasets such as CLDR for currency or Google’s dataset for address formats. Some of them are already used by other webshop solutions like Foxycart and developers even outside the Drupal community are giving feedback which makes maintaining them easier.

In the DC2 development process UI and UX has got a big emphasis already from the beginning. Based on research of existing webshop solutions the shop administration and checkout process has been redesigned by UX specialists. For example, the product creation process is quite confusing in DC1 and there’s not even a recommended way to do it. In DC2 this happens now in one single form which makes it super easy.

A new concept in DC2 is Stores. Stores represent billing locations and products can belong to one ore more stores. One use case is the need for different billing for customers from different countries. Another one is having a shop where sellers can open an account and sell their own products. In this case each seller has their own store.

There are many other new features and improvements like a new and flexible tax system (you can say things like: “from Jan 1st 2014 the VAT changes from 21% to 19%”), a redesigned checkout flow, different workflows for different order types etc.

DC2 is still in alpha phase and is not recommended for production use yet. Beta releases will already have upgrade paths between them and so can be considered for starting real sites with. Beta1 is expected for May.

Drupal Commerce is the most popular e-commerce solution for Drupal 7. Given the high quality code and responsiveness to developer, shop maintainer and customer needs I do not expect this to change in Drupal 8 either.

Sources:
Drupal Commerce 2 blog
Modules Unraveled podcast on Commerce

Oct 14 2015
Oct 14

Over 2000 people attended DrupalCon Europe 2015 last September at the Barcelona International Convention Center. Five days sharing about Drupal and its new version coming soon with developers, contributors, site builders, themers, project managers, well all the Drupal community ! Let’s talk about this big event !

DrupalCon Barelona photo group

Photo group of the attendees (Photo: Drupal Association)

We were 11 Liipers gathered to learn more about Drupal 8 and all its related topics. Three of us held sessions: if you want to learn more about testing, serialization with Symfony or self-managing organizations, check out the videos online !

The sessions started with an awesome prenote given by Jeff McGuire a.k.a. “Jam” and Robert Douglass, both respectively dressed up with fancy traditional Spanish clothes and cow-suit. This prenote was a really good way to start the sessions with learning a bit of Catalan culture, traditions, architecture and its similarities with the Drupal community. We also learned the most useful sentence in Catalan of the week  “Si us plau pots revisar el meu pegat?” which could be translated by “Please review my patch.”. Even if you have to wake up early, it’s worth it !

Then, the Driesnote started. Dries chose to face uncomfortable questions like “Is Drupal losing momentum?” or “Why is Drupal 8 not released?”.
Based on the lessons learned during the long development cycle of Drupal 8, he is proposing that the development process changes with time-based releases. Whenever a feature is shippable, it will be merged to finally get a shippable main branch. He announced that the first release candidate should be ready on the 7th of October 2015 and we now know that it is ready !
Then he tackled the market position of Drupal compared to other CMS such as WordPress. In reality, Drupal is the dominant platform for large and complex websites due to its scalability and flexibility. But Drupal has to improve user experience for non coders to get realized its full potential.
Finally, he talked about progressive decoupling that can achieve both traditional CMS and client-side apps advantages. Drupal 8 will offer the option of progressive decoupling through page-building tools and decoupled by feeding Drupal data to client-side apps.

The three-day sessions covered various topics such as business and strategy, development, core conversations, devops, content strategy, site building, front-end, project management etc. and about ten sessions were held at the same time. It was sometimes hard to pick one among all of these interesting subjects ! And the day always started with keynotes about web psychology, mental health and community contribution. DrupalCon is not only for developers but relies on all the fields related to Drupal projects. I made a selection of talks for you… Don’t hesitate to watch the videos of the sessions to learn more about each subject !

Drupal 8 multilingual site building hacks

Site building – videoslides

Gábor Hojtsy and Vijayachandran Mani demonstrated how the translation system works now in Drupal 8. They took all the contrib modules that handled translation from Drupal 7 such as entity translation, i18n, title, localization update etc. and included them into core.
English is now available for interface translation as an option and blocks are translatable so we can have different menu blocks depending on language. We also learned that every listing from the administration interface is built with the views module and can be overridden.

Drupal 8 theming

Front-end – videoslides

The class soup and div soup are features in Drupal. This is one of the statements Drupal 8 theming system is going to kill (and all the theme functions with it) by introducing Twig, a modern template language used in Symfony framework.
Morten Birch Heide-Jørgensen, maintainer of the “classy” theme, showed how we have complete control over the markup with several demonstrations on how to edit templates. For instance, there are now template files for menus and pagers ! We also can extend templates and override individual blocks easily avoiding duplicating markup. On the topic of theming debugging, there is a cool feature where the names of the templates used for the page are mentioned as HTML comments.

Making Drupal fly

Coding and development – videoslides

Fabian Franz and Wim Leers told us a little bit more about the caching system built in Drupal 8. It is all about tracking dependencies to know how we can invalidate cache and Drupal 8 manages to do this with cache tags (data dependencies), cache contexts (context dependencies) and cache max-age (time dependencies). But the cacheability was still not good enough as a page can contain static parts as also dynamic parts and these dynamic parts slow the page down. The solution that Drupal 8 core provides is placeholders and auto-placeholdering. Each dynamic block can automatically be a placeholder that has an independent lazy builder which is able to render elements in isolation. Auto-placeholdering allows to defer rendering to a later time and it is configurable. You can also define your own placeholder render strategy.

Self management organisations : teal is the new orange

Business and strategy – video

Our two Liipers, Lukas and Tonio presented what are self-managing organizations and how Liip is getting closer to become a teal company. According to Frederic Laloux, author of Reinventing organizations book, there are different organization color-levels from impulsive-red which represents an organization driven by one authority and division of labour, to evolutionary-teal which represents complete self-management without any hierarchy.
Liip is coming closer to teal as we have currently no middle management, partners are willing to let it go and every Liiper can easily take initiatives to empower business and technology. Plus, each team are cross-functional and self-organized: it produces its projects, maintains them, does innovation and is able to hire new people. In fact, each team takes responsibilities for its decisions. But there are still some missing points to reach teal… For instance, we still need management for big budget decisions, dealing with conflicts or salary topics. To become teal, we can find our own solutions or using existing ones such as “holocracy” which is a framework to implement self-management. We don’t know yet which solution to choose but teal is definitely going to happen within the next months !

How changing our estimation process took our project endgame from WTF? to FTW!

Project management – video

It is really rare that a client comes to you to build a project with endless budget. So how to deal with a client that requests for proposals but only has a fixed budget?
Ashleigh Thevenet’s solution is to integrate two estimate revisions in the project during discovery phase and design phase. It is very important to explain this process to the client and be transparent during all the phases: kickoff meeting, early technical planning (first rough estimation), UX sketches, wireframes and final technical planning (second revised and accurate estimation). And the results are deliverables shared with the client that include links to wireframes, implementation notes and a full feature list with estimates.
If there is over budget, some options are to de-scope items to a later phase, ask for more budget or divide work between your team and the client’s one.
Even if some budget has already burnt for doing the whole estimation process, this is time well spent to avoid rabbit holes with technical plannings and valuable deliverables. In fact, the project has already started during the estimation process.
There are still some disadvantages: the client has to understand the whole process and has to buy in. It also can be difficult to schedule meetings as a lot of people within the team are involved but this is still work in progress with a lot to improve.

Defense in Depth: Lessons learned securing 100,000 Drupal sites

Devops – videoslides

Starting with the statement that every online website can be hacked, David Strauss, Chris Teitzel and Luke Probasco showed us how to secure websites today from the hosting to the team. As the typical breach comes from human errors and bigger breaches are often built on smaller ones, you really have to build a security consciousness. You have to think about security when building a module, deploying a website and accessing emails… And be aware of all of the aspects of it: confidentiality, integrity of data and availability.
Then you need to have procedures within the company to limit your exposure. It is useless to have a secure Drupal website and a secure hosting environment while you’re emailing your server’s passwords. You also have to know if you’re vulnerable thanks to security announcements (Cert-EU, drupal.org/security, Twitter security groups etc.).
Keeping a backup, using version control to know if the code has changed, using secure passwords and two-factor authentications are some of the keys to ensure essential security. Keep in mind that the whole stack has to be secured: the hosting, the operating system, the web server, the database, the Drupal sources and the team.

Building semantic content models in Drupal 8

Content strategy – videoslides

Stéphane Corlosquet and Kevin Oleary’s session was about how to create semantic contents and models in Drupal 8.
First of all, schema.org standard was adopted by Drupal in 2012 and it provides schemas for structured data on the internet. Its vocabulary can be used with different formats including RDFa that simply adds HTML attributes to describe content and these are really useful for search engines such as Google to understand and reuse these values.
Currently, there are existing contrib modules in Drupal 8 (and 7) dealing with semantic content strategy. RDF UI enables to map each field with an RDF property using the regular field UI. RDF UI Builder go further by generating “ready-to-use” content types with existing fields built from the schema.org models. In the future, we could imagine content models bundled as shareable features and also have a UI with a content modelling tool. Plus, we could imagine connect the content with third-party APIs: for instance, we could use echonest to get a list of musical genres that is constantly updated instead of creating taxonomies for that.
There is also a WYSIWYG plugin for TinyIMCE called RDFaCE which enables to create semantic content. In that way, you can annotate words to avoid ambiguity, for instance using “London” for the city in Canada and not the one in the United Kingdom.
Using linked data tool module, you can map a node or a term to a particular ID from external sources of wikidata or freebase. And the idea behind this is to be able later to search related contents (videos, audio etc.) and pull them directly into the content edition page.
But the the whole work about semantic is still in progress and feedbacks are very welcomed.

The third day of conferences finished with the closing session where Holly Ross and Amanda Gonser from the Drupal association announced the next Drupal big events: DrupalCon Asia, DrupalCon New Orleans and DrupalCon Dublin ! The Indian community members warmly promoted their event as they prepared a traditional and colorful dancing for it.

On the last day, I attended the first-time sprinter workshop where you can learn about all the tools needed to have a computer ready-to-sprint. For more information, you can go to: https://www.Drupal.org/tools and also check out the page about mentoring: https://www.Drupal.org/core-office-hours.
It is important to say that everybody can be part of sprints as all skills are needed and of course mentors do a great job to push people in the right direction. During this DrupalCon, 70 mentors were there to help and there were about 200 sprinters working on the Drupal 8 issues.

There are also many social events organized at night where you can exchange with the community around a beer or some tapas like the Welcome party or the Women in Drupal meetup. Even if there are few, I could met some women in the community including site builders, themers, back-end developers and project managers and it was a pleasure to exchange with them about Drupal !

There is actually a really good spirit all around the Drupal community and we could feel it during  the whole DrupalCon event.

Oct 08 2015
Oct 08

11 Liipers attended the DrupalCon Barcelona in September 2015. We learned a lot about Drupal 8. And heard over and over again: “Start working with Drupal 8 now!”.

But let’s have a look back: The development of Drupal 8 took more than 4 years and for me as a developer it sometimes seems, like no stone was left unturned. Almost everything has changed. Symfony2 walked onto the stage and brought some paradigm shifts in Drupal 8 that may prove to be a challenge for us as developers.

But the good news is that there are so much new features and tools, that make developing enterprise websites so much easier, that I would like to start new projects with Drupal 8 exclusively and never use Drupal 7 again ;-)

At DrupalCon, there were a lot of interesting sessions about exciting features for developers. My personal favourites were the following:

But what about new projects with Drupal 8?

Sure, all these new features sound exciting. But what about building new customer projects with Drupal 8? The answer isn’t that easy.

We at Liip already started to work with Drupal 8 and we can share some experience with other companies out there, who would like to start working with Drupal 8.

New simple CMS project with Drupal 8 will work out of the box!

Simple CMS projects that rely on Drupal 8 Core without the help of too many contrib modules will work out of the box. So many great features like Views, Page Manager and CKEditor are now part of the core. Drupal 8 matured and is now a full featured enterprise content management system capable of building websites based on structured data.

Nevertheless, there is an initial investment you will have to do, because the learning curve of Drupal 8 is quite steep. If you are not familiar with Object Oriented Programming (OOP) and Symfony2, you will have to put some effort to learn the new concepts. But as soon you managed that, your Drupal installations will become easier to deploy and ways better maintainable.

In our first smaller projects, we calculate with a 10-20% additional costs, because of missing knowledge and experience. we also will have to fix some of these early adopter bugs. These costs will be covered mainly by my personal education budget and also by some extra budget Liip AG has reserved for Drupal 8 transition phase.

If you plan to start a new project you should try to go with Drupal 8 if it’s somehow possible. This will protect your investment for the next few years and make sure, that the long term costs (aka maintenance) stay low. You can expect that in the near future all new Drupal websites will be build with Drupal 8. Additionally you can profit from all the new features. Even as an end user you have a much better user experience and a lot of benefits. If you are uncertain, you may contact us. We have a lot of expertise in this field and will figure out together with you if your project will work out with Drupal 8. In the end, it will be up to you to decide which version is implemented.

For new and complex project you will have to calculate quite some extra efforts!

Bigger project often depend on a lot of custom module. For example complex media management (as we had it in Drupal 7) is not ready at all. For every contrib module you have to evaluate and test, if there is a working Drupal 8 version. If there is none, you will probably have to upgrade parts of the module or looking for an alternative.

We calculate with more than 50% additional costs, if we have to upgrade complex contrib modules or parts of it. As we would like to contribute these modules back to the community, this number can grow quickly if you try to implement the new module in a generic reusable way.

Upgrading existing big Drupal 7 project To Drupal 8: Stay away!

At the time of writing I consider it absolutely senseless to upgrade big projects to Drupal 8 that rely on a lot of contrib modules. You would have to upgrade a lot of contrib modules and also completely rewrite your custom modules. Updating a contrib module means, that you have to understand the code someone else has written and transform it into working module on Drupal 8 (where you probably are not as experienced) and in the same time discover and use the new system / APIs / services. Quite a brave mission that probably will end up in a mess. You should try to avoid that.

So what should I do now?

Every Drupal site is different and especially the customer needs are different and have to be considered on a case by case basis. You will certainly have to calculate with additional internal costs that you cannot bill to your customer in your first Drupal 8 project.

But you should definitely now start working and building websites with Drupal 8. You will enjoy all the new features and make your customer happy with a fast, testable and user friendly system. And we hope, that you also contribute and help upgrading key modules. This helps the Drupal community a lot and will enable all of us to do more complex Drupal 8 projects in the near future.

Apr 21 2015
Apr 21

Last week, I was at Drupal Dev Days in Montpellier with a few other Liipers. It was, as often is with such conferences in the Drupal community, greatly encouraging to see the passion and effort of the many Drupal developers there.

Throughout the week, hundreds of people were working on different projects in the “sprints” (as such code-marathons are called in the Drupal community). Drupal 8 core was (unsurprisingly) the biggest group, but there were many other efforts related to Drupal 8. I was primary involved in the Rules for Drupal 8 effort, which was a fairly large group. We managed to get quite a few issues solved, and the road to a user interface for Rules in Drupal 8 was begun, which is one of the biggest outstanding issues.

It was challenging work, at times, but there was a good mood in the group, and the D8rules team were really great at helping beginners getting started, so many thanks to them for that.

I was unfortunately forced to forgo the conference part of the event due to illness, but I hear it was great. In total, I think the organisers of DDD did a great job – especially with the food. So thanks to them as well. See you in the issue queues.

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web