Aug 18 2019
Aug 18

Learn about On-page SEO and which elements of your web page are important to increase your online visibility. Let's start!

On-page SEO is much more than title tags, meta descriptions and valuable content. Here is my actionable guide for digital marketers. I am an SEO Specialist and teamed up with one of my colleagues – a Content Marketing Specialist – for this article. Have fun reading it.

On-page SEO is about creating relevant signals to let search engines know what your page is about. Which improves the website’s ranking in search results.

There are no IT skills needed to implement on-page recommendations as most CMS have an extension for it. For example, if you use WordPress, download the Yoast SEO plugin, or add the Metatag module to Drupal.

On-Page SEO: Hypothetical case study

How to create those relevant signals? Let’s take the example of a florist. StarFlo is located in Lausanne and Zurich, Switzerland. StarFlo has a website in three languages (French, German and English). The flower shop decided to create a specific product page for wedding, in English. A product page is designed to provide information to users about a product and/or a service.

Find relevant keywords with the right search intent

The first step is to define keywords with the highest potential. The goal is to select words, which help to increase the ranking of the wedding product page.
Here are some examples of keywords (non-exhaustive list):

  • “wedding flowers lausanne”
  • “wedding flowers zurich”
  • “wedding table decorations”
  • “wedding bouquet”
  • “rose bouquet bridal”
  • “winter wedding flowers”
  • “wedding floral packages”
  • “orchid wedding bouquet”
  • “wedding flowers shop”

We will take the monthly volume of English keywords in Switzerland into consideration, because we are focusing on a flower shop located in Lausanne and Zurich whose product page is in English.

According to the image below, “wedding table decorations” and “wedding bouquet” have a higher volume (column Search) and a low difficulty score (column KD). Therefore, it could probably make sense to use those keywords. However, you need to investigate further.

If you check Google search results for the keyword “wedding table decorations”, you see a lot of images coming from Pinterest. People who are looking for “wedding table decorations” are looking for ideas and inspiration. As a result, “Wedding table decoration” might be a great blog post topic. As FloStar wants to create a product page, we suggest using “wedding flower shop” as a primary keyword, even if this keyword has a lower volume than “wedding table decorations”. The intent of the people searching “wedding flowers shop” is to buy wedding flowers. The intent of the new product page of FloStar is to sell wedding flowers. Therefore the goal is to align both the intent of the target public and the intent of the product page with this keyword.
Once you have the keywords, optimize the content of the page

On-page SEO structural elements

Title tags, H1, H2, and images are part of the on-page structural elements that communicate with search engines

Title tag best practices: clear and easy to understand

The title tag, is the page title and must contain the keyword in less than 60 characters (600 pixels). Ideally, the title tag is unambiguous and easy to understand. You define the title tag individually for each page.

For example:

Wedding flowers shop in Zurich & Lausanne | StarFlo

You do not need to end your title tag with your brand name. However, it helps to build awareness, even without raising the volume of clicks.

Meta description best practices: a short description with a call to action

The meta description describes the content of a page and appears in the search results. The purpose of the meta description is to help the user choose the right page among the results in Google Search. It must be clear, short and engaging. You have 160 characters at your disposal.

We recommend finishing your meta description with a clear call-to-action. Use a verb to describe what you want your target audience to do.

For example:

StarFlo is a flower shop located in Lausanne & Zurich which designs traditional & modern wedding flower arrangements. See our unique wedding creations.

SEO URL’s best practices

The URL is the address of your website. Its name describes both the content of the page and encompasses the page in the overall site map. The URL should contain the keyword and be short.
The structure of the URL is usually governed by rules in the CMS you are using.
Examples for StarFlo landing page about wedding flowers:
✔︎ https://starflo.ch/wedding-flowers
https://starflo.ch/node/357

Use secondary keywords to reinforce the semantic of your page

Startflow wants to be listed top for “wedding flower shop” and “Lausanne”. You can help this page improve its ranking by also using secondary keywords. Secondary keywords are keywords that relate to your primary keyword.

Ask yourself: what questions are your target audience looking to answer by searching for these keywords? What valuable information can you provide to help them?
Your text content must offer added value for your target audience. To ensure this, create a list of topics. In the case of StarFLo, you can include secondary keywords such as “wedding bouquet” and “wedding table decorations”. It may seem odd that the keyword used as the primary keyword has a lower volume than the secondary keywords, but it makes sense in this context. Because these secondary keywords reinforce the semantic of the page.

In the “wedding bouquet” section, you can give some examples of “Bridesmaid bouquets”, “Bridal bouquets” and “Maid of Honor bouquets”, as well as other services or products related to the proposed bouquets.

SEO H1 & H2 tags best practices: structure the text with several titles

A structured text with titles and subtitles is easier to read. Furthermore, titles support your organic referencing as they are considered strong signals by search engines. Start by defining your titles H1 and H2. Use only one H1. Your titles should be clear and descriptive. Avoid generic or thematic titles.

Here is an example:

  • H1: StarFlo, wedding flower shop specialized in nuptial floral design in Lausanne, Zurich & the surrounding area
  • H2: Outstanding wedding table decorations created by our wedding flower specialist in Lausanne & Zurich
  • H2: Wedding bouquet for the bride in Lausanne & Zurich
  • H2: Best seasonal flowers for your wedding

On-page content best practices: Write a text longer than 300 words

Keep in mind these three key points when you write your text:

  • Anything under 300 words is considered thin content.
  • Make sure that your primary keyword is part of the first 100 words in your text.
  • Structure your text with titles and subtitles to help your readers. Moreover, as said above H1 & H2 are strong signals

Images & videos best practices: Define file names, alt-texts and captions

Search engines don’t scan the content of a video or an image (yet). Search engines scan the content of file names, alt-texts and captions only.
Define a meaningful alt-text for each image and video. The alt-text should include your keyword in the file name. Google can then grasp what the image shows. Remember that you wish the website to load fast, so you may compress images.

SEO Internal linking best practices: create a thematic universe within your website using internal links

When writing your text, try to create links to other pages on your website. You can add links in the text or in teasers to race attention on more (or related) topics.

From a content point of view, when you link pages of your own website, you add value to your target audience as their attention is drawn to other pages of interest. Furthermore, the audience may stay longer on your website. Moreover, creating links gives the search engine a better understanding of the website and creates a thematic universe. Topics within such a universe will be preferred by search engines. Thematic universes help Google determine the importance of a page.

From an SEO point of view, internal linking is very important. Because it implies a transfer of authority between pages. A website with high domain authority will appear higher in the search engine results. Usually, homepages have the highest authority. In the case of StarFLo, you could add a hyperlink that connects the homepage to the wedding page. We also recommend adding hyperlinks between pages. For instance, you are writing about winter wedding flowers on your wedding page, and you have a dedicated page about seasonal bouquets. You could add a hyperlink from the wedding page to the seasonal flower page.

The result: the homepage will transfer the authority to the wedding page and the wedding page to the seasonal flower page. For each transfer of authority, there will be a slight dumping factor. This means that if a page has an authority of 10 when it links to another page, the authority transferred will be for example 8.5.

Outbound links Best practices: add relevant content

Link your content to external sources, when it makes sense. For example, StartFlo provided the floral decorations for a wedding in the Lausanne Cathedral. You can add a link to the website of Lausanne’s Cathedral while mentioning.

Bonus: write SEO-optimized blog posts with strong keywords

After publishing your product page, create more entry points to your website. For example, you can write blog posts about your main subject using powerful keywords.

Answer the needs of your readers

When we did the keyword research for StarFlo, we identified a list of topics connected to the main topic. As a reminder, when we were looking at wedding flowers, we discovered that people were very interested in wedding table decorations. We also noticed that people looked for different kinds of bouquets (types of flowers, etc.). You could, for instance, create a page about winter wedding flowers and use these related keywords on it. This strategy helps to define blog post topics.

On the winter wedding flowers page, you could describe the local flowers available in the winter months, the flowers that go best together, etc.

In this case, each of your pages should focus on a different keyword. If two pages are optimized for the same keyword, they compete with each other.

Prioritize your writing according to your business

Once you have a list of topics, it’s good practice not to start writing all at once. We recommend creating an editorial plan. Be honest with yourself: how many hours per week can you dedicate to writing? How long do you need to write a 500-word article? How long do you need to find or create suitable images?

Start with the strongest keywords and the topic with the highest priority for your business.

Here is an example of prioritization:

  • “Wedding table decoration”
  • “Wedding bouquet”
  • “Winter wedding flowers”
  • “Winter wedding floral packages”

If you start writing in September and the branding guidelines of your shop include ‘local’, ‘sustainable’ and ‘proximity’. You will, therefore, write about “Winter wedding flowers” first.

You decide to focus on:

  • “Winter wedding flowers”
  • “Winter wedding floral packages”

As a wrap-up, we prepared the checklist below for you.

Checklist

  • Main keyword is defined
  • Topic brings value to the target public
  • Meta Description and Title Page are written and contain the keyword
  • URL contains the keyword
  • H1 contains the keyword, at the beginning, if possible
  • Text contains a keyword density of 3%
  • Introduction and last paragraph have a particularly high keyword density
  • File names of photos and videos contain the keyword
  • Alt-Text of photos and videos contain the keyword
  • Photo captions contain the keyword
  • Page contains links to other pages on the site
  • Page contains links to valuable external resources

What’s next

On-page SEO is an important part of SEO. However, it’s not the only aspect. Technical SEO has also a tremendous impact. We work on a hands-on blog post about technical SEO. Reach out to us if you wish to be notified when our guide will be ready! Moreover, don’t miss our next SEO/ content meet-up taking place on the 26th of September. We are going to explain how to perform a keyword research. Contact our content expert if you want to be part of the meet up.

If you want to have a personalized workshop about on-page SEO or just want to increase your ranking on Google contact our SEO team:
for English, German and French.

Aug 18 2019
Aug 18

Building intelligent applications is not just about the technical challenge. How does AI become reality? We share here one of our tools for crafting intelligent, data-intensive software solutions.

While creating applications that heavily rely on data, I’ve gotten excited about many projects which soon ended up in the graveyard of good ideas.

Have you experienced this chicken-and-egg situation: no one wants to give you money to develop something until they know how good it will be, but you can’t promise anything before experimenting with the data first?

Smart Application Canvas

To get out of this impasse fast, Liip's Data team has developed a pragmatic "recipe" for data-intensive applications: the Smart Application Canvas.

It's simple and takes around 5 minutes: whenever we get started on a project idea, we ask the four questions below. And then we ask them again at every iteration to see if the answers have changed in the meantime.

It doesn't matter which of the four you start with - the trick is to have at least a rough answer to all four questions before investing more time and energy into an idea (for example, building a prototype, pitching a project proposal, etc). As long as one of these "ingredients" remain entirely blank, the project has a slim chance of succeeding.

>> Download this template as a PDF: template_data-application-idea_v1-0.pdf

*Released under a "Creative Commons" license (Attribution / ShareAlike). This means that you're free to use and change it, provided you mention www.liip.ch and share any modified version under the same terms.*

USE - Who are the users? What job does it do well for them?

Are you looking for a way to attract more new users, keep the current ones happy, or solve an annoying issue with an existing service? Do you want to make employees' life easier or to provide better public services for citizens?

Who to ask: Designers, product managers, line managers, frontline staff who are always in contact with users

Example: Customers can get immediate help 24/7 through a chatbot on our Facebook page.

Why: Avoid building something that nobody wants

BUSINESS CASE - Who might want to pay for this? How do they benefit in return?

Should the application help a person or an organization to save money, save time, or to generate more income? Should it help them to achieve something that isn't simply isn't possible with the current alternatives? Even with intangible benefits like happiness or social status, it's worth thinking of how much the current alternatives cost: this will give you an idea of how much you can afford to invest in the development of the application.

Who to ask: Businesspeople, salespeople, executives who have a bird's eye view of the most pressing concerns in their organization

Example: Online retailer XYZ can win more return customers thanks to a better after-sales service.

Why: Get someone to invest or give you a budget. The business benefits of innovative technology aren’t straightforward at first. Do you remember the time when smartphones were new and everyone thought they needed a native app just “because it’s the future”? By now it’s much easier to say whether an app is worth the cost, but what about a chatbot?

TECHNOLOGY - What main hardware / software are necessary to get the job done?

What is possible with current technology and with how much effort? Or how can a new technology be applied in a "real-world" application? The key part of the question here is "to get the job done". When innovative technologies come along, it's tempting to try to solve too many problems at once: if there are several possible use cases, pick one. You can always change it later if you see that it doesn't work.

Who to ask: Software engineers, data scientists, hardware engineers if applicable

Example: Facebook page, chatbot framework, natural language processing (NLP), search

Why: Avoid a very costly, never-ending implementation

DATA - What numbers, text, images are necessary for it to work?

Does your organization have a lot of great data but doesn't know what to do with it? What kind of data should you start collecting? Which external data sources can you use (e.g. open data)? Where is the data saved and in which format (e.g. in a database, in PDF files, in emails, ...)?

Who to ask: People who work with the data on a daily basis (e.g. Customer Care team), database specialist, data scientist

Example: Collection of past customer questions and matching support staff answers (e.g. in emails or in an issue tracking system)

Why: What makes software "smart" is data, rather than “only” predefined rules. There might be no / not enough data, its quality might not be sufficient, or it might be difficult to access because of security or technical issues. This makes the outcome of development unpredictable. Whether our data is good enough, we can only learn by trial-and-error.

Examples

Here are a few examples of early-stage ideas by participants of the Open Data Forum 2019. The goal was to come up with applications that make use of open data.

Some canvases already carry notes on the biggest risks the idea faces, so that this can be worked on as a next step.

Acknowledgements

The following were sources of inspiration:

  • Lean Canvas by Ash Maurya
  • Business Model Canvas by Strategyzer AG
  • Machine Learning Canvas by Louis Dorard
  • Migros's "Bananita" roulade cake enjoyed on a particularly creative coffee break
  • Our clients and colleagues in the past year with whom we've tested and refined our way of working

We're constantly working on improving our methods and would love to hear about your experience with the Smart Application Canvas. Drop us a line at [email protected].

Aug 12 2019
Aug 12

Booster durablement la performance des équipes en insufflant de la transparence, de la cohésion et du pragmatisme. Tel est mon rôle chez Liip en tant que Scrum Master depuis 5 ans. Aujourd'hui, j'aide d'autres organisations à délivrer, elles aussi, des projets de façon plus efficace et plus humaine.

Un retour au bon sens

Nous commençons l'accompagnement de nos clients par une journée de formation interactive. Nous présentons les idées et les valeurs qui constituent les bases d'un fonctionnement en Agilité. Lors du premier tour de table, les participants expriment ce qui les amène, ainsi que leurs propres objectifs d'apprentissage. Je me souviens des mots qu'une cheffe de projet expérimentée a partagé à cette occasion:

Il paraît que le bon sens ça s'appelle désormais Agilité, alors je suis venue me mettre au goût du jour.

Elle n'a pas tort. Les pratiques Agiles sont issues du sens commun. Alors, pourquoi faut-il les réapprendre et les nourrir régulièrement?

Des pratiques simples, s’inscrivant dans la durée

Une des métaphores les plus adaptées est celle du jardinage. Seul un verger entretenu un petit peu tous les jours donnera ses fruits malgré les variations du climat, l'appétit des insectes et la persistance des mauvaises herbes. Arroser le sol et arracher les plantes nuisibles, ce n’est pas compliqué. Le plus dur est de se discipliner à le faire de manière quotidienne. Faute de quoi, les circonstances et forces extérieures reprendront le dessus et dicteront la suite des évènements.

Dans le cas d'une équipe Agile, les pratiques sont elles aussi très simples et très fructueuses. Par exemple:

  • Réserver un quart d'heure tous les jours pour se coordonner – de vive voix – autour d'un but clairement exprimé ;
  • Visualiser le travail d'une façon compréhensible par tous ;
  • Faire le point de temps en temps pour sortir la tête du guidon et se demander ensemble ce qui marche et ce qui pourrait être amélioré.

L'Agilité, c'est avant tout une poignée de valeurs fondamentales telles que la transparence, la collaboration, la confiance, l'amélioration continue, l'apprentissage par l'expérience. Ces valeurs s'incarnent dans des rôles et des outils différents suivant l'approche que l'on choisit.

Les défis du "monde réel"

Nous avons tous fait l'expérience d'un projet mis à mal par la pression du marché, des investisseurs, ou celle exercée par la hiérarchie. L'état d'urgence est alors déclaré. Et l'on oublie notre bon sens. Celui qui soigne nos relations, la qualité de notre travail et de notre santé en général. Il y a alors des victimes: soit les collaborateurs, soit le projet, soit l'entreprise elle-même. Parfois les trois en même temps.

Des rôles clairs et précis

C'est pour cette raison que Scrum – une des variantes les plus connues de l'Agilité – créé un rôle dédié à "veiller au grain" dans une équipe, sur le long terme. C'est le "Scrum Master". Le Scrum Master est un leader au service du groupe. Dans mon rôle de Scrum Master, performance et épanouissement ne sont pas antinomiques mais forment un seul et même état de grâce. Je cherche à y amener mon équipe et l'y préserver.

Nous pratiquons Scrum chez Liip depuis 2009. Cette approche nous a permis de maximiser la valeur délivrée à nos clients, tout en construisant des équipes stables et soudées.

[embedded content]

Une organisation rigoureuse et souple à la fois

Avec Scrum, nous ne travaillons pas "contre" la hiérarchie, le marché, ou les investisseurs. Ils sont au contraire intégrés au processus. Il existe en tout temps un moyen de prendre en compte leurs signaux. Et ce, tout en conservant un cap stable pendant un laps de temps donné. Le cadre de travail Scrum permet à toute partie prenante d'écrire dans le "Backlog" – un document vivant qui recense tous les besoins et idées liés au projet.

Par contre, une et une seule personne a l'autorité de décider de l'ordre des priorités dans ce que l'équipe devra réaliser. Cette personne est le "Product Owner". Ce rôle est dédié à l'écoute permanente de tout ce qui constitue l'environnement du projet. Au coeur de l'Agilité se situe la notion de confiance. L'organisation fait confiance au Product Owner pour se porter garant de la vision du produit.

Un processus par itérations

La notion d'itération est fondamentale dans la pratique de Scrum. Nous ne cherchons pas à tout spécifier à l'avance dans un projet. Ce qui est d’ailleurs peine perdue dans un environnement complexe et changeant. L'attention de l'équipe est concentrée sur le "plus petit prochain pas" qui permettra de récolter des retours du marché ou des utilisateurs. On parle d'incrément – typiquement réalisé en une à quatre semaines. Ainsi, à intervalles réguliers, il est possible de tester la solution développée. C’est l’occasion de la confronter au monde réel et de prendre la meilleure décision possible sur ce qu'il faut construire ensuite pour apporter le plus de valeur.

L'équipe qui développe le produit a largement son mot à dire car c'est elle qui émet les prévisions de ce qu'elle pense délivrer lors de l'itération à venir. Ce qui génère très souvent des discussions créatives avec le Product Owner. Plutôt que de mettre l'équipe sous pression afin de délivrer un périmètre défini à l'avance et immuable, nous discutons ensemble de la façon de délivrer la plus grande valeur business avec le temps disponible.

Le premier pas

Tous ces principes et ces rôles semblent tenir de rêveries un brin naïves, jusqu'à ce que l'on en fasse l'expérience. Cela a été mon cas quand j'ai rejoint Liip. L'Agilité, ça marche. C'est un super-pouvoir pour les équipes et pour les projets. Essayer, c'est l'adopter. Nous nous rendons volontiers dans vos locaux pour une séance d'introduction. Ou passez simplement boire un café! Ce sera avec plaisir que nous imaginerons ensemble le premier pas pour votre organisation.

Jul 29 2019
Jul 29

I joined Liip for an internship in POing. To take the most out of this opportunity, I prepared myself for the Scrum Product Owner Certification, which I succeed. Here are my key take-aways.

Why a Scrum Product Owner certification ?

The first step, I would say, to get certified is to be ready to dedicate time for the preparation and training, and to commit to it. This wasn’t a problem for me as I was highly motivated in improving my skills and knowledge about Scrum. And here is why.

I worked more than 10 years in international corporations, using traditional models for product development. And I reached the point where I sensed that something wasn’t quite right.

As part of the product development team, we were asked to deliver a set of features based on specifications defined at the beginning of a project. All the specifications had to be delivered for a mid- to long term deadline. No need to say that it wasn’t possible to modify them. The sign off could only be possible if the defined specifications were delivered. I often experienced a big rush at the end of the project because nor the scope nor the deadline could be changed. The team members suffered, almost burning out sometimes.

During the latest projects I was working on, feedback from end-users was given during the testing phase. It was already too late though. End-users requested features that were not part of the specifications defined at first. As the project team, we were told that: “end-users can make an enhancement request later on, after the Go-live”. Like my former colleagues, I had this sad feeling. We worked really hard on delivering these features. But we weren’t proud of them as end-users were complaining.

There has to be something better. A way where users needs are at the core of the product. A way where inputs from the team members really count. That’s when I came across the Scrum Guide. It became clear to me that I wanted to be part of this. That I wanted to be part of a Scrum team.

Referring to my experience and skills, Product Owner was the role appealing to me. In order to do so, I set myself two objectives: gaining experience and getting certified.

Collaboration with Liipers

I had the chance to join Liip for a three-months training in POing. Witnessing practical POing and being immersed in the Scrum philosophy was part of the deal. I was on-boarded on different projects too, working with different teams and clients. This helped me integrate how the Scrum Guide should be applied in practice. I got a deeper understanding of the Scrum events, the Scrum artifacts as well as the roles of the Scrum Master, the Product Owner and the development team.

Yes, self-organized teams works ! I was strongly motivated by this environment where all team members are equally responsible for the success or failure of the product. It really makes everyone commit to the work to be done, and brings up new ideas and solutions.

What about the Product Owner ?

This role is the one which always keeps in mind the end-user, not only for the team but also for the client. In my opinion, one of its biggest challenges is to convince the team and the client that real end-users feedback is a must. The PO is the one prioritizing the features to be developed, expressing user stories in the most intuitive manner and identifying the needs of the client and end-users.

I believe that as a Product Owner you need to be empathic, synthetic, a good listener and a good communicator.

During my training, I was inspired and empowered by my fellow PO colleagues. I loved their reactivity and the way they reorganized work to seize each business opportunity. The user needs are evolving and as the Scrum team we have to adapt our developpements to them. The Scrum framework allows us to do so.

Sprint after sprint, I was amazed how the product was developed, refined and improved to perfectly meet the evolving user needs.

Becoming a Product Owner myself was definitely the right choice for me.

Training with Léo – a certified Scrum Master and Agile Coach

At Liip, I had the chance to meet Léo – a Scrum Master and Agile coach. He guided me through the different steps, advising me to several really interesting readings such as Scrum, A Smart Travel Companion by Gunther Verheyen. Thanks to his coaching, I gained a deeper understanding of Scrum’s essence. He challenged me and made me think about Scrum principles and why Scrum has to be fully implemented – not just bits and bites of this amazing framework.

Getting certified, what’s next ?

Beginning of July I felt ready for the Scrum Certification. And I nailed it!

Actually, I applied the Scrum principles to my own training. The vision I have for my product is “to become an efficient Product Owner.” My first iteration was “to understand the role of a Scrum Product Owner”. And the certification was the test. I gave myself three weeks to reach that goal (sprint goal set, sprint duration set, commitment of the team, self-organization). On the other hand, I was open to faillure. As we never fail, but only learn.

This first iteration was a team effort too. I even had a Scrum Master – you know Léo – on my team ;-). I improved my knowledge on my colleagues’ experience (empiricism). My minimum viable product evolved from “to understand the role of a Scrum Product Owner” to “being a certified Product Owner”.

I am proud to announce that my first product works ! And I’m already working on the next improved iteration. So that my (own) product fits the market needs.

I feel fully armed to embrasse a new work life as a Scrum Product Owner. I want to help people – used to work with traditional models – evolving to the use of the Scrum framework.

Last but not least, I will carry on learning and adapting fast. I will be Agile and help others to achieve this goal as well.

Jul 25 2019
Jul 25

Cognitive overload requires more time and effort to complete a task. Learn more about how to reduce mental effort for users and how you can expand your knowledge of Cognitive User Experience Design (Cognitive UXD).

Based on my overview blog post Cognitive User Experience Design - Where Psychology meets UX Design I give you a deeper insight into cognition and describe its role in design based on the Cognitive Load Theory. It's about how we consume information, how we think, how we learn or solve problems, and the strategies we use to make decisions.

In cognitive psychology, cognitive load is the total amount of mental effort that is used in the working memory. Although we have a huge brain capacity, the problem is that its capacity is limited. This approach is known as the Cognitive Load Theory (Sweller & Chandler).
Accordingly, a high cognitive load leads to a higher mental performance in the brain. If several high demanding processing things are going on at the same time, this becomes even worse. More time and effort is needed to complete a task. This leads us to the first question.

What happens if the cognitive load is too high?

As already mentioned, our brain has only a limited amount of mental power. If the cognitive load is too high, the user no longer reads the content of a website. He only scans it.
Compared to adults, this performance is much lower in children. For example, the normal attention span for 8-year-olds is about 20 minutes, for 13-year-olds about 30 minutes (Wöstmann et al., 2015). This should be taken into account when conducting an interview or a test with children.

How can a UX Designer support the user?

User support can be achieved by reviewing and optimizing each step. Here are some starting points that can help:

  • Show the user an overview of the entire setup and in which step he is at any time
  • Provide the information that the entire process or each individual step is beneficial to the user and worth the user's time
  • Give clear instructions on what to do next
  • Check whether certain information/steps are really necessary and delete everything that is not important
  • Provide the information in a simple and understandable way

In order to make these approaches more concrete and thus more tangible and to reduce the cognitive overload of the users, a UX designer can apply different strategies. I will give a short overview of the most important ones and add a short example or description:

KISS (Keep It Stupid Simple)

  • Avoid unnecessary elements, less is more
  • Reduce the number of complicated graphics

Use different techniques

  • Provide information in different ways; these can be verbal, visual and/ or auditory techniques

Provide “bite sized” information

  • Break the content into smaller pieces

Remove unnecessary content

  • Reduce repetitions by proofing if a text is really required or whether a picture fulfills the same task

Reduce choices

  • Too many choices increase the cognitive load, especially for forms, dropdowns and navigation

Place words close to the corresponding graphics

  • Long distances force the user to scan the screen

Build on existing mental models

  • Use labels and layouts that users already know from other websites

Taking these recommendations into account when creating designs reduces the amount of brain capacity. This has a direct impact on how easily the user finds content or performs tasks.
With this in mind: Happy designing!

Jul 24 2019
Jul 24

That's how we supported a team at PostLogistics in the tranformation from hierarchy to Holacracy® - an interview with Lucien Ecoffey.

At the end of the kick-off meeting in January 2019, Lucien Ecoffey – the then Head of French- and Italian-speaking Switzerland’s PostLogistics Sales Support body – signed the Holacracy® Constitution. He gave up his hierarchical authority and transferred it to the process that would guide his entire team.

Six months after this transformation began, the former manager and now Lead Link of the PostLogistics Sales Support circle takes a look back at the successful transition.

How did this transition project to Holacracy® happen?

Lucien Ecoffey: We had an internal reorganisation in 2017. At this point, we established a service unit providing back-office services to various divisions of Swiss Post. This change affected how we worked. We needed a more flexible structure to meet the various demands of our different stakeholders. In addition, the members of my team expected more autonomy, more responsibility and less monitoring.

Why did you choose Holacracy®?

For three reasons:

  1. I was convinced that Holacracy® represents democracy applied directly to a company.
  2. The members of my team wanted to play a bigger role in decision-making.
  3. Another team within Swiss Post had already successfully implemented Holacracy®.

The team voted, and Holacracy® was the unanimous choice.

The Constitution and the clear tools and processes enable us to oversee the quality of management. Holacracy® also enjoys a high level of credibility. In addition, there are various pieces of associated software available, such as Holaspirit.

Organising work by roles rather than sets of specifications was an attractive prospect for all team members. These roles develop in line with the changes we face, something that is perfect for our need for flexibility. The members also appreciate that the autority is given by a set of rules and not by a single person any more.

*Lucien – the team’s former manager – transferred his authority over to the Holacracy® Constitution.*

What would you say are the benefits of implementing Holacracy®?

Each member of the team is more deeply involved, in particular at tactical and governance sessions. Our meetings have become increasingly efficient. All problems are presented at the meeting and tackled immediately: decisions are taken, and everyone is involved. This new decision-making process also means that I no longer have to decide everything, and it has given me more respect for my colleagues’ opinions.

We are still at the learning stage. Everyone requires an adjustment period to fully incorporate the new responsibilities of a role into their work and learn how to manage the authority that comes from this. Clarifying these responsibilities and grouping these within specific roles has already been of great benefit. This enables everyone involved to examine their activities and optimise their tasks. Team members’ ability to identify with their roles is also a key benefit of this transition.

Holacracy® has another benefit, namely transparency. For example, we can integrate a new member of staff into the team easiler and faster. Processes, responsibilities and tasks are clear and transparent.

What did you think of Liip's coaching?

Your support – four days spread across six months – completely met our expectations. Laurent and Jonas did outstanding work.

The kick-off was intense but absolutely necessary, and it gave the team a new way of looking at hierarchy. I remember the rather formal moment when I transferred my authority over to the Constitution. I also enjoyed working on our circle's purpose and creating the first roles.

During this kick-off, you gave us the tools that would enable us to work with Holacracy®. The coaching showed us that we could view authority from a different angle. It took us out of our comfort zone and enabled us to optimise the roles within our team.

You have been working with Holacracy® since the beginning of the year. What are your plans for the future?

Our work with Holacracy® is only just beginning. It is great to see that it really works. The next step is to incorporate the idea of roles and people being separate from each other. Current roles will be developed (some will be wound up and others created) to bring them closer to our daily work. We still need to learn how to handle some tools to enable this structure to become a source of fulfilment, for each employees personal benefit.

The transition taking place within our team has piqued the company’s interest and been well viewed. Numerous other teams are beginning a similar process.

I think this is fantastic, and hope that Swiss Post as a whole will move in the same direction.
I would therefore be delighted if Liip comes back to see the progress we have made, coach us on the aspects we need to improve, and share our experiences.

Jul 18 2019
Jul 18

With uncluttered search functions and a particularly user-friendly design, the canton of St. Gallen has a new digital face - have a look sg.ch

Everything in one place, more streamlined and easier to understand

The canton of St. Gallen’s new web presence has been live since the end of April. Having an attractive design suitable for all devices, providing a quick and intuitive navigation system and search function, and considering the needs of a wide variety of users were all focal points of the redesign. The new website presents around 50 of the canton’s departments and authorities. The number of pages was reduced from 24,000 to 12,000, and some of the content is already available in simpler language.

Focus on end users

The canton wanted a new, modern website, and the platform needed to be reworked in terms of both design and content. The aim was to create a user-friendly online presence with an easy-to-use, intelligent concept and a responsive design.
The search function is a now a key element, and was therefore also a part of the redesign that sg.ch particularly wanted to focus on. An all-encompassing search function, with the majority of pages and authorities available to users in one place, simplifies the content from the user’s perspective.

Flexible and close collaboration as the key to success

The concept and design were provided by Liip in St. Gallen. This proximity, with their offices just a five-minute walk apart, enabled close collaboration between the canton and the web and mobile app agency. The company Online Consulting, responsible for the technical implementation of the platform, was also present at the workshops right from the very beginning, enabling us to continually ensure that the concept and design were practicable.
The high level of motivation from all involved (in particular the 300 people from the canton of St. Gallen) made the process very enjoyable and created the feeling of one large team working together to (successfully) achieve a major goal.

Simple navigation despite a complex organisational structure

The biggest challenge was to supply users with content in a clear and efficient way across all web pages. Whether looking for school holiday dates, applying for a learner driver’s licence, or planning an extension as a homeowner, all visitors should be able to find what they need quickly and easily. Liip developed a usability approach for this purpose, using a full-screen overlay to make the entire content structure visible. This enables visitors to directly access a variety of content quickly, easily and at any time.

Content guidelines, search engine optimisation and accessibility

To ensure that all texts were appealing to readers and easy to understand, rather than being written in ‘officialese’, Liip drew up content guidelines and ran training sessions. Making the over 270 editors aware of the importance of these texts was vital for the project’s success.
Another challenge was the complexity and retrievability of the online content. The content had to be accessible to search engines. Liip drew up a set of rules for sg.ch to enable it to optimise all of its content for search machines and ensure that it was accessible. Liip used these clearly comprehensible guidelines to help the team meet this challenge.

Open over closed

The Liip principle of ‘open over closed’ sums up the project very well. All content on the new sg.ch website is accessible, and there was also a focus on open communication and having a shared objective throughout the entire course of the project.

*Being able to get an overview of the topics, information and services detailed on the canton of St. Gallen's website and then consolidating and reorganising this content was a huge task. Liip’s concept has ensured that all citizens can quickly find what they are looking for at sg.ch.
Clemens Nef, Canton of St. Gallen Project Manager

Jul 14 2019
Jul 14

I had the honor to attend this year’s Swiss data science conference, that as usual took place at the Kursaal in Bern. In this blog post I'd like to share my insights on the excellent talks and the trends that emerged at the conference.

The big over-arching theme of the conference was human-machine collaboration: How can the results of an AI system be best communicated to its users. This touches topics like the perception, trust, and ethics of such AI systems. I was happy to see these questions at the core of the conference since they have been at our heart too for almost two years: See https://www.liip.ch/en/services/development/data.

In comparison to the last years, I had the impression that more and more corporations are also hooking up with the formerly mostly academic data-science community. That is an impression I had based on the number of booths and talks at the conference.

KeyNote

Ken Hughes gave an amazingly well-rehearsed keynote, that made me think about how the development of technology transcends our every-day-businesses or how he put it: Where silicon meets soul.

One of his key insights was that it is not enough to just satisfy the customer needs these days, but if companies can manage to give their users a tribal sense of belonging and provide more than their customers expect they can truly empower the customer. It might sound cliché but generally shifting towards a high consumer-centricity seems to be a thing not only for Jeff Bezos anymore.

Talks

The SDS conference offered up to seven simultaneous breakout session tracks - see the program here - which made it almost impossible to attend all of the talks that I was interested in.

There were technical tracks on Deep Learning and NLP containing lessons from hands-on experience with the newest AI models but there were also business-oriented tracks offering insights on integrating machine learning models into production. I was happy to see that there was a Data Ethics track, which created an opportunity for interested data scientists to discuss and shape the impact of AI models on society. Bravo to this! To get an feeling of what the trends are I attended presentations in different tracks. Here are my musings on the ones I attended. Feel free to check out the now available slides and recordings online here.

Principles and Best Practices for Applied Machine Learning Models

At Swiss Re, showed how the collective expertise and experience of numerous expert practitioners and managers from data science and risk management can be harnessed to to create a definitive set of principles and best practices that guides all our data science activities. In this talk they presented and discussed these principles and emphasized the principles which need much more care by the Data Scientist in industrial applications than in education and research.

I liked the birds-eye view segmenting those principles into “data-related”, “model-related”, “user-related” and “governance-related” areas, which forces us to think about all of these aspects at the same time.

Revenue Forecasting and Store Location Planning at Migros

The data scientists at Migros have presented their own algorithm that provides realistic revenue forecasts even for complex scenarios such as multiple new stores and/or alterations of the competitor store network. This algorithm combined a heuristic simulation of consumer behavior with machine learning methods to deliver both accurate and interpretable results.

It was interesting to learn how their in-house solution was developed in close collaboration with key users at Migros’ ten regional cooperatives. It bacame clear that interpretability for the planning expert was one of the main features that drove the adoption of the tool. Conrats to Bojan Škerlak from Migros, who won the Best Presentation award!

I was also excited to see that in order to create their tool they made use of a lot of open data from the BFS and SBB, which was then combined with their transactional cumulus data. In order to arrive at the end result, they ended up combining their “gravity model” with business logic to make the results more interpretable to the end-users.

Do You Have to Read All Incoming Documents?

In the NLP track Mark Cieliebak showed how NLP solutions can be used to provide automatic classification of incoming documents into pre-defined classes (such as medical reports, prescriptions etc.) and then showed how to extract the relevant information for further processing. Using real-world examples, they have provided an overview of the potential applications, a realistic assessment of the effort and the resulting quality to be expected.

I particularly liked his assessment of the effort needed for data preparation and implementation in regards to the different project cases that they have encountered. Unsurprisingly when a business owner already has a huge corpus of annotated material that greatly reduces the data preparation part, and so allows the team to focus on an excellent implementation of the project.

Also combining supervised with unsupervised learning methods during training and data processing seemed to be an fruitful approach for classes where not enough data is available.

GPU Acceleration with RAPIDS for Traditional Big Data Analytics or Traditional Machine Learning

In the technical track René Müller gave a very interesting talk about how the RAPIDS suite provides the freedom to execute end-to-end data science and analytics pipelines entirely on GPUs. I liked how a simple python library can be used to accelerate common data preparation tasks for analytics and data science without running into typical serialization costs.

I was surprised how many classical algorithms already are implemented in their library (see screenshot below), yet many of those can only run on a single GPU (of course there is always (text: DASK link: https://dask.org/) . It is worth to note that when using a model that has been trained with RAPIDS one needs to also have a GPU to run it in inference mode, which makes them less portable.

If you are inclined to give it a try, you can either install it on your laptop (if it has a GPU) or simply try the fastest thing that works out of the box: This is a jupyter notebook in the google colaboratory wich even gives you T4 instances for free.

Creating Value for Clients through Data & Analytics

In this talk, Michel Neuhaus (Head of advanced analytics) and Daniel Perruchoud (Professor at FHNW) showed their journey towards a corporate environment focused on creating value through analytics for UBS and clients. They shared interesting insights from their track towards offering a collaborative work place for data science professionals.

As an illustrative example, they have shown a simple use case of segmenting clients into groups of individuals with similar needs allowing us to offer the right service to the right client. I liked how they emphasized that the lessons learned along the way were embraced to drive the design and operation of the solution. For them, this meant going from a use-case oriented view to a vertical use case built on top of horizontal capabilities (see screenshot).

In the sample case that they provided, I liked how they were honest about how they modeled temporal client behavior, by just computing some very simple statistics, such as the minimum, mean, median or the average trend of a customers bank balance in time.

Final Thoughts

Overall the SDS 2019 was an excellent conference showing how the business applications and machine learning technologies are growing more closely together. Having seen a number of talks I am convinced that the winning formula of tech + business = ❤️ needs two additional components though: data and user experience.

Only if you really have the right data - and also have the right to use it in the way you intend to - you have a solid foundation for everything that builds on it. Collecting the right customer data and doing so in a way that preserves the user's privacy remains one of the biggest challenges today.

Focusing strongly on user experience is an aspect that gets neglected the most. Users are not interested in technology but instead in improving their user experience, thus any new type of algorithm has only one goal: to improve the user’s experience. Only then is the final result perceived as useful can stand a chance against the existing status quo.

So coming back to the key-note this means that indeed successful projects are those that use the existing ML-technology to delight, engage and empower the users.

Jul 11 2019
Jul 11

Switzerland’s largest corporate network has a new digital home: the Swiss Venture Club’s completely revised website went live in June – with a new design, fresh content and a comprehensive members’ portal.

Switzerland’s top business award

The Swiss Venture Club (SVC) serves SMEs and offers its 3000 members from all sectors and regions one of the largest and most important business networks in Switzerland.

At the 7 Prix SVC awards in all economic regions of Switzerland, a top-class jury selects the most successful companies in the region. The great and good of Switzerland’s business world all come to this awards ceremony – which has to take place in the large Hallenstadion stadium to fit the more than 3000 attendees. It is reported in the national press, and the companies involved enjoy nationwide publicity. The Swiss Venture Club is therefore a central driving force for the Swiss SME scene, setting an example of good practice. So, it was about time it set such an example for digital transformation too. In addition to the design of its digital presence, Swiss Venture Club’s entire event process has also been transformed.

From service design to implementation

A process that began with service design workshops ended with a comprehensive web platform. The service design process precisely analysed the needs of the Swiss Venture Club, its members and its sponsors in order to establish the basis for a successful web platform – because anyone paying a five-figure membership fee or sponsorship contribution is entitled to have their say on such a project. High quality services, user-friendly applications and informative content for about 3000 members as well as countless regional and national sponsors were in the foreground. Within the scope of the service design process, the following topics were dealt with:

  • Business requirements
  • Business and user objectives
  • Competitor analysis
  • Customer journeys for members and sponsors
  • Interviews with existing members and sponsors

The results of the service design process formed the basis for the next step, namely developing the UX concept and design.
The platform development process then built on these foundations. The CMS Drupal was used to implement the following core functionalities:

  • Website
  • Members’ and sponsors’ portal
  • Event management incl. ticketing

From the very first workshop, it became clear that we had begun an exciting and comprehensive collaboration. An innovative user experience and CRM integration were important in order to attract and inspire interested visitors, members, sponsors and also, of course, the national press.

Heralding digital transformation with good content

The project also included developing a content strategy. We defined messages, content formats and distribution channels. The new website comes with a few changes. We therefore started communicating with our stakeholders about the changes at a very early stage, and members and sponsors were regularly asked for feedback via our newsletter during the digital transformation process. Incorporating them into the change process in this way ensured a high degree of acceptance when the time came to launch the platform.

Compasses over maps

SEO and analytics help to measure, and thus optimise, the website’s performance, so we put them to good use for the SVC website: an audit and a needs analysis were used to define performance indicators and tracking requirements, which we then incorporated into the website. Target-oriented dashboards now provide regular information about the usage and performance of the new page. Key objectives from an SEO perspective were to comply with technical SEO best practices to attract visitors to the website, migrate content without any loss, and support the production of content geared towards users’ search needs.

User numbers and positive feedback show that the close interaction between concept, design, content, analytics and, of course, technical implementation has paid off. The Swiss Venture Club is ready for the future.

"The collaboration with Liip was intensive, geared to the needs of our target groups and resulted in a modern web platform. Our web platform was very well received by members, sponsors and the media. The Liip team skillfully moderated the agile development and supported us with high commitment and pragmatic solutions."
Alexander Saner, Head Services - Finance IT

Jul 07 2019
Jul 07

Six apprentices succeed their apprenticeship this summer. Congratulations to Niclas, Rami, Tim, Sina, Sonja and Gian!

They learnt how to code and developed websites and mobile applications using technologies such as PHP, Magento, Angular and OctoberCMS. Our business administration apprentices gained knowledge in various fields like Finance, Human Resources and Administration. All of them developed their communication skills and their self-organising attitude.

We are very proud of them. They are the experts of tomorrow. By taking on apprentices, we are investing in the future of not only young people but also the community and therefore the society.

Get to know them better. They tell you all about their time as a Liipers.

Gian Zgraggen

Developer - Zurich - Ping me
Motivated, helpful, sarcastic, coffee lover, metal head, nerd.

Tell me about a project or a task that you handled during your apprenticeship and that you liked.

My first client project was Meinplatz.ch, created with OctoberCMS. I really liked working on it since it was the first "real" project I was able to work on. Before working at Liip I was not able to work on any client project.

What do you like most about your job?

I really like working as a developer. I enjoy working as a dev at Liip, because I am able to work together with clients and directly take influence and help shaping a project. I enjoy the structure of the company and the possibilities it offers. The other employees are young, friendly, and easy to talk to, that way the whole atmosphere is way lighter than in other businesses.

What's your best memory with Liip?

Working on my first project for a client as well as both LiipConfs (our annual internal conference) I was able to attend.

What else do you want to learn?

I want to learn different technologies to create websites, increase my skills in mobile development as well as JavaScript. Additionally I would like to learn more about User Experience (UX).

Sonja Nydegger

Business administrator (Finance/Admin) - Fribourg - Ping me
Happy, motivated, ambitious, helpful, shy, unathletic, sensitive.

Tell me about a project or a task that you handled during your apprenticeship and that you liked.

I liked it very much to get an insight into the hiring process. I was able to document the applications after sending a confirmation of receipt. Handling the various applications not only gave me inspiration for my own application, but also gave me a lot of fun, as we also receive very creative applications. I was able to give my own opinion about each candidate.

What do you like most about your job?

I like the variety of my job. I get insights into the company and can carry very different tasks, which are associated with responsibilities (for example: cash management, applications recording and much more). I am convinced that this is a good apprenticeship and that it will bring a lot of opportunities.

What's your best memory with Liip?

Where to start? I have so many beautiful memories of my time at Liip. I love that all the Liipers are close, which results in a friendly and very nice working atmosphere. I will remember the opportunity I had to write my own blog post ;-) and the chance I had to take over Instagram for a week with other apprentices.

What else do you want to learn?

I would like to deepen my knowledge in the area of finance. I wish to pursue further education as a specialist in finance and accounting.

Sina Marty

Business administrator (Finance/HR/Admin) - Zurich - Ping me
Ambitious, motivated, helpful, funny, cheerful, athletic (gymnastics), impatient.

Tell me about a project or a task that you handled during your apprenticeship and that you liked.

During my three year apprenticeship, I liked the financial area the most. Most of all I liked to create the invoices. It was also exciting to experience the progress of the invoicing process. :-)

What do you like most about your job?

That despite an administrative job there is a lot of variety and contact with internal and external customers and partners.

What's your best memory with Liip?

I have many beautiful memories, but one of my most favorite is my first day at Liip. I was warmly welcomed and was able to work independently very quickly, which I really liked.

What else do you want to learn?

I would like to continue working and starting the vocational baccalaureate (Berufsmaturität) in order to study later.

Tim Keller

Backend & Frontend Developer - Zurich - Ping me
Eager to learn and discover, sarcastic, keen to debate, social, eloquent, helpful, tech-enthusiast.

Tell me about a project or a task that you handled during your apprenticeship and that you liked.

At the end of the IT apprenticeship, I was able to work on Kochoptik and I was part of a development team which integrated me fully.

What do you like most about your job?

The web development is incredibly varied and, depending on the complexity of the project, infinitely expandable and improvable. There are new challenges and problems every day, which have to be solved.

What's your best memory with Liip?

During every project I got to know and appreciate my colleagues better. I was able to work on exciting projects and get to know a lot of great people at the same time.

What else do you want to learn?

I want to expand my knowledge in backend and frontend development and explore the whole Linux world even better. Later I might want to continue my studies at vocational baccalaureate school (Berufsmaturitätsschule).

Rami Jumaah

Backend Developer - Zurich - Ping me
Motivated, helpful, funny, happy, fitness lover, impatient, tea lover.

Tell me about a project or a task that you handled during your apprenticeship and that you liked.

A website that I made for my IPA (Graduation Project) for school, because I built everything from scratch - UX till delivering. And every user who is gonna make use of the website have already liked the website.

What do you like most about your job?

The cooperation between the team members.

What's your best memory with Liip?

All the aperos that we have regularly.

What else do you want to learn?

Open for every piece of information that I can get from any person at the company.

Niclas Deplazes

Backend & Frontend Developer - St-Gallen
Patient, focused, eager to learn, hockey fan, hip hop enthusiast, kindle-worm, tea lover.

Tell me about a project or a task that you handled during your apprenticeship and that you liked.

I was able to work on a real customer project, right from the start of my internship. It was the Memberplus Portal of Raiffeisen Bank. I worked on the backend (Magento & PHP) as well as on the frontend (Angular).

What do you like most about your job?

Generally I like my daily work as an application developer because it's very varied and quite a challenge too. Furthermore, I learn something new every day, especially as a beginner. So I never get bored.

What's your best memory with Liip?

The highlight of my internship was the successful go-live of Memberplus, as I spent most of my internship (including my Graduation Project) on the project.

What else do you want to learn?

During my one-year training at Liip, I was able to deepen and expand my basic knowledge from three school years at IMS Frauenfeld. After the military service, I would like to continue working at Liip for a year in order to gain even more practical experience. Afterwards I will probably go to the ZHAW (Zurich University of Applied Sciences) to start a bachelor's degree in computer science.

Oct 28 2018
Oct 28

In 2017, Drupal Association decided not to host a DrupalCon Europe 2018 due to waning attendance and financial losses. They took some time to make the European event more sustainable. After this, the Drupal community decided to organise a Drupal Europe event in Darmstadt, Germany in 2018. My colleagues and I joined the biggest European Drupal event in October and here is my summary of few talks I really enjoyed!

Driesnote

By Dries Buytaert
Track: Drupal + Technology
Recording and slides

This year, Dries Buytaert focuses on improvements made for Drupal users such as content creators, evaluators and developers.

Compared to last year, Drupal 8 contributions increased by 10% and stable modules released by 46%. Moreover, a steady progress is noticeable. Especially in many core initiatives like the last version of Drupal 8 which is shipped with features and improvements created from 4 core initiatives.

Content creators are the key-decision makers in the selection of a CMS now. Their expectations have changed: they need flexibility but also simpler tools to edit contents. The layout_builder core module gives some solutions by enabling to edit a content inline and drag-and-dropping elements in different sections. The management of medias has been improved too and there is a possibility to prepare different “states” of contents using workspaces module. But the progress doesn’t stop here. The next step is to modernize the administrative UI with a refresh of the Seven administration theme based on React. Using this modern framework makes it familiar to Javascript (JS) developers and is building a bridge with the JS community.

Drupal took a big step forward for evaluators as it provides a demo profile called “Umami” now. Evaluators have a clear understanding of what kind of websites can be produced by Drupal and how it works by navigating through the demo website.
The online documentation on drupal.org has also been reorganized with a clear separation of Drupal 7 and Drupal 8. It provides some getting-started guides too. Finally, a quick-install link is available to have a website running within 3 clicks and 1 minute 27 seconds!

Developers experience has been improved as well: minor releases are now supported for 12 months instead of the former 4 weeks. Teams will have more time to plan their updates efficiently. Moreover, Gitlab will be adopted within the next months to manage the code contributions. This modern collaborative tool will encourage more people to participate to projects.

Regarding the support of the current Drupal versions, Dries shares that Symfony 3, the base component of Drupal 8 will be end-of-life by 2021. To keep the CMS secure, it implies to be end-of-life by November 2021 and Drupal 9 should be released in 2020. The upgrade from Drupal 8 to Drupal 9 should be smooth as long as you stay current with the minor releases and don’t use modules with deprecated APIs.
The support of Drupal 7 has been extended to November 2021 as the migration path from Drupal 7 to Drupal 8 is not stable with multilingualism yet.

This is a slide from Driesnote presentation showing a mountain with many tooltips: "Drupal 8 will be end-of-life by November 2021", "Drupal 7 will be supported until November 2021", "Drupal 9 will be released in 2020", "Drupal 8 became a better tool for developers", "You now have up to 12 months to upgrade your sites", "Drupal 8 became much easier to evaluate", "We've begun to coordinate the marketing of Drupal", "Drupal 8 became easier to use for content creators", "Drupal.org is moving to GitLab very soon".Slide from Driesnote showing current state of Drupal.

Last but not least, DrupalCon is coming back next year and will be held in Amsterdam!

JavaScript modernisation initiative

By Cristina Chumillas, Lauri Eskola, Matthew Grill, Daniel Wehner and Sally Young
Track: Drupal + Technology
Recording and slides

After a lot of discussions on which JS framework will be used to build the new Drupal administrative experience, React was finally chosen for its popularity.

The initiative members wanted to focus on the content editing experience. This affects a big group of Drupal users. The goal was to simplify and modernize the current interface. Furthermore, embracing practices that are familiar to JS developers so they can easier join the Drupal community.
On one hand, a UX team ran some user tests. Those showed that users like the flexibility they have with Drupal interface but dislike its complexity usually. A comparative study was ran to know what has been used in other tools or CMSs too. On the other hand, the User Interface (UI) team worked on the redesign of the administrative interface and built a design system based on components. The refreshment of the Seven administration theme is ongoing.
Another group worked on prototyping the User Experience (UX) and User Interface (UI) changes with React. For instance, if an editor quits a page without saving they's last changes, a popup appears to restore the last changes. This is possible due to contents stored to the state of the application.

You can see a demo of the new administrative UI in the video (go to 20 minutes 48 seconds):

[embedded content]

If you are interested, you can install the demo and of course join the initiative!

Drupal Diversity & Inclusion: Building a stronger community

By Tara King and Elli Ludwigson
Track: Drupal Community
Recording

Diversity in gender, race, ethnicity, immigration status, disability, religion etc. helps a lot. Proven it makes a team more creative, collaborative and effective.

Tara King and Elli Ludwigson who are part of the Drupal Diversity and Inclusion team presented how Drupal is building a stronger and smarter community. The initial need was to make Drupal a safer place for all. Especially for the less visible ones at community events such as women, minorities and people with disabilities.
The group addressed several issues, such as racism, sexism, homophobia, language barriers etc. with different efforts and initiatives. For example, diversity is highlighted and supported in Drupal events: pronoun stickers are distributed, #WeAreDrupal hashtag is used on Twitter and social events are organized for underrepresented people as well. Moreover, the group has released an online resource library, which collects articles about diversity. All of this is ongoing and new initiatives were created. Helping people finding jobs or attracting more diverse people as recruiters are only two to name.

Flyer put on a table with the text "Make eye Contact. Invite someone to join the conversation. Consider new perspectives. Call out exclusionary behavior. Be an ally at Drupal events."Diversity and Inclusion flyer, photo by Paul Johnson, license CC BY-NC 2.0 Sign mentionning "All-gender restrooms" at Drupal Europe venue.All-gender restrooms sign, photo by Gábor Hojtsy, license CC BY-SA 2.0

If you are interested in the subject and would like to be involved, there are weekly meetings in #diversity-inclusion Drupal Slack channel. You can join the contrib team or work on the issue queue too.

Willy Wonka and the Secure Container Factory

By Dave Hall
Track: DevOps + Infrastructure
Recording

Docker is a tool that is designed to create, deploy and run applications easily by using containers. It is also about “running random code downloaded from the internet and running it as root”. This quote points out how it is important to maintain secure containers. David Hall illustrates this with practical advice and images from the “Willy Wonka and the chocolate factory” movie. Here is a little recap:

  • Have a light image: big images will slow down deployments and also increase the attack surface. Install an Alpine distribution rather than a Debian which is about 20 times lighter;
  • Check downloaded sources very carefully: for instance, you can use wget command and validate checksum for a file. Plus you can scan your images to check vulnerabilities using tools like Microscanner or Clair;
  • Use continuous development workflows: build a plan to maintain your Docker images, using a good Continous Integration / Continous Delivery (CI/CD) system and document it;
  • Specify a user in your dockerfile: running root on a container is the same as running root on the host. You need to reduce the actions of a potential attacker;
  • Measure your uptime in hours/days: it is important to rebuild and redeploy often to potentially avoid having a compromised system for a long time.

Now you are able to incorporate these advice into your dockerfiles in order to build a safer factory than Willy Wonka’s.

Decoupled Drupal: Implications, risks and changes from a business perspective

By Michael Schmid
Track: Agency + Business
Recording

Before 2016, Michael Schmid and his team worked on fully Drupal projects. Ever since they are working on progressive and fully decoupled projects.
A fully decoupled website means that frontend is not handled with Drupal but with a JS framework such as React. This framework is “talking” to Drupal via an API such as GraphQL. It also means, that all interactions from Drupal are gone: views with filters, webforms, comments etc. If a module provides frontend, it is not useable anymore and needs to be somehow re-implemented.
When it comes to progressive decoupled websites, frontend stack is still built with Drupal. But some parts are implemented with a JS framework. You can have data provided by APIs or injected from Drupal too. The advantage is that you can benefit from Drupal components and don’t need to re-implement everything. A downside of it are conflicts with CSS styling and build systems handled on both sides. Therefore you need to have a clear understanding of what does what.

To be able to run such projects successfully, it is important to train every developer in new technologies: JS has evolved and parts of the logic can be built with it. We can say that backenders can do frontend now. In terms of hiring it means, you can hire full stack developers but also JS engineers. Attracting more developers as they love working with JS frameworks such as React on a global level.

Projects are investments which continue over time and expect failures at the beginning. These kinds of projects are more complex than regular Drupal ones, they can fail or go over budget. Learn from your mistakes and share them with your team in retrospectives. It is also very important to celebrate successes!
Clients request decoupled projects to have a faster and cooler experience for users. They need to understand that this is an investment that will pay off in the future.

Finally, fully decoupled Drupal is a trend for big projects and other CMSs are already using decoupled out of the box. Drupal needs to focus on a better editor experience and a better API. There might also be projects that require simple backend edition instead of Drupal.

Hackers automate but the Drupal Community still downloads updates on drupal.org or: Why we need to talk about Auto Updates

By Joe Noll and Hernani Borges de Freitas
Track: Drupal + Technology
Recording and slides

In 2017, 59% of Drupal users were still downloading modules from drupal.org. In other words, more than half of the users didn’t have any automatisation processes to install modules. Knowing that critical security updates were released in the past months and it is only a matter of hours until a website gets potentially hacked, it comes crucial to have a process to automate these updates.
The update can be quite complex and may take time: installing the update, reviewing the changes, deploying on a test environment, testing either automatically or manually and deploying on production. However this process can be simplify with automation in place.

There is a core initiative to support small-to-medium sites owners that usually are not taking care of security updates. The idea is a process to download the code and update sources in the Drupal directory.
For more complex websites, automating the composer workflow with a CI pipeline is recommended. Everytime a security update is released, the developer pushes it manually in the pipeline. The CI system builds an installation containing the security fix within a new branch. This will be deployed automatically to a non-productive environment where tests can be done and build approved. Changes can be merged and deployed on production afterwards.

A schema showing the update strategy through all steps from a CI pipelineUpdate strategy slide by Joe Noll and Hernani Borges de Freitas

To go further, the update_runner module focuses on automatizing the first part by detecting an update and firing up a push for an update job.

Conclusion

Swiss Drupal community members cheering at a restaurantMeeting the Swiss Drupal community, photo by Josef Dabernig, license CC BY-NC-SA 2.0

We are back with fresh ideas, things we are curious to try and learnings from great talks! We joined social events in the evenings too. Therefore we exchanged with other drupalists, in particular with the Swiss Drupal community! This week went so fast. Thank you Drupal Europe organizers for making this event possible!

Header image credits: Official Group Photo Drupal Europe Darmstadt 2018 by Josef Dabernig, license CC BY-NC-SA 2.0.

Aug 14 2018
Aug 14

Which face detection API is the best? Let’s have a look into Amazon Rekognition, Google Cloud Vision API, IBM Watson Visual Recognition and Microsoft Face API.

Part 1: SaaS vendors

This article is the first part of a series. Make sure to subscribe to receive future updates!
TLDR: If you want to use the API's as fast as possible, directly check out my code on GitHub.

Did you ever had the need for face detection?
Maybe to improve image cropping, ensure that a profile picture really contains a face or maybe to simply find images from your dataset containing people (well, faces in this case).
Which face detection SaaS vendor would be the best for your project? Let’s have a deeper look into the differences in success rates, pricing and speed.

In this blog post I'll be analyzing the face detection API's of:

How does face detection work anyway?

Before we dive into our analysis of the different solutions, let’s understand how face detection works today in the first place.

The Viola–Jones Face Detection

It’s the year 2001. Wikipedia is being launched by Jimmy Wales and Larry Sanger, the Netherlands becomes the first country in the world to make same-sex marriage legal and the world witnesses one of the most tragic terror attacks ever.
At the same time two bright minds, Paul Viola and Michael Jone, come together to start a revolution in computer vision.

Until 2001, face detection was something which didn’t work very precise nor very fast. That was, until the Viola-Jones Face Detection Framework was proposed which not only had a high success rate in detecting faces but could do it also in real time.

While face and object recognition challenges existed since the 90’s, they surely boomed even more after the Viola–Jones paper was released.

Deep Convolutional Neural Networks

One of such challenges is the ImageNet Large Scale Visual Recognition Challenge which exists since 2010. While in the first two years the top teams were working mostly with a combination of Fisher Vectors and Support vector machines, 2012 changed everything.

The team of the University of Toronto (consisting of Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton) used for the first time a deep convolutional neural network for object detection. They scored first place with an error rate of 15.4% while the second placed team had a 26.2% error rate!
A year later, in 2013, every team in the top 5 was using a deep convolutional neural network.

So, how does such a network work?
An easy-to-understand video was published by Google earlier this year:

[embedded content]

What do Amazon, Google, IBM and Microsoft use today?

Since then, not much changed. Today’s vendors still use Deep Convolutional Neural Networks, probably combined with other Deep Learning techniques though.
Obviously, they don’t publish how their visual recognition techniques exactly work. The information I found was:

While they all sound very similar, there are some differences in the results.
Before we test them, let’s have a look at the pricing models first though!

Pricing

Amazon, Google and Microsoft have a similar pricing model, meaning that with increasing usage the price per detection drops.
With IBM however, you always pay the same price per API call after your free tier usage volume is exhausted.
Microsoft provides you the best free tier, allowing you to process 30'000 images per month for free.
If you need more detections though, you need to use their standard tier where you start paying from the first image on.

Price comparison

That being said, let’s calculate the costs for three different profile types.

  • Profile A: Small startup/business processing 1’000 images per month
  • Profile B: Digital vendor with lots of images, processing 100’000 images per month
  • Profile C: Data center processing 10’000’000 images per month

| | Amazon | Google | IBM | Microsoft |
|-----------|--------------:|---------------:|---------------:||---------------:|
| Profile A | $1.00 USD | Free | Free | Free |
| Profile B | $100.00 USD | $148.50 USD | $396.00 USD | $100.00 USD |
| Profile C | $8’200.00 USD | $10’498.50 USD | $39’996.00 USD | 7’200.00 USD |

Looking at the numbers, for small customers there’s not much of a difference in pricing. While Amazon charges you starting from the first image, having 1’000 images processed still only costs one Dollar. However, if you don’t want to pay anything, then Google, IBM or Microsoft will be your vendor to go.

Note: Amazon offers a free tier on which you can process 5’000 images per month for the first 12 months for free! However, after this 12 month trial, you’ll have to start paying starting with the first image.

Large API usage

If you really need to process millions of images, it's important to compare how every vendor scales.
Here's a list of the minimum price you pay for the API usage after a certain amount of images.

  • IBM constantly charges you $4.00 USD per 1’000 images (no scaling)
  • Google scales down to $0.60 USD (per 1’000 images) after the 5’000’000th image
  • Amazon scales down to $0.40 USD (per 1’000 images) after the 100’000’000th image
  • Microsoft scales down to $0.40 USD (per 1’000 images) after the 100’000’000th image

So, comparing prices, Microsoft (and Amazon) seem to be the winner.
But can they also score in success rate, speed and integration? Let’s find out!

Hands on! Let’s try out the different API’s

Enough theory and numbers, let’s dive into coding! You can find all code used here in my GitHub repository.

Setting up our image dataset

First things first. Before we scan images for faces, let’s set up our image dataset.
For this blog post I’ve downloaded 33 images from pexels.com, many thanks to the contributors/photographers of the images and also to Pexels!
The images have been committed to the GitHub repository, so you don't need to search for any images if you simply want to start playing with the API's.

Writing a basic test framework

Framework might be the wrong word as my custom code only consists of two classes. However, these two classes help me to easily analyze image (meta-) data and have as few code as possibly in the different implementations.

A very short description: The FaceDetectionClient class holds general information about where the images are stored, vendor details and all processed images (as FaceDetectionImage objects).

Comparing the vendors SDK’s

As I’m most familiar with PHP, I've decided to stick to PHP for this test. I still want to point out what SDK’s each vendor provides (as of today):

| Amazon | Google | IBM | Microsoft |
|---------------|----------------|----------------|
|

  • Android
  • JavaScript
  • iOS
  • Java
  • .NET
  • Node.js
  • PHP
  • Ruby
  • Python
|
  • C#
  • Go
  • Java
  • Node.js
  • PHP
  • Python
  • Ruby
  • cURL examples
|
  • Node.js
  • Java
  • Python
  • cURL examples
|
  • C#
  • Go
  • Java
  • JavaScript
  • Node.js
  • PHP
  • Python
  • Ruby
  • cURL examples
|

Note: Microsoft doesn't actually provide any SDK's, they do offer code examples for the technologies listed above though.

If you’ve read the lists carefully, you might have noticed that IBM does not only offer the least amount of SDK’s but also no SDK for PHP.
However, that wasn’t a big issue for me as they provide cURL examples which helped me to easily write 37 lines of code for a (very basic) IBM Visual Recognition client class.

Integrating the vendors API’s

Getting the SDK's is easy. Even easier with Composer. However, I did notice some things that could be improved to make a developer’s life easier.

Amazon

I've started with the Amazon Rekognition API. Going through their documentation, I really felt a bit lost at the beginning. Not only did I miss some basic examples (or wasn’t able to find them?), but also I had the feeling that I have to click a few times until I was able to find what I was looking for. In one case I even gave up and simply got the information by directly inspecting their SDK source code.
On the other hand, it could just be me? Let me know if Amazon Rekognition was easy (or difficult) for you to integrate!

Note: While Google and IBM return the bounding boxes coordinates, Amazon returns the coordinates as ratio of the overall image width/height. I have no idea why that is, but it's not a big deal. You can write a helper function to get the coordinates from the ratio, just as I did.

Google

Next came Google. In comparison with Amazon, they do provide examples, which helped me a lot! Or maybe I was just already in the “investing different SDK’s” mindset.
Whatever the case may be, integrating the SDK felt a lot simpler and also I had to spend less clicks to retrieve information I was looking for.

IBM

As stated before, IBM doesn’t (yet?) provide a SDK for PHP. However, with the provided cURL examples, I had a custom client set up in no time. There’s not much that you can do wrong if a cURL example is provided to you!

Microsoft

Looking at Microsoft's code example for PHP (which uses Pear's HTTP_Request2 package), I ended up writing my own client for Microsoft's Face API.
I guess I'm simply a cURL person.

Inter-rater reliability

Before we compare the different face detection API’s, let's scan the images first by ourselves! How many faces would a human be able to detect?
If you already had a look on my dataset, you might have seen a few images containing tricky faces. What do I mean by "tricky"? Well, when you e.g. only see a small part of a face and/or the face is in an uncommon angle.

Time for a little experiment

I went over all images and wrote down how many faces I thought I've detected. I would use this number to calculate a vendor's sucess rate for an image and see if it was able to detect as many faces as I did.
However, setting the expected number of faces detected solely by me seemed a bit too biased to me. I needed more opinions.
This is when I kindly asked three coworkers to go through my images and tell me how many faces they would detect.
The only task I gave them was "Tell me how many faces, and not heads, you're able to detect". I didn't define any rules, I wanted to give them any imaginable freedom for doing this task.

What is a face?

When I ran through the images detecting faces, I just counted every face from which at least around a quarter was visible. Interestingly my coworkers came up with slightly different definitions of a face.

  • Coworker 1: I've also counted faces which I mostly wasn't able to see. But I did see the body, so my mind told me that there is a face
  • Coworker 2: If I was able to see the eyes, nose and mouth, I've counted it as a face
  • Coworker 3: I've only counted faces which I would be able to recognize in another image again

Example image #267885

My coworkers and me detected each 10, 13, 16 and 16 faces in this image. I've decided to continue with the average, thus 14.

It was very interesting to me to see how everyone came up with different techniques regarding face detection.
That being said, I've used the average face count of my results and the ones from my coworkers to set the expected number of faces detected for an image.

Comparing the results

Now that we have the dataset and the code set up, let’s process all images by all competitors and compare the results.
My FaceDetectionClient class also comes with a handy CSV export which provides some analytical data.

This is the first impression I've received:

Amazon Google IBM Microsoft Total faces detected 99 / 188
(52.66 %) 76 / 188
(40.43 %) 74 / 188
(39.36 %) 33 / 188
(17.55 %) Total processing time (ms) 57007 43977 72004 40417 Average processing time (ms) 1727 1333 2182 1225

Very low success rates?

Amazon was able to detect 52.66 % of the faces defined, Google 40.43 %, IBM 39.36 % and Microsoft even just 17.55 %.
How come the low success rates? Well, first off, I do have lots of tricky images in my dataset.
And secondly, we should not forget that we, as humans, do have a couple of million years worth of evolutionary context to help understand what something is.
While many people believe that we've mastered face detection in tech already, there's still room for improvement!

The need for speed

While Amazon was able to detect the most faces, Google’s and Microsoft’s processing times were clearly faster than the other ones. However, in average they still need longer than one second to process one image from our dataset.
Sending the image data from our computer/server to another server surely scratches on performance.

Note: We’ll find out in the next part of the series if (local) open source libraries could do the same job faster.

Groups of people with (relatively) small faces

After analyzing the images, Amazon seems to be quite good at detecting faces in groups of people and where the face is (relatively) small.

A small excerpt

Image # Amazon
(faces detected) Google
(faces detected) IBM
(faces detected) Microsoft
(faces detected) 109919 151088 34692 10868 889545 104nonenone

Example image #889545 by Amazon

Amazon was able to detect 10 faces in this image, while Google only found 4, IBM 0 and Microsoft 0.

Different angles, uncomplete faces

So, does it mean that IBM is simply less good than their competitors? Not at all. While Amazon might be good in detecting small faces in group photos, IBM has another strength:
Difficult images.

What do I mean with that? Well, images with faces where the head is in an uncommon angle or maybe not shown completely.
Here are three examples from our dataset from which IBM was the sole vendor to detect the face.

Example image #356147 by IBM

Image with a face only detected by IBM.

Example image #403448 by IBM

Image with a face only detected by IBM.

Example image #761963 by IBM

Image with a face only detected by IBM.

Bounding boxes

Yes, also the resulting bounding boxes are different.
Amazon, IBM and Microsoft are here very similar and return the bounding boxes of a person’s face.
Google is slightly different and focuses not on someone’s face but on the complete head (which makes more sense to me?).

Example image #933964 by Google

Google returns bounding boxes covering most of the head, not just the face.

Example image #34692 by Microsoft

Microsoft (as well as IBM and Amazon) focus on the face instead of the head.

What is your opinion on this? Should an API return the bounding boxes to the person's face or to the person's head?

False positives

Even though our dataset was quite small (33 images), it contains two images on which face detection failed for some vendors.

Example image #167637 by Amazon

Find the face!

In this (nice) picture of a band, Amazon and Google both didn’t detect the face of the front man but of his tattoo(!) instead. Microsoft didn't detect any face at all.
Only IBM succeeded and correctly detected the front man’s face (and not his tattoo).
Well played IBM!

Example image #948199 by Google

Two-Face, is that you?

In this image Google somehow detected two faces in the same region. Or the network sees something which is invisible to us. Which is even more scary.

Wait, there is more!

You can find the complete dataset with 33 source images, 4x 33 processed images and the metadata CSV export on GitHub.
Not only that, if you clone the repository and enter your API keys, you can even process your own dataset!
At last but not least, if you know of any other face detection API, feel free to send me a pull request to include it to the repository!

How come the different results?

As stated in the beginning of this blog post, none of the vendors completely reveal how they implemented face detection.
Let’s pretend for a second that they use the same algorithms and network configuration - they could still end up with different results depending on the training data they used to train their neural network.

Also there might be some wrappers around the neural networks. Maybe IBM simply rotates the image 3 times and processes it 4 times in total to also find uncommon face angles?
We may never find out.

A last note

Please keep in mind that I only focused on face detection. It’s not to confuse with face recognition (which can tell if a certain face belongs to a certain person) and also I didn’t dive deeper into other features the API’s may provide to you.
Amazon for example, tells you if someone is smiling, has a beard or their eyes open/closed. Google can tell you the likeliness if someone is surprised or wearing a headwear. IBM tries to provide you an approximately age range of a person including its likely gender. And Microsoft could tell you if a person is wearing any makeup.

The above points are only a few examples of what this vendors can offer to you. If you need more than just basic face detection, I highly recommend you to read and test their specs according to your purpose.

Conclusion

So, which vendor is now the best? There is really no right answer to this. Every vendor has its strengths and weaknesses. But for “common” images, Amazon, Google and IBM should do a pretty good job.
Microsoft didn't really convince me though. With 33 out of 188 faces detected, they had the lowest success rate of all four vendors.

Example image #1181562 by Google

For "common" images, Amazon, Google and IBM will be able to detect all faces.

Example image #1181562 by Microsoft

Microsoft, y u no detect faces?

What about OpenCV and other open source alternatives?

This question will be answered in the next part of this series. Feel free to subscribe to our data science RSS feed to receive related updates in the future and thank you so much for reading!

Aug 06 2018
Aug 06

After learning about the dirty tricks of deep learning for computer vision in part 1 of the blog post series, now we finally write some code to train an existing resnet50 network to distinguis llamas from oryxes. Learn two tricks that allow us to do deep learning with only 100 images.

Short Recap from Part 1

In the last blog post I briefly discussed the potential of using deep learning to build a zoo pokedex app that could be used to motivate zoo goers to engage with the animals and the information. We also discussed the imagenet competition and how deep learning has drastically changed the image recognition game. We went over the two main tricks that deep learning architectures do, namely convolutions and pooling, that allow such deep learning networks to perform extremely well. Last but not least we realized that all you have to do these days is to stand on the shoulders of giants by using the existing networks (e.g. Resnet50) to be able to write applications that have a similar state of the art precision. So finally in this blog post it’s time to put these giants to work for us.

Goal

The goal is to write an image detection app that will be able to distinguish animals in our zoo. Now for obvious reasons I will make our zoo really small, thus only containing two types of animals:

  • Oryxes and
  • LLamas (why there is a second L in english is beyond my comprehension).

Why those animals? Well they seem fluffy, but mostly because the original imagenet competition does not contain these animals. So it represents a quite realistic scenario of a Zoo having animals that need to be distinguished but having existing deep learning networks that have not been trained for those. I really have picked these two kinds of animals mostly by random just to have something to show. (Actually I checked if the Zürich Zoo has these so i can take our little app and test it in real life, but that's already part of the third blog post regarding this topic)

Getting the data

Getting data is easier than ever in the age of the internet. Probably in the 90ties I would have had to go to some archive or even worse take my own camera and shoot lots and lots of pictures of these animals to use them as training material. Today I can just ask Google to show me some. But wait - if you have actually tried using Google Image search as a resource you will realize that downloading their images in huge amounts is a pain in the ass. The image api is highly limited in terms of what you can get for free, and writing scrapers that download such images is not really fun. That's why I went to the competition and used Microsoft's cognitive services to download images for each animal.

Downloading image data from Microsoft

Microsoft offers quite a convenient image search API via their cogitive services. You can sign up there to get a free tier for a couple of days, which should be enough to get you started. What you basically need is an API Key and then you can already start downloading images to create your datasets.

# Code to download images via Microsoft cognitive api
require 'HTTParty'
require 'fileutils'

API_KEY = "##############"
SEARCH_TERM = "alpaka"
QUERY = "alpaka"
API_ENDPOINT  = "https://api.cognitive.microsoft.com/bing/v7.0/images/search"
FOLDER = "datasets"
BATCH_SIZE = 50
MAX = 1000

# Make the dir
FileUtils::mkdir_p "#{FOLDER}/#{SEARCH_TERM}"

# Make the request
headers = {'Ocp-Apim-Subscription-Key' => API_KEY}
query = {"q": QUERY, "offset": 0, "count": BATCH_SIZE}
puts("Searching for #{SEARCH_TERM}")
response = HTTParty.get(API_ENDPOINT,:query => query,:headers => headers)
total_matches = response["totalEstimatedMatches"]

i = 0
while response["nextOffset"] != nil && i < MAX
    response["value"].each do |image|
        i += 1
        content_url = image["contentUrl"]
        ext = content_url.scan(/^\.|jpg$|gif$|png$/)[0]
        file_name = "#{FOLDER}/#{SEARCH_TERM}/#{i}.#{ext}"
        next if ext == nil
        next if File.file?(file_name)
        begin
            puts("Offset #{response["nextOffset"]}. Downloading #{content_url}")
            r = HTTParty.get(content_url)
            File.open(file_name, 'wb') { |file| file.write(r.body) }
        rescue
            puts "Error fetching #{content_url}"
        end
    end
    query = {"q": SEARCH_TERM, "offset": i+BATCH_SIZE, "count": BATCH_SIZE}
    response = HTTParty.get(API_ENDPOINT,:query => query,:headers => headers)
end

The ruby code above simple uses the API in batches and downloads llamas and oryxes into their separate directories and names them accordingly. What you don’t see is that I went through these folders by hand and removed images that were not really the animal, but for example a fluffy shoe, that showed up in the search results. I also de-duped each folder. You can scan the images quickly on your mac using the thumbnail preview or use an image browser that you are familiar with to do the job.

Problem with not enough data

Ignoring probable copyright issues (Am i allowed to train my neural network on copyrighted material) and depending on what you want to achieve you might run into the problem, that it’s not really that easy to gather 500 or 5000 images of oryxes and llamas. Also to make things a bit challenging I tried to see if it was possible to train the neural networks using only 100 examples of each animal while using roughly 50 examples to validate the accuracy of the networks.

Normally everyone would tell you that you need definitely more image material because deep learning networks need a lot of data to become useful. But in our case we are going to use two dirty tricks to try to get away with our really small collection: data augmentation and reuse of already pre-trained networks.

Image data generation

A really neat handy trick that seems to be prevalent everyday now is to take the images that you already have and change them slightly artificially. That means rotating them, changing the perspective, zooming in on them. What you end up is, that instead of having one image of a llama, you’ll have 20 pictures of that animal, just every picture being slightly different from the original one. This trick allows you to create more variation without actually having to download more material. It works quite well, but is definitely inferior to simply having more data.

We will be using Keras a deep learning library on top of tensorflow, that we have used before in other blog posts to create a good sentiment detection. In the domain of image recognition Keras can really show its strength, by already having built in methods to do image data generation for us, without having to involve any third party tools.

# Creating a Image data generator
train_datagen = ImageDataGenerator(preprocessing_function=preprocess_input,
    shear_range=0.2, zoom_range=0.2, horizontal_flip=True)

As you can see above we have created an image data generator, that uses sheering, zooming and horizontal flipping to change our llama pictures. We don’t do a vertical flip for example because its rather unrealistic that you will hold your phone upside down. Depending on the type of images (e.g. aerial photography) different transformations might or might not make sense.

# Creating variations to show you some examples
img = load_img('data/train/alpaka/Alpacca1.jpg')
x = img_to_array(img) 
x = x.reshape((1,) + x.shape)  
i = 0
for batch in train_datagen.flow(x, batch_size=1,
                          save_to_dir='preview', save_prefix='alpacca', save_format='jpeg'):
    i += 1
    if i > 20:
        break  # otherwise the generator would loop indefinitely

Now if you want to use that generators in our model directly you can use the convenient flow from directory method, where you can even define the target size, so you don’t have to scale down your training images with an external library.

# Flow from directory method
train_generator = train_datagen.flow_from_directory(train_data_dir,
    target_size=(sz, sz),
    batch_size=batch_size, class_mode='binary')

Using Resnet50

In order to finally step on the shoulder of giants we can simply import the resnet50 model, that we talked about earlier. Here is a detailed description of each layer and here is the matching paper that describes it in detail. While there are different alternatives that you might also use the resnet50 model has a fairly high accuracy, while not being too “big” in comparison to the computationally expensive VGG network architecture.

On a side note: The name “res” comes from residual. A residual can be understood a a subtraction of features that were learned from the input a leach layer. ResNet has a very neat trick that allows deeper network to learn from residuals by “short-circuiting” them with the deeper layers. So directly connecting the input of an n-th layer to some (n+x)th layer. This short-circuiting has been proven to make the training easier. It does so by helping with the problem of degrading accuracy, where networks that are too deep are becoming exponentially harder to train.

#importing resnet into keras
from keras.models import load_model
base_model = ResNet50(weights='imagenet')

As you can see above, importing the network is really dead easy in keras. It might take a while to download the network though. Notice that we are downloading the weights too, not only the architecture.

Training existing models

The next part is the exciting one. Now we finally get to train the existing networks on our own data. The simple but ineffective approach would be to download or just re-build the architecture of the successful network and train those with our data. The problem with that approach is, that we only have 100 images per class. 100 images per class are not even remotely close to being enough data to train those networks well enough to be useful.

Instead we will try another technique (which I somewhat stole from the great keras blog): We will freeze all weights of the downloaded network and add three final layers at the end of the network and then train those.

Freezing the base model

Why is this useful you might ask: Well by doing so we can freeze all of the existing layers of the resnet50 network and just train the final layer. This makes sense, since the imagenet task is about recognizing everyday objects from everyday photographs, and it is already very good at recognising “basic” features such as legs, eyes, circles, heads, etc… All of this “smartness” is already encoded in the weights (see the last blog post). If we throw these weights away we will lose these nice smart properties. But instead we can just glue another pooling layer and a dense layer at the very end of it, followed by a sigmoid activation layer, that's needed to distinguish between our two classes. That's by the way why it says “include_top=False” in the code, in order to not include the initial 1000 classes layer, that was used for the imagenet competition. Btw. If you want to read up on the different alternatives to the resnet50 you will find them here.

# Adding three layers on top of the network
base_model = ResNet50(weights='imagenet', include_top=False)
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(1024, activation='relu')(x)
predictions = Dense(1, activation='sigmoid')(x)

Finally we can now re-train the network with our own image material and hope for it to turn out to be quite useful. I’ve had some trouble finding the right optimizer that had proper results. Usually you will have to experiment with the right learning rate to find a configuration that has an improving accuracy in the training phase.

#freezing all the original weights and compiling the network
from keras import optimizers
optimizer = optimizers.RMSprop(lr=0.00001, rho=0.9, epsilon=None, decay=0.0)
model = Model(inputs=base_model.input, outputs=predictions)
for layer in base_model.layers: layer.trainable = False
model.compile(optimizer=optimizer, loss='binary_crossentropy', metrics=['accuracy'])
model.fit_generator(train_generator, train_generator.n // batch_size, epochs=3, workers=4,
        validation_data=validation_generator, validation_steps=validation_generator.n // batch_size)

The training shouldn’t take long, even when you are using just a CPU instead of a GPU and the output might look something like this:

You’ll notice that we reached an accuracy of 71% which isn’t too bad, given that we have only 100 original images of each class.

Fine-tuning

One thing that we might do now is to unfreeze some of the very last layers in the network and re-train the network again, allowing those layers to change slightly. We’ll do this in the hope that allowing for more “wiggle-room”, while changing most of the actual weights, the network might give us better results.

# Make the very last layers trainable
split_at = 140
for layer in model.layers[:split_at]: layer.trainable = False
for layer in model.layers[split_at:]: layer.trainable = True
model.compile(optimizer=optimizers.RMSprop(lr=0.00001, rho=0.9, epsilon=None, decay=0.0), loss='binary_crossentropy', metrics=['accuracy'])    
model.fit_generator(train_generator, train_generator.n // batch_size, epochs=1, workers=3,
        validation_data=validation_generator, validation_steps=validation_generator.n // batch_size)

And indeed it helped our model to go from 71% accuracy to 82%! You might want play around with the learning rates a bit or maybe split it at a different depth, in order to tweak results. But generally I think that just adding more images would be the easiest way to achieve 90% accuracy.

Confusion matrix

In order to see how well our model is doing we might also compute a confusion matrix, thus calculating the true positives, true negatives, and the false positives and false negatives.

# Calculating confusion matrix
from sklearn.metrics import confusion_matrix
r = next(validation_generator)
probs = model.predict(r[0])
classes = []
for prob in probs:
    if prob < 0.5:
        classes.append(0)
    else:
        classes.append(1)
cm = confusion_matrix(classes, r[1])
cm

As you can see above I simply took the first batch from the validation generator (so the images of which we know if its a alpakka or an oryx) and then use the confusion matrix from scikit-learn to output something. So in the example below we see that 28 resp. 27 images of each class were labeled correctly while making an error in 4 resp. 5 images. I would say that’s quite a good result, given that we used only so little data.

#example output of confusion matrix
array([[28,  5],
       [ 4, 27]])

Use the model to predict images

Last but not least we can of course finally use the model to predict if an animal in our little zoo is an oryx or an alpakka.

# Helper function to display images
def load_image(img_path, show=False):

    img = image.load_img(img_path, target_size=(224, 224))
    img_tensor = image.img_to_array(img)                    # (height, width, channels)
    img_tensor = np.expand_dims(img_tensor, axis=0)         # (1, height, width, channels), add a dimension because the model expects this shape: (batch_size, height, width, channels)
    #img_tensor /= 255.                                      # imshow expects values in the range [0, 1]

    if show:
        plt.imshow(img_tensor[0]/255)                           
        plt.axis('off')
        plt.show()

    return img_tensor

# Load two sample images
oryx = load_image("data/valid/oryx/106.jpg", show=True)
alpaca = load_image("data/valid/alpaca/alpaca102.jpg", show=True)
model.predict(alpaka)
model.predict(oryx)

As you can see in the output, our model successfully labeled the alpaca as an alpaca since the value was less than 0.5 and the oryx as an oryx, since the value was > 0.5. Hooray!

Conclusion or What’s next?

I hope that the blog post was useful to you, and showed you that you don’t really need much in order to get started with deep learning for image recognition. I know that our example zoo pokedex is really small at this point, but I don’t see a reason (apart from the lack of time and resources) why it should be a problem to scale out from our 2 animals to 20 or 200.

On the technical side, now that we have a model running that’s kind of useful, it would be great to find out how to use it in on a smartphone e.g. the IPhone, to finally have a pokedex that we can really try out in the wild. I will cover that bit in the third part of the series, showing you how to export existing models to Apple mobile phones making use of the CoreML technology. As always I am looking forward to your comments and corrections and point you to the ipython notebook that you can download here.

Jul 17 2018
Jul 17

In this series of blog posts I will show you how to build a "zoo-pokedex app". That's an app that will tell different animals apart in a zoo. This blog post sells the idea of the app and demystifies the two most important basic concepts behind deep learning for image recognition.

We’ve all witnessed the hype in 2016 when people started hunting pokemons in “real-life” with the app Pokémon GO . It was one of the apps with the fastest rise in user-base and for a while with a higher addiction rate than crack - correction: I mean candycrush. Comparing it to technologies like telephone or email, it only took it 19 days to reach 50 mio users, vs. 75 years for the telephone.

Connecting the real with the digital world

You might be wondering, why I am reminiscing about old apps, we have certainly all moved on since the Pokemon GO hype in 2016 and are doing other serious things now. True, but I think though that the idea of “collecting” virtual things that are bound to real-life locations was a great idea and that we want to build more of it in the future. That’s why Pokemon is the starting point fort this blogpost. In fact If you are young enough to have watched the pokemon series, you are probably familiar with the idea of the pokedex.

The idea

The pokedex was a small device that Ash (the main character) could use to lookup information about certain pokemons in the animated series. He used it now and then to lookup some facts about them. While we have seen how popular the pokemon GO was, by connecting the real with the digital world, why not take the idea of the pokedex and apply it in real world scenarios, or:

What if we had such an app to distinguish not pokemons but animals in the zoo?

The Zoo-Pokedex

Imagine a scenario where kids have an app their parent’s mobile phones - the zoo-pokedex. They start it up when entering a zoo and they then go exploring. When they are at a cage they point the phones camera at the cage and try to film the animal with it. The app recognizes which animal they are seeing and gives them additional information on it as a reward.

Instead of perceiving the zoo as a educational place where you have to go from cage to cage and observe the animal, absorb the info material you could send them out there and let them “capture” all the animals with their Zoo-Pokedex.

Let’s have a look at the classic ways of learning about animals in a zoo:

Reading a plaque in the zoo feels boring and dated
Reading an info booklet you got at the cashier feels even worse
Bringing your own book about animals might be fun, when comparing the pictures of animals in the book with the real ones, but there is no additional information
Having a QR code at the cage that you need to scan, will never feel exciting or fun
Having a list of animals in my app that I can tap on to get more info could be fun, but more for parents in order to appear smart before their kids giving them facts about the animal

Now imagine the zoo-pokedex, you really need to go exploring the zoo in order to get information. In cases where the animals area is big and it can retreat you need to wait in front of it to take a picture of it. That takes endurance and perseverance. It might even be the case that you don’t get to see it and have to come back. When the animal appears in front of you, you’ll need to be quick - maybe even an element of surprise, excitement is there - you need to get that one picture of the animal in order to check of the challenge. Speaking of challenge, why not make it a challenge to have seen every animal in the zoo? That would definitely mean you need to come back multiple times, take your time and go home having ticked off 4-5 animals in your visit. This experience encourages you to come back and try again next time. And each time you learn something, you go home with a sense of accomplishment.

That would definitely be quite interesting, but how could such a device work? Well we would definitively use the phone’s camera and we could train a deep learning network to recognize the animals that are present in the zoo.

So imagine a kid walking up to an area and then trying to spot the animal in order to point his mobile phone to it and then magically a green check-mark appears next to it. We could display some additional info material like where they are originally from, what they eat, when they sleep etc.., but definitely those infos would feel much more entertaining than just reading them off a boring info plaque.

How train the Pokedex to distinguish new animals

Well nice idea you say, but how am I going to make that magical device that will recognize animals, especially the “weird” ones e.g. the oryx in the title :) . The answer is …. of course …. deep learning.

In recent years you have probably noticed the rise of deep learning in different areas of machine learning and noticed their practical applications in your everyday life. In fact I have covered a couple of these practical applications such as state of the art sentiment detection or survival rates for structured data or automatic speech recognition and text to speech applications in our blog.

Deep learning image categorization task

The area we need for our little zoo-pokedex is image categorization. Image categorization tasks have advanced tremendously in the last years, due to deep learning outperforming all other machine learning approaches (see below). One good indicator of this movement is the yearly imagenet competition, which is about letting machine learning algorithms compete about the best way of finding out what can be seen on an image. The task is simple: there are 1000 categories of everyday objects such as cats, elephants, tea-cattles and millions of images that need to be mapped to one of these categories. The algorithm that makes the lowest error wins. Below is an example of the output on the sample images. You’ll notice that the algorithm displays the label of which it thinks the image belongs to.

Now this ILSVRC competition has been going on for a couple of years now and while the improvements that have been made have been astonishing each year, in the last 5 years especially in 2012 and 2013 deep learning appeared with a big bang on the horizon. As you can see on the image below the amount of state of the art solutions exploded and outperformed all other solutions in this area. It even goes so far that the ability of the algorithm to tell the contents apart is better than this of a competing human group. This super-human ability of deep learning networks in these areas is what the hype is all about.

How does it work?

In this blog post I don’t want to be technical but just show you how two easy concepts of convolution (kernels) and pooling are applied in a smart way to really achieve outstanding results in image recognition tasks with deep learning. I don’t want to go into details how deep learning works in the way of how it learns in the form of updating of weights, backpropagation but abstract all of this stuff away from you. In fact if you have 20 minutes and are a visual learner I definitely recommend that video below that does an extremely good job at explaining the concepts behind it:

[embedded content]

Instead I will quickly cover the two basic tricks that are used to make things really work.

We’ll start by looking at a common representation of a deep learning network above and you’ll notice that two words appear a lot there, namely convolution and pooling. While it seems obvious that the image data has to travel through these layers from left to right, it would be cool if we only knew what these layers do.

Convolutions and Kernels

If you are not a native speaker you’ve probably have never heard of the word convolution before and might be quite puzzled when you hear it. For me it also sounded like some magic procedure that apparently does something very complicated and apparently makes the deep learning work :).

After getting into the field I realized that it's basically its an image transformation that is almost 20 years old (e.g. Computer Vision. From Prentice Hall book by Shapiro) and present in your everyday image editing software. Things like sharpening an image or blurring it, or finding edges are basically a convolution. It's a process of applying a small e.g. 3x3 Matrix over each pixel of your image and multiply this value with the neighbouring pixels and then collect the results of that manipulation in a new image.

To make this concept more understandable I stole some examples of how a 3x3 matrix, also called a kernel, transforms an image after being applied to every pixel in your image.

In the image below the kernel gives you the top-edges in your image. The numbers in the grey boxes represent the gray image values (from 0 black to 255 white) and the little numbers after the X represent how these numbers are multiplied when added together. If you change these numbers you get another transformation.

Here is another set of numbers in the 3x3 matrix that will blur your image.

Now normally the way of create such “filters” is to hand-tune these numbers by hand to achieve the desired results. With some logical thinking you can easily come up with filters that sharpen or blur an image and then apply those to the image. But how are these applied in the context of deep learning?

With deep learning we do things the other way round, we teach the neural network to find filters that are somewhat useful in regards to the final result. So for example to tell a zebra apart from an elephant it would really be useful if we had a filter that detects diagonal edges. And if the image has diagonal edges e.g. the stripes of the zebra, it's probably not an elephant. So we train the network on our training images of zebras and elephants and let it learn these filters or kernels on its own. If the emerging kernels are helpful with the task they have a tendency to stay, if not, they keep on updating themselves until they become useful.

So one layer that applies such filters or kernels or convolutions is called a convolutional layer. And now comes another cool property. If you keep on stacking such layers on top of each other, each of these layers will find own filters that are helpful. And on top of that each of these filters will become more and more complicated and be able to detect more detailed features.

In the image above (which is from a seminal paper, you see gray boxes and images. A great way to show these filters is to show the activations or convolutions which are these gray boxes. The images are samples that “trigger” these filters the most. Or said the other way round, these are images that these filters detect well.

So for example in the first layer you’ll notice that the network detects mostly vertical, horizontal and diagonal edges. In the second layer its already a bit “smarter” and is able to detect round things, e.g. eyes or corners of frames etc.. In the third layer its already a bit smarter and is able to detect not only round things but things that look like car tires for example. This layering often goes on and on for many layers. Some networks have over 200 of these layers. That's why they are called deep. Now you know. So usually adding more and more of these layers makes the network better at detecting things but also it makes it slower and sometimes less able to generalize for things it had not seen yet.

Pooling

The second word that you might see a lot in those architecture above is the word pooling. Here the trick is really simple: You look at a couple of pixels next to each other e.g. 2x2 and simply take the biggest value - also called max-pooling. In the image below this trick has been applied for each colored 2x2 area and the output is a much smaller image. Now why are we doing this?

The answer is simple, in order to be size invariant. We try to scale the image down and up multiple times in order to be able to detect a zebra that is really close to the camera vs. one that might only be viewable in the far distance.

Putting things together

After the small excursion into the two main principles of inner workings of state of the art deep learning networks we have to ask the question of how we are going to use these tricks to detect our animals in the zoo.

While a few years ago you would have had to write a lot of code and hire a whole machine team to do this task, today you can already stand on the shoulders of giants. Thanks to the Imagenet competitions (and I guess thanks to Google, Microsoft and other research teams constantly outputting new research) we can use some of these pretrained networks to do our job for us. What does this mean?

The networks that are often used in these competitions can be obtained freely (In fact they even come https://github.com/pytorch/pytorch pre-bundled to the deep-learning frameworks) and you can use networks these without any tuning in order to be able to categorize your image into the 1000 categories that are used in the competition. As you can see in the image below the bigger in terms of layers the network the better it performs, but also the slower it is and the more data it needs to be trained.

Outlook - Part 2 How to train state of the art image recognition networks to categorize new material

The cool thing now is that in the next blog post we will use these pretrained networks and teach them new tricks. In our case teach them to tell apart a llama from an oryx, for our zoo pokedex. So basically train these network to recognize things these networks have never been trained to do. So obviously we will need training data and we have to find a way to somehow teach them new stuff without “destroying” their properties of being really good at detecting common things.

Finally after that blog post I hope to leave you with at least one the takeaway of demystifying deep learning networks in the image recognition domain. So hopefully whenever you see these weird architecture drawings of image recognition deep learning networks and you see those steps saying “convolution” and “pooling” you’ll hopefully know that this magic sauce is not that magic after all. It’s just a very smart way of applying those very old techniques to achieve outstanding results.

Jun 11 2018
Jun 11

Learn how to combine automatic speech recognition (ASR) with text to speech solutions (TTS) in a simple hands free recipe assistant prototype that we've build in an innoday at Liip. Part three of three provides the code and shows how we put everything together into a small flask socket.io prototype.

Welcome to part three of three in our mini blog post series on how to build a recipe assistant with automatic speech recognition and text to speech to deliver a hands free cooking experience. In the first blog post we gave you a hands on market overview of existing Saas and opensource TTS solutions, in the second post we have put the user in the center by covering the usability aspects of dialog driven apps and how to create a good conversation flow. Finally it's time to get our hands dirty and show you some code.

Prototyping with Socket.IO

Although we envisioned the final app to be a mobile app and run on a phone it was much faster for us to build a small Socket.io web application, that is basically mimicking how an app might work on the mobile. Although socket.io is not the newest tool in the shed, it was great fun to work with it because it was really easy to set up. All you needed is a js library on the HTML side and tell it to connect to the server, which in our case is a simple python flask micro-webserver app.

#socket IO integration in the html webpage
...
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/2.1.0/socket.io.js"></script>
</head>
<body>
<script>
$(document).ready(function(){
    var socket = io.connect('http://' + document.domain + ':' + location.port);
    socket.on('connect', function() {
        console.log("Connected recipe");
        socket.emit('start');
    });
    ...

The code above connects to our flask server and emits the start message, signaling that our audio service can start reading the first step. Depending on different messages we can quickly alter the DOM or do other things in almost real time, which is very handy.

To make it work on the server side in the flask app all you need is a python library that you integrate in your application and you are ready to go:

# socket.io in flask
from flask_socketio import SocketIO, emit
socketio = SocketIO(app)

...

#listen to messages 
@socketio.on('start')
def start_thread():
    global thread
    if not thread.isAlive():
        print("Starting Thread")
        thread = AudioThread()
        thread.start()

...

#emit some messages
socketio.emit('ingredients', {"ingredients": "xyz"})

In the code excerpt above we start a thread that will be responsible for handling our audio processing. It starts when the web server receives the start message from the client, signalling that he is ready to lead a conversation with the user.

Automatic speech recognition and state machines

The main part of the application is simply a while loop in the thread that listens to what the user has to say. Whenever we change the state of our application, it displays the next recipe state and reads it out loudly. We’ve sketched out the flow of the states in the diagram below. This time it is really a simple mainly linear conversation flow, with the only difference, that we sometimes branch off, to remind the user to preheat the oven, or take things out of the oven. This way we can potentially save the user time or at least offer some sort of convenience, that he doesn’t get in a “classic” recipe on paper.

The automatic speech recognion (see below) works with wit.ai in the same manner like I have shown in my recent blog post. Have a look there to read up on the technology behind it and find out how the RecognizeSpeech class works. In a nutshell we are recording 2 seconds of audio locally and then sending it over a REST API to Wit.ai and waiting for it to turn it into text. While this is convenient from a developer’s side - not having to write a lot of code and be able to use a service - the downside is the reduced usability for the user. It introduces roughly 1-2 seconds of lag, that it takes to send the data, process it and receive the results. Ideally I think the ASR should take place on the mobile device itself to introduce as little lag as possible.

#abbreviated main thread

self.states = ["people","ingredients","step1","step2","step3","step4","step5","step6","end"]
while not thread_stop_event.isSet():
    socketio.emit("showmic") # show the microphone symbol in the frontend signalling that the app is listening
    text = recognize.RecognizeSpeech('myspeech.wav', 2) #the speech recognition is hidden here :)
    socketio.emit("hidemic") # hide the mic, signaling that we are processing the request

    if self.state == "people":
        ...
        if intro_not_played:
            self.play(recipe["about"])
            self.play(recipe["persons"])
            intro_not_played = False
        persons = re.findall(r"\d+", text)
        if len(persons) != 0:
            self.state = self.states[self.states.index(self.state)+1]
        ...
    if self.state == "ingredients"
        ...
        if intro_not_played:
            self.play(recipe["ingredients"])
            intro_not_played = False
        ...
        if "weiter" in text:
            self.state = self.states[self.states.index(self.state)+1]
        elif "zurück" in text:
            self.state = self.states[self.states.index(self.state)-1]
        elif "wiederholen" in text:
            intro_not_played = True #repeat the loop
        ...

As we see above, depending on the state that we are in, we play the right audio TTS to the user and then progress into the next state. Each step also listens if the user wanted to go forward (weiter), backward (zurück) or repeat the step (wiederholen), because he might have misheard.

The first prototype solution, that I am showing above, is not perfect though, as we are not using a wake-up word. Instead we are offering the user periodically a chance to give us his input. The main drawback is that when the user speaks when it is not expected from him, we might not record it, and in consequence be unable to react to his inputs. Additionally sending audio back and forth in the cloud, creates a rather sluggish experience. I would be much happier to have the ASR part on the client directly especially when we are only listening to mainly 3-4 navigational words.

TTS with Slowsoft

Finally you have noticed above that there is a play method in the code above. That's where the TTS is hidden. As you see below we first show the speaker symbol in the application, signalling that now is the time to listen. We then send the text to Slowsoft via their API and in our case define the dialect "CHE-gr" and the speed and pitch of the output.

#play function
    def play(self,text):
        socketio.emit('showspeaker')
        headers = {'Accept': 'audio/wav','Content-Type': 'application/json', "auth": "xxxxxx"}
        with open("response.wav", "wb") as f: 
            resp = requests.post('https://slang.slowsoft.ch/webslang/tts', headers = headers, data = json.dumps({"text":text,"voiceorlang":"gsw-CHE-gr","speed":100,"pitch":100}))
            f.write(resp.content)
            os.system("mplayer response.wav")

The text snippets are simply parts of the recipe. I tried to cut them into digestible parts, where each part contains roughly one action. Here having an already structured recipe in the open recipe format helps a lot, because we don't need to do any manual processing before sending the data.

Wakeup-word

We took our prototype for a spin and realized in our experiments that it is a must to have a wake-up. We simply couldn’t time the input correctly to enter it when the app was listening, this was a big pain for user experience.

I know that nowadays smart speakers like alexa or google home provide their own wakeup word, but we wanted to have our own. Is that even possible? Well, you have different options here. You could train a deep network from scratch with tensorflow-lite or create your own model by following along this tutorial on how to create a simple speech recognition with tensorflow. Yet the main drawback is that you might need a lot (and I mean A LOT as in 65 thousand samples) of audio samples. That is not really applicable for most users.

Luckily you can also take an existing deep network and train it to understand YOUR wakeup words. That means that it will not generalize as well to other persons, but maybe that is not that much of a problem. You might as well think of it as a feature, saying, that your assistant only listens to you and not your kids :). A solution of this form exists under the name snowboy, where a couple of ex-Googlers created a startup that lets you create your own wakeup words, and then download those models. That is exactly what I did for this prototype. All you need to do is to go on the snowboy website and provide three samples of your wakeup-word. It then computes a model that you can download. You can also use their REST API to do that, the idea here is that you can include this phase directly in your application making it very convenient for a user to set up his own wakeup- word.


#wakeup class 

import snowboydecoder
import sys
import signal

class Wakeup():
    def __init__(self):
        self.detector = snowboydecoder.HotwordDetector("betty.pmdl", sensitivity=0.5)
        self.interrupted = False
        self.wakeup()

    def signal_handler(signal, frame):
        self.interrupted = True

    def interrupt_callback(self):
        return self.interrupted

    def custom_callback(self):
        self.interrupted = True
        self.detector.terminate()
        return True

    def wakeup(self):
        self.interrupted = False
        self.detector.start(detected_callback=self.custom_callback, interrupt_check=self.interrupt_callback,sleep_time=0.03)
        return self.interrupted

```

All it needs then is to create a wakeup class that you might run from any other app that you include it in. In the code above you’ll notice that we included our downloaded model there (“betty.pmdl”) and the rest of the methods are there to interrupt the wakeup method once we hear the wakeup word.

We then included this class in your main application as a blocking call, meaning that whenever we hit the part where we are supposed to listen to the wakeup word, we will remain there unless we hear the word:

```python
#integration into main app
...
            #record
            socketio.emit("showear")
            wakeup.Wakeup()
            socketio.emit("showmic")
            text = recognize.RecognizeSpeech('myspeech.wav', 2)
…
```
So you noticed in the code above that we changed included the *wakeup.Wakeup()* call that now waits until the user has spoken the word, and only after that we then record 2 seconds of audio to send it to processing with wit.ai. In our testing that improved the user experience tremendously. You also see that we signall the listening to the user via graphical clues, by showing a little ear, when the app is listening for the wakeup word, and then showing a microphone when the app is ready is listening to your commands. 

### Demo

So finally time to show you the Tech-Demo. It gives you an idea how such an app might work and also hopefully gives you a starting point for new ideas and other improvements. While it's definitely not perfect it does its job and allows me to cook handsfree :). Mission accomplished! 

<figure class="embed-responsive embed-responsive--16/9"><iframe allowfullscreen src="https://player.vimeo.com/video/270594859"></iframe></figure>

## What's next?

In the first part of this blog post series we have seen quite an <a href="https://www.liip.ch/en/blog/betti-bossi-recipe-assistant-prototype-with-automatic-speech-recognition-asr-and-text-to-speech-tts-on-socket-io">extensive overview</a> over the current capabilities of TTS systems. While we have seen an abundance of options on the commercial side, sadly we didn’t find the same amount of sophisticated projects on the open source side. I hope this imbalance catches up in the future especially with the strong IoT movement, and the need to have these kind of technologies as an underlying stack for all kinds of smart assistant projects. Here is an <a href="https://www.kickstarter.com/projects/seeed/respeaker-an-open-modular-voice-interface-to-hack?lang=de">example</a> of a Kickstarter project for a small speaker with built in open source ASR and TTS.

In the <a href="https://www.liip.ch/en/blog/recipe-assistant-prototype-with-asr-and-tts-on-socket-io-part-2-ux-workshop">second blog post</a>, we discussed the user experience of audio centered assistants. We realized that going audio-only, might not always provide the best user experience, especially when the user is presented with a number of alternatives that he has to choise from. This was especially the case in the exploration phase, where you have to select a recipe and in the cooking phase where the user needs to go through the list of ingredients.  Given that the <a href="https://www.amazon.de/Amazon-Echo-2nd-Generation-Anthrazit-Stoff-/dp/B06ZXQV6P8">Alexas</a>, <a href="https://www.apple.com/homepod/">Homepods</a> and the <a href="https://www.digitec.ch/de/s1/product/google-home-weiss-grau-multiroom-system-6421169">Google Home</a> smart boxes are on their way to take over the audio-based home assistant area, I think that their usage will only make sense in a number of very simple to navigate domains, as in “Alexa play me something from Jamiroquai”. In more difficult domains, such as cooking, mobile phones might be an interesting alternative, especially since they are much more portable (they are mobile after all), offer a screen and almost every person already has one. 

Finally in the last part of the series I have shown you how to integrate a number of solutions together - wit.ai for ASR, slowsoft for TTS, snowboy for wakeupword and socket.io and flask for prototyping - to create a nice working prototype of a hands free cooking assistant. I have uploaded the code on github, so feel free to play around with it to sketch your own ideas. For us a next step could be taking the prototype to the next level, by really building it as an app for the Iphone or Android system, and especially improve on the speed of the ASR. Here we might use the existing <a href="https://developer.apple.com/machine-learning/">coreML</a> or <a href="https://www.tensorflow.org/mobile/tflite/">tensorflow light</a> frameworks or check how well we could already use the inbuilt ASR capabilities of the devices. As a final key take away we realized that building a hands free recipe assistant definitely is something different, than simply having the mobile phone read out the recipe out loud for you. 

As always I am looking forward to your comments and insights and hope to update you on our little project soon.
Jun 03 2018
Jun 03

Learn how to combine automatic speech recognition (ASR) with text to speech solutions (TTS) in a simple hands free recipe assistant prototype that we've build in an innoday at Liip. Part two of three provides insights from our small ideation and UX workshop, that paved the way for the prototype.

Welcome to part two of three in our mini blog post series on how to build a recipe assistant with automatic speech recognition and text to speech to deliver a hands free cooking experience. In the last blog post we provided you with an exhaustive hands on text to speech (TTS) market review, now its time to put the user in the center.

Workshop: Designing a user experience without a screen

Although the screen used to dominate the digital world, thanks to the rapid improvement of technologies, there are more options emerging. Most of mobile users have used or heard Siri from Apple iOS and Amazon Echo and almost 60 Mio Americans apparently already own a smart speaker. Until recently sill unheard of, nowadays smart voice based assistants are changing our life quickly. This means that user experience has to think beyond screen based interfaces. Actually it has always defined a holistic experience in a context where the user is involved, and also in speech recognition and speech as main input source, UX is needed to prevent potential usability issues in its interaction.

Yuri participated in our innoday workshop as an UX designer, where her goal was to help the team to define a recipe assistant with ASR and TTS, that help the user to cook recipes in the kitchen without using his hands, and is a enjoyable to use. In this blog post Yuri helped me to write down our UX workshop steps.

Ideation

We started off with a brainstorming of our long term vision and short term vision and then wrote down down our ideas and thoughts on post its. We then grouped the ideas into three organically emerging topics, which were Business, Technology and User needs. I took the liberty to highlight some of the aspects that came to our minds:

  • User
    • Privacy: Users might not want to have their voice samples saved on some google server. Click here to listen to all your samples, if you have an Android phone.
    • Alexa vs. Mobile or is audio only enough?: We spent a lot of discussion thinking if a cookbook could work in an audio only mode. We were aware that there is for example an Alexa Skill from Chefkoch, but somehow the low rating made us suspicious if the user might need some minimal visual orientation. An app might be able to show you the ingredients or some visual clues on what to do in a certain step and who doesn't like these delicious pictures in recipes that lure you in to give a recipe a try?
    • Conversational Flow: An interesting aspect, that is easy to overlook was how to design the conversational flow in order to allow the user enough flexibility when going through each step of recipe but also not being to rigid.
    • Wakeup Word: The so called wakeup word is a crucial part of every ASR system, which triggers the start of the recording. I've written about it in a recent blog post.
    • Assistant Mode: Working with audio also gives interesting opportunities for features that are rather unusual on normal apps. We thought of a spoken audio alert, when the app notifies you to take the food from the oven. Something that might feel very helpful, or very annoying, depending on how it is solved.
  • Technology
    • Structured Data: Interestingly we soon realized that breaking down a cooking process means that we need to structure our data better than a simple text. An example is simply multiplying the ingredients by the amount of people. An interesting project in this area is the open recipe format that simply defines a YAML to hold all the necessary data in a structured way.
    • Lag and Usability: Combining TTS with ASR poses an interesting opportunity to combine different solutions in one product, but also poses the problem of time lags when two different cloud based systems have to work together.
  • Business
    • Tech and Cooking: Maybe a silly idea, but we definitely thought that as men it would feel much cooler to use a tech gadget to cook the meal, instead of a boring cookbook.

User journey

From there we took on the question: “How might we design an assistant that allows for cooking without looking at recipe on the screen several times, since the users’ hands and eyes are busy with cooking.”

We sketched the user journey as a full spectrum of activities that go beyond just cooking, and can be described as:

  • Awareness of the recipes and its interface on App or Web
  • Shopping ingredients according to selected recipe
  • Cooking
  • Eating
  • After eating

Due to the limited time of an inno-day, we decided to focus on the cooking phase only, while acknowledging that the this phase is definitely part of a much bigger user journey, where some parts, such as exploration, might be hard to tackle with an audio-only assistant. We tried though to explore the cooking step of the journey and broke it down into its own sub-steps. For example:

  • Cooking
    • Preparation
    • Select intended Recipe to cook
    • Select number of portions to cook
    • Check ingredients if the user has them all ready
  • Progress
    • Prepare ingredients
    • The actual cooking (boiling, baking, etc)
    • Seasoning and garnishing
    • Setting on a table

This meant for our cooking assistant that he needs to inform the user when to start each new sub-step and introduce the next steps in a easy unobtrusive way. He has also to track the multiple starts and stops from small actions during cooking, to for example remind the user to preheat the baking oven at an early point in time, when the user might not think of that future step yet (see below)

User experience with a screen vs. no screen

Although we were first keen on building an audio only interface, we found that a quick visual overview helps to make the process faster and easier. For example, an overview of ingredients can be viewed at a glance on the mobile screen without listening every single ingredient from the app. As a result we decided that a combination of a minimal screen output and voice output will ease out potential usability problems.

Since the user needs to navigate with his voice easy input options like “back”, “stop”, “forward”, “repeat” we decided to also show the step that the user is currently in the screen. This feedback helps the user to solve small errors or just orient himself more easily.

During the ux-prototyping phase, we also realised that we should visually highlight the moments when the user is expected to speak and when he is expected to listen. That's why immediately after a question from the app, we would like to show an icon with a microphone meaning “Please tell me your answer!”. In a similar way we also want to show an audio icon when we want the user to listen carefully. Finally since we didn’t want the assistant to permanently listen to audio, but listen to a so called “wake-up-word”, we show a little ear-icon, signalling that the assistant is now listening for this wake-up-word.

While those micro interactions and visual cues, helped us to streamline the user experience, we still think that these are definitely areas that are central to a user experience and should be improved in a next iteration.

Conclusion and what's next

I enjoyed that instead of starting to write code right away, we first sat together and started to sketch out the concept, by writing sticky notes, with ideas and comments that came to our mind. I enjoyed having a mixed group where we had UX people, Developers, Data Scientists and Project owners sitting at one table. Although our ambitious goal for the day was to deliver a prototype that was able to read recipes to the user we ran out of time and I couldn’t code the prototype on that day, but in exchange I think we gathered very valuable insights on a user experiences that work and that don’t work without a screen. We realized that going totally without a screen is much harder than it seems. It is crucial for the user experience that the user has enough orientation to know where he is in the process in order for him not to feel lost or confused.

In the final and third blog post of this mini series I will finally provide you with the details on how to write a simple flask and socket.io based prototype that combines automatic speech recognition, text to speech and wake-up-word detection to create a hands-free cooking experience.

May 28 2018
May 28

This article is aiming at giving an introduction to CKAN. What kind of projects we use it for and which Plugins we use to implement the features we need for our customers.

CKAN's Main Goal and Key Features

CKAN is an open source management system whose main goal is to provide a managed data-catalog-system for Open Data. It is mainly used by public institutions and governments. At Liip we use CKAN to mainly help governments to provide their data-catalog and publish data in an accessible fashion to the public. Part of our work is supporting data owners to get their data published in the required data-format. We’re doing this by providing interfaces and useable standards to enhance the user experience on the portal to make it easier to access, read and process the data.

bookcase

Metadata-Catalog

Out of the box CKAN can be used to publish and manage different types of datasets. They can be clustered by organizations and topics. Each dataset can contain resources which themself consist of Files of different formats or links to other Data-Sources. The metadata-standard can be configured to represent the standard you need but the Plugin already includes a simple and useful Meta-Data-Standard that already can get you started. The data is saved into a Postgres-Database by default and is indexed using SOLR.

Powerful Action-API

CKAN ships with an API which can be used to browse through the metadata-catalog and create advanced queries on the metadata. With authorization the API can also be used to add, import and update data with straight-forward requests.

Cli-Commands

The standard also includes a range of Cli-Commands which can be used to process or execute different tasks. Those can be very useful, e.g. to manage, automate or schedule backend-jobs.

Preview

CKAN offers the functionality to configure a preview of a number of different file-types, such as tabular-data (e.g. CSV, XLS), Text-Data (e.g. TXT), Images or PDFs. That way interested citizens can get a quick overview into the data itself without having to download it first and having to use local Software to merely get an better idea on how the data looks.

Preview von Daten auf Statistik Stadt Zürich

Plugins

While CKAN itself acts as a CMS but for data, it really shines when making use of its extensibility and configure and develop it to your business needs and requirements. There is already a wide-ranging list of plugins that have been developed for CKAN, which covers a broad range of additional features or make it easier to adjust CKAN to fit your use cases and look and feel. A collection of most of the plugins can be found on CKAN-Extensions and on Github.

At Liip we also help maintaining a couple of CKAN's plugins. The most important ones that we use in production for our customers are:

ckanext-harvest

The ckanext-harvest-plugin offers the possibility to export and import data. First of all, it enables you to exchange data between Portals that both use CKAN.

Furthermore we use this plugin to harvest data in a regular manner from different data-sources. At opendata.swiss we use two different types of harvesters. Our DCAT-Harvester consumes XML-/RDF-endpoints in DCAT-AP Switzerland-Format which is enforced on the Swiss Portal.

The Geocat-Harvester consumes data from geocat.ch. As the data from geocat is in ISO-19139_che-Format (Swiss version of ISO-19139) the harvester converts the data to the DCAT-AP Switzerland format and imports it.

Another feature of this plugin we use, is our DCAT-AP endpoint, to allow other portals to harvest our data and also serves as an example to Organizations that want to build an export that can be harvested by us.

How our Harvesters interact with the different Portals

ckanext-datastore

The plugin ckanext-datastore stores the actual tabular data (opposing to 'just' the meta-data) in a seperate database. With it, we are able to offer an easy to use API on top of the CKAN-Standard-API to query the data and process it further. It provides basic functionalities on the resource-detail-page to display the data in simple graphs.

The datastore is the most interesting one for Data-Analysts, who want to build apps based on the data, or analyze the data on a deeper level. This is an API-example of the Freibäder-dataset on the portal of Statistik Stadt Zürich.

ckanext-showcase

We use ckanext-showcase to provide a platform for Data-Analysts by displaying what has been built, based on the data the portal is offering. There you can find a good overview on how the data can be viewed in meaningful ways as statistics or used as sources in narrated videos or even in apps for an easier everyday life. For example you can browse through the Showcases on the Portal of the City of Zurich.

ckanext-xloader

The ckanext-xloader is a fairly new plugin which we were able to adopt for the City of Zurich Portal. It enables us to automatically and asynchronously load data into the datastore to have the data available after it has been harvested.

CKAN Community

The CKAN-Core and also a number of its major plugins are maintained by the CKAN-Core-Team. The developers are spread around the globe, working partly in companies that run their own open-data portals. The community that contribute to CKAN and its Plugins is always open to developers that would like to help with suggestions, report issues or provide Pull-Requests on Github. It offers a strong community which helps beginners, no matter their background. The ckan-dev-Mailing-List provides help in developing CKAN and is the platform for discussions and ideas about CKAN, too.

Roadmap and most recent Features

Since the Major-Release 2.7 CKAN requires Redis to use a new system of asynchronous background jobs. This helps CKAN to be more performant and reliable. Just a few weeks ago the new Major-Release 2.8 was released. A lot of work on this release went into driving CKAN forward by updating to a newer Version of Bootstrap and also deprecating old features that were holding back CKAN's progress.

Another rather new feature is the datatables-feature for tabular data. Its intention is to help the data-owner to describe the actual data in more detail by describing the values and how they gathered or calculated.

In the Roadmap of CKAN are many interesting features ahead. One example is the development of the CKAN Data Explorer which is a base component of CKAN. It allows to converge data from any dataset in the DataStore of a CKAN instance to analyze it.

Conclusion

It is important to us to support the Open Data Movement as we see value in publishing governmental data to the public. CKAN helps us to support this cause by working with several Organizations to publish their data and consult our customers while we develop and improve their portals together.

Personally, I am happy to be a part of the CKAN-Community which has always been very helpful and supportive. The cause to help different Organizations to make their data public to the people and the respectful CKAN-Community make it a lot of fun to contribute to the code and also the community.

Open Data auf opendata.swiss
May 27 2018
May 27

Learn how to combine automatic speech recognition (ASR) with text to speech solutions (TTS) in a simple hands free recipe assistant prototype that we've build in an innoday at Liip. Part one of two, provides an exhaustive overview of commercial and open source TTS solutions.

Intro

In one of our monthly innodays, where we try out new technologies and different approaches to old problems, we had the idea to collaborate with another company. Slowsoft is a provider of text to speech (TTS) solutions. To my knowledge they are the only ones who are able to generate Swiss German speech synthesis in various Swiss accents. We thought it would be a cool idea to combine it with our existing automatic speech recognition (ASR) expertise and build a cooking assistant that you can operate completely hands free. So no more touching your phone with your dirty fingers only to check again how many eggs you need for that cake. We decided that it would be great to go with some recipes from a famous swiss cookbook provider.

Overview

Generally there are quite a few text to speech solutions out there on the market. In the first out of two blog posts would like to give you a short overview of the available options. In the second blog post I will then describe at which insights we arrived in the UX workshop and how we then combined wit.ai with the solution from slowsoft in a quick and dirty web-app prototype built on socket.io and flask.

But first let us get an overview over existing text to speech (TTS) solutions. To showcase the performance of existing SaaS solutions I've chosen a random recipe from Betty Bossi and had it read by them:

Ofen auf 220 Grad vorheizen. Broccoli mit dem Strunk in ca. 1 1/2 cm dicke Scheiben schneiden, auf einem mit Backpapier belegten Blech verteilen. Öl darüberträufeln, salzen.
Backen: ca. 15 Min. in der Mitte des Ofens.
Essig, Öl und Dattelsirup verrühren, Schnittlauch grob schneiden, beigeben, Vinaigrette würzen.
Broccoli aus dem Ofen nehmen. Einige Chips mit den Edamame auf dem Broccoli verteilen. Vinaigrette darüberträufeln. Restliche Chips dazu servieren. 

But first: How does TTS work?

The classical way works like this: You have to record at least dozens of hours of raw speaker material in a professional studio. Depending on the task, the material can range from navigation instructions to jokes, depending on your use case. The next trick is called "unit-selection", where recorded speech is sliced into a high number (10k - 500k) of elementary components called phones, in order to be able to recombine those into new words, that the speaker has never recorded. The recombination of these components is not an easy task because the characteristics depend on the neighboring phonemes and the accentuation or prosody. These depend on a lot on the context. The problem is to find the right combination of these units that satisfy the input text and the accentuation and which can be joined together without generating glitches. The raw input text is first translated into a phonetic transcription which then serves as the input to selecting the right units from the database that are then concatenated into a waveform. Below is a great example from Apple's Siri engineering team showing how the slicing takes place.

Using an algorithm called Viterbi the units are then concatenated in such a way that they create the lowest "cost", in cost resulting from selecting the right unit and concatenating two units together. Below is a great conceptual graphic from Apple's engineering blog showing this cost estimation.

Now in contrast to the classical way of TTS new methods based on deep learning have emerged. Here deep learning networks are used to predict the unit selection. If you are interested how the new systems work in detail, I highly recommend the engineering blog entry describing how Apple crated the Siri voice. As a final note I'd like to add that there is also a format called speech synthetisis markup language, that allows users to manually specify the prosody for TTS systems, this can be used for example to put an emphasis on certain words, which is quite handy. So enough with the boring theory, let's have a look at the available solutions.

SaaS / Commercial

Google TTS

When thinking about SaaS solutions, the first thing that comes to mind these days, is obviously Google's TTS solution which they used to showcase Google's virtual assistant capabilities on this years Google IO conference. Have a look here if you haven't been wowed today yet. When you go to their website I highly encourage you to try out their demo with a German text of your choice. It really works well - the only downside for us was that it's not really Swiss German. I doubt that they will offer it for such a small user group - but who knows. I've taken a recipe and had it read by Google and frankly liked the output.

[embedded content]

Azure Cognitive Services

Microsoft also offers TTS as part of their Azure cognitive services (ASR, Intent detection, TTS). Similar to Google, having ASR and TTS from one provider, definitely has the benefit of saving us one roundtrip since normally you would need to perform the following trips:

  1. Send audio data from client to server,
  2. Get response to client (dispatch the message on the client)
  3. Send our text to be transformed to speech (TTS) from client to server
  4. Get the response on client. Play it to the user.

Having ASR and TTS in one place reduces it to:

  1. ASR From client to server. Process it on the server.
  2. TTS response to client. Play it to the user.

Judging the speech synthesis quality, I personally I think that Microsoft's solution didn't sound as great as Googles synthesis. But have a look for yourself.

[embedded content]

Amazon Polly

Amazon - having placed their bets on Alexa - of course has a sophisticated TTS solution, which they call Polly. I love the name :). To be where they are now, they have acquired a startup called Ivona already back in 2013, which were back then producing state of the art TTS solutions. Having tried it I liked the soft tone and the fluency of the results. Have a check yourself:

[embedded content]

Apple Siri

Apple offers TTS as part of their iOS SDK in the name of SikiKit. I haven’t had the chance yet to play in depth with it. Wanting to try it out I made the error to think that apples TTS solution on the Desktop is the same as SiriKit. Yet SiriKit is nothing like the built in TTS on the MacOS. To have a bit of a laugh on your Macbook you can do a really poor TTS in the command line you can simply use a command:

say -v fred "Ofen auf 220 Grad vorheizen. Broccoli mit dem Strunk in ca. 1 1/2 cm dicke Scheiben schneiden, auf einem mit Backpapier belegten Blech verteilen. Öl darüberträufeln, salzen.
Backen: ca. 15 Min. in der Mitte des Ofens."

While the output sounds awful, below is the same text read by Siri on the newest iOS 11.3. That shows you how far TTS systems have evolved in the last years. Sorry for the bad quality but somehow it seems impossible to turn off the external microphone when recording on an IPhone.

[embedded content]

IBM Watson

In this arms race IBM also offers a TTS system, with a way to also define the prosody manually, using the SSML markup language standard. I didn't like their output in comparison to the presented alternatives, since it sounded quite artificial in comparison. But give it a try for yourself.

[embedded content]

Other commercial solutions

Finally there are also competitors beyond the obvious ones such as Nuance (formerly Scansoft - originating from Xerox research). Despite their page promising a lot, I found the quality of the TTS in German to be a bit lacking.

[embedded content]

Facebook doesn't offer a TTS solution, yet - maybe they have rather put their bets on Virtual Reality instead. Other notable solutions are Acapella, Innoetics, TomWeber Software, Aristech and Slowsoft for Swiss TTS.

OpenSource

Instead of providing the same kind of overview for the open source area, I think it's easier to list a few projects and provide a sample of the synthesis. Many of these projects are academic in nature, and often don't give you all the bells and whistles and fancy APIs like the commercial products, but with some dedication could definitely work if you put your mind to it.

  • Espeak. sample - My personal favorite.
  • Festival a project from the CMU university, focused on portability. No sample.
  • Mary. From the german "Forschungszentrum für Künstliche Intelligenz" DKFI. sample
  • Mbrola from the University of Mons sample
  • Simple4All - a EU funded Project. sample
  • Mycroft. More of an open source assistant, but runs on the Raspberry Pi.
  • Mimic. Only the TTS from the Mycroft project. No sample available.
  • Mozilla has published over 500 hours of material in their common voice project. Based on this data they offer a deep learning ASR project Deep Speech. Hopefully they will offer TTS based on this data too someday.
  • Char2Wav from the University of Montreal (who btw. maintain the theano library). sample

Overall my feeling is that unfortunately most of the open source systems have not yet caught up with the commercial versions. I can only speculate about the reasons, as it might take a significant amount of good raw audio data to produce comparable results and a lot of fine tuning on the final model for each language. For an elaborate overview of all TTS systems, especially the ones that work in German, I highly recommend to check out the extensive list that Felix Burkhardt from the Technical University of Berlin has compiled.

That sums up the market overview of commercial and open source solutions. Overall I was quite amazed how fluent some of these solutions sounded and think the technology is ready to really change how we interact with computers. Stay tuned for the next blog post where I will explain how we put one of these solutions to use to create a hands free recipe reading assistant.

May 08 2018
May 08

I recently had the honor to be invited on a small conference about the future of web development recently. I want to share my insights with you in this quick review of the talks, including insights on progressive web apps, meteor.js, microsoft azure and the data science stack.

Back to the future

Although the conference (hosted in Zürich last week in the Crown Plaza) had explicitly the word future in the title, I found that often the new trends felt a bit like "back to the future". Why ? Because it seems that some rather old concepts like plain old SQL, "offline first" or pure javascript frameworks or are making a comeback in web development - but with a twist. This already brings us to the first talk.

Modern single page apps with meteor

Timo Horstschaefer from Ledgy showed how to create modern single page apps with meteor.js. Although every framework promises to "ship more with less code", he showed that for their project Ledgy - which is a mobile app to allocate shares among stakeholders - they were able to actually write it in less than 3 months using 13'000 lines of code. In comparison to other web frameworks where there is a backend side, that is written in one language (e.g. ruby - rails, python - django etc..) and a js-heavy frontend framework (e.g. react or angular) meteor does things differently by also offering a tightly coupled frontend and a backend part written purely in js. The backend is mostly a node component. In their case it is really slim, by only having 500 lines of code. It is mainly responsible for data consistency and authentication, while all the other logic simply runs in the client. Such client projects really shine especially when having to deal with shaky Internet connections, because meteor takes care of all the data transmission in the backend, and catches up on the changes once it has regained accessibility. Although meteor seemed to have had a rough patch in the community in 2015 and 2016 it is heading for a strong come back. The framework is highly opinionated, but I personally really liked the high abstraction level, which seemed to allow the team a blazingly fast time to market. A quite favorable development seems to be that Meteor is trying to open up beyond MongoDB as a database by offering their own GraphQL client (Apollo) that even outshines Facebook's own client, and so offers developers freedom on the choice of a database solution.

I highly encourage you to have a look at Timo's presentation.

The data science stack

Then it was my turn to present the data science stack. I won't bother you about the contents of my talk, since I've already blogged about it in detail here. If you still want to have a look at the presentation, you can download it of course. In the talk offered a very subjective birds eyes view on how the data centric perspective touches modern web standards. An interesting feedback from the panel was the question if such an overview really helps our developers to create better solutions. I personally think that having such maps or collections for orientation helps especially people in junior positions to expand their field of view. I think it might also help senior staff to look beyond their comfort zone, and overcome the saying "if everything you have is a hammer, then every problem looks like a nail to you" - so using the same set of tools for every project. Yet I think the biggest benefit might be to offer the client a really unbiased perspective on his options, of which he might have many more than some big vendors are trying to make him believe.

From data science stack to data stack

Meinrad Weiss from Microsoft offered interesting insights into a glimpse of the Azure universe, showing us the many options on how data can be stored in an azure cloud. While some facts were indeed surprising, for example Microsoft being unable to find two data centers that were more than 400 miles apart in Switzerland (apparently the country is too small!) other facts like the majority of clients still operating in the SQL paradigm were less surprising. One thing that really amazed me was their "really big" storage solution so basically everything beyond 40 peta!-bytes: The data is spread into 60! storage blobs that operate independently of the computational resources, which can be scaled for demand on top of the data layer. In comparison to a classical hadoop stack where the computation and the data are baked into one node, here the customer can scale up his computational power temporarily and then scale it down after he has finished his computations, so saving a bit of money. In regards to the bill though such solutions are not cheap - we are talking about roughly 5 digits per month entrance price, so not really the typical KMU scenario. Have a look at the presentation if you want a quick refresher on current options for big data storage at Microsoft Azure. An interesting insight was also that while a lot of different paradigms have emerged in the last years, Microsoft managed to include them all (e.g. Gremlin Graph, Cassandra, MongoDB) in their database services unifying their interfaces in one SQL endpoint.

Offline First or progressive web apps

Nicro Martin, a leading Web and Frontend Developer from the Say Hello agency showcased how the web is coming back to mobile again. Coming back? Yes you heard right. If thought you were doing mobile first for many years now, you are right to ask why it is coming back. As it turns out (according to a recent comscore report from 2017) although people are indeed using their mobile heavily, they are spending 87% of their time inside apps and not browsing the web. Which might be surprising. On the other hand though, while apps seem to dominate the mobile usage, more than 50% of people don't install any new apps on their phone, simply because they are happy with the ones the have. Actually they spend 80% of their time in the top 3 apps. That poses a really difficult problem for new apps - how can they get their foot into the door with such a highly habitualized behavior. One potential answer might be Progressive Web apps, a standard defined by Apple and Google already quite a few years ago, that seeks to offer a highly responsive and fast website behavior that feels almost like an application. To pull this off, the main idea is that a so called "service worker" - a piece of code that is installed on the mobile and continues running in the background - is making it possible for these web apps to for example send notifications to users while she is not using the website. So rather something that users know from their classical native apps. Another very trivial benefit is that you can install these apps on your home screen, and by tapping them it feels like really using an app and not browsing a website (e.g. there is no browser address bar). Finally the whole website can operate in offline mode too, thanks to a smart caching mechanism, that allows developers to decide what to store on the mobile in contrast to what the browser cache normally does. If you feel like trying out one of these apps I highly recommend to try out mobile.twitter.com, where Google and Twitter sat together and tried to showcase everything that is possible with this new technology. If you are using an Android phone, these apps should work right away, but if you are using an Apple phone make sure to at least have the most recent update 11.3 that finally supports progressive apps for apple devices. While Apple slightly opened the door to PWAs I fear that their lack of support for the major features might have something to do with politics. After all, developers circumventing the app store and interacting with their customers without an intermediary doesn’t leave much love for Apples beloved app store. Have a look at Martin's great presentation here.

Conclusion

Overall although the topics were a bit diverse, but I definitely enjoyed the conference. A big thanks goes to the organizers of Internet Briefing series who do an amazing job of constantly organizing those conferences in a monthly fashing. These are definitely a good way to exchange best practices and eventually learn something new. For me it was the motivation to finally get my hands dirty with progressive web apps, knowing that you don't really need much to make these work.

As usual I am happy to hear your comments on these topics and hope that you enjoyed that little summary.

May 03 2018
May 03

Read this blog post to get an overview over SaaS and open source options for sentiment detection. Learn an easy and accurate method relying on word embeddings with LSTMs that allows you to do state of the art sentiment analysis with deep learning in Keras.

Overview SaaS

When it comes to sentiment detection it has become a bit of a commodity. Especially the big 5 vendors offer their own sentiment detection as a service. Google offers an NLP API with sentiment detection. Microsoft offers sentiment detection through their Azure platform. IBM has come up with a solution called Tone Analyzer, that tries to get the "tone" of the message, which goes a bit beyond sentiment detection. Amazon offers a solution called comprehend that runs on aws as a lambda. Facebook surprisingly doesn't offer an API or an open source project here, although they are the ones with user generated content, where people often are not so nice to each other. Interestingly they do not offer any assistance for page owners in that specific matter.

Beyond the big 5 there are a few noteworthy of companies like Aylien and Monkeylearn, that are worth checking out.

Overview Open Source Solutions

Of course there are are open source solutions or libraries that offer sentiment detection too.
Generally all of these tools offer more than just sentiment analysis. Most of the outlined SaaS solutions above as well as the open source libraries offer a vast amount of different NLP tasks:

  • part of speech tagging (e.g. "going" is a verb),
  • stemming (finding the "root" of a word e.g. am,are,is -> be),
  • noun phrase extraction (e.g. car is a noun),
  • tokenization (e.g. splitting text into words, sentences),
  • words inflections (e.g. what's the plural of atlas),
  • spelling correction and translation.

I like to point you to pythons NLTK library, TextBlob, Pattern or R's Text Mining module and Java's LingPipe library. Finally, I encourage you to have a look at the latest Spacy NLP suite, which doesn't offer sentiment detection per se but has great NLP capabilities.

If you are looking for more options I encourage you to take a look at the full list that I have compiled in our data science stack.

Let's get started

So you see, when you need sentiment analysis in your web-app or mobile app you already have a myriad of options to get started. Of course you might build something by yourself if your language is not supported or you have other legal compliances to meet when it comes to data privacy.

Let me walk you through all of the steps needed to make a well working sentiment detection with Keras and long short-term memory networks. Keras is a very popular python deep learning library, similar to TFlearn that allows to create neural networks without writing too much boiler plate code. LSTM networks are a special form or network architecture especially useful for text tasks which I am going to explain later.

Step 1: Get the data

Being a big movie nerd, I have chosen to classify IMDB reviews as positive or negative for this example. As a benefit the IMDB sample comes already with the Keras datasets library, so you don't have to download anything. If you are interested though, not a lot of people know that IMDB offers its own datasets which can be downloaded publicly. Among those we are interested in the ones that contain movie reviews, which have been marked by hand to be either positive or negative.

#download the data
from keras.datasets import imdb 
top_words = 5000 
(X_train, y_train), (X_test, y_test) = imdb.load_data(num_words=top_words)

The code above does a couple of things at once:

  1. It downloads the data
  2. It downloads the first 5000 top words for each review
  3. It splits the data into a test and a training set.

If you look at the data you will realize it has been already pre-processed. All words have been mapped to integers and the integers represent the words sorted by their frequency. This is very common in text analysis to represent a dataset like this. So 4 represents the 4th most used word, 5 the 5th most used word and so on... The integer 1 is reserved reserved for the start marker, the integer 2 for an unknown word and 0 for padding.

If you want to peek at the reviews yourself and see what people have actually written, you can reverse the process too:

#reverse lookup
word_to_id = keras.datasets.imdb.get_word_index()
word_to_id = {k:(v+INDEX_FROM) for k,v in word_to_id.items()}
word_to_id["<PAD>"] = 0
word_to_id["<START>"] = 1
word_to_id["<UNK>"] = 2
id_to_word = {value:key for key,value in word_to_id.items()}
print(' '.join(id_to_word[id] for id in train_x[0] ))

The output might look like something like this:

<START> this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert <UNK> is an amazing actor and now the same being director <UNK> father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for <UNK> and would recommend it to everyone to watch and the fly <UNK> was amazing really cried at the end it was so sad and you know w

One-hot encoder

If you want to do the same with your text (e.g. my example are some work reviews) you can use Keras already built in "one-hot" encoder feature that will allow you to encode your documents with integers. The method is quite useful since it will remove any extra marks (e.g. !"#$%&...) and split sentences into words by space and transform the words into lowercase.

#one hot encode your documents
from numpy import array
from keras.preprocessing.text import one_hot
docs = ['Gut gemacht',
        'Gute arbeit',
        'Super idee',
        'Perfekt erledigt',
        'exzellent',
        'naja',
        'Schwache arbeit.',
        'Nicht gut',
        'Miese arbeit.',
        'Hätte es besser machen können.']
# integer encode the documents
vocab_size = 50
encoded_docs = [one_hot(d, vocab_size) for d in docs]
print(encoded_docs)

Although the encoding will not be sorted like in our example before (e.g. lower numbers representing more frequent words), this will still give you a similar output:

[[18, 6], [35, 39], [49, 46], [41, 39], [25], [16], [11, 39], [6, 18], [21, 39], [15, 23, 19, 41, 25]]

Step 2: Preprocess the data

Since the reviews differ heavily in terms of lengths we want to trim each review to its first 500 words. We need to have text samples of the same length in order to feed them into our neural network. If reviews are shorter than 500 words we will pad them with zeros. Keras being super nice, offers a set of preprocessing routines that can do this for us easily.

# Truncate and pad the review sequences 
from keras.preprocessing import sequence 
max_review_length = 500 
X_train = sequence.pad_sequences(X_train, maxlen=max_review_length) 
X_test = sequence.pad_sequences(X_test, maxlen=max_review_length) 

As you see above (I've just output the padded Array as a pandas dataframe for visibility) a lot of the reviews have padded 0 at the front which means, that the review is shorter than 500 words.

Step 3: Build the model

Surprisingly we are already done with the data preparation and can already start to build our model.

# Build the model 
embedding_vector_length = 32 
model = Sequential() 
model.add(Embedding(top_words, embedding_vector_length, input_length=max_review_length)) 
model.add(LSTM(100)) 
model.add(Dense(1, activation='sigmoid')) 
model.compile(loss='binary_crossentropy',optimizer='adam', metrics=['accuracy']) 
print(model.summary()) 

The two most important things in our code are the following:

  1. The Embedding layer and
  2. The LSTM Layer.

Lets cover what both are doing.

Word embeddings

The embedding layer will learn a word embedding for all the words in the dataset. It has three arguments the input_dimension in our case the 500 words. The output dimension aka the vector space in which words will be embedded. In our case we have chosen 32 dimensions so a vector of the length of 32 to hold our word coordinates.

There are already pre-trained word embeddings (e.g. GloVE or Word2Vec) that you can download so that you don't have to train your embeddings all by yourself. Generally, these word embeddings are also based on specialized algorithms that do the embedding always a bit different, but we won't cover it here.

How can you imagine what an embedding actually is? Well generally words that have a similar meaning in the context should be embedded next to each other. Below is an example of word embeddings in a two-dimensional space:

Why should we even care about word embeddings? Because it is a really useful trick. If we were to feed our reviews into a neural network and just one-hot encode them we would have very sparse representations of our texts. Why? Let us have a look at the sentence "I do my job" in "bag of words" representation with a vocabulary of 1000: So a matrix that holds 1000 words (each column is one word), has four ones in it (one for I, one for do one for my and one for job) and 996 zeros. So it would be very sparse. This means that learning from it would be difficult, because we would need 1000 input neurons each representing the occurrence of a word in our sentence.

In contrast if we do a word embedding we can fold these 1000 words in just as many dimensions as we want, in our case 32. This means that we just have an input vector of 32 values instead of 1000. So the word "I" would be some vector with values (0.4,0.5,0.2,...) and the same would happen with the other words. With word embedding like this, we just need 32 input neurons.

LSTMs

Recurrent neural networks are networks that are used for "things" that happen recurrently so one thing after the other (e.g. time series, but also words). Long Short-Term Memory networks (LSTM) are a specific type of Recurrent Neural Network (RNN) that are capable of learning the relationships between elements in an input sequence. In our case the elements are words. So our next layer is an LSTM layer with 100 memory units.

LSTM networks maintain a state, and so overcome the problem of a vanishing gradient problem in recurrent neural networks (basically the problem that when you make a network deep enough the information for learning will "vanish" at some point). I do not want to go into detail how they actually work, but here delivers a great visual explanation. Below is a schematic overview over the building blocks of LSTMs.

So our output of the embedding layer is a 500 times 32 matrix. Each word is represented through its position in those 32 dimensions. And the sequence is the 500 words that we feed into the LSTM network.

Finally at the end we have a dense layer with one node with a sigmoid activation as the output.

Since we are going to have only the decision when the review is positive or negative we will use binary_crossentropy for the loss function. The optimizer is the standard one (adam) and the metrics are also the standard accuracy metric.

By the way, if you want you can build a sentiment analysis without LSTMs, then you simply need to replace it by a flatten layer:

#Replace LSTM by a flatten layer
#model.add(LSTM(100)) 
model.add(Flatten()) 

Step 4: Train the model

After defining the model Keras gives us a summary of what we have built. It looks like this:

#Summary from Keras
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_1 (Embedding)      (None, 500, 32)           160000    
_________________________________________________________________
lstm_1 (LSTM)                (None, 100)               53200     
_________________________________________________________________
dense_1 (Dense)              (None, 1)                 101       
=================================================================
Total params: 213,301
Trainable params: 213,301
Non-trainable params: 0
_________________________________________________________________
None

To train the model we simply call the fit function,supply it with the training data and also tell it which data it can use for validation. That is really useful because we have everything in one call.

#Train the model
model.fit(X_train, y_train, validation_data=(X_test, y_test), nb_epoch=3, batch_size=64) 

The training of the model might take a while, especially when you are only running it on the CPU instead of the GPU. When the model training happens, what you want to observe is the loss function, it should constantly be going down, this shows that the model is improving. We will make the model see the dataset 3 times, defined by the epochs parameter. The batch size defines how many samples the model will see at once - in our case 64 reviews.

To observe the training you can fire up tensor board which will run in the browser and give you a lot of different analytics, especially the loss curve in real time. To do so type in your console:

sudo tensorboard --logdir=/tmp

Step 5: Test the model

Once we have finished training the model we can easily test its accuracy. Keras provides a very handy function to do that:

#Evaluate the model
scores = model.evaluate(X_test, y_test, verbose=0) 
print("Accuracy: %.2f%%" % (scores[1]*100))

In our case the model achieved an accuracy of around 90% which is excellent, given the difficult task. By the way if you are wondering what the results would have been with the Flatten layer it is also around 90%. So in this case I would use Occam's razor and in case and in doubt: go with the simpler model.

Step 6: Predict something

Of course at the end we want to use our model in an application. So we want to use it to create predictions. In order to do so we need to translate our sentence into the corresponding word integers and then pad it to match our data. We can then feed it into our model and see if how it thinks we liked or disliked the movie.

#predict sentiment from reviews
bad = "this movie was terrible and bad"
good = "i really liked the movie and had fun"
for review in [good,bad]:
    tmp = []
    for word in review.split(" "):
        tmp.append(word_to_id[word])
    tmp_padded = sequence.pad_sequences([tmp], maxlen=max_review_length) 
    print("%s. Sentiment: %s" % (review,model.predict(array([tmp_padded][0]))[0][0]))
i really liked the movie and had fun. Sentiment: 0.715537
this movie was terrible and bad. Sentiment: 0.0353295

In this case a value close to 0 means the sentiment was negative and a value close to 1 means its a positive review. You can also use "model.predict_classes" to just get the classes of positive and negative.

Conclusion or what’s next?

So we have built quite a cool sentiment analysis for IMDB reviews that predicts if a movie review is positive or negative with 90% accuracy. With this we are already quite close to industry standards. This means that in comparison to a quick prototype that a colleague of mine built a few years ago we could potentially improve on it now. The big benefit while comparing our self-built solution with an SaaS solution on the market is that we own our data and model. We can now deploy this model on our own infrastructure and use it as often as we like. Google or Amazon never get to see sensitive customer data, which might be relevant for certain business cases. We can train it with German or even Swiss German language given that we find a nice dataset, or simply build one ourselves.

As always I am looking forward to your comments and insights! As usual you can download the Ipython notebook with the code here.

P.S. The people from monkeylearn contacted me and pointed out that they have written quite an extensive introduction to sentiment detection here: https://monkeylearn.com/sentiment-analysis/ so I point you to that in case you want to read up on the general concepts.

Apr 12 2017
Apr 12

As a Swiss-based Drupal Agency, we have to create a lot of multilingual sites. Since Switzerland has three official languages (German, French, Italian) and even one more national language (Rumantsch), we are used to this requirement and we found our way with Drupal to make this an easy task (usually). We mainly used node translations in Drupal 7 for maximum flexibility. We used to separate languages from each other using the various i18n modules, language specific menus, blocks, URL-patterns, terms and so on.

With Drupal 8, things changed.
I struggled a little doing multilingual sites in Drupal 8 the same way I was used to in Drupal 7 because node translation is not available anymore (which is good) so I had to find another way to achieve the same easy to handle translations system. For us and for our clients. Let me explain, what I have learned.

Drupal 8 multilanguage

Image: drupal8multilingual.org

Drupal 8 issues multilanguage challenges

Challenge 1: Node add / edit menu handling

The main challenge I had using Drupal 8, was the ease to build your menus directly from the node creation page. You can do it, but only for the initial language. If you try to add a translated node to another menu or rename the item, it always ends up moving / renaming the source node instead of adding a link to the translation. So it can become quite confusing building a navigation directly from the node creation page or to add translations to the menu. A workaround was to add all navigation items manually in the menu administration if you are using a menu per language. With lots of languages and menus / items, this is not really a convenient task. Fortunately, translations from the node creation page have been implemented with a later release of Drupal 8.

Challenge 2: Untranslated Nodes show up in Menu

Another thing which bothered me was that untranslated nodes show up in the navigation (if you use only one menu). This can be quite confusing since most of the times not every page is translated in every language. Or in some languages, you need a little more than in others. You can read a lot about this topic and the reasons behind (e.g. here and here). However you do it, it’s always wrong in some situations and perfectly fine in others. But to be “limited” and “locked in” to a certain way is not nice and you have to deal with it. To sum up, once a node is put into a menu, it will show up everywhere. Regardless if there are translations or not.

Challenge 3: Language Switcher shows all languages – always.

Somewhat confusing is the Language Switcher. In Drupal 7, a language link was not available or strikethrough if there was no translation available. In Drupal 8, every language is always visible and linked. So if you look on a German page which is only available in German, the language switcher will present you all language links to the same node. A click on those language links mainly changes the interface language but the node content remains the same (since not translated). Usually also with a drupalish URL (node/xxxx) because there is no translation for the node and therefore also no URL alias available. This behavior is confusing and wrong in my point of view

An example to illustrate the above-written challenges.

multilanguage issues with Drupal 8

English Front-Page with mixed navigation items.

The screen above shows an installation with 2 languages (English and German). The English Page is a basic page which has a translation. English is selected. If you choose Deutsch on the language switcher, the English Page becomes Deutsche Seite (see image below) and shows the German content. So far so good. But the second menu item you see with the title Über uns (nur Deutsch) should not appear here since it’s only available in German. But it does. And if you actually go on this page, you will see the German text with everything English around it and no URL-Alias (/node/2 in this example). This is usually not very useful for us.

multilanguage issues with Drupal 8

German only Page – Language Switcher visible.

Also, the language switcher shown in the image above is from my point of view wrong or not very useful. It shows a link to the English version, but there is no English translation for this node. So why is it there? To see a German page with English decoration? Not sure. But I want to get rid of this link or at least modify it to be stroked through if the language is not available.

How to fix improve this?

Luckily, the Drupal community is always good for help. After some “research” on the web, I finally found (besides lots of discussions and comments in the issue queues) a way to achieve the desired setup.

To sum up again: I want to see only menu items which are available in my language and only see a link to another language, if a translation is available.

Since there is no patch and still some ongoing discussions on drupal.org you need to implement it on your own. Implement the following two modules.

Hide untranslated menu items

Code from https://www.drupal.org/node/2466553#comment-11991690. Credits go to michaelkoehne.

<?php use Drupal\Core\Menu\MenuLinkInterface; use Drupal\menu_link_content\Plugin\Menu\MenuLinkContent; use Drupal\Core\Language\LanguageInterface; /** * Implements hook_preprocess_menu(). */ function MYMODULE_preprocess_menu(&$variables) { if ($variables['menu_name'] == 'main') { $language = Drupal::languageManager() ->getCurrentLanguage(LanguageInterface::TYPE_CONTENT) ->getId(); foreach ($variables['items'] as $key => $item) { if (!$variables['items'][$key] = MYMODULE_checkForMenuItemTranslation($item, $language)) { unset($variables['items'][$key]); } } } } function MYMODULE_checkForMenuItemTranslation($item, $language) { $menuLinkEntity = MYMODULE_load_link_entity_by_link($item['original_link']); if ($menuLinkEntity != NULL) { $languages = $menuLinkEntity->getTranslationLanguages(); // Remove links which are not translated to the current language. if (!array_key_exists($language, $languages)) { return FALSE; } else { if (count($item['below']) > 0) { foreach ($item['below'] as $subkey => $subitem) { if (!$item['below'][$subkey] = MYMODULE_checkForMenuItemTranslation($subitem, $language)) { unset($item['below'][$subkey]); } } } return $item; } } } function MYMODULE_load_link_entity_by_link(MenuLinkInterface $menuLinkContentPlugin) { $entity = NULL; if ($menuLinkContentPlugin instanceof MenuLinkContent) { $menu_link = explode(':', $menuLinkContentPlugin->getPluginId(), 2); $uuid = $menu_link[1]; $entity = \Drupal::service('entity.repository') ->loadEntityByUuid('menu_link_content', $uuid); } return $entity; }

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

<?php

use Drupal\Core\Menu\MenuLinkInterface;

use Drupal\menu_link_content\Plugin\Menu\MenuLinkContent;

use Drupal\Core\Language\LanguageInterface;

/**

* Implements hook_preprocess_menu().

*/

function MYMODULE_preprocess_menu(&$variables) {

  if ($variables['menu_name'] == 'main') {

    $language = Drupal::languageManager()

      ->getCurrentLanguage(LanguageInterface::TYPE_CONTENT)

      ->getId();

    foreach ($variables['items'] as $key => $item) {

      if (!$variables['items'][$key] = MYMODULE_checkForMenuItemTranslation($item, $language)) {

        unset($variables['items'][$key]);

      }

    }

  }

}

function MYMODULE_checkForMenuItemTranslation($item, $language) {

  $menuLinkEntity = MYMODULE_load_link_entity_by_link($item['original_link']);

  if ($menuLinkEntity != NULL) {

    $languages = $menuLinkEntity->getTranslationLanguages();

    // Remove links which are not translated to the current language.

    if (!array_key_exists($language, $languages)) {

      return FALSE;

    }

    else {

      if (count($item['below']) > 0) {

        foreach ($item['below'] as $subkey => $subitem) {

          if (!$item['below'][$subkey] = MYMODULE_checkForMenuItemTranslation($subitem, $language)) {

            unset($item['below'][$subkey]);

          }

        }

      }

      return $item;

    }

  }

}

function MYMODULE_load_link_entity_by_link(MenuLinkInterface $menuLinkContentPlugin) {

  $entity = NULL;

  if ($menuLinkContentPlugin instanceof MenuLinkContent) {

    $menu_link = explode(':', $menuLinkContentPlugin->getPluginId(), 2);

    $uuid = $menu_link[1];

    $entity = \Drupal::service('entity.repository')

      ->loadEntityByUuid('menu_link_content', $uuid);

  }

  return $entity;

}

Hide untranslated languages in language switcher

Code from https://www.drupal.org/node/2791231#comment-12004615 (slightly adapted. Links get a class, not removed by default). Credits to Leon Kessler.

<?php /** * @file * Hide language switcher links for untranslated languages on an entity. */ use Drupal\Core\Entity\ContentEntityInterface; /** * Implements hook_language_switch_links_alter(). */ function MYOTHERMODULE_language_switch_links_alter(array &$links, $type, $path) { if ($entity = MYOTHERMODULE_get_page_entity()) { $new_links = array(); foreach ($links as $lang_code => $link) { try { if ($entity->getTranslation($lang_code)->access('view')) { $new_links[$lang_code] = $link; } } catch (\InvalidArgumentException $e) { // This language is untranslated so do not add it to the links. $link['attributes']['class'][] = 'not-translated'; $new_links[$lang_code] = $link; } } $links = $new_links; // If we're left with less than 2 links, then there's nothing to switch. // Hide the language switcher. if (count($links) < 2) { $links = array(); } } } /** * Retrieve the current page entity. * * @return Drupal\Core\Entity\ContentEntityInterface * The retrieved entity, or FALSE if none found. */ function MYOTHERMODULE_get_page_entity() { $params = \Drupal::routeMatch()->getParameters()->all(); $entity = reset($params); if ($entity instanceof ContentEntityInterface) { return $entity; } return FALSE; }

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

<?php

/**

* @file

* Hide language switcher links for untranslated languages on an entity.

*/

use Drupal\Core\Entity\ContentEntityInterface;

/**

* Implements hook_language_switch_links_alter().

*/

function MYOTHERMODULE_language_switch_links_alter(array &$links, $type, $path) {

  if ($entity = MYOTHERMODULE_get_page_entity()) {

    $new_links = array();

    foreach ($links as $lang_code => $link) {

      try {

        if ($entity->getTranslation($lang_code)->access('view')) {

          $new_links[$lang_code] = $link;

        }

      }

      catch (\InvalidArgumentException $e) {

        // This language is untranslated so do not add it to the links.

        $link['attributes']['class'][] = 'not-translated';

        $new_links[$lang_code] = $link;

      }

    }

    $links = $new_links;

    // If we're left with less than 2 links, then there's nothing to switch.

    // Hide the language switcher.

    if (count($links) < 2) {

      $links = array();

    }

  }

}

/**

* Retrieve the current page entity.

*

* @return Drupal\Core\Entity\ContentEntityInterface

*   The retrieved entity, or FALSE if none found.

*/

function MYOTHERMODULE_get_page_entity() {

  $params = \Drupal::routeMatch()->getParameters()->all();

  $entity = reset($params);

  if ($entity instanceof ContentEntityInterface) {

    return $entity;

  }

  return FALSE;

}

Please note: The code above is from Drupal.org and therefore thanks to the original authors linked above.

Enable those two modules and you’re all set!

I did not encounter any issues yet using those two modules. If ever something changes in the way Drupal handles those cases, you just need to switch off the modules and everything should be back to normal. So nothing to lose right?

There are other attempts to this by altering the menu block. One of them is Menu Block Current Language but I had no luck with this one. On my most recent project, it worked with one menu but not if you separate your menu by two blocks (different starting levels).

I would love to hear how you guys handle those cases or how you deal with I18N in general. I’m sure there are a gazillion other ways to do it.

Apr 11 2017
Apr 11

As a Swiss-based Drupal Agency, we have to create a lot of multilingual sites. Since Switzerland has three official languages (German, French, Italian) and even one more national language (Rumantsch), we are used to this requirement and we found our way with Drupal to make this an easy task (usually). We mainly used node translations in Drupal 7 for maximum flexibility. We used to separate languages from each other using the various i18n modules, language specific menus, blocks, URL-patterns, terms and so on.

With Drupal 8, things changed.

I struggled a little doing multilingual sites in Drupal 8 the same way I was used to in Drupal 7 because node translation is not available anymore (which is good) so I had to find another way to achieve the same easy to handle translations system. For us and for our clients. Let me explain, what I have learned.

Drupal 8 multilanguage

Image: drupal8multilingual.org

Drupal 8 issues multilanguage challenges

Challenge 1: Node add / edit menu handling

The main challenge I had using Drupal 8, was the ease to build your menus directly from the node creation page. You can do it, but only for the initial language. If you try to add a translated node to another menu or rename the item, it always ends up moving / renaming the source node instead of adding a link to the translation. So it can become quite confusing building a navigation directly from the node creation page or to add translations to the menu. A workaround was to add all navigation items manually in the menu administration if you are using a menu per language. With lots of languages and menus / items, this is not really a convenient task. Fortunately, translations from the node creation page have been implemented with a later release of Drupal 8.

Challenge 2: Untranslated Nodes show up in Menu

Another thing which bothered me was that untranslated nodes show up in the navigation (if you use only one menu). This can be quite confusing since most of the times not every page is translated in every language. Or in some languages, you need a little more than in others. You can read a lot about this topic and the reasons behind (e.g. here and here). However you do it, it's always wrong in some situations and perfectly fine in others. But to be “limited” and “locked in” to a certain way is not nice and you have to deal with it. To sum up, once a node is put into a menu, it will show up everywhere. Regardless if there are translations or not.

Challenge 3: Language Switcher shows all languages – always.

Somewhat confusing is the Language Switcher. In Drupal 7, a language link was not available or strikethrough if there was no translation available. In Drupal 8, every language is always visible and linked. So if you look on a German page which is only available in German, the language switcher will present you all language links to the same node. A click on those language links mainly changes the interface language but the node content remains the same (since not translated). Usually also with a drupalish URL (node/xxxx) because there is no translation for the node and therefore also no URL alias available. This behavior is confusing and wrong in my point of view

An example to illustrate the above-written challenges.

multilanguage issues with Drupal 8

English Front-Page with mixed navigation items.

The screen above shows an installation with 2 languages (English and German). The English Page is a basic page which has a translation. English is selected. If you choose Deutsch on the language switcher, the English Page becomes Deutsche Seite (see image below) and shows the German content. So far so good. But the second menu item you see with the title Über uns (nur Deutsch) should not appear here since it's only available in German. But it does. And if you actually go on this page, you will see the German text with everything English around it and no URL-Alias (/node/2 in this example). This is usually not very useful for us.

German only Page – Language Switcher visible.

Also, the language switcher shown in the image above is from my point of view wrong or not very useful. It shows a link to the English version, but there is no English translation for this node. So why is it there? To see a German page with English decoration? Not sure. But I want to get rid of this link or at least modify it to be stroked through if the language is not available.

How to fix improve this?

Luckily, the Drupal community is always good for help. After some “research” on the web, I finally found (besides lots of discussions and comments in the issue queues) a way to achieve the desired setup.

To sum up again: I want to see only menu items which are available in my language and only see a link to another language, if a translation is available.

Since there is no patch and still some ongoing discussions on drupal.org you need to implement it on your own. Implement the following two modules.

Hide untranslated menu items

Code from drupal.org/node/2466553. Credits go to michaelkoehne.

<?php

use Drupal\Core\Menu\MenuLinkInterface;
use Drupal\menu_link_content\Plugin\Menu\MenuLinkContent;
use Drupal\Core\Language\LanguageInterface;

/**
 * Implements hook_preprocess_menu().
 */
function MYMODULE_preprocess_menu(&$variables) {
  if ($variables['menu_name'] == 'main') {
    $language = Drupal::languageManager()
      ->getCurrentLanguage(LanguageInterface::TYPE_CONTENT)
      ->getId();
    foreach ($variables['items'] as $key => $item) {
      if (!$variables['items'][$key] = MYMODULE_checkForMenuItemTranslation($item, $language)) {
        unset($variables['items'][$key]);
      }
    }
  }
}

function MYMODULE_checkForMenuItemTranslation($item, $language) {
  $menuLinkEntity = MYMODULE_load_link_entity_by_link($item['original_link']);

  if ($menuLinkEntity != NULL) {
    $languages = $menuLinkEntity->getTranslationLanguages();

    // Remove links which are not translated to the current language.
    if (!array_key_exists($language, $languages)) {
      return FALSE;
    }
    else {
      if (count($item['below']) > 0) {
        foreach ($item['below'] as $subkey => $subitem) {
          if (!$item['below'][$subkey] = MYMODULE_checkForMenuItemTranslation($subitem, $language)) {
            unset($item['below'][$subkey]);
          }
        }
      }
      return $item;
    }

  }
}

function MYMODULE_load_link_entity_by_link(MenuLinkInterface $menuLinkContentPlugin) {
  $entity = NULL;
  if ($menuLinkContentPlugin instanceof MenuLinkContent) {
    $menu_link = explode(':', $menuLinkContentPlugin->getPluginId(), 2);
    $uuid = $menu_link[1];
    $entity = \Drupal::service('entity.repository')
      ->loadEntityByUuid('menu_link_content', $uuid);
  }
  return $entity;
}

Hide untranslated languages in language switcher

Code from drupal.org/node/2791231 (slightly adapted. Links get a class, not removed by default). Credits to Leon Kessler.

<?php

/**
 * @file
 * Hide language switcher links for untranslated languages on an entity.
 */
use Drupal\Core\Entity\ContentEntityInterface;

/**
 * Implements hook_language_switch_links_alter().
 */
function MYOTHERMODULE_language_switch_links_alter(array &$links, $type, $path) {
  if ($entity = MYOTHERMODULE_get_page_entity()) {
    $new_links = array();
    foreach ($links as $lang_code => $link) {
      try {
        if ($entity->getTranslation($lang_code)->access('view')) {
          $new_links[$lang_code] = $link;
        }
      }
      catch (\InvalidArgumentException $e) {
        // This language is untranslated so do not add it to the links.
        $link['attributes']['class'][] = 'not-translated';
        $new_links[$lang_code] = $link;
      }

    }
    $links = $new_links;

    // If we're left with less than 2 links, then there's nothing to switch.
    // Hide the language switcher.
    if (count($links) < 2) {
      $links = array();
    }
  }
}

/**
 * Retrieve the current page entity.
 *
 * @return Drupal\Core\Entity\ContentEntityInterface
 *   The retrieved entity, or FALSE if none found.
 */
function MYOTHERMODULE_get_page_entity() {
  $params = \Drupal::routeMatch()->getParameters()->all();
  $entity = reset($params);
  if ($entity instanceof ContentEntityInterface) {
    return $entity;
  }
  return FALSE;
}

Please note: The code above is from Drupal.org and therefore thanks to the original authors linked above.

Enable those two modules and you're all set!

I did not encounter any issues yet using those two modules. If ever something changes in the way Drupal handles those cases, you just need to switch off the modules and everything should be back to normal. So nothing to lose right?

There are other attempts to this by altering the menu block. One of them is Menu Block Current Language but I had no luck with this one. On my most recent project, it worked with one menu but not if you separate your menu by two blocks (different starting levels).

I would love to hear how you guys handle those cases or how you deal with I18N in general. I'm sure there are a gazillion other ways to do it.

Apr 07 2017
Apr 07

After implementing some larger enterprise Drupal 8 websites, I would like to share some insights, how to solve common issues in the deployment workflow with Drupal 8 CMI.

Introduction to Drupal CMI

First of all, you need to understand, how the configuration management in Drupal 8 works. CMI allows you to export all configurations and its dependencies from the database into yml text files. To make sure, you never end up in an inconsistent state, CMI always exports everything. By default, you cannot exclude certain configurations.

Example:

If you change some configuration on the live database, these configurations will be reverted in the next deployment when you use

drush config-import

1

drush config-import

This is helpful and will make sure, you have the same configuration on all your systems.

How can I have different configurations on local / stage / live environments?

Sometimes, you want to have different configurations on your environments. For example, we have installed a “devel” module only on our local environment but we want to have it disabled on the live environment.

This can be achieved by using the configuration split module: https://www.drupal.org/project/config_split

What does Configuration Split?

This module slightly modifies the CMI by implementing a Config Filter (https://www.drupal.org/project/config_filter). Importing and exporting works the same way as before, except some configuration is read from and written to different directories. Importing configuration still removes configuration not present in the files. Thus, the robustness and predictability of the configuration management remains. And the best thing is: You still can use the same drush commands if you have at least Drush 8.1.10 installed.

Configuration Split Example / Installation Guide

Install config_split using composer. You need need at least “8.x-1.0-beta4” and > drush 8.1.10 for this guide.

composer require drupal/config_split "^1.0"

1

composer require drupal/config_split "^1.0"

Enable config_split and navigate to “admin/config/development/configuration/config-split”

drush en config_split -y

1

drush en config_split -y

Optional: Installing the chosen module will make the selection of blacklists / greylists way more easier. You can enable chosen only on admin pages.

composer require drupal/chosen "^1.0"

1

composer require drupal/chosen "^1.0"

I recommend you to create an “environments” subfolder in your config folder. Inside this folder you will have a separate directory for every environment:

Drupal 8 Configuration Management Folders

Now you can configure your environments:

Config Split in Drupal 8 Configuration Management

The most important thing is, that you set every environment to “Inactive”. We will later activate them according to the environment via settings.php

Config Split settings with the Drupal 8 Configuration Management

Here is my example where I enable the devel module on local:

Dev Environment Example

Activate the environments via settings.php

This is the most important part of the whole setup up. Normally, we never commit the settings.php into git. But we have a [environment]-settings.php in git for every environment:

settings.php (not in git) variables-dev.php (in git and included in the settings.php of dev) variables-live.php (in git and included in the settings.php of live) settings.local.php (in git and included locally)

settings.php (not in git)

variables-dev.php (in git and included in the settings.php of dev)

variables-live.php (in git and included in the settings.php of live)

settings.local.php (in git and included locally)

You need to add the following line to the variables-[environment].php. Please change the variable name according to your environment machine name:

// This enables the config_split module $config['config_split.config_split.dev']['status'] = TRUE;

// This enables the config_split module

$config['config_split.config_split.dev']['status'] = TRUE;

If you have done everything correctly and cleared the cache you will see “active (overriden)” in the config_split overview next to the current environment.

Now you can continue using

drush config-import -y drush config-export -y

drush config-import -y

drush config-export -y

and config_split will do the magic.

How can I exclude certain Config Files and prevent them to be overridden / deleted on my live environment?

The most prominent candidates for this workflow are webforms and contact forms. In Drupal 7, webforms are nodes and you were able to give your CMS administrator the opportunity to create their own forms.

In Drupal 8 webforms are config entities, which means that they will be deleted while deploying if the yml files are not in git.

After testing a lot of different modules / drush scripts, I finally came up with an easy to use workflow to solve this issue and give CMS administrators the possibility to create webforms without git knowledge:

Set up an “Excluded” environment

First of all, we need an “excluded” environment. I created a subfolder in my config-folder and added a .htaccess file to protect the content. You can copy the .htaccess from an existing environment, if you are lazy. Don’t forget to deploy this folder to your live system before you do the next steps.

Folders

Excluded

Now you can exclude some config files to be excluded / grey-listed on your live environment:

webform.webform.* contact.form.*

webform.webform.*

contact.form.*

Greylist Webform in Config Split

Set the excluded environment to “Inactive”. We will later enable it on the live / dev environment via settings.php.

Enable “excluded” environment and adapt deployment workflow

We enable the “excluded” environment on the live system via variables-live.php (see above):

// This will allow module config per environment and exclude webforms from being overridden $config['config_split.config_split.excluded']['status'] = TRUE;

// This will allow module config per environment and exclude webforms from being overridden

$config['config_split.config_split.excluded']['status'] = TRUE;

In your deployment workflow / script you need to add the following line before you do a drush config-import:

#execute some drush commands echo "-----------------------------------------------------------" echo "Exporting excluded config" drush @live config-split-export -y excluded echo "-----------------------------------------------------------" echo "Importing configuration" drush @live config-import -y

1

2

3

4

5

6

7

8

#execute some drush commands

echo "-----------------------------------------------------------"

echo "Exporting excluded config"

drush @live config-split-export -y excluded

echo "-----------------------------------------------------------"

echo "Importing configuration"

drush @live config-import -y

The drush command “drush @live config-split-export -y excluded” will export all webforms and contact forms created by your CMS administrators into the folder “excluded”. The “drush config-import” command will therefore not delete them and your administrators can happily create their custom forms.

Benefit of disable “excluded” on local environment

We usually disable the “excluded” environment on our local environment. This allows us to create complex webforms on our local machine for our clients and deploy them as usual. In the end you can have a mix of customer created webforms and your own webforms which is quite helpful.

Final note

The CMI is a great tool and I would like to thank the maintainers of the config_split module for their great extension. This is a huge step forward making Drupal 8 a real Enterprise CMS Tool.

If you have any questions, don’t hesitate to post a comment.

Apr 06 2017
Apr 06

After implementing some larger enterprise Drupal 8 websites, I would like to share some insights, how to solve common issues in the deployment workflow with Drupal 8 CMI.

Introduction to Drupal CMI

First of all, you need to understand, how the configuration management in Drupal 8 works. CMI allows you to export all configurations and its dependencies from the database into yml text files. To make sure, you never end up in an inconsistent state, CMI always exports everything. By default, you cannot exclude certain configurations.

Example:

If you change some configuration on the live database, these configurations will be reverted in the next deployment when you use

drush config-import

This is helpful and will make sure, you have the same configuration on all your systems.

How can I have different configurations on local / stage / live environments?

Sometimes, you want to have different configurations on your environments. For example, we have installed a “devel” module only on our local environment but we want to have it disabled on the live environment.

This can be achieved by using the configuration split module.

What does Configuration Split?

This module slightly modifies the CMI by implementing a Config Filter. Importing and exporting works the same way as before, except some configuration is read from and written to different directories. Importing configuration still removes configuration not present in the files. Thus, the robustness and predictability of the configuration management remains. And the best thing is: You still can use the same drush commands if you have at least Drush 8.1.10 installed .

Configuration Split Example / Installation Guide

Install config_split using composer. You need need at least “ 8.x-1.0-beta4” and > drush 8.1.10 for this guide.

composer require drupal/config_split "^1.0"

Enable config_split and navigate to “admin/config/development/configuration/config-split”

drush en config_split -y

Optional: Installing the chosen module will make the selection of blacklists / greylists way more easier. You can enable chosen only on admin pages.

composer require drupal/chosen "^1.0"

I recommend you to create an “environments” subfolder in your config folder. Inside this folder you will have a separate directory for every environment:

Drupal 8 Configuration Management Folders

Now you can configure your environments:

Config Split in Drupal 8 Configuration Management

The most important thing is, that you set every environment to “Inactive”. We will later activate them according to the environment via settings.php

Config Split settings with the Drupal 8 Configuration Management

Here is my example where I enable the devel module on local:

Dev Environment Example

Activate the environments via settings.php

This is the most important part of the whole setup up. Normally, we never commit the settings.php into git. But we have a [environment]-settings.php in git for every environment:

settings.php (not in git)

variables-dev.php (in git and included in the settings.php of dev)
variables-live.php (in git and included in the settings.php of live)
settings.local.php (in git and included locally)

You need to add the following line to the variables-[environment].php. Please change the variable name according to your environment machine name :

// This enables the config_split module
$config['config_split.config_split.dev']['status'] = TRUE;

If you have done everything correctly and cleared the cache you will see  “active (overriden)” in the config_split overview next to the current environment.

Now you can continue using

drush config-import -y
drush config-export -y

and config_split will do the magic.

How can I exclude certain Config Files and prevent them to be overridden / deleted on my live environment?

The most prominent candidates for this workflow are webforms and contact forms . In Drupal 7, webforms are nodes and you were able to give your CMS administrator the opportunity to create their own forms.

In Drupal 8 webforms are config entities , which means that they will be deleted while deploying if the yml files are not in git.

After testing a lot of different modules / drush scripts, I finally came up with an easy to use workflow to solve this issue and give CMS administrators the possibility to create webforms without git knowledge:

Set up an “Excluded” environment

First of all, we need an “excluded” environment. I created a subfolder in my config-folder and added a .htaccess file to protect the content. You can copy the .htaccess from an existing environment, if you are lazy. Don't forget to deploy this folder to your live system before you do the next steps.

FoldersExcluded

Now you can exclude some config files to be excluded / grey-listed on your live environment:

webform.webform.*
contact.form.*
Greylist Webform in Config Split

Set the excluded environment to “Inactive” . We will later enable it on the live / dev environment via settings.php.

Enable “excluded” environment and adapt deployment workflow

We enable the “excluded” environment on the live system via variables-live.php (see above):

// This will allow module config per environment and exclude webforms from being overridden
$config['config_split.config_split.excluded']['status'] = TRUE;

In your deployment workflow / script you need to add the following line before you do a drush config-import:

#execute some drush commands
echo "-----------------------------------------------------------"
echo "Exporting excluded config"
drush @live config-split-export -y excluded

echo "-----------------------------------------------------------"
echo "Importing configuration"
drush @live config-import -y

The drush command “ drush @live config-split-export -y excluded ” will export all webforms and contact forms created by your CMS administrators into the folder “excluded”. The “drush config-import” command will therefore not delete them and your administrators can happily create their custom forms.

Benefit of disable “excluded” on local environment

We usually disable the “excluded” environment on our local environment. This allows us to create complex webforms on our local machine for our clients and deploy them as usual. In the end you can have a mix of customer created webforms and your own webforms which is quite helpful.

Final note

The CMI is a great tool and I would like to thank the maintainers of the config_split module for their great extension. This is a huge step forward making Drupal 8 a real Enterprise CMS Tool.

If you have any questions, don't hesitate to post a comment.

Jan 10 2017
Jan 10

Drupal is an open source project and really depends on its community to move forward. It is all about getting to know the CMS, spreading the knowledge and contribute to projects.
I will give you some ways to get involved, even if you are not a developer there is a task for you!

A group of Drupal mentors at DrupalCon 2016 in Dublin

Drupal Mentors – DrupalCon Dublin 2016 by Michael Cannon is licenced under CC BY-SA 2.0

Participating in user support

Sharing your knowledge with others is very important to the community: it is a nice thing to do and you might also learn some things by doing so. Whatever your skill level, you can give back to the community with online support. There are many places where you can give support starting with the Support Forums. You can also go to Drupal Answers which is more active than the forums or subscribe to the Support Mailing list. If you prefer real-time chat, you can also join #drupal-support channel on IRC or the Slack channels.

Helping out on documentation

Community members can write, review and improve different sorts of documentation for the project: community documentation on drupal.org, programming API reference, help pages inside the core software, documentation embedded in contributed modules and themes etc.
Contributing is a good way to learn more about Drupal and share your knowledge with others. Beginners are particularly encouraged to participate as they are more likely to know where documentation is lacking.
If you are interested, check out the new contributor tasks for anyone and writers.

Translating Drupal interface in your own language

The default language for the administration interface is English but there are about 100 available languages for translations. There is always a need for translations as many of these translation sets are incomplete or can be improved for core and contributed modules.
All translations are now managed by the translation server. If you are willing to help, all you have to do is logging into drupal.org and join a language team. There is even a video to learn how the translation system works and a documentation.

You can also help to translate documentation into your language. Most language-specific communities have their own documentation so you should get in touch with them directly. To learn more, see the dedicated page.

Improving design and usability

The idea is to improve the usability especially in Drupal 8 regarding the administration interface. The focus is mainly on content creation and site building. The community has done many research to understand the problems that users run into and how the new improvements performs. The purpose is also to educate developers and engage designers in order to grow the UX-team. You can visit the Drupal 8 UX page for more details and join the usability group.

Writing a blog post about Drupal

Writing a blog post about Drupal is a good way to share your knowledge and expertise. There are many subjects to explore, technical or not: talking about a former project you developed or writing a tutorial, telling about the state of a version or sharing about an event you attended… And if you are lucky enough your post can be published on the Weekly Drop, the official Drupal newsletter!

Don’t forget to reference your blog post on Planet Drupal, this platform is an aggregated list of feeds from around the web which shares relevant Drupal-related knowledge and information.

You can also find our Drupal related blog posts on the Liip blog.

Testing core and modules

Testing Drupal projects is necessary to make the platform stable and there are many things to test! If you have a technical background, you can help to review patches or to write unit tests.
For non-technical people, you can provide some feedback about usability of the administration interface that will help to improve the user experience. Follow the process to give a proper feedback.

Contributing to development

There are many ways to contribute code in core and “contrib” projects such as modules or themes.
You can first help to improve existing projects by submitted patches. This would be the natural thing to do when you work with a module and you notice a bug or a missing feature: search in the corresponding issue queue if the problem have been noticed before. If not, post a message explaining the issue and add a snippet of code if you found a potential fix. Then you can create a patch and submit it into the issue queue.
You can also contribute to new projects by creating your very own module or theme or create a sandbox for more experimental projects.

Attending events

The Drupal association organizes many events all around the world to promote the CMS and gather the community.

One of the biggest events are the Drupalcons. A Drupalcon gathers thousands of people and lasts about one week including 3 full days of conferences. These conferences cover many topics: site building, user experience, security, content authoring etc. You can also join sprints to contribute to Drupal projects and social events to meet the members of the community. Check out our report about DrupalCon Barcelona 2015!

“Drupal Dev Days” conferences occur once a year and gather developers to discuss and present topics technically relevant to the community. You can join sprints, intensive coding sessions and technical conferences.

You can also join DrupalCamps to meet your local community. These events last one or two days and focus on sharing knowledge amongst the community. You can attend conferences and sprints.

There are also many Drupal meetups which are free events happening in many cities in the world. Presentations and discussions finish around nice drinks and appetizers.

Sponsoring events

The community holds conventions and meetups in many countries and being a sponsor will not only help Drupal development but it will also enable you to be noticeable within the community. There are different levels of sponsorings that will offer from mentions on social media to advertising online and at the exhibition space of the event. All you have to do is getting in touch with the event organizers. By the way, Liip will sponsor the Drupal Mountain Camp in Davos this year!

Offering a donation

You can give donations to the Drupal association through the website in order to support drupal.org infrastructure and maintenance, worldwide events such as Drupalcons. The donations are either in Euros or Dollars.

You can also become a member of the Drupal Association for the same purpose, as an individual member or an organization member. The minimal fees are 15 Euros. Find more information about membership on drupal.org.

Conclusion

Drupal projects are constantly improving thanks to passionate volunteers who work on many subjects: development, documentation, marketing, events organization, supports… There is for sure a task that will suit you and it only takes small time commitment to make changes.
So join the great Drupal community and start getting involved!

Jan 09 2017
Jan 09

Drupal is an open source project and really depends on its community to move forward. It is all about getting to know the CMS, spreading the knowledge and contribute to projects.

I will give you some ways to get involved, even if you are not a developer there is a task for you!

A group of Drupal mentors at DrupalCon 2016 in Dublin

Drupal Mentors – DrupalCon Dublin 2016 by Michael Cannon is licenced under CC BY-SA 2.0

Participating in user support

Sharing your knowledge with others is very important to the community: it is a nice thing to do and you might also learn some things by doing so. Whatever your skill level, you can give back to the community with online support. There are many places where you can give support starting with the Support Forums. You can also go to Drupal Answers which is more active than the forums or subscribe to the Support Mailing list. If you prefer real-time chat, you can also join #drupal-support channel on IRC or the Slack channels.

Helping out on documentation

Community members can write, review and improve different sorts of documentation for the project: community documentation on drupal.org, programming API reference, help pages inside the core software, documentation embedded in contributed modules and themes etc.

Contributing is a good way to learn more about Drupal and share your knowledge with others. Beginners are particularly encouraged to participate as they are more likely to know where documentation is lacking.

If you are interested, check out the new contributor tasks for anyone and writers.

Translating Drupal interface in your own language

The default language for the administration interface is English but there are about 100 available languages for translations. There is always a need for translations as many of these translation sets are incomplete or can be improved for core and contributed modules.

All translations are now managed by the translation server. If you are willing to help, all you have to do is logging into drupal.org and join a language team. There is even a video to learn how the translation system works and a documentation.

You can also help to translate documentation into your language. Most language-specific communities have their own documentation so you should get in touch with them directly. To learn more, see the dedicated page.

Improving design and usability

The idea is to improve the usability especially in Drupal 8 regarding the administration interface. The focus is mainly on content creation and site building. The community has done many research to understand the problems that users run into and how the new improvements performs. The purpose is also to educate developers and engage designers in order to grow the UX-team. You can visit the Drupal 8 UX page for more details and join the usability group.

Writing a blog post about Drupal

Writing a blog post about Drupal is a good way to share your knowledge and expertise. There are many subjects to explore, technical or not: talking about a former project you developed or writing a tutorial, telling about the state of a version or sharing about an event you attended… And if you are lucky enough your post can be published on the Weekly Drop, the official Drupal newsletter!

Don't forget to reference your blog post on Planet Drupal, this platform is an aggregated list of feeds from around the web which shares relevant Drupal-related knowledge and information.

You can also find our Drupal related blog posts on the Liip blog.

Testing core and modules

Testing Drupal projects is necessary to make the platform stable and there are many things to test! If you have a technical background, you can help to review patches or to write unit tests.

For non-technical people, you can provide some feedback about usability of the administration interface that will help to improve the user experience. Follow the process to give a proper feedback.

Contributing to development

There are many ways to contribute code in core and “contrib” projects such as modules or themes.

You can first help to improve existing projects by submitted patches. This would be the natural thing to do when you work with a module and you notice a bug or a missing feature: search in the corresponding issue queue if the problem have been noticed before. If not, post a message explaining the issue and add a snippet of code if you found a potential fix. Then you can create a patch and submit it into the issue queue.

You can also contribute to new projects by creating your very own module or theme or create a sandbox for more experimental projects.

Attending events

The Drupal association organizes many events all around the world to promote the CMS and gather the community.

One of the biggest events are the Drupalcons. A Drupalcon gathers thousands of people and lasts about one week including 3 full days of conferences. These conferences cover many topics: site building, user experience, security, content authoring etc. You can also join sprints to contribute to Drupal projects and social events to meet the members of the community. Check out our report about DrupalCon Barcelona 2015!

“Drupal Dev Days” conferences occur once a year and gather developers to discuss and present topics technically relevant to the community. You can join sprints, intensive coding sessions and technical conferences.

You can also join DrupalCamps to meet your local community. These events last one or two days and focus on sharing knowledge amongst the community. You can attend conferences and sprints.

There are also many Drupal meetups which are free events happening in many cities in the world. Presentations and discussions finish around nice drinks and appetizers.

Sponsoring events

The community holds conventions and meetups in many countries and being a sponsor will not only help Drupal development but it will also enable you to be noticeable within the community. There are different levels of sponsorings that will offer from mentions on social media to advertising online and at the exhibition space of the event. All you have to do is getting in touch with the event organizers. By the way, Liip will sponsor the Drupal Mountain Camp in Davos this year!

Offering a donation

You can give donations to the Drupal association through the website in order to support drupal.org infrastructure and maintenance, worldwide events such as Drupalcons. The donations are either in Euros or Dollars.

You can also become a member of the Drupal Association for the same purpose, as an individual member or an organization member. The minimal fees are 15 Euros. Find more information about membership on drupal.org.

Conclusion

Drupal projects are constantly improving thanks to passionate volunteers who work on many subjects: development, documentation, marketing, events organization, supports… There is for sure a task that will suit you and it only takes small time commitment to make changes.

So join the great Drupal community and start getting involved!

Jan 05 2017
Jan 05

As a follow-up to my previous blog post about the usage of Migrate API in Drupal 8, I would like to give an example, how to import multilingual content and translations in Drupal 8.

Prepare and enable translation for your content type

Before you can start, you need to install the “Language” and “Content Translation” Module. Then head over to “admin/config/regional/content-language” and enable Entity Translation for the node type or the taxonomy you want to be able to translate.

As a starting point for setting up the migrate module, I recommend you my blog post mentioned above. To import data from a CVS file, you also need to install the migrate_source_csv module.

Prerequisites for migrating multilingual entities

Before you start, please check the requirements. You need at least Drupal 8.2 to import multilingual content. We need the destination option “translations”, which was added in a patch in Drupal 8.2. See the corresponding drupal.org issue here.

Example: Import multilingual taxonomy terms

Let’s do a simple example with taxonomy terms. First, create a vocabulary called “Event Types” (machine name: event_type).

Here is a simplified dataset:

Id Name Name_en 1 Kurs Course 2 Turnier Tournament

You may save this a csv file.

Id;Name;Name_en 1;Kurs;Course 2;Turnier;Tournament

Id;Name;Name_en

1;Kurs;Course

2;Turnier;Tournament

The recipe to import multilingual content

As you can see in the example data,  it contains the base language (“German”) and also the translations (“English”) in the same file.

But here comes a word of warning:

Don’t try to import the term and its translation in one migration run. I am aware, that there are some workarounds with post import events, but these are hacks and you will run into troubles later.

The correct way of importing multilingual content, is to

  1. create a migration for the base language and import the terms / nodes. This will create the entities and its fields.
  2. Then, with an additional dependent migration for each translated language, you can then add the translations for the fields you want.

In short: You need a base migration and a migration for every language. Let’s try this out.

Taxonomy term base language config file

In my example, the base language is “German”. Therefore, we first create a migration configuration file for the base language:

This is a basic example in migrating a taxonomy term in my base language ‘de’.

Put the file into <yourmodule>/config/install/migrate.migration.event_type.yml and import the configuration using the drush commands explained in my previous blog post about Migration API.

id: event_type label: Event Types source: plugin: csv # Full path to the file. Is overriden in my plugin path: public://csv/data.csv # The number of rows at the beginning which are not data. header_row_count: 1 # These are the field names from the source file representing the key # uniquely identifying each node - they will be stored in the migration # map table as columns sourceid1, sourceid2, and sourceid3. keys: - Id ids: id: type: string destination: plugin: entity:taxonomy_term process: vid: plugin: default_value default_value: event_type name: source: Name language: 'de' langcode: plugin: default_value default_value: 'de' #Absolutely necessary if you don't want an error migration_dependencies: {}

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

id: event_type

label: Event Types

source:

  plugin: csv

  # Full path to the file. Is overriden in my plugin

  path: public://csv/data.csv

  # The number of rows at the beginning which are not data.

  header_row_count: 1

  # These are the field names from the source file representing the key

  # uniquely identifying each node - they will be stored in the migration

  # map table as columns sourceid1, sourceid2, and sourceid3.

  keys:

    - Id

ids:

  id:

    type: string

destination:

  plugin: entity:taxonomy_term

process:

  vid:

   plugin: default_value

   default_value: event_type

  name:

    source: Name

    language: 'de'

  langcode:

    plugin: default_value

    default_value: 'de'

#Absolutely necessary if you don't want an error

migration_dependencies: {}

Taxonomy term translation migration configuration file:

This is the example file for the English translation of the name field of the term.

Put the file into <yourmodule>/config/install/migrate.migration.event_type_en.yml and import the configuration using the drush commands explained in my previous blog post about Migration API.

id: event_type_en label: Event Types english source: plugin: csv # Full path to the file. Is overriden in my plugin path: public://csv/data.csv # The number of rows at the beginning which are not data. header_row_count: 1 keys: - Id ids: id: type: string destination: plugin: entity:taxonomy_term translations: true process: vid: plugin: default_value default_value: event_type tid: plugin: migration source: id migration: event_type name: source: Name_en language: 'en' langcode: plugin: default_value default_value: 'en' #Absolutely necessary if you don't want an error migration_dependencies: required: - event_type

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

id: event_type_en

label: Event Types english

source:

  plugin: csv

  # Full path to the file. Is overriden in my plugin

  path: public://csv/data.csv

  # The number of rows at the beginning which are not data.

  header_row_count: 1

  keys:

    - Id

ids:

  id:

    type: string

destination:

  plugin: entity:taxonomy_term

  translations: true

process:

  vid:

    plugin: default_value

    default_value: event_type

  tid:

    plugin: migration

    source: id

    migration: event_type

  name:

    source: Name_en

    language: 'en'

  langcode:

     plugin: default_value

     default_value: 'en'

#Absolutely necessary if you don't want an error

migration_dependencies:

  required:

    - event_type

Explanation and sum up of the learnings

The key in the migrate configuration to import multilingual content are the following lines:

destination: plugin: entity:taxonomy_term translations: true

destination:

  plugin: entity:taxonomy_term

  translations: true

These configuration lines instruct the migrate module, that a translation should be created.

tid: plugin: migration source: id migration: event_type

tid:

  plugin: migration

  source: id

  migration: event_type

This is the real secret. Using the process plugin migration,  we maintain the relationship between the node and its translation.The wiring via the tid field make sure, that Migrate API will not create a new term with a new term id. Instead, the existing term will be loaded and the translation of the migrated field will be added. And thats exactly what we need!

Now go ahead and try to create a working example based on my explanation. Happy Drupal migrations!

Jan 04 2017
Jan 04

As a follow-up to my previous blog post about the usage of Migrate API in Drupal 8, I would like to give an example, how to import multilingual content and translations in Drupal 8.

Prepare and enable translation for your content type

Before you can start, you need to install the “Language” and “Content Translation” Module. Then head over to “admin/config/regional/content-language” and enable Entity Translation for the node type or the taxonomy you want to be able to translate.

As a starting point for setting up the migrate module, I recommend you my blog post mentioned above. To import data from a CSV file, you also need to install the migrate_source_csv module.

Prerequisites for migrating multilingual entities

Before you start, please check the requirements. You need at least Drupal 8.2 to import multilingual content. We need the destination option “translations”, which was added in a patch in Drupal 8.2. See the corresponding drupal.org issue here.

Example: Import multilingual taxonomy terms

Let's do a simple example with taxonomy terms. First, create a vocabulary called “Event Types” (machine name: event_type).

Here is a simplified dataset:

Id Name Name_en 1 Kurs Course 2 Turnier Tournament

You may save this a csv file.

Id;Name;Name_en
1;Kurs;Course
2;Turnier;Tournament

The recipe to import multilingual content

As you can see in the example data,  it contains the base language (“German”) and also the translations (“English”) in the same file.

But here comes a word of warning:

Don't try to import the term and its translation in one migration run. I am aware, that there are some workarounds with post import events, but these are hacks and you will run into troubles later.

The correct way of importing multilingual content, is to

  1. create a migration for the base language and import the terms / nodes. This will create the entities and its fields.
  2. Then, with an additional dependent migration for each translated language, you can then add the translations for the fields you want.

In short: You need a base migration and a migration for every language. Let's try this out.

Taxonomy term base language config file

In my example, the base language is “German”. Therefore, we first create a migration configuration file for the base language:

This is a basic example in migrating a taxonomy term in my base language ‘de'.

Put the file into <yourmodule>/config/install/migrate.migration.event_type.yml and import the configuration using the drush commands explained in my previous blog post about Migration API.

id: event_type
label: Event Types
source:
  plugin: csv
  # Full path to the file. Is overriden in my plugin
  path: public://csv/data.csv
  # The number of rows at the beginning which are not data.
  header_row_count: 1
  # These are the field names from the source file representing the key
  # uniquely identifying each node - they will be stored in the migration
  # map table as columns sourceid1, sourceid2, and sourceid3.
  keys:
    - Id
ids:
  id:
    type: string
destination:
  plugin: entity:taxonomy_term
process:
  vid:
   plugin: default_value
   default_value: event_type
  name:
    source: Name
    language: 'de'
  langcode:
    plugin: default_value
    default_value: 'de'

#Absolutely necessary if you don't want an error
migration_dependencies: {}

Taxonomy term translation migration configuration file:

This is the example file for the English translation of the name field of the term.

Put the file into <yourmodule>/config/install/migrate.migration.event_type_en.yml and import the configuration using the drush commands explained in my previous blog post about Migration API.

id: event_type_en
label: Event Types english
source:
  plugin: csv
  # Full path to the file. Is overriden in my plugin
  path: public://csv/data.csv
  # The number of rows at the beginning which are not data.
  header_row_count: 1
  keys:
    - Id
ids:
  id:
    type: string
destination:
  plugin: entity:taxonomy_term
  translations: true
process:
  vid:
    plugin: default_value
    default_value: event_type
  tid:
    plugin: migration
    source: id
    migration: event_type
  name:
    source: Name_en
    language: 'en'
  langcode:
     plugin: default_value
     default_value: 'en'

#Absolutely necessary if you don't want an error
migration_dependencies:
  required:
    - event_type

Explanation and sum up of the learnings

The key in the migrate configuration to import multilingual content are the following lines:

destination:
  plugin: entity:taxonomy_term
  translations: true

These configuration lines instruct the migrate module, that a translation should be created.

tid:
  plugin: migration
  source: id
  migration: event_type

This is the real secret. Using the process plugin migration,  we maintain the relationship between the node and its translation.The wiring via the tid field make sure, that Migrate API will not create a new term with a new term id. Instead, the existing term will be loaded and the translation of the migrated field will be added. And thats exactly what we need!

Now go ahead and try to create a working example based on my explanation. Happy Drupal migrations!

Nov 30 2016
Nov 30

Freitag logoOur latest site with Drupal Commerce 1.x went live in July 2016. It is Freitag. Since then we’ve been adding several new commerce related features. I feel it’s time to write a wrap-up. The site has several interesting solutions, this article will focus on commerce.

First a few words about the architecture. platform.sh hosts the site. The stack is Linux + nginx +  MySQL + PHP, the CMS is Drupal 7. Fastly caches http responses for anonymous users and also for authenticated users having no additional role (that is, logged-in customers). Authcache module takes care of lazy-loading the personalized parts (like the user menu and the shopping cart). Freitag has an ERP system to which we connect using the OCI8 PHP library. We write Behat and simpletest tests for QA.

We use the highly flexible Drupal Commerce suite. 23 of the enabled Freitag contrib modules have a name starting with ‘commerce’. We applied around 45 patches on them. Most of the patches are authored by us and 15 of them have already been committed. Even with this commitment to solve everything we could in an open-source way we wrote 30.000+ lines of commerce-related custom code. Still, in March 2016 Freitag was the 3rd largest Drupal customer contributor.

The words ‘product’ and ‘product variation’  I’ll be using throughout the article correspond to ‘product display node’ and ‘product’ in Drupal Commerce lingo.

ERP

ERP is the source of all products and product variations. We import this data into Drupal on a regular basis using Feeds. (Now I would use Migrate instead, it’s better supported and easier to maintain.) ERP also lets Drupal know about order status changes, sends the shipping tracking information and informs Drupal about products sent back to Freitag by customers.

There is data flowing in the opposite direction as well. ERP needs to know about all Drupal orders. Also, we create coupons in Drupal and send them to ERP too for accounting and other reasons.

Emails

We send commerce-related emails using the Message stack. This way we can have order-related tokens in our mails and we can manage and translate them outside the Rules UI. Mandrill takes care of the mail delivery.

Payment gateway

It was a client requirement to use the Swiss Datatrans payment gateway. However, at the time of starting  the project, Commerce Datatrans (the connector module on drupal.org) was in dev state and lacked several features we needed. Pressed for time we opted for buying a Datatrans Drupal module from a company offering this solution. It turned out to be a bad choice. When we discovered that the purchased module still does not cover all our needs and looked at the code we found that it was obfuscated and pretty much impossible to change. Also, the module could be used only on one site instance which made it impossible to use it on our staging sites.

We ended up submitting patches to the Commerce Datatrans module hosted on drupal.org. The module maintainer, Sascha Grossenbacher (the well-known Drupal 8 core contributor) helped us solving several issues and feature requests by reviewing our patches. This process has lead to a stable release of Commerce Datatrans with a dozen of feature improvements and bugfixes.

Additional to Datatrans we use Commerce Custom Offline Payments to enable offline store purchases by store staff and bank transfer payments.

Currencies

The site works with 7 different currencies, some of them having two different prices depending on the shipping country. Prices come from ERP and we store them in a field collection field on the product. We do not use the commerce_price field on the product variation.

Tax

Freitag ships to countries all around the world. VAT calculations are performed for EU, Switzerland, UK, Japan, South Korea and Singapore. To implement this functionality our choice fell on the commerce_vat module. Adding commerce_eu_vat and commerce_ch_vat released us from having to maintain VAT rates for EU and Switzerland ourselves. For the 3 Asian countries we implemented our own hook_commerce_vat_rate_info().

We have two different VAT rates for most of the countries. This is because usually a lower VAT rate applies to books. Drupal imports the appropriate VAT rate from the ERP with the product variation data. This information is handled by price calculation rules in Drupal.

Shipping

Freitag delivers its products using several shipping providers (like UPS, Swiss Post) all around the world. Most shipping providers have many shipping rates depending on the destination country, speed and shipped quantity (weight or volume). On checkout the customer can choose from a list of shipping services. This list needs to be compatible with the order.

We used rules to implement the shipping services in Drupal based on the Commerce Flat Rate module. For this end we trained our client to set up and maintain these rules themselves. It was not easy: shipping rules are daunting even for experienced commerce developers. First we needed to set up the “Profile Address” rules components. Then we configured the “Place of Supply” components. We applied these in turn in the condition part of the shipping rules components themselves.

The weakest point of any  implementation based on Rules is the maintenance. It’s not easy to find a specific rule after you created it. Having 250 rules components for only shipping made this feeling stronger.

The shipping line item receives the VAT rate of the product with the highest VAT rate in the order.

Coupons

Freitag has 6 different coupon types. They differ in who can create them, who and where (online/offline) can redeem them, whether partial redemption is possible, whether it’s a fixed amount or percentage discount and whether Freitag accounting needs to know about them or not.

Based on these criteria we came up with a solution featuring Commerce Coupon. Coupons can be discount coupons or giftcards. Giftcard coupons can only have a fixed value. Discount based coupons can also apply a percentage discount. The main difference between them is that customers can partially redeem giftcards, while discount-based coupons are for one-time use.

To make coupons work with VAT was quite tricky. (To make things simpler we only allowed one coupon per order.) Some coupon types work as money which means that from an accounting point of view they do not actually decrease the order total (and thus the VAT) but work as a payment method. Other coupon types however do decrease the order total (and thus the VAT). At the same time Drupal handles all coupons as line items with a negative price and the Drupal order total does decrease in either case.

The solution we found was to use Commerce proportional VATAxel Rutz maintains this brilliant little module and he does this in a very helpful and responsive manner. All the module does is adding negative VAT price components to coupon line items to account for VAT decrease. It decreases the order total VAT amounts correctly even if we have several different VAT rates inside the order.

Conclusion

Although there’s always room for increasing the complexity of the commerce part of the site (let’s find some use case for recurring payments!), it’s already the most complicated commerce site I’ve worked on.  For this Drupal Commerce provided a solid foundation that is pleasant to work with. In the end, Drupal enabled us to deliver a system that tightly integrates content and commerce.

I also would like to say thanks to Bojan Živanović from Commerce Guys who provided me with valuable insights on the legal aspects of tax calculation.

Nov 29 2016
Nov 29

Our latest site with Drupal Commerce 1.x went live in July 2016. It is Freitag. Since then we've been adding several new commerce related features. I feel it's time to write a wrap-up. The site has several interesting solutions, this article will focus on commerce.

First a few words about the architecture. platform.sh hosts the site. The stack is Linux + nginx +  MySQL + PHP, the CMS is Drupal 7. Fastly caches http responses for anonymous users and also for authenticated users having no additional role (that is, logged-in customers). Authcache module takes care of lazy-loading the personalized parts (like the user menu and the shopping cart). Freitag has an ERP system to which we connect using the OCI8 PHP library. We write Behat and simpletest tests for QA.

We use the highly flexible Drupal Commerce suite. 23 of the enabled Freitag contrib modules have a name starting with ‘commerce'. We applied around 45 patches on them. Most of the patches are authored by us and 15 of them have already been committed. Even with this commitment to solve everything we could in an open-source way we wrote 30.000+ lines of commerce-related custom code. Still, in March 2016 Freitag was the 3rd largest Drupal customer contributor.

The words ‘product' and ‘product variation'  I'll be using throughout the article correspond to ‘product display node' and ‘product' in Drupal Commerce lingo.

ERP

ERP is the source of all products and product variations. We import this data into Drupal on a regular basis using Feeds. (Now I would use Migrate instead, it's better supported and easier to maintain.) ERP also lets Drupal know about order status changes, sends the shipping tracking information and informs Drupal about products sent back to Freitag by customers.

There is data flowing in the opposite direction as well. ERP needs to know about all Drupal orders. Also, we create coupons in Drupal and send them to ERP too for accounting and other reasons.

Emails

We send commerce-related emails using the Message stack. This way we can have order-related tokens in our mails and we can manage and translate them outside the Rules UI. Mandrill takes care of the mail delivery.

Payment gateway

It was a client requirement to use the Swiss Datatrans payment gateway. However, at the time of starting  the project, Commerce Datatrans (the connector module on drupal.org) was in dev state and lacked several features we needed. Pressed for time we opted for buying a Datatrans Drupal module from a company offering this solution. It turned out to be a bad choice. When we discovered that the purchased module still does not cover all our needs and looked at the code we found that it was obfuscated and pretty much impossible to change. Also, the module could be used only on one site instance which made it impossible to use it on our staging sites.

We ended up submitting patches to the Commerce Datatrans module hosted on drupal.org. The module maintainer, Sascha Grossenbacher (the well-known Drupal 8 core contributor) helped us solving several issues and feature requests by reviewing our patches. This process has lead to a stable release of Commerce Datatrans with a dozen of feature improvements and bugfixes.

Additional to Datatrans we use Commerce Custom Offline Payments to enable offline store purchases by store staff and bank transfer payments.

Currencies

The site works with 7 different currencies, some of them having two different prices depending on the shipping country. Prices come from ERP and we store them in a field collection field on the product. We do not use the _commerceprice field on the product variation.

Tax

Freitag ships to countries all around the world. VAT calculations are performed for EU, Switzerland, UK, Japan, South Korea and Singapore. To implement this functionality our choice fell on the commerce_vat module. Adding commerce_eu_vat and commerce_ch_vat released us from having to maintain VAT rates for EU and Switzerland ourselves. For the 3 Asian countries we implemented our own _hook_commerce_vat_rateinfo().

We have two different VAT rates for most of the countries. This is because usually a lower VAT rate applies to books. Drupal imports the appropriate VAT rate from the ERP with the product variation data. This information is handled by price calculation rules in Drupal.

Shipping

Freitag delivers its products using several shipping providers (like UPS, Swiss Post) all around the world. Most shipping providers have many shipping rates depending on the destination country, speed and shipped quantity (weight or volume). On checkout the customer can choose from a list of shipping services. This list needs to be compatible with the order.

We used rules to implement the shipping services in Drupal based on the Commerce Flat Rate module. For this end we trained our client to set up and maintain these rules themselves. It was not easy: shipping rules are daunting even for experienced commerce developers. First we needed to set up the “Profile Address” rules components. Then we configured the “Place of Supply” components. We applied these in turn in the condition part of the shipping rules components themselves.

The weakest point of any  implementation based on Rules is the maintenance. It's not easy to find a specific rule after you created it. Having 250 rules components for only shipping made this feeling stronger.

The shipping line item receives the VAT rate of the product with the highest VAT rate in the order.

Coupons

Freitag has 6 different coupon types. They differ in who can create them, who and where (online/offline) can redeem them, whether partial redemption is possible, whether it's a fixed amount or percentage discount and whether Freitag accounting needs to know about them or not.

Based on these criteria we came up with a solution featuring Commerce Coupon. Coupons can be discount coupons or giftcards. Giftcard coupons can only have a fixed value. Discount based coupons can also apply a percentage discount. The main difference between them is that customers can partially redeem giftcards, while discount-based coupons are for one-time use.

To make coupons work with VAT was quite tricky. (To make things simpler we only allowed one coupon per order.) Some coupon types work as money which means that from an accounting point of view they do not actually decrease the order total (and thus the VAT) but work as a payment method. Other coupon types however do decrease the order total (and thus the VAT). At the same time Drupal handles all coupons as line items with a negative price and the Drupal order total does decrease in either case.

The solution we found was to use Commerce proportional VATAxel Rutz maintains this brilliant little module and he does this in a very helpful and responsive manner. All the module does is adding negative VAT price components to coupon line items to account for VAT decrease. It decreases the order total VAT amounts correctly even if we have several different VAT rates inside the order.

Conclusion

Although there's always room for increasing the complexity of the commerce part of the site (let's find some use case for recurring payments!), it's already the most complicated commerce site I've worked on.  For this Drupal Commerce provided a solid foundation that is pleasant to work with. In the end, Drupal enabled us to deliver a system that tightly integrates content and commerce.

I also would like to say thanks to  Bojan Živanović from Commerce Guys who provided me with valuable insights on the legal aspects of tax calculation.

Oct 24 2016
Oct 24

In this blog post I will present how, in a recent e-Commerce project built on top of Drupal7 (the former version of the Drupal CMS), we make Drupal7, SearchAPI and Commerce play together to efficiently retrieve grouped results from Solr in SearchAPI, with no indexed data duplication.

We used the SearchAPI and the FacetAPI modules to build a search index for products, so far so good: available products and product-variations can be searched and filtered also by using a set of pre-defined facets. In a subsequent request, a new need arose from our project owner: provide a list of products where the results should include, in addition to the product details, a picture of one of the available product variations, while keep the ability to apply facets on products for the listing. Furthermore, the product variation picture displayed in the list must also match the filter applied by the user: this with the aim of not confusing users, and to provide a better user experience.

An example use case here is simple: allow users to get the list of available products and be able to filter them by the color/size/etc field of the available product variations, while displaying a picture of the available variations, and not a sample picture.

For the sake of simplicity and consistency with Drupal’s Commerce module terminology, I will use the term “Product” to refer to any product-variation, while the term “Model” will be used to refer to a product.

Solr Result Grouping

We decided to use Solr (the well-known, fast and efficient search engine built on top of the Apache Lucene library) as the backend of the eCommerce platform: the reason lies not only in its full-text search features, but also in the possibility to build a fast retrieval system for the huge number of products we were expecting to be available online.

To solve the request about the display of product models, facets and available products, I intended to use the feature offered by Solr called Result-Grouping as it seemed to be suitable for our case: Solr is able to return just a subset of results by grouping them given an “single value” field (previously indexed, of course). The Facets can then be configured to be computed from: the grouped set of results, the ungrouped items or just from the first result of each group.

Such handy feature of Solr can be used in combination with the SearchAPI module by installing the SearchAPI Grouping module. The module allows to return results grouped by a single-valued field, while keeping the building process of the facets on all the results matched by the query, this behavior is configurable.

That allowed us to:

  • group the available products by the referenced model and return just one model;
  • compute the attribute’s facets on the entire collection of available products;
  • reuse the data in the product index for multiple views based on different grouping settings.

Result Grouping in SearchAPI

Due to some limitations of the SearchAPI module and its query building components, such plan was not doable with the current configuration as it would require us to create a copy of the product index just to apply the specific Result Grouping feature for each view.

The reason is that the features implemented by the SearchAPI Grouping are implemented on top of the “Alterations and Processors” functions of SearchAPI. Those are a set of specific functions that can be configured and invoked both at indexing-time and at querying-time by the SearchAPI module. In particular Alterations allows to programmatically alter the contents sent to the underlying index, while the Processors code is executed when a search query is built, executed and the results returned.
Those functions can be defined and configured only per-index.

As visible in the following picture, the SearchAPI Grouping module configuration could be done solely in the Index configuration, but not per-query.

SearchAPI: processor settings

Image 1: SearchAPI configuration for the Grouping Processor.

As the SearchAPI Grouping module is implemented as a SearchAPI Processor (as it needs to be able to alter the query sent to Solr and to handle the returned results), it would force us to create a new index for each different configuration of the result grouping.

Such limitation requires to introduce a lot of (useless) data duplication in the index, with a consequent decrease of performance when products are saved and later indexed in multiple indexes.
In particular, the duplication is more evident as the changes performed by the Processor are merely an alteration of:

  1. the query sent to Solr;
  2. the handling of the raw data returned by Solr.

This shows that there would be no need to index multiple times the same data.

Since the the possibility to define per-query processor sounded really promising and such feature could be used extensively in the same project, a new module has been implemented and published on Drupal.org: the SearchAPI Extended Processors module. (thanks to SearchAPI’s maintainer, DrunkenMonkey, for the help and review :) ).

The Drupal SearchAPI Extended Processor

The new module allows to extend the standard SearchAPI behavior for Processors and lets admins configure the execution of SearchAPI Processors per query and not only per-index.

By using the new module, any index can now be used with multiple and different Processors configurations, no new indexes are needed, thus avoiding data duplication.

The new configuration is exposed, as visible in the following picture, while editing a SearchAPI view under “Advanced > Query options”.
The SearchAPI processors can be altered and re-defined for the given view, a checkbox allows to completely override the current index setting rather than providing additional processors.

Drupal SearchAPI: view's extended processor settings

Image 2: View’s “Query options” with the SearchAPI Extended Processors module.

Conclusion: the new SearchAPI Extended Processors module has now been used for a few months in a complex eCommerce project at Liip and allowed us to easily implement new search features without the need to create multiple and separated indexes.
We are able to index Products data in one single (and compact) Solr index, and use it with different grouping strategies to build both product listings, model listings and model-category navigation pages without duplicating any data.
Since all those listings leverages the Solr FilterQuery query parameter to filter the correct set of products to be displayed, Solr can make use of its internal set of caches and specifically the filterCache to speed up subsequent searches and facets. This aspect, in addition to the usage of only one index, allows caches to be shared among multiple listings, and that would not be possible if separate indexes were used.

For further information, questions or curiosity drop me a line, I will be happy to help you configuring Drupal SearchAPI and Solr for your needs.

Oct 23 2016
Oct 23

In this blog post I will present how, in a recent e-Commerce project built on top of Drupal7 (the former version of the Drupal CMS), we make Drupal7, SearchAPI and Commerce play together to efficiently retrieve grouped results from Solr in SearchAPI, with no indexed data duplication.

We used the SearchAPI and the FacetAPI modules to build a search index for products, so far so good: available products and product-variations can be searched and filtered also by using a set of pre-defined facets. In a subsequent request, a new need arose from our project owner: provide a list of products where the results should include, in addition to the product details, a picture of one of the available product variations, while keep the ability to apply facets on products for the listing. Furthermore, the product variation picture displayed in the list must also match the filter applied by the user: this with the aim of not confusing users, and to provide a better user experience.

An example use case here is simple: allow users to get the list of available products and be able to filter them by the color/size/etc field of the available product variations, while displaying a picture of the available variations, and not a sample picture.

For the sake of simplicity and consistency with Drupal's Commerce module terminology, I will use the term “Product” to refer to any product-variation, while the term “Model” will be used to refer to a product.

Solr Result Grouping

We decided to use Solr (the well-known, fast and efficient search engine built on top of the Apache Lucene library) as the backend of the eCommerce platform: the reason lies not only in its full-text search features, but also in the possibility to build a fast retrieval system for the huge number of products we were expecting to be available online.

To solve the request about the display of product models, facets and available products, I intended to use the feature offered by Solr called Result-Grouping as it seemed to be suitable for our case: Solr is able to return just a subset of results by grouping them given an “single value” field (previously indexed, of course). The Facets can then be configured to be computed from: the grouped set of results, the ungrouped items or just from the first result of each group.

Such handy feature of Solr can be used in combination with the SearchAPI module by installing the SearchAPI Grouping module. The module allows to return results grouped by a single-valued field, while keeping the building process of the facets on all the results matched by the query, this behavior is configurable.

That allowed us to:

  • group the available products by the referenced model and return just one model;
  • compute the attribute's facets on the entire collection of available products;
  • reuse the data in the product index for multiple views based on different grouping settings.

Result Grouping in SearchAPI

Due to some limitations of the SearchAPI module and its query building components, such plan was not doable with the current configuration as it would require us to create a copy of the product index just to apply the specific Result Grouping feature for each view.

The reason is that the features implemented by the SearchAPI Grouping are implemented on top of the “ Alterations and Processors” functions of SearchAPI. Those are a set of specific functions that can be configured and invoked both at indexing-time and at querying-time by the SearchAPI module. In particular Alterations allows to programmatically alter the contents sent to the underlying index, while the Processors code is executed when a search query is built, executed and the results returned.

Those functions can be defined and configured only per-index.

As visible in the following picture, the SearchAPI Grouping module configuration could be done solely in the Index configuration, but not per-query.

SearchAPI: processor settings

Image 1: SearchAPI configuration for the Grouping Processor.

As the SearchAPI Grouping module is implemented as a SearchAPI Processor (as it needs to be able to alter the query sent to Solr and to handle the returned results), it would force us to create a new index for each different configuration of the result grouping.

Such limitation requires to introduce a lot of (useless) data duplication in the index, with a consequent decrease of performance when products are saved and later indexed in multiple indexes.

In particular, the duplication is more evident as the changes performed by the Processor are merely an alteration of:

  1. the query sent to Solr;
  2. the handling of the raw data returned by Solr.

This shows that there would be no need to index multiple times the same data.

Since the the possibility to define per-query processor sounded really promising and such feature could be used extensively in the same project, a new module has been implemented and published on Drupal.org: the SearchAPI Extended Processors module. (thanks to SearchAPI's maintainer, DrunkenMonkey, for the help and review :) ).

The Drupal SearchAPI Extended Processor

The new module allows to extend the standard SearchAPI behavior for Processors and lets admins configure the execution of SearchAPI Processors per query and not only per-index.

By using the new module, any index can now be used with multiple and different Processors configurations, no new indexes are needed, thus avoiding data duplication.

The new configuration is exposed, as visible in the following picture, while editing a SearchAPI view under “Advanced > Query options”.

The SearchAPI processors can be altered and re-defined for the given view, a checkbox allows to completely override the current index setting rather than providing additional processors.

Drupal SearchAPI: view's extended processor settings

Image 2: View's “Query options” with the SearchAPI Extended Processors module.

Conclusion: the new SearchAPI Extended Processors module has now been used for a few months in a complex eCommerce project at Liip and allowed us to easily implement new search features without the need to create multiple and separated indexes.

We are able to index Products data in one single (and compact) Solr index, and use it with different grouping strategies to build both product listings, model listings and model-category navigation pages without duplicating any data.

Since all those listings leverages the Solr cwiki.apache.org/confluence/display/solr/Common+Query+ParametersParameter text: FilterQuery query parameter) to filter the correct set of products to be displayed, Solr can make use of its internal set of caches and specifically the filterCache to speed up subsequent searches and facets. This aspect, in addition to the usage of only one index, allows caches to be shared among multiple listings, and that would not be possible if separate indexes were used.

For further information, questions or curiosity drop me a line, I will be happy to help you configuring Drupal SearchAPI and Solr for your needs.

Aug 22 2016
Aug 22

The Drupal accessibility initiative started with some advancements in Drupal 7 to ensure that Drupal core followed the World Wide Web Consortium (W3C) guidelines: WCAG 2.0 (Web Content Accessibility Guidelines) and ATAG 2.0 (Authoring Tool Accessibility Guidelines).
Many elements introduced in Drupal 7 were improved and bugs discovered through intensive testing were addressed and integrated to Drupal 8 core as well. Let’s take a tour of the accessibility in Drupal 8 !

Contrasts improved

Drupal’s accessibility maintainers improved contrasts in core themes so people that suffer from colorblindness are able to visit websites clearly. It is also good when visiting the website under bright sunlight, on mobile for instance.

A screenshot that compares Bartik headers in Drupal 7.43 and Drupal 8

Color contrasts in Bartik theme in Drupal 7.43 and Drupal 8.

See the related WCAG 2.0 section about contrasts.

Alternative texts for images

The alternative text for images is really useful for blind people who use screen readers. They can understand the meaning of an image through short descriptive phrases. This alternative text is now by default a required field in Drupal 8.

A screenshot showing that the alternative text is required when uploading an image in Drupal 8.

The alternative text for an image is required by default in Drupal 8 content edition.

See the related WCAG 2.0 section about alternative texts.

More semantics

Many accessibility improvements are hard to see as it involves semantics. Drupal 8 uses HTML5 elements in its templates which add more meaning into the code. For instance, assistive technology such as screen readers can now interpret elements like <header>, <footer> or <form>.

Moreover, WAI-ARIA (Web Accessibility Initiative – Accessible Rich Internet Applications) additional markup really improved semantics using:

  • landmarks to identify regions in a page, for instance: role="banner" ;
  • live regions to indicate that an element will be updated, for instance: aria-live="polite";
  • roles to describe the type of widgets presented, for instance: role="alert";
  • properties: attributes that represent a data value associated with the element.

See the related WCAG 2.0 sections about semantics.

Tabbing order

Drupal 8 introduces the TabbingManager javascript feature. It enables to constrain tabbing order on the page and facilitates navigation with keyboards. It is really helpful to guide a non-visual user to the most important elements on the page and minimize confusion with screen readers.

See the related WCAG 2.0 section about keyboard operations.

Forms

Drupal 8 accessibility involves many improvements regarding forms in Drupal 8.

In Drupal 7, all errors were displayed by default on top of the form and fields were highlighted in red. It was not right for colorblind people to understand where the errors were.
In Drupal 8, there is an experimental option to enable form inline errors and an error icon is displayed next to the field. Nevertheless, note that this feature is not enabled by default as there are still some pending issues.

Screenshot of a password field highlighted in red with an error message below "The specified passwords do not match"

The error message is displayed below the field when the Form Inline Error module is enabled.

See the related WCAG 2.0 sections about error identification.

Regarding the form API, radios and checkboxes are now embedded in fieldsets to meet WCAG compliance. Indeed, grouping related elements will help screen readers to navigate in complex forms. Plus, all fields have a label associated with the right element using the “for” attribute.

Here is an example of HTML code for radio buttons:

Poll status

See the related WCAG technical section about fieldsets

Tables and views

As Views UI module is in core now, it became accessible.
The views tables markup is more semantic. Data cells are associated with header cells through “id” and “headers” attributes. It is also possible to add a <caption> element to explain the purpose of the table and a <summary> element to give an overview on how the data is organized and how to navigate the table.
Plus, the “scope” attribute enables to explicitly mark row and column headings.

Here is an example of a table HTML code generated by a view:

Content type <a href="https://blog.liip.ch/admin/structure/types/manage/article">Article</a> <a href="https://blog.liip.ch/admin/structure/types/manage/article">Article</a>

Content type

<a href="/admin/structure/types/manage/article">Article</a>

<a href="/admin/structure/types/manage/article">Article</a>

Details for the table

Description for details

See the related WCAG section about tabular information.

Hidden elements

Using "display:none;" CSS styling can be problematic as it will hide elements for both visual and non-visual users and consequently, screen readers will not be able to read them.
Drupal 8 accessibility maintainers decided to standardize in the naming convention of HTML5 Boilerplate using different classes to hide elements:

  • hidden“: hide an element visually and from screen readers;
  • visually-hidden“: hide an element only visually but available for screen readers;
  • invisible“: hide an element visually and from screen readers without affecting the layout.

Aural alerts

Users with visual impairment will not be able to see all visual updates of the page such as color changes, animations or texts appended to the content. In order to make these changes apparent in a non-visual way, Drupal provides the Drupal.announce() JavaScript method which creates an “aria-live” element on the page. This way, text appended to the node can then be read by a screen reading user agent.

Here is an example of a code using the aural alert:

Drupal.announce('Please fill in your user name', 'assertive');

1

Drupal.announce('Please fill in your user name', 'assertive');

The first parameter is a string for the statement, the second is the priority:

  • polite“: this is the default, polite statements will not interrupt the user agent;
  • assertive“: assertive statements will interrupt any current speech.

See the related WCAG technical section about how to use live regions to identify errors.

CKEditor WYSIWYG accessibility

Drupal community helped improving CKEditor accessibility.
First of all, the WYSIWYG editor now comes with keyboard shortcuts which are beneficial for both power users and keyboard-only users.
Drupal 8 implements more semantic elements. For instance, the user can create HTML tables with headers, caption and summary elements. <figure> and <figcaption> HTML5 tags are also available to add captions to images.
Moreover, every image added through CKEditor are required by default, as it is on image fields.

CKEditor module also introduces a language toolbar button so that users can select a part of text and specify the language used. Screen readers will be able then to choose the appropriate language for each content.

See the related WCAG technical section about language attributes.

Finally, there is an accessibility checker plugin for CKEditor. It is not in core yet as a CKEditor issue blocks its integration, you can find more information on the related Drupal issue queue. However, you will find a module that implements it currently: ckeditor_a11checker.

All these options will definitely help users to generate accessible contents.

Conclusion

Drupal core maintainers accomplished great enhancements regarding accessibility in Drupal 8. These accessibility features will definitively be beneficial to keyboard-only users, low-vision users and colorblind people but will also be good for the usability and the SEO of your website.
Nevertheless, there is still work to be done to make Drupal 8 core fully accessible and Drupal needs contributors to tackle the remaining issues.

If you want to learn more about Drupal 8 accessibility, you can watch the presentation about “How Drupal 8 makes your website more easily accessible” given by Mike Gifford, one of the main accessibility core maintainer for Drupal.

Aug 21 2016
Aug 21

The Drupal accessibility initiative started with some advancements in Drupal 7 to ensure that Drupal core followed the World Wide Web Consortium (W3C) guidelines: WCAG 2.0 (Web Content Accessibility Guidelines) and ATAG 2.0 (Authoring Tool Accessibility Guidelines).

Many elements introduced in Drupal 7 were improved and bugs discovered through intensive testing were addressed and integrated to Drupal 8 core as well. Let's take a tour of the accessibility in Drupal 8 !

Contrasts improved

Drupal's accessibility maintainers improved contrasts in core themes so people that suffer from colorblindness are able to visit websites clearly. It is also good when visiting the website under bright sunlight, on mobile for instance.

A screenshot that compares Bartik headers in Drupal 7.43 and Drupal 8

Color contrasts in Bartik theme in Drupal 7.43 and Drupal 8.

See the related WCAG 2.0 section about contrasts.

Alternative texts for images

The alternative text for images is really useful for blind people who use screen readers. They can understand the meaning of an image through short descriptive phrases. This alternative text is now by default a required field in Drupal 8.

A screenshot showing that the alternative text is required when uploading an image in Drupal 8.

The alternative text for an image is required by default in Drupal 8 content edition.

See the related WCAG 2.0 section about alternative texts.

More semantics

Many accessibility improvements are hard to see as it involves semantics. Drupal 8 uses HTML5 elements in its templates which add more meaning into the code. For instance, assistive technology such as screen readers can now interpret elements like <header>, <footer> or <form>.

Moreover, WAI-ARIA (Web Accessibility Initiative – Accessible Rich Internet Applications) additional markup really improved semantics using:

See the related WCAG 2.0 sections about semantics.

Tabbing order

Drupal 8 introduces the TabbingManager javascript feature. It enables to constrain tabbing order on the page and facilitates navigation with keyboards. It is really helpful to guide a non-visual user to the most important elements on the page and minimize confusion with screen readers.

See the related WCAG 2.0 section about keyboard operations.

Forms

Drupal 8 accessibility involves many improvements regarding forms in Drupal 8.

In Drupal 7, all errors were displayed by default on top of the form and fields were highlighted in red. It was not right for colorblind people to understand where the errors were.

In Drupal 8, there is an experimental option to enable form inline errors and an error icon is displayed next to the field. Nevertheless, note that this feature is not enabled by default as there are still some pending issues.

Screenshot of a password field highlighted in red with an error message below "The specified passwords do not match"

The error message is displayed below the field when the Form Inline Error module is enabled.

See the related WCAG 2.0 sections about error identification.

Regarding the form API, radios and checkboxes are now embedded in fieldsets to meet WCAG compliance. Indeed, grouping related elements will help screen readers to navigate in complex forms. Plus, all fields have a label associated with the right element using the “for” attribute.

Here is an example of HTML code for radio buttons:

Poll statusClosedActive

See the related WCAG technical section about fieldsets

Tables and views

As Views UI module is in core now, it became accessible.

The views tables markup is more semantic. Data cells are associated with header cells through “id” and “headers” attributes. It is also possible to add a <caption> element to explain the purpose of the table and a <summary> element to give an overview on how the data is organized and how to navigate the table.

Plus, the “scope” attribute enables to explicitly mark row and column headings.

Here is an example of a table HTML code generated by a view:

Content type

<a href="https://www.liip.ch/admin/structure/types/manage/article">Article</a> <a href="https://www.liip.ch/admin/structure/types/manage/article">Article</a>

<table class="views-table views-view-table cols-2">Caption for the table Details for the tableDescription for details
  <tbody>
    <tr>
      <th id="view-title-table-column" class="views-field views-field-title" scope="col">Title</th>
    </tr>
    <tr>
      <td class="views-field views-field-title" headers="view-title-table-column">Premo Quae Vero</td>
      <td class="views-field views-field-title" headers="view-title-table-column">Capto Dolor</td>
    </tr>
  </tbody>
</table>

See the related WCAG section about tabular information.

Hidden elements

Using "display:none;" CSS styling can be problematic as it will hide elements for both visual and non-visual users and consequently, screen readers will not be able to read them.

Drupal 8 accessibility maintainers decided to standardize in the naming convention of HTML5 Boilerplate using different classes to hide elements:

  • hidden“: hide an element visually and from screen readers;
  • visually-hidden“: hide an element only visually but available for screen readers;
  • invisible“: hide an element visually and from screen readers without affecting the layout.

Aural alerts

Users with visual impairment will not be able to see all visual updates of the page such as color changes, animations or texts appended to the content. In order to make these changes apparent in a non-visual way, Drupal provides the Drupal.announce() JavaScript method which creates an “aria-live” element on the page. This way, text appended to the node can then be read by a screen reading user agent.

Here is an example of a code using the aural alert:

Drupal.announce('Please fill in your user name', 'assertive');

The first parameter is a string for the statement, the second is the priority:

  • polite“: this is the default, polite statements will not interrupt the user agent;
  • assertive“: assertive statements will interrupt any current speech.

See the related WCAG technical section about how to use live regions to identify errors.

CKEditor WYSIWYG accessibility

Drupal community helped improving CKEditor accessibility.

First of all, the WYSIWYG editor now comes with keyboard shortcuts which are beneficial for both power users and keyboard-only users.

Drupal 8 implements more semantic elements. For instance, the user can create HTML tables with headers, caption and summary elements. <figure> and <figcaption> HTML5 tags are also available to add captions to images.

Moreover, every image added through CKEditor are required by default, as it is on image fields.

CKEditor module also introduces a language toolbar button so that users can select a part of text and specify the language used. Screen readers will be able then to choose the appropriate language for each content.

See the related WCAG technical section about language attributes.

Finally, there is an accessibility checker plugin for CKEditor. It is not in core yet as a CKEditor issue blocks its integration, you can find more information on the related Drupal issue queue. However, you will find a module that implements it currently: ckeditor_a11checker.

All these options will definitely help users to generate accessible contents.

Conclusion

Drupal core maintainers accomplished great enhancements regarding accessibility in Drupal 8. These accessibility features will definitively be beneficial to keyboard-only users, low-vision users and colorblind people but will also be good for the usability and the SEO of your website.

Nevertheless, there is still work to be done to make Drupal 8 core fully accessible and Drupal needs contributors to tackle the remaining issues.

If you want to learn more about Drupal 8 accessibility, you can watch the presentation about “ How Drupal 8 makes your website more easily accessible” given by Mike Gifford, one of the main accessibility core maintainer for Drupal.

Aug 17 2016
Aug 17

Last week Drupalaton 2016 took place. With about 150 registrations this was the largest Drupalaton so far. The organizers did an amazing job in coping with this mass. There were two session threads and a sprint room. Of the many interesting presentations I would like to mention Fabian Bircher’s “Configuration Management: theory and practice” (a must for everyone who gets lost while trying to work in a team on a Drupal8 project) , Pieter Frenssen’s “Working with REST APIs”  (it was good to see how simple it is in Drupal8) and “Drupal 8 Media” from Pónya Péter, Szanyi Tamás and Rubén Teijeiro (seems we have a huge step forward in media handling since Drupal7!). I held a session on caching in Drupal 8 which was the shortened version the one I did on Drupal Developer Days in Milan.

Liip was a silver sponsor of the event.

Finally, some pictures on the Friday ship cruise. Thanks to Brainsum for sponsoring it!

Aug 16 2016
Aug 16

Last week Drupalaton 2016 took place. With about 150 registrations this was the largest Drupalaton so far. The organizers did an amazing job in coping with this mass. There were two session threads and a sprint room. Of the many interesting presentations I would like to mention Fabian Bircher's “Configuration Management: theory and practice” (a must for everyone who gets lost while trying to work in a team on a Drupal8 project) , Pieter Frenssen's “Working with REST APIs”  (it was good to see how simple it is in Drupal8) and “Drupal 8 Media” from Pónya Péter, Szanyi Tamás and Rubén Teijeiro (seems we have a huge step forward in media handling since Drupal7!). I held a session on caching in Drupal 8 which was the shortened version the one I did on Drupal Developer Days in Milan.

Liip was a silver sponsor of the event.

Finally, some pictures on the Friday ship cruise. Thanks to Brainsum for sponsoring it!

Jul 11 2016
Jul 11

After having contributed to the official styleguide of the Swiss Federal Government and having implemented it on a couple of websites, we decided to go further and bring these styleguide into a theme for Drupal, a well-known, pluripotent and robust CMS we implement regularly at Liip.

Screenshot of Drupal theme for the Swiss Confederation

The current result is a starterkit providing the essential bricks to start in a snap a website project for the federal government running with Drupal 8, based on the version 3 of the official styleguide.

Navigation modules, multilingual environnement per default (German, French, Italian, Rumantch and English), responsive layout following the Web Content Accessibility Guidelines, we threw the fundamental stones for bootstraping a web platform for the Confederation.

con~foederatio : to build a league, together.

In other words, joining forces, to support a common cause. From the very start of the project we decided to opensource the code, as a participatory initiative.
Learn more about this intent.

Any developer working on a new website for the swiss government can now quickly start developing with this Drupal starterkit, then modify, contribute and improve it collegially. Pulling requests and opening issues on GitHub is the recommended way to help us extend further the project.

What’s inside the box

The Bund-Starterkit provides theme and elements based on the official styleguide (version 3.0.0) of the Swiss Federal Administration.

This starterkit also contains a base to quickly implement a website running on Drupal 8 for the Swiss Federal Administration. Currently, it provides the following Drupal and frontend elements:

  • Multilingual main navigation blocks
  • Multilingual service navigation blocks
  • Multilingual footer service navigation blocks
  • Logo block
  • Language switcher block with German, French, Italian, Rumantsch enabled
  • All the assets (CSS, SASS. JS files) provided by the official styleguide
  • A ready-to-use SASS workflow

Installation process, an overview

Please check the Readme file to quickly start your project. But let’s have a look at the details of the installation process. First of all, Composer (a PHP dependencies manager) is binding together for us the following repositories:

After downloading the sources with Composer and setting your vhost and hosts files, you have two options. Continuing with a few drush commands to run the Drupal installation process, or following the installation wizard in the browser. If you choose this last option, don’t forget to select the «Bund profile» option when the wizard ask you to choose a profile:

Chose a profile for the Drupal theme for the Swiss Confederation

Continue with the last steps of the wizard and that’s it. you should be able to see an empty Drupal 8 website, painted with the swiss administration’s corporate sauce.

Inserting menus content

With the help of a .CSV file and some drush commands, you can quickly import your menu structure. Once done, create and assign your content the the freshly created menu items through the Drupal administration interface.

Theming

Don’t forget to create a personal Drupal sub-theme from the bund_drupal_starterkit_theme, as a Drupal best practice.  Don’t edit the existing theme directly or you could loose your changes after a future update.

Frontend

This starterkit use the official styleguide (version 3.0.0) as a submodule. All existing CSS/JS files and assets are imported and available per default, but not necessary integrated as a drupal module at the moment. We highly encourage you to check the official styleguide before adding any new CSS style or JS files to your project. Based on the existing styles, it should be possible to create a lot of Drupal templates without modifying or extending any CSS. And as already said, we invite you to share any Drupal template matching the styleguide you would develop for your project.

Further reading

Jul 10 2016
Jul 10

After having contributed to the official styleguide of the Swiss Federal Government and having implemented it on a couple of websites, we decided to go further and bring these styleguide into a theme for Drupal, a well-known, pluripotent and robust CMS we implement regularly at Liip.

After having contributed to the official styleguide of the Swiss Federal Government and having implemented it on a couple of websites, we decided to go further and bring these styleguide into a theme for Drupal, a well-known, pluripotent and robust CMS we implement regularly at Liip.

Screenshot of Drupal theme for the Swiss Confederation

The current result is a starterkit providing the essential bricks to start in a snap a website project for the federal government running with Drupal 8, based on the version 3 of the official styleguide.

Navigation modules, multilingual environnement per default (German, French, Italian, Rumantch and English), responsive layout following the Web Content Accessibility Guidelines, we threw the fundamental stones for bootstraping a web platform for the Confederation.

con~foederatio : to build a league, together.

In other words, joining forces, to support a common cause. From the very start of the project we decided to opensource the code, as a participatory initiative.

Learn more about this intent.

Any developer working on a new website for the swiss government can now quickly start developing with this Drupal starterkit, then modify, contribute and improve it collegially. Pulling requests and opening issues on GitHub is the recommended way to help us extend further the project.

What's inside the box

The Bund-Starterkit provides theme and elements based on the official styleguide (version 3.0.0) of the Swiss Federal Administration.

This starterkit also contains a base to quickly implement a website running on Drupal 8 for the Swiss Federal Administration. Currently, it provides the following Drupal and frontend elements:

  • Multilingual main navigation blocks
  • Multilingual service navigation blocks
  • Multilingual footer service navigation blocks
  • Logo block
  • Language switcher block with German, French, Italian, Rumantsch enabled
  • All the assets (CSS, SASS. JS files) provided by the official styleguide
  • A ready-to-use SASS workflow

Installation process, an overview

Please check the Readme file to quickly start your project. But let's have a look at the details of the installation process. First of all, Composer (a PHP dependencies manager) is binding together for us the following repositories:

bund_drupal_starterkit_dummycontent

After downloading the sources with Composer and setting your vhost and hosts files, you have two options. Continuing with a few drush commands to run the Drupal installation process, or following the installation wizard in the browser. If you choose this last option, don't forget to select the «Bund profile» option when the wizard ask you to choose a profile:

Chose a profile for the Drupal theme for the Swiss Confederation

Continue with the last steps of the wizard and that's it. you should be able to see an empty Drupal 8 website, painted with the swiss administration's corporate sauce.

Inserting menus content

With the help of a .CSV file and some drush commands, you can quickly import your menu structure. Once done, create and assign your content the the freshly created menu items through the Drupal administration interface.

Theming

Don't forget to create a personal Drupal sub-theme from the bund_drupal_starterkit_theme, as a Drupal best practice.  Don't edit the existing theme directly or you could loose your changes after a future update.

Frontend

This starterkit use the official styleguide (version 3.0.0) as a submodule. All existing CSS/JS files and assets are imported and available per default, but not necessary integrated as a drupal module at the moment. We highly encourage you to check the official styleguide before adding any new CSS style or JS files to your project. Based on the existing styles, it should be possible to create a lot of Drupal templates without modifying or extending any CSS. And as already said, we invite you to share any Drupal template matching the styleguide you would develop for your project.

Further reading

Jul 07 2016
Jul 07

Eight months ago Drupal 8.0.0 was released. Exciting news for drupalists. Since then comparing D8’s features to its predecessor is a topic in daily business. «Can drupal 8 do what we can do now with 7 today?”. After playing around with D8 i get the feeling some crucial features are missing. Dries invited people to tell ”why we not use or migrate to drupal 8” – and got a clear answer: A majority of drupalist (60%) are waiting for certain modules. So the follow up question would be what are these modules.

On the fly my top 10 wishlist would be:

  • pathauto
  • token
  • webform
  • metadata
  • views_bulk_operations
  • flag
  • rules
  • xmlsitemap
  • redirect
  • search_api

Today it seems quite difficult to get a good overview of D8 ecosystem. Also because some module development moved to github to have a better collaboration tool. I was irritated to see no D8 version of the webform module in the download section on drupal.org  – That’s a module with 1/2 million downloads for D7. Comments on this issue gives some answers. Without committed maintainers from the beginning the porting takes much longer. A highly complex module like webform probably needs almost complete rewrite to fit into the new core of D8. Porting module from D7 to D6 was much easier. For forms we could use in some cases the core Form API, core contact forms or the eform module. But our clients would most likely miss out on the experience of D7s webform module.

Under the hood Drupal 8 core changed significantly. Symfony2 for example is now playing its music to give us new possibilities. I guess in some cases there are new solutions we have yet to discover. From a suitebuilder point of view, D8 is delightfully similar to what we know from D7. However, it will take some getting used to not trying to add the old modules we know to this new architecture.

In the end the importance of a variety of mature modules that play together nicely is crucial when it comes to efficiency, maintainability and stability of a project…

“I am confident that Drupal 8 will be adopted at “full-force” by the end of 2016.”
Dries Buytaert

Drupal is a registered trademark of Dries Buytaert.

Jul 06 2016
Jul 06
  • 07 July 2016
  • Lennart Jegge

Eight months ago Drupal 8.0.0 was released. Exciting news for drupalists. Since then comparing D8's features to its predecessor is a topic in daily business. «Can drupal 8 do what we can do now with 7 today?”. After playing around with D8 i get the feeling some crucial features are missing. Dries invited people to tell ” why we not use or migrate to drupal 8” – and got a clear answer: A majority of drupalist (60%) are waiting for certain modules. So the follow up question would be what are these modules.

On the fly my top 10 wishlist would be:

  • pathauto
  • token
  • webform
  • metadata
  • views_bulk_operations
  • flag
  • rules
  • xmlsitemap
  • redirect
  • search_api

Today it seems quite difficult to get a good overview of D8 ecosystem. Also because some module development moved to github to have a better collaboration tool. I was irritated to see no D8 version of the webform module in the download section on drupal.org  – That's a module with 1/2 million downloads for D7. Comments on this issue gives some answers. Without committed maintainers from the beginning the porting takes much longer. A highly complex module like webform probably needs almost complete rewrite to fit into the new core of D8. Porting module from D7 to D6 was much easier. For forms we could use in some cases the core Form API, core contact forms or the eform module. But our clients would most likely miss out on the experience of D7s webform module.

Under the hood Drupal 8 core changed significantly. Symfony2 for example is now playing its music to give us new possibilities. I guess in some cases there are new solutions we have yet to discover. From a suitebuilder point of view, D8 is delightfully similar to what we know from D7. However, it will take some getting used to not trying to add the old modules we know to this new architecture.

In the end the importance of a variety of mature modules that play together nicely is crucial when it comes to efficiency, maintainability and stability of a project…

“I am confident that Drupal 8 will be adopted at “full-force” by the end of 2016.”

Dries Buytaert

Drupal is a registered trademark of Dries Buytaert.

Lennart Jegge

Software Developer, Drupalista

Related services
  • Topics
  • Tags

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web