Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Jun 23 2020
Jun 23

With a simplified translation flow with a balanced use of both human and machine translation, you can reduce your costs and reach even more markets.

In this blog, we’ll further explore a few user stories (introduced in part 1 of this series) and how we resolved their translation needs with the help of the Translation Management (TMGMT) module.

We will see how to:

  • Simplify and accelerate the translation process to empower content editors

  • Delegate the translation task to a machine and human translation to feed other translation systems (like e.g. Trados or Memsource)

  • Prevent data loss or duplication by choosing the content model that fits your expectations, from the beginning

  • Identify possible translation processes in a publishing workflow

  • Set a deadline, word count, and allow non-Drupal users to receive translation files by mail


These user stories can be grouped into three topics: Content Moderation, Paragraphs translation, and UX experiments. The implementation described below is Drupal 9 ready.

Content moderation
 

The very basic requirement for a “translation flow” would be to mark the translation as outdated.
This is working perfectly fine for non-moderated content, and there is also work in progress in Drupal 9.1 to set this feature back for moderated content.

Translation-outdated

It could also be a valid use case to set a state for a translation while making use of a publication workflow (e.g. content moderation). So if the source is published, it might not necessarily be the case for the translation.

There is work in progress with TMGMT to support pending revisions and accept translation as a specific moderation state, so this option is available while doing a Job review.

Content-moderation-translation-state

Additionally, we are experimenting with the following features with the TMGMT Content Moderation module:

  • Display the current moderation state close to the published status in the content translate form

  • Enable translation operation only when the source content reaches a specific state (example: “Published” state)

  • Exclude states from the translated entity (example: “For translation” state)

  • Redirect on Job completion to the latest revision of the entity


Combined with the Moderation State Columns module, it can also produce this kind of view.

Content Dashboard

Paragraphs asymmetric translation


Paragraphs can be configured to have asymmetric translations. In this case, the structure from the source entity can differ from the translations. It also allows to not have a translation fallback to the source on the frontend.

Example:

  • English source

    • Paragraph text 1 EN

    • Paragraph text 2 EN

  • French translation

    • Paragraph text 1 FR


While working with this setup, we need to be aware of several possible issues.

  • Data “loss” and dangling references can occur

    With existing content, switching to and from symmetric to asymmetric can cause data to not appear in the backend or the frontend, and then produce dangling references as the Paragraph entity ID will or will not be the same depending on the chosen setup. So this needs to be taken carefully into account before giving access to the content editors.



     
  • While using TMGMT, the flow might not be the expected one while doing several translations

    Let’s continue with our minimal example.



     
  • For our English source, the first edit is
    • Paragraph text 1 EN

    • Paragraph text 2 EN



  • A first French translation Job via TMGMT produces the expected result
    • Paragraph text 1 FR

    • Paragraph text 2 FR



  • A second edit of the English source adds a 3rd paragraph, so we have
    • Paragraph text 1 EN

    • Paragraph text 2 EN

    • Paragraph text 3 EN


We translate the same content again with a TMGMT Job: the 3rd paragraph appears in the Job review, but it will never appear on the frontend nor while editing the translation. This is probably not expected. If it is still accepted in the flow, it consumes unnecessary translation resources.

If we manually edit the French translation, to say remove the 2 originally translated paragraphs then add 2 others, we end up with

  • Paragraph text 4 FR

  • Paragraph text 5 FR


If we translate again the source at this stage, the situation between the TMGMT Job and the frontend will become quite unclear.

  • Work in progress

    We are close to a solution but the integration of TMGMT with Paragraphs asymmetric translation (including content moderation) is still a work in progress.


Conclusion:
Paragraphs asymmetric integration with TMGMT is feasible but if you plan to translate the content several times/update translations with TMGMT it will most likely not be the right fit for content editors.

UX experiments


A client requested to adapt the TMGMT translation flow for content editors. The assumption was that the Job review will be done right after the translation, so it allowed us to propose a simplification of the UI and skip some steps. A similar request came from another client then we’ve decided to start a proof of concept and gather some of these requirements to see how they could be generalised.

Here are a few examples of alternate flows and simplified UX that could be used by roles that do not need the whole TMGMT stack.

Original TMGMT flow: Machine translation with DeepL

[embedded content]

The alternate flows described below are combined with the TMGMT Content Moderation features that were previously mentioned.

Alternate flow 1: Machine translation with DeepL

Once a source reaches the translatable state (Published in this case), the translation occurs in 3 steps: Create the Job > Review the translation > View the result as the Latest revision.
The second step could even be skipped by accepting translations without review.

[embedded content]

Alternate flow 2: Send a file by mail to a translation service

Here we presume that we can send the XLF file attached.

[embedded content]

 

 


Combined flow

 

In some cases, we can also combine the 2 flows and first do a machine translation via e.g. DeepL, and then export the result in the XLF output to populate the initial translation in another translation solution (like Trados or Memsource).

List of other features provided by the Simple TMGMT module:

  • Translate with the usual translate operations

  • Disable operations links when the user selects multiple languages

  • Limit operation links to the supported translators

  • Optionally disable translation once already translated (edit only and do not translate again via a Job)

  • Per content translation flag to keep track of automatic translation

  • Optionally add a delivery date. A default delivery date is calculated based on a certain amount of open days

  • Integration with the Swift Mailer module (HTML mail templates with attachments)

  • Integration with the Disable Language module (filter languages with permissions)

  • If a Job is left as unprocessed allow to continue or delete it via the translation form

  • On Job delete or Job item delete, redirect to the entity translation form

  • Re-route machine translation error messages to a specific email address (e.g. [email protected]) and simplify the error messages for content editors


Next steps
 

  • With the deadline, word count, and non-Drupal users in the flow, it opens the door to integration with other information systems and stakeholders. For example, a CRM might contain your translator contact details and translation skills while an accounting platform will get the cost reporting once the translation Job is finished.

  • Introduce a generic way to deal with notifications, as suggested in this issue.


If you need help with any of the above including adding translation functionality to your site - get in touch with us today!

May 27 2020
May 27

In this series about building multilingual sites, we will use Drupal as our central content management system. In this blog, we will dive into how we can implement translation for different features based on defined user stories with Drupal core and contributed modules.

Getting started: elaborate on a process
 

Having a good translation strategy starts at the early beginning of a project. Here are a few topics that could help to elaborate a process.

The translation strategy could be approached in several ways depending on the project:

With a minimum viable product (MVP) or a new project build

  • Fully translated, with several languages, for the first release

  • A proof of concept in a single initial language

  • A subset of languages first, then add more languages at a later phase

  •  ...



With upcycling an existing project or doing a project migration

  • The source and destination content model might not be the same

  • Some translations could be partial or outdated

  • ...


For both content creation and update, we should be able to answer the following questions:

  • Who are the personas? They could be represented by Drupal roles: content editor, translator, reviewer, publisher, …



  • What are their relations with the translation system, which features can help their work?
  • How will the content and features access be handled?



  • How will the content flow between personas, is there a publication workflow? Example: when the source is ready for translation is a simple workflow enough (translation review then publication) or do we need a more elaborate process that allows back and forth between reviews?

  • Are revisions required? Would it be possible to roll back a previous revision and how does it affect translations? Also what happens when the source is updated?

  • What are the relations between languages? Are we able to translate from another language than the source?



  • For locale, how can I make use of a first contributed version then override it, handle change and provide flexibility when more context is needed?



  • What are the available resources? In house translators, one or several specific translation agencies that might be business-specific, automated translation services? How do we want to deal with deadlines and scheduled publications?



  • How does it integrate with third parties (if any)? Translators could be part of a CRM or translation jobs might need to be reported in an accounting system.



  • What is the translation scope? It could be frontend and/or backend, do we want URL aliases to be translated?



  • Do we have to deal with external entities (e.g. content being imported and updated from an external service)?



  • Do we need a different structure between the source and the translation?



  • How does the translation fallback behave? Do we want to display the source, display a message, redirect, …?


Drupal multilingual concepts


Let’s see what Drupal brings out of the box. Here is a quick recap of the Drupal translation concepts.

  • Interface translation: also known as “locale”, it can be handled via Gettext / .po files. It includes strings passed through the t() function or trans Twig tag, these strings are also contained in Yaml files (module name in the info file, schema label, title in routing and annotations. They are not synced between environments by default, but their import can be automated.
Example: the label of a button in a Twig template



  • Configuration entities: core, contributed and custom modules are providing them via their schema. They can be synced via the configuration export/import between environments.
Example: strings set for Views, Blocks, settings, …



  • Content entities: their blueprints are made from configuration. They are specific to each environment by default.
Examples: Nodes, Terms, Users, Media, ...


Expectations


With the Drupal capabilities in mind, we could continue by refining some parts of our process as common user stories, by project stakeholders. Then see how it can be covered with the core or contributed ecosystem. Here are some examples of possible expectations.

As a Product Owner, I can

  • have an overview of all the translations and their status

  • access each node translation without having the admin UI translated

  • edit translations that are defined by code


As a Translator, I can

  • be notified of new translation requests without having to access a Drupal site

  • receive structured files like .xlf or .po that I can use with other translation solutions than Drupal


As a Translation Reviewer, I can

  • let know the Publisher that a translation is ready for publication


As a Site Builder, I can

  • limit the edit / view access to several languages to translate content before publishing it, as a whole for a language

  • limit the access to translation languages, by user


As a Developer, I can

  • add metadata to locale by specifying context related strings

  • have support for plural

  • update locales between environments

  • extract translations from custom code

  • make sure that every string is translatable

  • migrate content with translations and map them to a new content model


Translations in the wild


Content translation will be developed in the second part of this series, so let’s see first how some of these user stories can be covered by the contributed ecosystem and which tools are available for developers.

Drupal core (>= 8, 9)
 

Each core concept is basically covered by its own module with the Language module as the common requirement: Configuration Translation, Content Translation, Interface Translation.



Make the source language editable

Since 8.5.0, the Interface translation module provides an easy way to enable interface translation for the site default language. Given that English is the default language: edit the English language and check “Enable interface translation to English”. In a sense, it can be regarded as a successor of String Overrides.



Migrate

Multilingual migrations are stable (8.9.0, 9.0.0). Migrate Drupal Multilingual module is no longer required.

Translation context


Context can be used to provide or identify variations of the same string.
So it can also be used to group translations for business-specific translations (e.g. custom code) that might not fall in the scope of contributed translations by adding metadata.

This issue aims to provide context filtering in core (> 9.x) so it will replace modules like Locale Translation Context.

Examples of context definition

  • PHP $this->t('May', [], ['context' => 'Long month name']);

  • Twig <span>{% trans 'May' with {'context': 'Long month name'} %}</span>

  • Javascript Drupal.t('May', {}, {context: 'Long month name'});

  • PHP annotation label = @Translation("Month", context="Date"),


POEdit with context

The core API

The core provides great services to deal with translations, like 

Contributed solutions
 

Translation Management Tool (TMGMT)

TMGMT facilitates the translation process and really shines if you need to export the data structure with the content while outsourcing a translation Job.
It supports a large variety of translation service providers (Lionbridge, DeepL, Google, ...) and provides support for content, configuration and interface translation. These 3 sources are displayed as an overview, so it replaces solutions like Translation Overview.

The beauty of this unified solution is that you can outsource translations with any of the provider (like DeepL), and e.g. create a Job that groups each locale string as Job items.

Translation Extractor (POTX)

The Drupal core interface translation export (admin/config/regional/translate/export) will export basically everything that is available in the interface translation UI. This module provides

  • string extraction from code, instead of strings that have previously been registered

  • filtered extractions as .pot (template) or .po files by language with optional inclusion of the translations.


The Drupal 8 version is still a working in progress, make sure to review these issues:
String context not taken into account when retrieving a translation and Plural translation values are not exported.

TMGMT Source

Disable Language
Filters out disabled languages from the language switcher and sitemap redirects users that don’t have the permission to view disabled languages and much more.

Allowed Languages
Set restrictions on which content a user can edit by language.

Administration Language Negotiation
Its main use case is to allow displaying the frontend of the site in one language and still keep most of the backend in English (or another language of your choice).

Since 8.6.4, this behaviour can still be partially achieved for the backend, with core configuration, by using the Account administration pages as a detection method for interface text.

Paragraphs Asymmetric Translation Widgets
While using Paragraphs, it could be required to have a different structure between the source and the translations. Make sure to carefully test it if you switch from Paragraphs default widgets as data loss might occur if you already have translations. Also, there might be some limitations while using TMGMT.

Drush
Drush supports locale check, update and import (see https://localize.drupal.org/translate/drupal8)

Quick tips


Having an overview of the content model (entities and relations) can help to answer accurately some site-building questions:

  • which entities are the subject of a content moderation

  • which ones are translatable (entity and field level)

  • which tools might answer the requirements (Paragraphs, Blocks, …)

  • how does it integrate with site search (e.g. while using Search API)


Troubleshooting common errors:

  • Interface: Check if the string is wrapped in a t() function or trans tag. Reduce duplicates early (interface translation strings are case sensitive).

  • Content: check the translatability of your entities and fields, the Content language UI is a great tool for this (/admin/config/regional/content-language)

  • Configuration: the schema or the .config_translation.yml files might be missing.


Making use of context early for business-specific translations might help later to include translation metadata to export or filter.

Resources
 

More to come on this topic
 

In the next blog, we'll adapt our content translation flow and delegate this to the Translation Management Tool (TMGMT), where we will tackle different approaches to solving complex issues such as using content moderation for symmetric and asymmetric translation.
 
Finally, we'll finish off the series with a third instalment to look into how we've approached and solved a complex Gatsby multilingual integration with Drupal 7, and how this can be applied to a decoupled Drupal 8 site. 

Questions? Comments? Want to learn more? Get in touch with us today!
 

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web