Sep 02 2019
Sep 02

Drupal 8.7.7 expected later this week is the first Drupal release to support extensions compatible with multiple major versions of Drupal. This is huge!

Drupal 9 is planned for June 3, 2020 and we are aiming to make the transition as smooth as possible. In fact we build Drupal 9 in Drupal 8 with API deprecations and optional support for new dependencies (Symfony 4.4 is getting very close, while Drupal 8.7.0 was already Twig 2 compatible). Extensions following these deprecations and making sure they are compatible with the updated dependencies could have codebases that are both compatible with Drupal 8 and 9 at the same time, meaning less support burden and earlier Drupal 9 compatibility. In March 48% of contributed projects on Drupal.org were already "Drupal 9 compatible" (as we could detect at the time), and that increased 54% by July (highfive!). These projects will now have one more thing to do, a one line info.yml file addition, to allow their existing codebase to also run on Drupal 9.

✋Drupal contrib maintainers, thank you! You are already contributing a lot to a successful Drupal 9. Projects that are "compatible with Drupal 9" (as much as we know now) are up to 54% from 48% since March. Deprecated API use is decreasing overall.

You are awesome, keep it up! pic.twitter.com/mHpw4ET0P4

— Gábor Hojtsy (@gaborhojtsy) August 2, 2019

The core: 8.x key did not cut it anymore

Drupal 8 looks at extension .info.yml files and checks for the core: 8.x key to check if the extension can be used with Drupal 8. Unfortunately this was not adequate already due to API additions in Drupal 8 minor releases. You could not specify an extension was only compatible with Drupal 8.4+. People devised clever tricks with dependency on certain versions of the system module, but that was still limiting due to inability to depend on specific patch releases. In case your extension depended on a bugfix being present, you cannot set a dependency on Drupal 8.4.1+ in any way.

Introducing the core_version_requirement key

While solving this issue it became apparent that solving it with the existing core: 8.x key is going to cause backwards compatibility problems with existing Drupal 8 sites. The format of the value could not be changed without causing issues, so a new core_version_requirement key was introduced. This now allows to depend on major and minor versions of core as well as specify multiple version requirements even between multiple minor Drupal versions. For example if you would need a bugfix that is to be included in both Drupal 8.7.23 and 8.8.3 (imaginary future versions) but you wouldn't care for the minor version, then you would use the following, making your module incompatible with versions before the particular bugfix you need on both minor branches, as well as incompatible with all older Drupal 8 branches:

name: My Module
type: module
core_version_requirement: ~8.7.23 || ~8.8.3

Such precision was never possible before!

The new feature is made available in 8.7.7 to help prepare for Drupal 9, as this makes it possible to also declare Drupal 9 compatibility with the existing codebase you already have. Drupal 8.7.x will get security support up until the planned release of Drupal 9 on June 3, 2020. For extensions that only support Drupal 8.7.7 and later you can now exclusively use the core_version_requirement key. If the extension is also Drupal 9 compatible:

name: My Module
type: module
core_version_requirement: ^8.7.7 || ^9

For extensions that also support older versions, you should keep the core: key as well. The core_version_requirement key will be ignored by older Drupal versions, and new versions will validate that you specified non-conflicting requirements in the two keys (with regards to Drupal 8 support) so for extensions supporting older core versions, core_version_requirement also needs to be permissive, but could also allow compatibility with Drupal 9 too:

name: My Module
type: module
core: 8.x
core_version_requirement: ^8 || ^9

Read more about this change in the change notice. There is also work ongoing to support dependency metadata from composer.json to drive Drupal's dependency resolution as well. That will coexist with this simpler approach and will provide a more composer-native solution for those who already adopted composer for their extensions.

What happens to Drupal.org project version numbers then?

So a 8.x-1.3 version of an extension on drupal.org may only be compatible with a subset of core, but that was already the case earlier. Now an 8.x-1.3 version of an extension on drupal.org may also be compatible with Drupal 9. How will this work? Drupal's composer integration already ignores the 8.x prefix from version numbers and only cares about the project specific version. Drupal core already only installs extensions where the core compatibility requirements are met regardless of which version number was the package you used to download (8.x modules may be only compatible with 8.4+ already). So the ultimate plan is to do away with the 8.x prefix in extension numbers entirely on drupal.org and support semantic versions for contributed extensions instead. In the short term, the Drupal.org project browser also needs to be updated to take the dependency metadata and use that to power compatibility filters, instead of merely using the version number prefix.

Jul 30 2019
Jul 30

I posted my analysis of top uses of deprecated code in Drupal contributed projects in May 2019 two months ago. There are some great news since then. First of all, Matt Glaman of Centarro implemented support for deprecation messages directly in reports, so we can analyse reports much better in terms of actionability. Second, Ryan Aslett at the Drupal Association implemented running all-contrib deprecation testing on drupalci.org (Drupal's official CI environment). While showing that data on projects themselves is in the works, I took the data to crunch some numbers again and the top deprecated API uses are largely the same as two months ago. However, we have full analysis of all deprecation messages and their fixability which gets us two powerful conclusions.

Stop using drupal_set_message() now!

If there is one thing you do for Drupal 9 compatibility, make it getting rid of using drupal_set_message(). As the API documentation for drupal_set_message() explains you should use the messenger service and invoke the addMessage() method on it.

Of the total of 23750 deprecated API use instances found in 7561 projects, 29% of them were instances of drupal_set_message(). So statistically speaking if you stop using this single function, you are already 29% on your way to Drupal 9 compatibility (likely more for most projects).

Dezső Biczó already built automated deprecation fixers covering drupal_set_message() and more, based on rector.

Figure visualising the data explained in the text.

76% of deprecated API use can already be resolved

On top of the 29% of drupal_set_message(), there is another 47% of various other API uses that can be fixed now. This means that those deprecated APIs were marked before Drupal 8.6.0 was released and are therefore in currently unsupported Drupal core versions. So stopping the use of them would still keep your code compatible with all currently supported Drupal core versions. In other words, as of today, drupal.org project maintainers can resolve three quarters of the outstanding code changes for Drupal 9 compatibility. Considering we are still ten months from before Drupal 9's planned release date, this is quite good news!

Upgrade Status is a nice visual tool to explicitly list all the errors in the projects you scan. It provides instructions on how to fix them and immediately fixable issues are highlighted.

Work with project maintainers

You are not a drupal.org project maintainer but want to help? Yay! Based on the above it may be tempting to run to the issue queue and submit issues for fixing all the things. Good plan! One thing to keep in mind though is to work with the maintainers of projects so your help benefits the project most effectively. Drupal.org project owners may specify Drupal 9 plan information that should help you engage with them the best way (use the right meta issue, know of their timeline plans, and so on). Check the project page of projects you are interested to be involved with to make sure you contribute the best way.

https://t.co/hf2ENvlZSo projects can now specify Drupal 9 porting information, so *you* can direct *your* contributors to provide the most valuable help on the way to Drupal 9, fund the process or just step back (for now). Edit your project to help your contributors help you! pic.twitter.com/l1OWwOllBK

— The Drop is Always Moving (@DropIsMoving) May 21, 2019

More to come

I think the data is super-interesting and I plan to do more with it, for example cross-referencing with project usage. Stay tuned for more information in the future. In the meantime all the source data is available for your mining as well.

Correction: An earlier version of this post said there were 43% additional fixable deprecations for a total of 72%. Thanks to Ryan Aslett for corrections, the correct numbers are 47% and 76% respectively. Images and text fixed.

Disclaimer: The data is based on the state of contributed projects on July 29, 2019 based on Drupal core's 8.8.x development branch on July 29, 2019. As contributed module maintainers commit fixes, the data will get better. As core introduces more deprecations (until Drupal 8.8.0-alpha1), the data could get worse. There may also be phpstan Drupal integration bugs or missing features, such as not finding uses of deprecated global constants yet. This is a snapshot as the tools and state of code on all sides evolve, the resulting data will be different.

Jul 19 2019
Jul 19

On our way to Drupal 9 every site will need to take care of making updates to their custom code as well as updating their contributed projects. However this time, instead of needing to rewrite code, only smaller changes are needed. Most contributed modules will only need to deal with a couple changes. Collaborating with project maintainers is the best way to get to Drupal 9. The first beta of the Upgrade Status module alongside recent drupal.org changes focus on making this much easier.

Upgrade Status beta provides better insight into Drupal 9 readiness

Take the first beta of the Upgrade Status module and run it on your site. It will provide executive summaries of results about all scanned projects and lets you inspect each individually.

Custom and contributed projects are grouped and summarised separately. You should be able to do all needed changes to your custom code, while for contributed projects you should keep them up to date in your environment and work with the maintainers to get to Drupal 9. The later is facilitated by displaying available update information inline and by pulling the Drupal 9 plan information from drupal.org projects and displaying it directly on the page.

This is how the summary looks like after scanning a few projects:

Upgrade Status summary page after scanning several projects.

Digging deeper from the executive summary, you can review each error separately. The beta release now categorizes issues found to actionable (Fix now) and non-actionable (Fix later) categories with a Check manually category for items where it cannot decide based on available information. For custom projects, any deprecation is fixable that has replacements in your environment while for contributed projects supporting all core versions with security support the window is shifted by a year. Only deprecations from two or more releases earlier can be fixed (compared to the latest Drupal release) while keeping Drupal core support. So somewhat ironically, Upgrade Status itself has deprecated API uses that it cannot yet fix (alongside ones it could fix, but we have them for testing purposes specifically):

Upgrade Status project issue list categories

The module is able to catch some types of PHP fatal errors (unfortunately there are still some in projects that we need to figure out the best way to catch). The @deprecated annotation information guiding you on how to fix the issues found are also displayed thanks to lots of work by Matt Glaman.

Own a Drupal.org project? Direct contributors to help you the way you prefer!

If you own a Drupal.org project that has Drupal 8 code, you should specify your Drupal 9 plans. It is worth spending time to fill in this field to direct contributors to the best way you prefer them help you, so contributions can be a win-win for you and your users alike. Whether it is a META issue you plan to collect work or a specific time in the future you will start looking at Drupal 9 deprecations or a funding need to be able to move forward, letting the world know is important. This allows others to engage with you the way you prefer them to. Additionally to it being displayed in Upgrade Status's summary it is also displayed directly on your project page!

Go edit your project and find the Drupal 9 porting info field to fill in. Some suggestions are provided for typical scenarios:

Drupal 9 porting info field on a project

This will then be displayed on your project page alongside usage and security coverage information. For example, check it out on the Upgrade Status project page.

Thanks

Special thanks for dedicated contributors and testers of the Upgrade Status module who helped us get to beta, especially Karl Fritsche (bio.logis), Nancy Rackleff (ThinkShout), Tsegaselassie Tadesse (Axelerant), Bram Goffings (Nascom), Travis Clark (Worthington Libraries), Mats Blakstad (Globalbility), Tony Wilson (UNC Pembroke), Alex Pott (AcroMedia, Thunder), Charlie ChX Negyesi (Smartsheet), Meike Jung (hexabinær Kommunikation). Thanks to Neil Drumm (Drupal Association) and Angela Byron (Acquia) for collaboration on the Drupal 9 plan field.

Jun 24 2019
Jun 24

The planned Drupal 9 release date is June 3, 2020. While the Drupal 9 branch is not yet open, we've been working on Drupal 9 ever since the release of Drupal 8.0.0 through our deprecation processes. In fact, that is our process to clean up and refactor our APIs.

For a PHP API to be removed we need to deprecate and introduce a replacement (as appropriate). For a module to be removed, we need to deprecate it and at least move it to a contributed module (or provide a replacement in core). For a JavaScript dependency to be removed (such as the already end of life jQuery UI), we need to deprecate it and provide a replacement, and so on. So what can we get rid of in Drupal 9 finally? Well, all the things marked deprecated.

Other than third party dependency updates, things not deprecated in Drupal 8 will stay in Drupal 9 and be subject to backwards compatibility support in Drupal 9 until its end of life (likely around the end of 2023).

And this is a good thing. We want stability with Drupal, we want people to invest and find their investments work with Drupal for a reasonable time. To completely answer my clickbait title though: how long do we have to "finally" get rid of something? Well as per Dries's DrupalCon Seattle keynote, we have until Drupal 8.8 to define what we are deprecating for (and thus removing in) Drupal 9. Here is the video snippet:

What does that mean in practice? 8.8.0-alpha1 is by when we need to get the API for Drupal 8.8.0 done and therefore the API of Drupal 9 done (other than third party dependency updates). And 8.8.0-alpha1 is scheduled for the week of October 14th, 2019.

In other words, you have 111 days to get your least favourite thing removed from Drupal. If not, well, we will live with backwards compatibility to what we have in October 2019 for 4 more years to come.
Jun 07 2019
Jun 07

I created and shared an open source set of editable slides with plenty speaker notes titled "State of Drupal 9" early in May, based on my webinar with Dries and then the Drupal 9 parts of the DrupalCon Seattle Driesnote. I hope that we can bring this know-how to a lot of conferences, meetups and to companies so people are more aware of what is coming, what happens to Drupal 8 and 7, what's so great about Drupal 9 and what are the key tools involved.

Before I even presented the slideshow first, I already got improvement suggestions from people presenting it elsewhere. So the first time I got to present this session in Minsk, Belarus at the kind invitation of DrupalCamp Belarus organizers, it was already improved from when I first published it. I will keep updating the slides and present where I can in the coming months. Please, do it yourself too! Translate, shorten, lengthen, etc. to match your audience and timing needs, as long as the key messages get through. Be an important contributor to Drupal 9 this way!

Here is the recording from DrupalCamp Belarus:

May 24 2019
May 24

Dwayne McDaniel did some thorough reporting of deprecated code use in all Drupal 8 contributed modules in March. Ultimately this kind of reporting would be best to have on drupal.org but while that is figured out, Dwayne's data set provides a very nice way to mine data about Drupal 9 readiness of contributed modules and to inform our tooling to improve the process. His original numbers showed that almost 44% of contributed modules had no deprecated code use at the time. What I was interested in was how to help the rest of the 56%.

Dwayne created an updated process and a new repository this week with fresh data. I was still curious so I delved right into the data and started mining it. A key question I was interested in is how much of the most widespread deprecations are actionable right now. An actionable deprecation is something core deprecated in a previous version that is not supported anymore, so you can update your code to remove the use of that API. Currently Drupal 8.6 and 8.7 are supported, so deprecations there should only be acted on for your custom code. However deprecations in and before 8.5 are entirely fine to act on.

First I counted the top list of deprecated APIs used from Dwayne's data across all of contributed projects. Then I wrote a script to collate api.drupal.org documentation to the deprecation notices. Ideally phpstan itself would report these messages directly and Matt Glaman is working on that. However since that is still blocked on the phpstan side, one needs a different data source to find the deprecation documentation for each occurrence, so I took to api.drupal.org to get that for now. Once that is found, we can categorize the deprecations into actionable, not actionable and actionable for custom code only. For the later case you know which core version you are using, and that should be an up to date minor version. So you don't need to deal with what core branches the community supports otherwise.

The results look really promising so far in terms of how much contributed modules can make progress on even today. If all already actionable deprecations get resolved, there will be very little left at least of the deprecations we already know.

A month ago Dezső Biczó created a set of proof of concept Rector fixes to automate some of these deprecation fixes, so I opened an issue with this new data set to try and cover the top ones that are not just actionable but approachable to automate.

Screenshot of deprecation status PDF

As with all interesting data sets, this summary is just the tip of the iceberg. There is huge potential to mine this data set for other uses, such as finding modules that potential contributors at an event could contribute fixes to. I don't know yet how much I can continue to work with this data myself, and of course others doing analysis of their own would be more than welcome.

Are you a drupal.org project maintainer? Now would be a good time to fill in your Drupal 9 porting information in your project, so you can let contributors know how to best engage with your in the process towards Drupal 9.

https://t.co/hf2ENvlZSo projects can now specify Drupal 9 porting information, so *you* can direct *your* contributors to provide the most valuable help on the way to Drupal 9, fund the process or just step back (for now). Edit your project to help your contributors help you! pic.twitter.com/l1OWwOllBK

— The Drop is Always Moving (@DropIsMoving) May 21, 2019

Disclaimer: The data is based on the state of contributed projects on May 20, 2019 based on Drupal core's 8.8.x development branch on May 20, 2019. The lookup on api.drupal.org was not perfect and I found some bugs there that are being resolved as well so the data gets more accurate. Also, as contributed modules will get updated, there will be less uses of deprecated APIs. As core will introduce more deprecations, the data could get worse. There may also be phpstan Drupal integration bugs or missing features, such as not finding uses of deprecated global constants yet. This is a snapshot as the tools and state of code on all sides evolve, the resulting data will be different.

May 03 2019
May 03

I am about to present about Drupal 9 at DrupalCamp Belarus in May and then at Drupal Developer Days Transylvania in June . I already presented an Acquia webinar with Dries Buytaert on the topic, and was on the Lullabot Podcast discussing Drupal 9 with Angie Byron and Nathaniel Catchpole. I am a firm believer that this know-how should spread as far and wide as possible. I should not be needed to travel around the globe to present the topic and people should not spend the same time again to redo slides for their local presentations. There is no intellectual property here to hide, as many people should be aware and excited and participating as possible. The topic should be presented at Drupal Meetups, Camps, and inside your own companies. So the natural next step for me was to create an open source slideshow.

Screenshot of the first 16 slides of the 1.1 version of the slideshow

I took all that we learned from the webinar and Dries' keynote at DrupalCon Seattle as well as new technology that emerged since then. I also used a free slide template and Google Slides so you can make a copy for yourself and add your own contact information as well as edit the slides down to shorter or longer timeslots. The 51 slides in my test run for about 35 minutes, leaving 10 minutes for discussion in a 45 minute slot. You would likely need to cut content for shorter sessions. There are only basic buildup animations, so if you need to present offline that is also an option. Edit in your contact/introduction info and export and present as PDF.

The 1.0 version of the slides have been presented by Christian Fritsch at DrupalCamp Munich last week and I updated some content to the current 1.1 version as it is available now. I'll keep updating slides based on all your feedback. I shared the slides with public comments allowed, so keep the feedback coming there, comments here or some other way you can get ahold of me.

Resources to watch/listen to learn more include:

  1. Dries' State of Drupal presentation from DrupalCon Seattle
  2. Lullabot Podcast on Drupal 9
  3. Acquia Webinar on Drupal 9

Thanks to Acquia for funding me to create this slideshow and thank you for presenting it!

Apr 29 2019
Apr 29

Dries Buytaert recently published a great post on how to prepare for Drupal 9. He explains how we build Drupal 9 in Drupal 8 using deprecations and the tools to use to detect use of deprecated code. Acquia funded me and Zoltán Herczog to work on one of the tools in the past few weeks. Zoltán just released the second alpha of Upgrade Status. It is definitely worth a try!

Here is how it works:

The module provides an overview of all the contributed and custom projects you have on a Drupal 8 site. It groups custom modules based on the directory structure and takes contributed projects from the update module. For contributed projects it also displays the available updates information inline. This is because if you keep contributed modules up to date, they should eventually improve Drupal 9 compatibility. We also plan to allow project maintainers to inform users about the best way to engage with them on the way to Drupal 9. This is dependent on new project fields on drupal.org which we plan to use to pull information to this dashboard.

Looking at the project list, the UI allows you to run a report on all of the projects to find any known Drupal 9 incompatibilities. Because running the full report takes a really long time, the UI also allows you to run reports on projects one by one. Or rerun the report on specific projects as you are working on improving their codebase. You can export the full report and each report individually.

The reporting backend and the report output is based entirely on the fantastic work of Matt Glaman on drupal-check at Centarro (formerly Commerce Guys) which is in itself based on phpstan by Ondřej Mirtes. We use the same tools that are used in drupal-check, the best command line tool available to check for deprecation errors (and other code quality issues). If you plan to include Drupal 9 compatibility checking in your build or continuous integration system, then drupal-check is your best bet.

Matt is also working with Ondřej to include the @deprecation annotation text in errors, which would improve the usefulness of the reporting of both tools immensely. Until then Upgrade Status links to api.drupal.org for documentation where you can find the deprecation documentation as well and learn about how to fix it.

The alpha2 version of the module was just released including support for recovering from PHP fatal errors (that legitimately happen in parsing some projects), as well as better chunking of project parsing so huge projects like Drupal Commerce will not result in timeouts or memory limit issues. We plan to improve the interactive experience further as well as fix any bugs reported, so please keep the reports coming. Looking forward to your experience with Upgrade Status!

Feb 06 2019
Feb 06

Traditionally Drupal was developed in two parallel branches. Take the example of Drupal 7 and 8. There was the supported stable branch of Drupal 7 that received bugfixes only and there was a development branch of Drupal 8 where developers went free to change and improve things the way they have seen best. As the development of Drupal 8 was nearing release, contributed module and theme authors were asked to take a look and release new versions as well in preparation. While the new branch was built with great care, it was not proven yet in various production environments. It also often took quite some time for contributed modules and themes to update.

Drupal 7 to Drupal 8 API update
Traditional Drupal code vs. module compatibility

Drupal 9 gets built in Drupal 8

With Drupal 8 however, we adopted a key change in our process. Instead of working on Drupal 9 in its own confined space, we work on it within Drupal 8. What does that mean? Drupal 8 allows new backwards compatible changes and additions as well as provides ways to mark outdated functionality as deprecated. This means that Drupal 9 is built step by step in Drupal 8 and so large parts of it are already running in production environments.

There are various good reasons for this. We get our improvements into user's hands much faster this way and also get validation of the new additions and changes faster. We can improve the version people actually use while creating the next version. It hopefully also means that modules and themes get updated with the new better APIs. This way the upgrade effort is spread out from one big upgrade to several smaller ones for site owners and contributed project developers alike. Ultimately this will lead to an easy and relatively small upgrade path to Drupal 9, because Drupal 9 will mainly be updating dependencies and cleaning up our codebase by removing deprecated APIs.

Keeping module/theme/custom code up to date

So theoretically contributed modules and themes and custom code could follow this update process:

Drupal 8.x to Drupal 8.x and 9 API updates
Contributed module that keeps up with core updates

This sounds good in theory, however it is worth thinking about a bit. As shown, the simplest case is the module keeps releasing versions of its own with new features and bugfixes (those are the gaps in numbers between versions shown), while it also updates regularly to new core APIs in the same stream. This is not a problem at all if it is about your custom code and you have full control over the site being deployed to. However it could lead to problems for users of modules and themes.

If you closely follow new core APIs, that also means that you keep updating the required core version number for your code. While Drupal 8.7 may be backwards compatible to Drupal 8.6, once you start using the new things from Drupal 8.7 you are not compatible with lower version numbers anymore. Keeping Drupal 8's one year security support for minor versions in mind, that also means that you would force people using your updated code to update from a Drupal core version that is otherwise security supported just to receive bugfixes as well for the module.

I would say that is fine as long as you only drop support for core versions that are not supported anymore (out of the one year support period), because sites should not be running those anyway. Any security fixes will only be released for the supported versions.

Alternatively a contributed module or theme may consider opening new branches for each core compatibility change. That puts a lot of burden on the maintainer, to keep fixing 8.x-4.x, 8.x-5.x, etc. bugs as well and I would say likely would die off the same way core support gets removed.

Why not do it all at once?

So why not do it the same way contributed module and theme developers always did? All at once!

That is exactly what we are trying to avoid. While Drupal 9 will be mostly the same as Drupal 8 (no new paradigms of development, etc.), the extent of deprecated code will not be trivial to process all at once. We only get the changes proven and Drupal 9 battle tested if developers do keep their code updated. That is why we keep publishing change notices, marking deprecated code and build tooling to support the process. If you are to do everything at once after Drupal 8.8 is released at the end of 2019 you will have quite a limited time to update before Drupal 9 is released 6 months later. For simpler modules and themes this may not be an issue, but for code with some complexity, it is best to start assessing now.

How to figure out what work am I up to?

1. If your code has automated tests, great! Not just great for the tests themselves, but because you can use deprecation testing to find issues. This will not exactly prove that your code is ready for Drupal 9 but if it is not, that will be revealed. Once you fix the issues found, you will be a few steps closer.

2. An additional tool even if you have automated tests, but especially if you don't is static code analysis. Matt Glaman implemented Drupal integration for PHPStan to do deprecation testing effectively. The good thing about this is that it relies on phpdoc annotations, which core consistently uses (as opposed to the tests which rely on trigger_error() that is missing still in many places). This is great for your custom code especially.

For example, Dries recently ran Matt's tools on his site's custom code and fixed his issues:

3. Finally we have a third path under discussion where your live site would record use of deprecated code. This could find some more deprecated code use on top of what is found with tests and static analysis. A tool like this could empower site builders to asses Drupal 9 readiness as well while the prior two methods are entirely developer focused.

Working with core and contributed project maintainers

Drupal core has various issues to properly mark deprecated code now and then clean up our technical debt once Drupal 9 opens (eg. remove deprecated modules). Please search the issue queue and submit issues when existing ones are not found. The api.drupal.org listing is a good place to start to help clean up our own use of deprecated code.

Some of the methods above will get you deprecated code use in a contributed modules or themes you don't maintain. In this case it is best to coordinate with the maintainer. Open an issue if there is not one already to remove deprecated code use and discuss next steps with the maintainer. Submitting a report from the phpstan tool or test failures you found would go a long way to help the maintainer asses the extent of work needed. Depending on how many modules does the maintainer own and the extent of work required, they may or may not prefer to keep it up to date every 6 months. Offering your help updating use of deprecated code is the best way to get closer to the outcome you are looking for.

Further opportunities to learn and participate

Comments are welcome on this post. I am also happy to see you on the Drupal 9 meetings every other week in the #d9readiness channel and the Drupal deprecations office hours held by Lee Rowlands and me in the same channel every Thursday. Exact dates and times of meetings are on the Drupal core calendar. If you can make it to DrupalCon Seattle, it would be lovely to meet you at one of the daily Drupal 9 office hours or the "Drupal 9 is coming: Getting your code ready" session.

Dec 17 2018
Dec 17

Drupal 9 is planned to be only 18 months away now, wow! It is already being built in Drupal 8 by marking APIs to be removed in Drupal 9 as deprecated and eventually upgrading some dependency version requirements where needed. Once the Drupal 9 git branch will be open, you will be able to test directly against Drupal 9. That should not stop you from assessing the compatibility of your module with Drupal 9 now. To prepare for compatibility with Drupal 9, you need to keep up with deprecated functionality and watch out for upgraded dependencies (when we know which are those exactly). Of these two, automation can go a long way to help you keep up with deprecated APIs.

What does deprecated code look like?

Deprecations include a trigger_error() call that allows catching when you use deprecated functionality. Here is an abbreviated example from the documentation:

<?php
/**
* [...]
*/
function drupal_clear_css_cache() {
@
trigger_error('drupal_clear_css_cache() is deprecated in Drupal 8.0.0 and will be removed before Drupal 9.0.0. Use \Drupal\Core\Asset\AssetCollectionOptimizerInterface::deleteAll(). See https://www.drupal.org/node/2317841.', E_USER_DEPRECATED);
\
Drupal::service('asset.css.collection_optimizer')->deleteAll();
}
?>

The trigger_error() call here only fires when the error reporting level includes E_USER_DEPRECATED errors, and the message provides full explanation as to what exactly is deprecated, which Drupal version will stop providing it and what should be used instead.

Unfortunately not all deprecated code is yet updated to this standard, but help is more than welcome. We were already deprecating APIs before we instituted the clever trigger_error() format, and the old deprecations need some work to be updated. However usages of those that are already marked are possible to catch with automated testing.

Make tests catch usage of deprecated APIs

Drupal.org has a great automated testing system. Whenever someone uploads a patch for consideration, the test suite of the corresponding project runs and results are posted back on the issue. Changes that fail with the test suite of the project are never committed. You can use this system to try out how far is your contributed module from working on what we currently know to be Drupal 9. Let's see how this works.

First of all, your module should have automated testing enabled. Make sure you have at least one test in your project and configure automated testing for your module on the Automated testing tab of your project page.

Drupal.org's automated testing is driven from an (optional) drupalci.yml file in projects. Core's drupalci.yml file for example always exposes usages of deprecated APIs in test fails. On the other hand, contributed modules are not tested for usages of deprecated functionality by default. (Note no suppress-deprecations: false clauses in the file).

However you can choose to add a drupalci.yml file to your project repository to expose these failures or keep an issue on your project queue with a patch including the file to learn about the failures. Adding the file to your project outright ensures that the project keeps being Drupal 9 compatible as much as possible. On the other hand once a new minor version of Drupal 8 comes out, new deprecations could fail existing releases of your module in this case. If the newly deprecated APIs were replaced with newly added APIs, then fixing those failures would mean you increase the Drupal minor version requirement of your module, thus not being compatible with older minor versions. That may be problematic. For now you can at least add a testing issue to see what you are up to to begin with.

To make your contributed module tests fail when your module uses deprecated functionality, you need to take the default build.yml file for contributed projects and follow the instructions on drupal.org to take the assessment part of the file and add a suppress-deprecations: false directive to each testing section. The resulting file looks like as follows (everything being a straight copy-paste from the assessment section of the default build.yml file other than the deprecations directives). Save this in a drupalci.yml file in the root of your project:

build:
  assessment:
    validate_codebase:
      phplint:
      container_composer:
      csslint:
      eslint:
      phpcs:
    testing:
      run_tests.standard:
        types: 'Simpletest,PHPUnit-Unit,PHPUnit-Kernel,PHPUnit-Functional'
        # Test for Drupal 9 compatibility
        suppress-deprecations: false
      run_tests.js:
        concurrency: 1
        types: 'PHPUnit-FunctionalJavascript'
        # Test for Drupal 9 compatibility
        suppress-deprecations: false
      nightwatchjs:

If you already had a drupalci.yml file, then adapt its content adding the suppress-deprecations: false directives only. Create a patch file out of it for your project, or take mine from https://www.drupal.org/files/issues/2018-12-17/3020957-2.patch, which should be universally applicable to projects that did not have a drupalci.yml before. Once the patch is uploaded to an issue and set to needs review, you will get deprecations results.

My results and what do they mean

I tried this out on Configuration inspector, a module I help maintain, that provides insights about configuration structures and schemas for developers. As a developer focused module, it would be key to have it up to date with Drupal 9 so it will be trivial to use when the time comes. And the results were green, everything passed.

Ok, does that mean my module is Drupal 9 compatible? Well, not really. This method is only capable to tell me if there are incompatibilities, a green result does not guarantee that the module is Drupal 9 compatible. Why?

  • The method is dependent on all of core (and any other contrib dependencies) marking their deprecated code properly. As stated above this is not yet complete even for core. Help with the core issue and encouragement for contributed module authors to adopt the same deprecation format is welcome.
  • There will be more deprecations until Drupal 9 is released, and we don't know yet the full extent of those, so the result can only tell us so much about the present state.
  • Even in the present state, the results only represent code that my contributed module's test suite actually ran from my module. If my test suite is not very thorough (which in this case unfortunately is the case), then I cannot trust this green result either that it proves Drupal 9 compatibility. I know you are better in test coverage for your contributed modules, so your results would be better.
  • Drupal 9 will bring updated dependencies like Symfony 4 or 5, possibly Twig 2, etc. Usage of those dependency APIs directly would only fail tests if they would use the same trigger_error() deprecation mechanism. I believe that is not the case, for example, Symfony has its own Deprecation Detector that uses code sniffing instead of a test suite.

On the other hand if there are failures, those should be genuine compatibility issues that you should fix for Drupal 9 compatibility. Therefore the title of this post: How to automate testing whether your Drupal 8 module is incompatible with Drupal 9?

Keep in mind that some failures may be results of contributed module dependencies of your module not being up to date. You may not be able to do anything with those failures directly, however letting the maintainer know and submitting fixes to those projects is very welcome.

How will this improve in the future?

As more code in core and contributed modules is marked properly deprecated with the current standard, more of the incompatibilities will be surfaced by this testing method. Also a checkbox for Drupal 9 deprecation testing is planned to be added to the automated testing configuration screen of projects, to make it easier to configure environments to test for deprecated code use. That will make the drupalci.yml creation of this tutorial obsolete, but all caveats and conclusions will still apply.

How does this test work out for your module?

Apr 19 2017
Apr 19

(Disclaimer: Dries did not ask/order/suggest/request me to post this neither to make any changes whatsoever.)

I was reading Drupal Confessions with great interest, because my primary job for quite a while now is to help enable the best efforts in the Drupal community to work together to reach new heights. I am really privileged to be able to do this and make a living out of it, it is the job I always dreamed of. I also hope that I am being useful to the Drupal community in doing so. I kept nodding in agreement with the open letter's statements on diversity so I could not get rid of my cognitive dissonance on their connection with reality for several days now. The open letter says "We ask you to fight for us, Dries, to protect us from intolerance, harassment, smearing, bullying, and discrimination, no matter why or where it originates from."

As someone who is getting a paycheck at least indirectly from Dries in the past 10 years, and working as directly with him as possible in Acquia's Office of the CTO for the second half of that period, I have the opportunity to look at his practices in terms of diversity. Which would be the most observable group to learn more from if not his personal department where he directly appoints people?

In the past 5 years I've been closely working with Dries as a remote employee in the Office of the CTO at Acquia, which consisted of 15 people at most at any given time. In this time I worked with at least the following types of people directly in this group (in random order): blind, stutterer, person with ADHD, person with serious sleep problems, person with even more serious sleep problems, divorced, gay, bi, lesbian, asexual, polyamorous, gospel singer, BDSM, religiously raised, atheist, clinically infertile, cosplayer, very tall, short, people from Eastern Europe, Western Europe, US and India, native-borns, immigrants, people in offices, people always working remotely, capitalist, socialist, pacifist, people from a military family, adopting parent, transgender, people with tics, people who don't drink alcohol, etc. The teams I worked on usually had relatively high percentage of women in technical roles. My current team is 50% female, and 1/3rd of the department overall is female. This could not happen by accident.

While none of this represent the whole rainbow of humanity, and it could definitely be further improved, our limited group of 15 people already covered outstanding diversity in my view. Also this is a group that cares, we discuss and live through our struggles and support each other on every step where we can. I would challenge many of those signing the open letter to practice this kind of diversity in their teams. I for one really enjoy the varying values that people brought in and am sad that some of those great people have left to pursue other technical challenges.

In this five years, I never experienced that when hiring people either of these things were considered as a negative or that any person was treated or felt treated badly for any personal life matter. Neither that any of the amazing people who unfortunately left and are covered in the list left because of any of those.

Given that I just cannot get over my cognitive dissonance. Who are being convinced of what?

Feb 17 2017
Feb 17

We started regular Drupal usability meetings twice a week almost a year ago in March 2016. That is a long time and we succeeded in supporting many key initiatives in this time, including reviews on new media handling and library functionality, feedback on workflow user experience, outside-in editing and place block functionality. We helped set scope for the changes required to inline form errors on its way to stability. Those are all supporting existing teams working on their respective features where user interfaces are involved.

However, we also started to look at some Drupal components and whether we can gradually improve them. One of the biggest tasks we took on was redesigning the status page, where Drupal's system information is presented and errors and warnings are printed for site owners to resolve. While that looks like a huge monster issue, Roy Scholten in fact posted a breakdown of how the process itself went. If we were to start a fresh issue (which we should have), the process would be much easier to follow and would be more visible. The result is quite remarkable:

New status page in Drupal 8.3

While the new status page is amazing, for me the biggest outcome is how eager people were for this refresh and the example we set as to what is possible to do in Drupal 8. We can take a page, redesign it completely and get it released in a minor release for everyone to use! Some feedback on the result:

This looks great! #drupalnerd https://t.co/UgB7C2t3rN

— tiny red flowers (@tinyredflowers) February 12, 2017

Wow, just wow. https://t.co/Slq2bn7AIe

— Thomas Donahue (@dasginganinja) February 9, 2017

It's about damn time! https://t.co/56hGMKfwdQ

— Jerry Low (@jerrylowm) February 8, 2017

Wow! Just wow! Sometimes Christmas comes early. Sweet! https://t.co/9cYQbFdK18

— Adam Evertsson (@AdamEvertsson) February 8, 2017

@DrupalUx pic.twitter.com/BoCp720AUo

— Jan Laureys (@JanLaureys) February 8, 2017

Wow this looks really nice @DrupalUx. https://t.co/tyr6NfNZOm

— Eric Heydrich (@ericheydrich) February 8, 2017

A status page that does justice for the elegance of Drupal 8. Looks very nice! https://t.co/Kt75wR40U6

— David Lanier (@nadavoid) February 8, 2017

THANK YOU, LORD! https://t.co/dVU49QPS6D

— Phéna Proxima (@djphenaproxima) February 8, 2017

My mistakes are now much nicer to look at. Great work @DrupalUx ! pic.twitter.com/7LKNGxI3FA

— dawehner (@da_wehner) February 7, 2017

This is the best https://t.co/vTErYfMVc2

— drupteg (@drupteg) February 7, 2017

If this is not enough proof that we can make significant improvements and that people are more than open to receive them, not sure what else could be it.

To join these efforts, there are several smaller things in the works currently, including improving the bulk operations UI on views forms (or for the more adventurous redesigning filters and bulk operations, which would affect the content, users and even the media admin pages). We are working to update the throbber in the Seven theme, make the add content link finger friendly, and so on. There are many smaller to bigger issues for anyone to work on, we can match you with an issue. We need designers, developers, testers, etc.

Want to be a part of the next celebrated improvement? Join the UX slack on drupal.slack.com (get an invite at http://drupalslack.herokuapp.com/). Meetings are 9pm CEST every Tuesday and 9am CEST every Wednesday. See you there!

Oct 03 2016
Oct 03

Starting with Drupal 8, we decided to make more rapid innovation possible by releasing minor versions every 6 months that may come with new features and backwards compatible changes. Now that we released Drupal 8.1.0 and almost 8.2.0 as well, how did we do? Also what else is possible and what is blocking us to make those moves? What do all the changes mean for how might Drupal 9 unfold?

Dries Buytaert posted last Wednesday The transformation of Drupal 8 for continuous innovation and on the same day I presented Checking on Drupal 8's rapid innovation promises at DrupalCon Dublin. Here is a video recording of my session, which should be good for those looking to get to know Drupal's release process and schedule, as well as how we made it possible to experiment within Drupal core directly with Drupal 8. While I did hope for more discussion on the possibilities within Drupal 8 with the participants, somehow the discussion pretty much ended up focusing on Drupal 9, when it should be released and how much change should it come with.

[embedded content]
Aug 24 2016
Aug 24

In my previous post I explained why there will be a Drupal 9 even though we have previously unseen possibilities to add new things within Drupal 8.x.y. Now I'd like to dispel another myth, that initiatives are only there to add those new things.

Drupal 8 introduced initiatives to the core development process with the intention that even core development became too big to follow, understand or really get involved with in general. However because there are key areas that people want to work in, it makes sense to set up focused groups to organize work in those areas and support each other in those smaller groups. So initiatives like Configuration Management, Views in Core, Web Services, Multilingual, etc. were set up and mostly worked well, not in small part because it is easier to devote yourself to improving web services capabilities or multilingual support as opposed to "make Drupal better". Too abstract goals are harder to sign up for, a team with a thousand people is harder to feel a member of.

Given the success of this approach, even after the release of Drupal 8.0.0, we continued using this model and there are now several groups of people working on making things happen in Drupal 8.x. Ongoing initiatives include API-first, Media, Migrate, Content Workflows and so on. Several of these are primarily working on fixing bugs and plugging holes. A significant part of Migrate and API-first work to date was about fixing bugs and implementing originally intended functionality for example.

The wonder of these initiatives is they are all groups of dedicated people who are really passionate about that topic. They not only have plan or meta issues linked in the roadmap but also have issue tags and have regular meeting times. The Drupal 8 core calendar is full of meetings happening almost every single workday (that said, somehow people prefer Wednesdays and avoid Fridays).

If you have an issue involving usability, a bug with a Drupal web service API, a missing migration feature and so on, your best choice is to bring it to the teams already focused on the topics. The number and diverse areas of teams already in place gives you a very good chance that whatever you are intending to work on is somehow related to one or more of them. And since no issue will get done by one person (you need a reviewer and a committer at minimum), your only way to get something resolved is to seek interested parties as soon as possible. Does it sound like you are demanding time from these folks unfairly? I don't think so. As long as you are genuinely interested to solve the problem at hand, you are in fact contributing to the team which is for the benefit of everyone. And who knows, maybe you quickly become an integral team member as well.

Thanks for contributing and happy team-match finding!

Ps. If your issue is no match for an existing team, the friendly folks at #drupal-contribute in IRC are also there to help.

Aug 09 2016
Aug 09

Earlier this week Steve Burge posted the intriguingly titled There Will Never be a Drupal 9. While that sure makes you click on the article, it is not quite true.

Drupal 8.0.0 made several big changes but among the biggest is the adoption of semantic versioning with scheduled releases.

Scheduled releases were decided to happen around twice a year. And indeed, Drupal 8.1.0 was released on time, Drupal 8.2.0 is in beta and Drupal 8.3.x is already open for development and got some changes committed that Drupal 8.2.x will never have. So this works pretty well so far.

As for semantic versioning, that is not a Drupalism either, see http://semver.org/. It basically means that we have three levels of version numbers now with clearly defined roles. We increment the last number when we make backwards compatible bug fixes. We increment the middle number when we add new functionality in a backwards compatible way. We did that with 8.1.0 and are about to do it with 8.2.0 later this year. And we would increment the first number (go from 8.x.x to 9.0.0) when we make backwards incompatible changes.

So long as you are on some version of Drupal 8, things need to be backwards compatible, so we can just add new things. This still allows us to modernize APIs by extending an old one in a backwards compatible way or introducing a new modern API alongside an old one and deprecate (but not remove!) the old one. This means that after a while there may be multiple parallel APIs to send emails, create routes, migrate content, expose web services and so on, and it will be an increasingly bigger mess.

There must be a balance between increasing that mess in the interest of backwards compatibility and cleaning it up to make developer's lives easier, software faster, tests easier to write and faster to run and so on. Given that the new APIs deprecate the old ones, developers are informed about upcoming changes ahead of time, and should have plenty of time to adapt their modules, themes, distributions. There may even be changes that are not possible in Drupal 8 with parallel APIs, but we don't yet have an example of that.

After that Drupal 9 could just be about removing the bad old ways and keeping the good new ways of doing things and the first Drupal 9 release could be the same as the last Drupal 8 release with the cruft removed. What would make you move to Drupal 9 then? Well, new Drupal 8 improvements would stop happening and Drupal 9.1 will have new features again.

While this is not a policy set in stone, Dries Buytaert had this to say about the topic right after his DrupalCon Barcelona keynote in the Q&A almost a year ago:

[embedded content]

Read more about and discuss when Drupal 9 may be open at https://www.drupal.org/node/2608062

Dec 08 2015
Dec 08

Drupal 8 makes huge-huge strides in terms of reaching more people on the globe by making everything translatable and automating translation downloads and updates from the community among hundreds of other improvements. What about software translation contribution though? We did not make progress on that front in core, but it is not very hard to do as a contributed module.

The Localization client module is available for Drupal 7 and 6. It does two things: it provides an alternate nicer user interface to translate Drupal (modules) and it allows you to contribute to the community right from that interface. Drupal 8 already improved the built-in translation interface considerably, so it made sense to start with porting the contribution functionality and integrate that first. The initial Drupal 8 port of that is available for your testing now!

To try it out:

  • Join the https://localize.drupal.org/ team you want to contribute translations to (if not already). For testing, you can use the test language. Or better: just test with good translation suggestions so your reviewers will be happy.
  • Grab the Drupal 8.x-1.0-alpha1 version of Localization client. Install the Localization Client Contributor submodule only (you would not be able to install the other one yet anyway).
  • The module comes with contribution enabled by default, but you need to set your API key so localize.drupal.org can identify you. Edit your user account on your site and enter the API key there. (Follow the instructions on the API key field on your user account edit screen).
  • If you want to grant other people to contribute, grant them the Contribute translations to localization server permission to contribute and tell them to enter their respective API keys on their user profiles.
  • Now when you go to the built-in software translation UI it will tell you that all things changed will also be submitted to localize.drupal.org and the submit button will reflect that too, to make it super-clear. If you did not set your API key yet, it will tell you to do that instead.
  • Once submitting the translation changes, it will let you know how many strings were successfully submitted and any errors encountered as well. All messages are also logged for later review.

The resulting effect of submitting changes locally are visible remotely by the community as well, yay!

Happy translating! See known issues and report more at https://www.drupal.org/project/issues/l10n_client?version=8.x Further improvements are of course possible, issues are welcome!

Nov 18 2015
Nov 18

In the previous tidbits we covered each language and translation capability one by one. The community translates the software interface on http://localize.drupal.org/ which you can customize with Interface translation. You can translate your local configuration and content with the Configuration translation and Content translation modules respectively. However, actual real life use cases are never clear cut like that. Content shows up with some shipped interface elements, local configuration and content. Menus contain elements from code, content and configuration. It is good to know how these pieces relate so you can translate every piece and know the right place to do it.

The connection between interface and configuration translation

One of the most common complaints in earlier Drupal core versions is that when you install in a foreign language, if the translation was not complete, later translation updates will not fix untranslated content type names, field names, etc. They are translated with the words available at time of installation. That is because Drupal 7 and before treat configuration as something entered by the administrator on a site. Once it is saved, it is not touched again by the system. Drupal 8 strives to solve this problem by connecting interface translation with configuration translation. This means that configuration that was part of projects (such as Drupal core) when you installed them will be kept updated with translations from interface translation as well. Translations for the site default language (if not English or if English is configured to be translatable) are kept in the active configuration, while other translations are kept as translation overrides.

What does this practically mean? If you are on a translated site and are to use a built-in content type like Article for example, the name, description, title label, etc. for the content type will be managed by both interface translation and configuration translation. If you add a new content type, that will entirely be managed by configuration translation only, given that it was not shipepd with Drupal or a module. Also if you add more fields to the article content type, those will be managed by configuration translation only.

All content created with that content type when output will display a mix of things that are managed by interface translation, configuration translation and content translation. Let's consider this example where I use the article content type and added a region field (Hungary, Europe, etc. to target my content) and a subtitle field:

Of course all the configuration is Hungarian because I installed in Hungarian and created the new fields in Hungarian. All the built-in fields got translations as part of interface translation. The newly added fields are created in Hungarian originally. Drupal 8 is smart enough to handle this situation. Now if I am about to translate this content type configuration to another language, adding French for example on the site would pull in interface translations for the built-in fields in French already and I would only need to manually add translations for the two custom fields.

When seeing these fields on the content editing form, the form will use the language of the page to display the options. For example, the default value configured for the subtitle will show up in the language of the form, the select box will show options in the language of the form. The help text of the select box uses the language of the form. When editing content in Hungarian:

When using the French content translation form, of course all of these will show up in French. It is important to note here that the default value of the subtitle is translated for this input form, but then the subtitle of this content will be independent of original the default value. Once I save this content (even if I keep using the copied default subtitle), the field will be translated with content translation onwards. (On the screenshot I actually changed the field value).

Finally, when the content is display, that uses a mix of configuration and content translation as well. I made the region field a select box, so the actual printed value will depend on the translated configuration for the field (Magyarország in Hungarian and Hongrie in French). That is because the internal value stored is the key for this field (hu), and not the label. For the subtitle field, that is stored as-is in the content, so it will purely use content translation. For the label of the fields though, the configuration translation applies in all cases. I don't think the label would look good on the subtitle, so I removed it, but the region makes sense to lead with a label. All-in-all the French translated version looks like the following:

In this scenario I did not make the region field value translatable, in other words I did not need the Hungarian and French translations to be able to target different regions. If I would need that too, then the resulting translation would depend first on which region key the concrete translation selected, which would then be turned into the proper translated label for the display language of the content based on the field's configuration translation.

That is how interface translation, configuration translation and content translation works together for the translation of a content entity. When it just works, you don't need to care about it, when you set it up, knowing where to look saves you lots of time.

Menus, the black belt master level

If that was a bit complex, consider what a feat it was to make it all work. Also, there is at least one more level up to black belt mastery of translation and that would be menus. Again this is a place where content, configuration and interface translation meet to form interesting combinations.

Let's get shipped menu items out of the way first. Drupal core ships with numerous menu items, the most obvious for translation however is the 'Home' (Címlap in Hungarian) one in the main navigation menu. You'll see that you cannot edit the title or path of this menu item, you can only disable it if you don't want to see it. This menu item is shipped with Drupal as part of the package. That means that it is translated with interface translation.

Imagine I want to add three more menu items to this menu:

  • I want to create a view for my old articles and expose the page under an 'Archive' (Archívum in Hungarian) menu item.
  • I want to create an about us page and expose it as a menu item under the 'About us' (Rólunk in Hungarian) title.
  • I want to have a quick 'Contact us' (Kapcsolat in Hungarian) menu item that leads to the contact form.

All of these end up in my navigation menu. I created them from views, the node form and the menu item user interface respectively. So where are they translated? Well, the same place they are stored. The menu item exposed by views is translatable as part of the view, and the contact link is translated as the custom menu item itself. Custom menu items are content entities in core, so you need to configure content translation is enabled for custom menu items to make it possible to translate them. After that you just have a translate operation on your custom items when editing the menu.

The 'About us' item created from the node form is the most intricate. That is also stored as a content item and requires custom menu items to be translatable the same way the contact link does. However, I entered this on the node form, so I have a natural expectation to translate it there as part of my French translation. Unfortunately the node form's menu widget was not made ready for menu item translations (yet), so if you provide the French menu item title there, it will be updated in the original item and not going to create a translation. Until that is fixed, use the menu item translation process as with the contact menu item.

That allowed me to translate the shipped 'Home' menu item with interface translation, the 'Archive' view menu item with configuration translation and the 'About us' and 'Contact' menu items with content translation. So all is good? Let me show you one more trick.

You may want to display this link at a different place, for example, moving this block into a sidebar block. For better visual separation, we would want to enable displaying the block title for this block. Because I moved this from the header to the sidebar, this is still the block shipped with Drupal core. That means that the block title will already be translated to French, so when switching language, the title shows up properly translated.

However if you are to make another placement of this block in the sidebar instead of moving the menu, that would be a block of your own creation. In that case, while the block title is prefilled by the menu title, it will not be automatically translated (the block's title does not follow the translated menu title dynamically), so you will need to manually translate it to other languages as the block title (with configuration translation of the block placement).

Different menu items for different languages

Now we know all the details of where to translate menu items. But what if we want to add/substract some menu items from our menus for certain languages. Unfortunately that is not possible in Drupal 8 core. However, we can easily create entirely new menus for various languages or language groups. For example, I may want to translate this menu to French (from my Hungarian original) and then create a different menu for German where only 'About us' and 'Contact us' will be present because I am not doing German translation yet.

I can use block visibility settings to limit each block to their respective languages, so the original menu block will only show for French and Hungarian and the new menu's block will only show for German. Then I only need to translate the items in the original menu and can create German items in the specific menu. Later on if I need to adapt this to changing needs, Drupal will know the respective source languages for all items, so I can move to whatever direction I need from there.

In closing

With these directions in mind, I am sure you can figure out even the most puzzling corners of how Drupal 8's multilingual system works. That concludes coverage of the core features and APIs in Drupal 8 for multilingual needs. Next up we'll cover some contributed modules, because yes, however surprising that may be, you may still want to use contributed modules to round out some multilingual features.

Issues to work on

  • The most important of course would be to make the menu widget work properly in content forms, so the right translation may be edited without leaving the node form. https://www.drupal.org/node/2315773 had a grand plan to implement it with fields but unfortunately did not get done.
  • Some configuration elements, while translatable are not exposed on the translation interface. For example the default value for the text field is not. See https://www.drupal.org/node/2546212 for a resolution.
Nov 11 2015
Nov 11

Up to date as of March 14th, 2017.

Now that we covered how content translation workflow works in Drupal 8, its time to look a bit at the API side. In Drupal 7 this meant dealing with scary seemingly infinitely nested arrays with language codes, field names, deltas, etc. Drupal 8 makes this a whole lot simpler.

Translation basics on an entity

Drupal 8's entity API handles entities as full fledged objects. Translations of entities may be requested from this object and returned as a reusable entity object as well, which can be treated the same way as the original entity that we loaded. Here are some quick examples:

<?php
use Drupal\node\Entity\Node;// Load node 4. In terms of language, this will get us an entity
// in the original submission language.
$node = Node::load(4);// Get a list of all translations and collect titles.
$titles = [];
$languages = $node->getTranslationLanguages();
foreach (
$languages as $langcode => $language) {
// The object returned by getTranslation() behaves the same way as $node.
$translation = $node->getTranslation($langcode);
$titles[$langcode] = $translation->title;
}
// If the node has no Hungarian translation, add one.
if (!$node->hasTranslation('hu')) {
$translation = $node->addTranslation('hu', array('title' => 'Hungarian title'));
$translation->save();
}
// If the node has a Spanish translation, update the title. In case of a missing
// Spanish translation this will throw an InvalidArgumentException.
$node->getTranslation('es')->setTitle('Spanish title')->save();// Remove the Hungarian translation.
$node->removeTranslation('hu');
$node->save();
?>

Loading the right translation

Drupal 8 also comes with several supporting systems for dealing with content language. For example, a content language is selected for the page to be used for translated content display. You can retrieve that language from the language manager and use to deal with the right translation.

<?php
use Drupal\Core\Language\LanguageInterface;
use
Drupal\taxonomy\Entity\Term;$term = Term::load(23);
$langcode = \Drupal::languageManager()->getCurrentLanguage(LanguageInterface::TYPE_CONTENT->getId());
$translation = $term->getTranslation($langcode);
// Now deal with the right language from here on.
?>

A possible problem with that is getTranslation() will return an exception if the translation requested does not exist. If you want to be sure to load only from existing translations and apply a fallback mechanism for loading suitable translations when something is not available, you need entity language negotiation. That is provided by the entity repository and helps you figure out which translation to load.

<?php
use Drupal\user\Entity\User;$account = User::load(15);
$langcode = \Drupal::languageManager()->getCurrentLanguage(LanguageInterface::TYPE_CONTENT)->getId();
// Get the user profile with language fallback, if it did not have a translation for $langcode.
$translation = \Drupal::service('entity.repository')->getTranslationFromContext($account, $langcode);
?>

The entity repository uses the language manager and possibly contributed modules to figure out which language to load for the entity. Ideally $langcode would be that language, but if that is not available, modules get a chance to provide fallback schemes for which other language may be best suitable. See LanguageManager::getFallbackCandidates() as well as hook_language_fallback_candidates_alter() and hook_language_fallback_candidates_OPERATION_alter() for more information.

That's it for a short summary of how the much improved content translation API works in Drupal 8. There is of course a lot more to this topic from creating translatable entity types, new field types to dealing with revisions and entity field queries. For some of those deeper details on how content translations are stored and queried, see Francesco Placella's (unfortunately already slightly outdated) articles in Drupal Watchdog titled Wait, $langcode? What the Heck? and Entity Storage, the Drupal 8 Way.

Next up, I plan to look at complex core use cases where interface, content and configuration translation meet. If you know how the pieces map together, you can really translate practically anything!

Issues to work on

None at the moment.

Nov 10 2015
Nov 10

Up to date as of November 10th, 2015.

In the previous tidbit, we covered content translation basics. In short now you can configure translatability on any subtype of any entity type, so for example articles or specific taxonomy vocabularies may be configured to have all their entities support translation. Then each entity structure may be configured on the field and in some cases subfield level to support translation. The question is how does it all work then, what do we do to translate content?

When can an entity have translations?

When editing an entity, there is a set of conditions to be met for it to have translation support:

  • The bundle (subtype) of the entity should have translation support configured.
  • The site should have more than one configured language.
  • The entity should have a configured language (not one of the "Not applicable" and "Not specified" built-in languages)

There is of course no requirement that the original content entity be of any specific language. Drupal can translate from varying source languages to other target languages.

Example: translate articles

A translate tab would appear on content like this. For example, in this case, I am editing an original Hungarian entity (actual words in English for sake of readers). Behind the translation tab is a list of languages on the site to use as target languages.

At this time, there is no English translation, so I can add one.

The translation form uses the same content editing experience and it even prefills all the original Hungarian field values. In this case, I configured the tags field to not be translatable. A subtle "(all languages)" note appears alongside the tags field. I have access to edit all fields for the translation as a content administrator. (See below for details on permissions).

Once the translation is saved, the content administration list now displays both. It is possible to act on each translation individually to make them published / unpublished (which in my case is per language as is by default). The list may also be filtered by language. What's really great is that this page is a view, so I can customize the content administration experience to support translators better. I can even clone this view/display and create a translation dashboard tab. That would allow me to provide distinct content management and translation management screens.

Other uses of translated content

But where else would the articles show up? Well, the front page is configured by default to show promoted articles in the language selected for the page. So if you view the front page in English, the English translation will show up there. If you view the front page in Hungarian, the Hungarian original will be displayed. Again, the front page is a view, so you can customize how it filters what it shows and what language is used to render the content it finds to display. We covered how views are configured with language options in the blocks and views tidbit.

Finally translated content is also integrated in search. The core search module indexes translations separately. So if you search for "English" in my example, you'll find the English translation immediately. (If your search found no results, make sure to run cron for the indexing to happen. One option is from Administration >> Reports >> Status report). Drupal's search integration APIs are also fully equipped with language information, so third party search integrations will know about translations just as well.

The place and life of translations

Translations exist under their respective original content entities. When editing the original content entity you get buttons to "Save and keep published (this translation)", "Save and unpublish (this translation)", "Preview" and "Delete". In this case "Delete" will remove the entire set of content with all its translations. The confirmation message makes note of that as well.

On the other hand, when editing a translation, the options are a little different. The last one there is "Delete translation" meaning you can delete the translation itself from its parent entity, keeping the parent and all other translations intact. This is possible because a translation is dependent on the "host entity" that it is a translation of. Without that, the translation could not exist. But the translation could be removed without any side effects on the rest of the content.

Drupal also tracks whether each translation is up to date. When editing a translation, one of the sidebar options is to mark all other translations as outdated. When making significant changes to the original content for example, you may want to signal to translators that they need to take a longer look on what to update in their translations. All this does is it puts a flag on the rest of the translations. Translators may filter to this or see this in an overview. When editing a translation, this flag can be unchecked if the translator believes the translation was sufficiently updated.

Permissions around content translation

In the configuration translation tidbit, we covered that there is one single flat permission for translating all of configuration. This is also true for interface translation. However, for content translation, the situation is more difficult. That is because content is created by a much wider audience. In the screenshots above when I created and edited my content translations, I have seen all the fields because I had all permissions on the site.

Content translation module provides a combination of permissions to allow you to configure who has access to edit translations in a much more gragular fashion.

  • The create/edit/delete translations permissions govern whether the respective actions may be done to translations (of some kind). For example, if someone only has permission to edit translations, they will not be able to add new ones or delete existing ones.
  • The create/edit/delete permissions need to be combined with type specific permissions. The overall "Translate any entity" permission gives global permissions to do what was granted of the above three permissions on any entity. For people who should not get that permission, each entity bundle configured to be translatable gets its own permission as well. For example, in my case I configured articles and tags to be translatable, so I get the respective permissions for them.

This way I can configure my proofreaders to only be able to edit translations of articles and tags. If I have legal papers as well, I can restricts their translatability to a more limited group. These permissions do not allow editing non-translatable fields (none of the fields marked with "all languages" would show up for these users). Such shared fields require permission to edit the content itself, because changing them would change the original content as well.

This is the extent of content translation permissions in core. You may agree it is already pretty advanced. However, for example, you cannot limit permissions to specific languages. As with search integration, the content permission system also got expanded in terms of language support, so it is possible to have entity access on a per language fashion as well. Contributed modules will fill in this role.

I hope this summary helped understand the inner workings, features and limitations of the new core content translation features. I think its a really huge step forward for Drupal, especially in the way it applies to all kinds of content entities in core, but is definitely not perfect yet.

In the next tidbit, we will look at some of the content translation API, so you can orient yourself where to look for dealing with content translations. After that I plan to look at complex core use cases where interface, content and configuration translation meet. If you know how the pieces map together, you can really translate practically anything!

Issues to work on

Nov 02 2015
Nov 02

I had the chance two weeks ago to talk about Drupal and Drupal 8 at the Free Software Foundation's conference in Budapest for a whole of 21 minutes. While there is this amazing 63 screen slideshow about all things new in Drupal 8 that I help keep up to date, having such short time really made me focus my message and think long and hard about how to summarize what Drupal 8 is really about for a wide range of people attending. Here is my take in written form.

Drupal has always been amazing as a structured content management tool. With content types and then entities and fields it allows us to really structure our content. Drupal 8 steps up this game several ways. First, it makes more things able to get structured. Whether it is a block or the categorization of content itself, it can be structured further with fields now. Drupal also has a history of using this structural system for flexible functionality. For example, ratings, workflows, user groups, selling content, etc. are all supported with fields (in contributed modules). In Drupal 8, comments are fields too (for example, you can take comments on user profiles) and there are more reusable field types like date, email, references, etc. built in. Best of all everything supports multiple languages and is translatable without further modules required. In short, Drupal 8 is improving on the system's key strength in all directions.

This is amazing for an enterprise because content needs to show up in a lot of places and a lot of ways these days. The more structured the content, the easier it is to pull out and display things for the environment needed. Drupal 8 makes this easy by building in entity view modes for display variants, Views for pulling data in whatever way from entities, and responsive output for flexible display on the web. Integration with third party systems and decoupled site implementations is enabled by web service support. Since Drupal knows so much more about your content structure internally as well, it can also intelligently cache (and invalidate the caches) when needed, and serve pages with much faster perceived performance (enabling the BigPipe contributed module). The markup generated is significantly better for accessibility too.

What about the small sites though? I think the changes are even more exciting there, because they lead to a lot more consistency on the site building front as well. You now use blocks to place everything on your pages (including branding, navigation and even the page title). You can use Views to customize even your administration experience and quick in-place editing and WYSIWYG integration for fields allows you to get further, faster. On top of that, rolling out changes is a whole lot easier with the built-in configuration deployment system.

Drupal 8 also grew the core community manyfold. While Drupal 7 had less than a thousand contributors, Drupal 8 has almost 3300. That is pretty remarkable, because it means the new version starts out with many more people already in the know.

All-in-all Drupal 8 really doubles down on our commitment to structured content and flexible functionality around it with a focus on making it easier to both enter and output that content however it fits, whoever the consumer is. It truly empowers you and me (as the tagline says) to build something amazing, for anyone.

Sep 19 2015
Sep 19

If you are interested in to learn about, solve your problems with and/or contribute to multilingual Drupal, DrupalCon Barcelona is the place to be. Here is a quick summary of things happening so you don't miss what is going on.

  1. Extended sprints before and after DrupalCon (19-21 and 26-27 September) are happening, and the multilingual team is there.
  2. Lingotek is holding a free training on Monday 1-5pm to translate Drupal sites with all the benefits of the lingotek module and services. Also a BoF on the same topic on Thu 10:45 to 11:45am
  3. Intersted in localize.drupal.org and translating Drupal itself? Come to and discuss at Bring Drupal 8 to all in their native languages on Monday 11am to noon.
  4. If you'd rather see the real life experience of Acronis, there is also a BoF presentation at the same time (Monday 11am to noon) titled Drupal 7 - Internationalisation for custom enterprise environment
  5. Drupal 8 comes with a whole new set of multilingual features, which combined with other improvements is even more powerful. Drupal 8 multilingual site building hacks is the place to be to see that in action.
  6. Just want your problems solved and move on? The multilingual therapy BoF is on for that on Wednesday lunchtime. Bring your lunch and questions/problems.
  7. If you don't have time or the opportunity to come to the Drupal 8 site building session, get the executive summary at the Acquia booth at 3:15-3:45pm on Wednesday (in the break).
  8. Tired of all this multilingual stuff? Want to learn how we managed to be so active for four years on so you can get your movement going better too? Your place is at Open source project management in the Drupal community, lessons from the multilingual team Thursday 1pm to 2pm.
  9. Just want to get stuff done (fix UI text, solve bugs, figure out issues) in core so you don't need to solve them over and over for your customers? We are also at the sprints on Friday and the community is even offering plenty of mentoring if you are entirely new. No excuses to not contribute!
  10. While not primarily multilingual content, there is also multilingual coverage in Entity storage, the Drupal 8 way, in Our experience with building Drupal 8 Sites in Alpha and Beta, in Configuration management in Drupal 8, in Building sites in Drupal 7 with an eye on Drupal 8, as well as in Let's build it on Drupal 8 and some others I likely missed (sorry for that).

Hope to see you at some of these places!

Jul 07 2015
Jul 07

Drupal 8 packs a historic amount of site building features which make producing websites easier than ever with core or just a couple contributed modules only. There are already various live Drupal 8 multilingual sites using little more but core.

It is hard to grasp the many things with useful levers and knobs in Drupal 8. Think about combining views with entity view modes and blocks; block language visibility with menus; user preferences with comment submission; language filtering and entity rendering; translatable fields with administration views; and so on and on.

Wouldn't it be fun to experiment with the possibilities and come up with clever ways to combine core features to solve common problems? You may be familiar with the name and format of O'Reilly's Hacks Series which reclaims the term "hacking" for the good guysfolks — innovators who explore and experiment, unearth shortcuts, create useful tools, and come up with fun things to try on their own. The excellent series inspired the name and format of our contest.

Long story short, hereby, we announce the Drupal 8 multilingual site building hacks contest!

Rules

  1. Come up with clever ways to combine Drupal 8 core features (and if needed one or at most two contributed modules) to fulfill a multilingual site building need.
  2. Write up the steps taken. See an example in hack #1. (We'll do light editing of the post if needed, don't let perfection be the enemy of good).
  3. Register on http://drupal8multilingual.org/user to submit entries (requires approval for spam protection).
  4. Submit entries by end of day (CEST) July 31st.
  5. One person may submit as many entries as they wish.
  6. All entries will be published after review (and possible light editing).

What is in it for you?

The top 3 best hacks will receive unique presents from Hook42 and Amazee Labs! (Further sponsors welcome). You'll either receive the presents at DrupalCon Barcelona or we'll mail it to you if you are not coming to DrupalCon. This is of course additionally to the joy of getting to play with some of the less frequented but definitely no less fun features of Drupal 8.

What is in it for us?

All hacks will be published under Creative Commons Attribution-ShareAlike 4.0, so the community will benefit. Additionally to that Gábor Hojtsy and Vijayachandran Mani are building an open source presentation with the best tips (same license). This will be presented at Drupalaton Hungary and DrupalCon Barcelona. Similar to our existing open source workshop, everyone will be able to present this at local meetups and camps or follow along at home at their own pace.

What kind of hacks are we looking for?

Hack #1 is hopefully a good example. Really the only common thread between the hacks would be to satisfy a multilingual site need or use multilingual features in some other clever way (even for features that are not necessarily multilingual). Some ideas for hacks that may help you start off experimenting:

  1. Swap textual site logo Need to swap a site logo with text on it for different languages? Use a translatable custom block with an image field. Configure the display mode and add some custom CSS if needed.
  2. Translator todo helper Create a views block for content translators to summarize the number of outdated translations they have to update (and link to content administration filtered to that language)
  3. Language dependent front page Use block visibility to display up to date content on a well maintained language while an About us / Contact us page on languages where resources are limited to maintain useful fresh content.

Of course these are just some things we made up (although still eligible for the contest). Looking for your creative ideas and solutions!

Questions, concerns? Contact us!

This is a crosspost from http://www.drupal8multilingual.org/hacks.

May 22 2015
May 22

The Drupal 8 multilingual team is really great in spreading know-how on the new things in the upcoming version, so we had our session (1h) and workshop (2h) recordings published and widely available. While we of course love our baby and can talk all day about it, who has hours when they just want to explore what is coming up? We just addressed that this week with the following.

1. New 2m22s introduction video with the key benefits

[embedded content]

2. A quick summary of key benefits and an easy to skim features list

http://www.drupal8multilingual.org/#topbenefits lists the top 12 benefits and http://www.drupal8multilingual.org/features provides the more detailed information in an easy to skim text form. And yeah, that 1h session video if you have the time.

3. Easy to launch demo to try features out

Thanks to our work on the multilingual workshops for DrupalCons, BADCamp and DrupalCamps, we have a demo with sample content in 4 languages that you can try out in your browser for 30 minutes without any registration or local software install required thanks to simplytest.me.

4. Check out who voted with their feet already

Drupal 8 is not yet released, yet there are numerous live multilingual Drupal 8 sites helping with nature preservation, finding health professionals or concert tickets among other good uses. Now there is a handy list to review at http://www.drupal8multilingual.org/showcase.

If you like what you see, we still have guided workshops (those that last 2h). The next one is coming up right this Sunday at DrupalCamp Spain. We also believe that the multilingual team is one of the best to get involved with if you want to know Drupal 8 better and give back some to improve the new version as well. We have weekly meetings and a huge sprint coming up at DrupalCon Barcelona. Maybe we'll have some opportunity to celebrate as well. See you there!

Feb 09 2015
Feb 09

I finally stopped putting it off and took the opportunity to test myself on the Acquia Certified Developer exam. To be honest I put it off for quite long. As a household name in the community I had fears it will prove I am not good enough and funnily enough, I did worst on back end development (ooops!) and 10% better on site building. My overall result is actually the same as Angie Byron at 85%. I'm flawless with fundamental web concepts at least. Ha!

As a computer science major who transferred into more of a mix of development, leadership, events and content production, I don't have much of an experience with tech certification exams. My only encounter was with the CIW certifications 13 or so years ago, which I took back in the day to be able to teach the CIW courses at a local private school. Judging from that experience and common wisdom, I expected paperbook style questions where I need to know the order and name of arguments and options on l() as well recite row styles of views and available options of date fields. The reality cannot be farther from that.

While there were one or two questions that actually required knowing Drupal API functions, most were very practical site building questions where you put yourself in the shoes of the architect / site builder and need to figure out a solution ideal for the eventual site users and content producers. I really liked the site builder questions because most are nudging you to best practices to use Drupal's features to their fullest. They also line up all the wrong things that Drupal beginners may do including client side workarounds to server problems and code in blocks. I can imagine how the vast experience of the Acquia support group contributed to "popular" wrong answers.

The code questions are not always ideally written (vs. Drupal's code style) so if you are really picky about your code style then that may make it a bit harder to process them. It is true, that not using Drupal's code style helps the questions make the answers a little bit harder to figure out, so you get one more twist and will not be able to instinctively pick the right answer. I think it would still be valuable for the code examples to conform more to the code style expected in Drupal.

Overall I found that the site building, frontend and backend questions covered a very good spectrum of Drupal 7. In particular I was pleased to find performance and security questions on different levels, which is great validation that you are making the right choices on both the code and the configuration of your site to ensure best security.

In most bigger projects I was involved with, I worked with other professionals, and therefore I encountered some siloing of different skills. I worked little with Javascript or templates even on sites I could put on my resume. As someone who interviewed candidates before, the listing of prior references is not enough for me to tell if the person had a wide or limited involvement in the projects. The Acquia Certified Developer exam tests people on a wide range of skills and helps prove more rounded capabilities. It also helped me see where some of my shortcomings are. I think it may be valuable to take the exam for that result itself.

90 minutes sounds like plenty of time for the test and I hoped to finish in around an hour but it took 83 minutes nonetheless. Not sure it is because one would use all the time available to make sure they pick the right answers, or it just takes that much time anyway. The question review tool was really great to get back to some items and it would even be better to be able to look at which questions I did wrong. I can see how having that opportunity would let people spread a study-book for the exam, which would not help reach the original goals. So I am left with a feeling of which frontend areas I need to develop and wondering which of my back end answers may have been wrong.

Disclosure: I am an Acquia employee and Acquia sponsored me to take the exam. As Angie wrote in her rightly famous post about it, Acquia employees can take the Acquia certifications for free (and yes, we are still hiring!).

Jan 27 2015
Jan 27

Up to date as of October 29th, 2015.

In the introduction to content and configuration translation piece we discussed what is considered content in Drupal 8. This is a somewhat misleading term because custom blocks, custom menu items and even user entities (user profiles) are considered content in terms of their implementation.

Content is always stored as entities. The main difference between configuration and content entities is configuration is usually created on the backend (think views, vocabularies, etc.) while content is usually created on the frontend (think free tagging taxonomy terms, comments, blog posts). This is not a black and white differentiation but it helps think of the categories. The other key differentiator is content entities usually get to have configurable fields. You can add new fields to user profiles, taxonomy terms or comments. Again there are exceptions, for example custom menu items cannot get configurable fields in core. Finally, there are even content entities that will not be stored, in Drupal 8 contact form submissions are content entities that live only until they are sent via email. For this tidbit we are concerned for content entities that are stored and multilingual.

In the fifth tidbit, we covered language assignment to content entities. That showed the Content language page under Regional and language configuration which lists all the content entity types and lets you configure language defaults and language selector visibility for bundles of each. That is very useful on multilingual sites where you don't translate posts, like a multilingual blog where you sometimes post in one language and then another.

If you also need content translation support, all you need to do is to enable the Content translation module and have multiple languages configured. The same screen can be used to configure content translatability that you already used to configure content language defaults. With content translation module enabled, the menu item changes from Content language to Content language and translation. Bingo!

This screen now lets you turn on and off translatability as well on an entity type and bundle (subtype) level. So you can configure nodes per content type, taxonomy terms per vocabulary, custom blocks per type, etc. Configuring a bundle to be translatable then opens a whole set of configuration on the field level. Built-in (base) fields are supported, so you can translate the title of nodes and name of taxonomy terms for example. Publishing metadata like author, creation date and change date "translation" lets you keep accountability on translations. Publication status tracking per language lets you implement workflows for translations, so you can keep some languages published while others are not (yet). Promotion and stickiness per language lets you keep different metadata per language variants. You can of course uncheck the ones which you do not intend to keep different per language.

Going further down on the field list, you'll notice that image fields even support translation on a sub-field level. That means that by default they offer to translate alt text and titles but keep the image itself the same across translations. This makes sense for product pictures for example. If you also need to have separate files per language, you can configure that too.

Finally, the article type also has a taxonomy tags reference field, which stores all related taxonomy terms. By making this field translatable, you can keep a different list of related taxonomy terms per language (Case A). It is also possible that you only want to translate the terms themselves, in which case you should uncheck this box and set the tags vocabulary terms to be translatable on the same page (Case B). That would mean you keep the same tags for all languages but translate the terms themselves. You can do both at once also (Case A+B) but that may be more confusing than useful.

If you already built multilingual sites with Drupal 7 that had content translation, you may notice this model is a refined version of the Drupal 7 Entity translation module (without awkward modules required like Title) rather than the Drupal 7 core content translation module. The core module in Drupal 7 keeps different copies of nodes and relates them together in a translation set. In Drupal 8 this feature is not available, instead you can configure translatability on a field (and in some cases subfield) basis. However if you configure all fields to be fully translatable, you can essentially reproduce the Drupal 7 behavior. Compared to the Drupal 7 core solution with different entity identifiers then, Drupal 8 will have the same entity identifier but a different language variant.

The biggest advantage of the Drupal 8 solution is you can configure to translate as much as needed and not more. With one unified solution to translate only some fields or all fields, (contributed) modules only need to deal with one system instead of two. With storing translations under one entity, modules that don't want or need to deal with multilingual scenarios can still consider the entity as one, no need for special translation set support. That is way more flexible than Drupal 7 and best of all it applies to all kinds of content, not just nodes. It is future proof for modules you enable like rules and commerce.

That's it for the basics. Next, we'll cover the user interfaces, permissions and basic workflow supported in core.

Issues to work on

  • DONE! Unfortunately language-aware entity types that cannot be translated (like Aggregator feed or File) will not show up in the master list for configuration. That is an oversight. Help at https://www.drupal.org/node/2397729
  • It is also possible to configure translatability for entity bundles on their own edit screen and then their fields on the field edit screens respectively. This is a lot more tedious and error prone compared to the huge overview screen we covered in this article. It may lead to incorrect settings combinations, see https://drupal.org/node/1893596. It also leads to bugs such as https://drupal.org/node/1894596.
  • PARTLY DONE: Miro Dietiker outlined some concerns with the in-entity translation system, some of which have since been resolved. The document is by far not up to date but would be interesting for anyone looking at possible technological challenges with the Drupal 8 approach in more complex environments: http://techblog.md-systems.ch/tutorial-howto/2012-06-drupal-8-multilingu.... There is a meta issue at https://www.drupal.org/node/2465901 with links to some of the problems opened as drupal.org issues.
Jan 06 2015
Jan 06

Whew! 2014 was a fantastic year for the Drupal multilingual team. We had some great events with huge sprints, including but not limited to: Global Sprint Weekend, the amazing Drupal Dev Days Europe, NYC Camp, DrupalCon Austin, DrupalCon Amsterdam and BADCamp.

A fun fact about people on the multilingual team is that even though we usually turn out in big numbers at sprints, there are numerous great mentors among us, so we don't work on code that much at mentored sprints. We do a great job helping people get started and move into more serious core work though. Our most famous mentee this past year is 2014th Drupal 8 core contributor Holly Ross, Executive Director of the Drupal Association who contributed her first and second core patches fixing multilingual issues.

Check out this video on her first contribution:

[embedded content]

While there is clearly a group of usual suspects opening and fixing issues, there is a really long tail of contributors as well. There were 80 different people opening issues and lots more, 131 contributing significantly to fix them (based on commit messages). While the top 6 contributed to fixing over 10 issues each, there is lots of room for helping in the initiative:

The multilingual team remains to be a friendly and fun place to start contributing to Drupal core.

As for whether we are working on the right things, 55% of our issues were bug reports and 44% were tasks. The remaining 1% is feature requests with only one support request. This is no surprise given the current phase of the release.

How good are we in fixing things? We opened 351 new multilingual issues in 2014, that is almost one every single day! We resolved 229 (65%) within the year. I am sure there are duplicates and some outdated reports among the ones still open but there are definitely remaining legitimate open issues too. But do we need experts to resolve them? Turns out we are very good at resolving the big problems. Although help is still welcome there if you have the capacity, there is even more space with the normal and minor issues especially for newcomers. These are great targets to bring to the Global Sprint Weekend in 2015 and further mentored sprints:

We do put in a lot of effort into resolving these issues. The total comment count on the 351 issues opened is 8033, so on average 23 comments per issue. The total contributor count on Drupal 8 multilingual issues based on comments to date is above 1100 people (including all previous years), so we have lots of commenters who may not post patch updates but help test and provide feedback to move issues forward. Compared to the above mentioned 134 people who were highlighted in commit messages in 2014.

Looking at how much overall impact our changes have on Drupal 8, in terms of numbers, out of the 3736 total Drupal 8 commits made last year, 186 commits(*) referred an issue from the multilingual initiative, that is about 5%. I am sure the actual impact of the changes made is much more significant. We filed at least 10 issues each in key components like the language system, configuration system, language.module, content_translation.module, locale.module, config_translation.module, the install system, the entity system, the menu system, CSS and views.module. The most important multilingual changes are indeed found in these areas.

Given the pieces were coming together, we got published in the DrupalCon Austin edition of Drupal Watchdog. Check out Drupal in Babel for a good overview of all the new things for foreign language and multilingual sites in Drupal 8. We also created a nice Drupal 8 distribution to demonstrate the new multilingual features with sample content in multiple languages. Even better, we developed a workshop around this distribution so you can immerse yourself in the new elements and walk through building foreign language or multilingual sites with Drupal 8 yourself or deliver the workshop to a local audience. The workshop handout, slides, demonstration distribution and the two hours of original recording is all open sourced. It is easy and free to try it out on simplytest.me for 30 minutes.

Let me extend my thanks to all 1100+ contributors on the initiative! Here's for a strong 2015, let's bring all our innovation to live sites soon!

(*) The number of commits is lower than the number of closed issues because we have issues for discussions and identify duplicate and outdated issues from time to time.

Photo of fran seva sprinting on multilingual issues in Amsterdam by Pedro Lozano from Flickr.

Dec 12 2014
Dec 12

After over a month of concentrated work, Drupal 8 was ready today to finally flip the switch and enforce strict configuration schema adherence in all TestBase derived tests in core. See the announcement in the core group.

If you are a Drupal 8 contrib developer and provided some configuration schema earlier (or you integrate with an existing core system like blocks, views, fields, etc.) then your tests may now fail with configuration schema errors. Unless of course all your configuration schema is correct: #highfive for you then.

Otherwise I thought you'll have questions. There is of course the existing configuration schema documentation that I helped write. However if you are a visual person and want to get an understanding of the basics fast, I thought a cheat sheet would be a great tool. So sat down today and produced this one in the hopes it will help you all! Enjoy!

Please provide feedback for improvements in the comments, and I'll update the sheet as appropriate.

Changes:

  • 1.0: Initial version
  • 1.1: Different background colour for schemas for readability; Fixed incorrect [key] and [%parent] examples; some small fixes; thanks Wim Leers
  • 1.2: Better wording for parent example labels, clarify naming of internal type vs. top level type; thanks Wim Leers
  • 1.3: Fixed config name matching in both examples on second page; fixed 'messages' key matching in last example; minor typos; thanks Michael Prasuhn
  • 1.4: Update for simplified sequence format introduced in https://www.drupal.org/node/2414539
  • 1.5: Update for new base type for configuration files, config_object, see https://www.drupal.org/node/2460751
Oct 17 2014
Oct 17

I just had the time to watch Larry Garfield's DrupalCon Amsterdam core conversation on managing complexity today. I did not have the chance to attend his session live due to other obligations, but it is nonetheless a topic I am very interested in.

The key point of the talk in my understanding is the Drupal community needs to decouple and be component based (evolving around more independent components), so responsibility and authority is distributed and local. Larry specifically calls out that Drupal 8 initiative leads and component maintainers are "glorified secretaries" with responsibility but no explicitly granted authority.

I am not a native speaker and although I had an idea, I wanted to clear up what authority would mean. According to Google the power or right to give orders, make decisions, and enforce obedience. (I'll use hard power as a synonym). Larry alludes to parts of this definition in the talk with examples. While the talk is well worth the hour to get insights form one of the Drupal 8 initiative leads on some of the struggles we had in the Drupal 8 cycle so far, I think there are fundamental issues with the premise. The biggest fallacy is the gross generalisation and picking one facet out of a very multi-faceted situation, so the proposed solutions don't stand deeper scrutiny.

Let's assume that leaving everything else as given, we grant authority (by the above definition) to some selected people in Drupal core. How would they be selected? Probably the same way leads are now, coming up the ranks, showing expertise and availability in a given technical area. It is hard to see that without giving in lots of those 1-hour 1-votes that Larry is concerned about, anyone would grow to be recognised as an authority. What would be other ways?

Let's assume they are there. Now what? Would everyone suddenly agree with their set direction and go and implement them? Why would they? If Dries says Alice and Bob should now have authority in X that does not change the incentives of contributors. Alice and Bob still need to spend all the time architecting solutions, communicating them clearly, rally people around their idea and then being available to gatekeep changes in their area, so to ensure they conform (see enforce obedience). What if contributors don't agree or Bob and Alice are not good communicators to get the word out? If Dries gave them authority, how does that give them the power to give orders? Where are the people waiting to implement Alice and Bob's great ideas? The only way the model proposed by Larry would work in this case is if the component in question is so small that Alice and Bob themselves can move it forward (or if they have or at least also control money to hire people to work on it, despite disagreements with contributors).

So the authority solution would solve Larry's concerns with those winning who have "time they don't know how to spend better" in two scenarios:

  1. If the components are (a) so decoupled (b) so small and (c) interface with each other so rarely that you can become an authority without needing to put in so much time ahead and then constantly, to exercise your authority
  2. Or if there is abundant money to pay both authorities for their time and then implementors to abide the authority even if they disagree

Both scenarios are interesting. I don't think the first will happen in several years (way into the middle of Drupal 9 earliest), and I doubt all three subpoints would apply. I really wish the funding situation will improve, there are so many people working on it, but I am not sure paying people to work on things they are not excited about / agree with is the spirit of open source anymore.

In short, hard power and a volunteer based open source community are not compatible on the long run. You either need to lose the volunteerism or gain soft power which authority does not help you with.

And initiatives in Drupal 8 were especially set up to be "soft power centers". Larry says initiative leads are merely "glorified secretaries" which I think is a very unfortunate misunderstanding from an initiative lead. The official definition of initiative leads says they coordinate work on the initiative and communicate the plans, progress, and needs of the initiative to the community and the branch maintainers. While this does not give them authority, as shown above giving authority in itself would not have helped.

Initiatives are the first try to set up bigger areas of work with appointed leaders. By highlighting these topics, end users know some of the major improvements to expect and contributors know where to contribute effectively. An initiative is a self organised unit with meetings, possibly its own website, issue tags, sprints, blog posts, sessions, etc. If a contributor cannot figure out how to get involved that is then a problem with the initiative.

So my understanding of initiative leadership was that I have responsibility to show the community there is a (1) shared goal, that (2) it is an achievable goal, and then (3) help people get there. There are a myriad of ways an initiative lead can do these, I did at least the following in the multilingual initiative:

  • I set up a website to promote the goals and progress of the initiative and a twitter account to follow progress
  • I held "Multilingual therapy" BoFs at DrupalCons to gather common problems from real scenarios to inform our decisions
  • I worked with people experienced in user testing to help us develop a user testing script for key actions and coordinated several user tests; circled back results to improvements
  • I distributed power in my initiative to three key areas relying on topic experts Erik Stielstra (Sutharsan), Francesco Placella (plach) and Jose Reyero (in alphabetical order)
  • I made a point of not being attached to specific architectures so that I can scale myself (but it also turned out to be great to stay sane)
  • I figured out a tagging system for the three areas and adopted one for the current focus issues (thanks Jacine Luisi for the idea)
  • I built a custom visualisation tool to help communicate our goals / progress after trying mindmaps and other tools that failed
  • I grabbed new people coming to sprints; the most famous example is Cathy Theys (YesCT) at DrupalCon Denver who is a fabulous team player and mentor
  • I made an effort to try to make people on my team successful by finding reviewers, pairing people up, etc.
  • I organised sprints to get the team together, Denver being the first "extended DrupalCon sprint" but only for multilingual; it grew into a generic extended sprint and transformed some events such as Drupal Dev Days
  • I did my best to find funding for these sprints so at least we had a good venue and if possible people got fed
  • I wrote a blog post series to communicate changes and point out missing pieces and lingering flaws at the same time, which helped drive some contribution.
  • I made effort to find replacement for people when they left (for a while), including myself
  • I held a fast paced talk of all the changes and the new multilingual structure in Drupal 8 at DrupalCons and Camps to inform people and recruit new contributors
  • I developed a two hour lab with Aimee Degnan and Kristen Pol of Hook42 for DrupalCon Amsterdam which we "open sourced" (slides, handout, demo script, Drupal 8 distribution) to be presented around the world
  • I wanted to acknowledge all issue contributors, not just patch contributors, so I created a page to credit all of them (above 1100 people) and bring them to life with photos from events
  • I organize weekly IRC meetings (more inclusive for foreign language folks than video meetings) to discuss progress, find people to help each other with issues
  • I developed a bad case of RSI two weeks before DrupalCon Amsterdam and was in pain; at the sprints therefore I focused on bringing food/coffee to discussions and clearing out tables of trash instead of typing, so I can help people not spill soda on their laptops
  • I read a lot of literature on leadership and suggested my favourite find in practically all related conversations: Switch - how to make change when change is hard (protip: the whole book is about getting your way in things you don't have authority about, which is most things in life)
  • I called core maintainer attention to important issues and helped explain decisions when needed (more on this below)

All of these one by one helped immensely to make great progress on the initiative. All of them required time to implement, some more significant than others. None of them would have come any easier if I was given authority. Not a single one of them. (Unless authority comes with a sizeable budget that I can direct so others do these). And all of these are well within the definition of the initiative lead published as coordinate work on the initiative and communicate the plans, progress, and needs of the initiative to the community and the branch maintainers. As for whether initiative leads like myself are glorified secretaries then I leave to the reader to decide.

But what about power then? Are initiative leads powerless people with way more responsibility then they should have? As a matter of fact leaders have unparalleled access to core committers. Leads had a 2 hour phone meeting with Dries every other week and Dries asked leads to attend separate issue review phone calls for high stakes issues. Core committers in general were eager to discuss pressing issues on IRC with leads. Longer sprint events made a point to have core committers sit in person on prescheduled meetings for several days with initiative leads discussing important topics (special thanks to xjm for organizing these!). That is a level of access that is far above what anybody else in the community could attain. (This in itself did not guarantee an initiative lead supported solution would get committed, and I had a major counter-example as well). So while saying an initiative lead has no formally documented power may be true, that it does not actually have power is way too far from the truth.

Several people said the multilingual initiative, that is my experience is a special flower and the same tools or processes would not apply to other topics in Drupal (or even Open Source in general). Let's return to my understanding of initiative leadership that one needs to show the community there is a (1) shared goal, that (2) it is an achievable goal, and then (3) help people get there. I don't think it can be said for any of the initiatives that a huge number of people would not share the goal (even if they disagree on some steps). In my view if a lead looks to defining that shared goal and help drive people there, then they can use all these tools effectively and get to good results regardless of whether they are (also) granted authority or not. Authority is not what gets you there, appreciating leadership to its fullest (not as being a secretary) does.

Aug 18 2014
Aug 18

Drupalaton 2014 was amazing. I got involved pretty late in the organization when we added sprinting capacity on all four days, but I must say doing that was well worth it. While the pre-planned schedule of the event focused on longer full day and half day workshops on business English, automation, rules, commerce, multilingual, etc. the sprint was thriving with backend developer luminaries such as Wim Leers, dawehner, fago, swentel, pfrennsen, dasjo as well as sizable frontend crew such as mortendk, lewisnyman, rteijeiro, emmamaria, etc. This setup allowed us to work on a very wide range of issues.

The list of 70+ issues we worked on shows our work on the drupal.org infrastructure, numerous frontend issues to clean up Drupal's markup, important performance problems, several release critical issues and significant work on all three non-postponed beta blockers at the time.

Our coordinated timing with the TCDrupal sprints really helped in working on some of the same issues together. We successfully closed one of the beta blockers shortly after the sprint thanks to coordinated efforts between the two events.

Our list of issues also shows the success of the Rules training on the first day in bringing new people in to porting Rules components, as well as work on other important contributed modules: fixing issues with the Git deploy module's Drupal 8 port and work on the Drupal 8 version of CAPTCHA.

Thanks to the organizers, the sponsors of the event including the Drupal Association Community Cultivation Grants program for enabling us to have some of the most important Drupal developers work together on pressing issues, eat healthy and have fun on the way.

Ps. There is never a lack of opportunity to work with these amazing people. Several days of sprints are coming up around DrupalCon Amsterdam in a little over a month! The weekend sprint locations before/after the DrupalCon days are also really cool! See you there!

Aug 14 2014
Aug 14

DrupalCon Amsterdam is coming up in just a few weeks and it is full of opportunities to learn about and get all your questions answered when it comes to multilingual Drupal. What's better, you can get involved making things happen and learn from those implementing the features firsthand. Here are my picks:

Multilingual Drupal 8 site building and programming

  • There is no excuse to not attend some of the sprints at and around DrupalCon. Sprints start two days ahead of the start of the conference on Saturday the week before. And there are still sprints going on the Sunday after the conference. It is not just the last day of DrupalCon itself where you can get involved and make a difference. In fact the leads are actually focusing more on the sprint on the weekend days. Also the weekend sprints are in a really cool venue. The best way to learn is to do!
  • You are looking for more of a directed guide of Drupal 8 still with the possibility to do it all hands-on? Look no further than the Drupal 8 multilingual hands-on lab presented by Aimee Degnan of Hook42 and myself from Acquia. The schedule info is a bit misleading, this session spans two timeslots and lasts two hours. Bring your laptop with Drupal 8 freshly installed!
  • Dive deeper into the APIs of Drupal 8! Francesco Placella from Tag1 presents Multilingual Content in D8: a Highly Evolved Permutated API showing how to code with the new system. While not strictly multilingual, in Field API is dead. Long live Entity Field API! swentel, yched and amateescu show how the most essential content element storage system changed and this is full of multilingual support of course.

Multilingual Drupal 7 site building and programming

Moving localize.drupal.org forward

The localize.drupal.org site seriously needs people who care about it enough to devote time to maintaining and fixing bugs. I set up one more BoF to gather people interesting in the well-being of this site titled We love localize.drupal.org. We need to upgrade to Drupal 7, support the whole range of new Drupal 8 APIs, drastically improve performance and then get new features going.

These are all the multilingual pieces that I collected. There may still be more, BoF scheduling just started and I may have missed a session or two. Let us know in the comments what other great events happen around multilingual Drupal. See you in Amsterdam!

Jun 24 2014
Jun 24

Drupal.org provides an amazingly flexible issue queue and is the backbone of most community activity around code, community, policies, drupal.org itself and so on. Each issue has a priority value which can be one of Critical, Major, Normal and Minor. Even more interesting is the tagging system we use with some commonly used tags like 'beta blocker' or 'beta target' or 'revisit before release' which add extra priority on top of the single value field. The drupal.org issues however don't lend themselves to supporting working on your priorities. Here are some options and tools I used so far that help solve this issue.

Strategic tagging

Some people have their own tags, but this is very limited. You likely would not get away with using your own tag like Favorite-of-Dries. Instead people use tag combinations to express priority on other scales (eg. time vs. severity). I've first seen Jacine Luisi (HTML 5 initive lead at the time) using the 'sprint' tag in combination with other tags to express priority in time. Issues tagged 'sprint' would be reviewed on meetings. This is good for expressing priority of a team.

Following issues and the dashboard

However, if you are not working as part of an integral team working in sprints and maybe just want to keep tabs on issues important for your projects, this will not work. Enter the Follow button. On each issue (when logged in) you can find a follow button to follow updates on those issues. All the issues you follow show up on the Your issues dashboard block. Access the dashboard using the big blue Your Dashboard tab from the header. Several people use this to keep tabs on issues they are interested in.

The first problem with this approach is this does not allow segmentation of the issues (eg. ones you work on vs. ones you follow out of interest). A much bigger problem is that the Your issues block/view in fact reflects priorities of others. Issues that people work on will show up higher in the list pushing others down.

Externalizing issues #1: email

So how to reflect your own priorities then? Well, drupal.org has a very easy way for you to get emails about all the updates on issues. This is super-useful even if it sounds very old-school. Who wants issue updates pushed to them? Well, several people in the core development community. Why? This easily creates copies of issue data in your own system that you can then use to tag, prioritize and follow up on as you see fit without issues scrolling by in the Your issues view.

To get email updates, go to your profile and in editing, hit the Notifications tab. This will let you subscribe to issues per project (and even have a global default). Then you can use your own email system's ways to tag issues and prioritise them. You don't even need to make them show up in your inbox. Cathy Theys (@YesCT) says she likes this solution because it lets her search in issues she cares about locally, which is blazing fast. Lee Rowlands (@larowlan) manages his daily issue contribution with tagged emails. I like it because I can keep issues I care about tagged and reply on my own schedule while I can ignore issue updates I merely follow on the side. Here is how autotagged emails look in my inbox:

I set up filters for tags showing up in the tag list in the email, for example contents matching "Configuration system [", "Entity Field API [", etc. get their respective cmi and entity tags. The drupal.org tags show up suffixed with a number in brackets (therefore the opening bracket in the filter). These filters are not 100% accurate but they almost never failed for me.

Externalizing issues #2: Google docs (temporarily not applicable)

Email is good if you need your personal priorities reflected. However if you need to coordinate a team working on a project and contribute to drupal.org at the same time, sharing an email account sounds stone-age. And it is. Roger López (@zroger) originally created this quick little Google sheet script that used to pull JSON issue data from drupal.org into Google Spreadsheets:

function drupal_org_issue_fetch(issue_number) {
  if (issue_number == "") {
    return "";
  }

  var response = UrlFetchApp.fetch("http://drupal.org/node/" + issue_number + "/project-issue/json");
  if (response.getResponseCode() !== 200) {
    return {};
  }

  return Utilities.jsonParse(response.getContentText());
}

function drupal_org_issue_property(issue_number, property) {
  if (issue_number == "") {
    return "";
  }

  if (typeof issue_number != "number") {
    return("error: input must be a number");
  }

  var issue = drupal_org_issue_fetch(issue_number);
  return issue[property] || "";
}

(This script can be added to a Google Sheet in Tools > Script editor...). At Acquia we used to use this script a lot for reflecting priority of the team when the priority of the team did not necessarily blend with global priorities. A bug may be minor for a module, if the success of your project depends on it, it will be critical for you. It was very easy to use this to build a sheet with all kinds of additional information on issues. Just enter the issue numbers verbatim in cells A2, 3, 4, 5, etc. (assuming the first row is used for a header) and use the drupal_org_issue_property() function provided to fill in dynamic values in other cells. Such as fill in B2 with =drupal_org_issue_property($A2, "title"), C3 with =drupal_org_issue_property($A4, "status"), etc. Add any cells that reflect local information to your team, such as your internal priority for your convenience. Of course you should always, always post updates on the issue each time you have something useful to say and not use this internal copy as an opportunity to fork the issue. That would be against your best interests.

Unfortunately this method cannot be used right now. The issue JSON output went away with the Drupal 7 upgrade of drupal.org quite a while back. So there is no way to programatically request issue metainformation now. Deploy RestWS for D7 project issue JSON is an issue to get this feature back. As soon as that is fixed, you will be able to use this technique (adjusted for the format of the JSON at the time).

Externalizing issues #3: Rocketship

While I am happy with the email solution for my own priorities, I also built a tool to help the multilingual team manage our priorities and to communicate them to the wide world. That is why I built Rocketship, a tool to pull in issues from drupal.org to a site and create custom issue boards out of them. This tool builds on the structured tagging concept for teams (although by treating status, priority, etc. as tags it expands the boundaries a bit). Several people see the main benefit of Rocketship as a Kanban-like board builder that is a better visualization of issues tagged in certain ways. On top of that I see it as a great way to augment issues with external information and put them into context. For example Multilingual novice issues are augmented with information on where and how to get help. The base language services page explains the overall plan in Drupal 8 for that area.

What's even better is that overall we are putting the issues in context with all the rest of the information of the initiative. The whole site that has the Rocketship generated boards serves as an overview for visitors on what we do (video and text summary), who we are (a friendly and welcoming team ideal for Drupal 8 first-timers to work with) as well as where and when we meet. In summary this allows us to put issues in context, reflect our own priorities and augment with information most relevant for the initiative. Basically the same as the Google sheet or the email solution but this is for the wider world.

Your solution

The common element in these solutions is externalizing issues in another system so you can maintain/list issues with your priority in mind. It is not very likely that drupal.org will get personal / shared team tags anytime soon so these tools will have great use for quite a while. Again, I'd like to point out that all updates should be posted on the issues themselves, it would be against your best interest to not maintain the best information as applicable to drupal.org so you can work with other interested parties in the community best. What methods are you using to reflect personal / team priorities which cannot be represented on drupal.org?

May 26 2014
May 26

We learned how configuration translation works on the conceptual level and through the Drupal 8 built-in user interfaces in the previous article of the series. In this article, we'll cover how can developers integrate with configuration translation.

Explaining the structure of configuration

We used your main site configuration file as an example in the previous article. Let's review that again (system.site.yml):

uuid: ''
name: Drupal
mail: ''
slogan: ''
page:
  403: ''
  404: ''
  front: user/login
admin_compact_mode: false
weight_select_max: 100
langcode: en

There are clearly some translatable elements here. Which ones? Well, at least the site name and slogan would be. How would Drupal know though? There is nothing in this file to tell Drupal about that. There is also no code in handling this configuration that needs to deal with that. We wanted to introduce language support in the most transparent way. Instead Drupal supports a static description format to describe the structure of configuration, that we call configuration schema.

The corresponding configuration schema snippet for this configuration is as follows:

system.site:
  type: mapping
  label: 'Site information'
  mapping:
    uuid:
      type: string
      label: 'Site UUID'
    name:
      type: label
      label: 'Site name'
    mail:
      type: email
      label: 'E-mail address'
    slogan:
      type: label
      label: 'Slogan'
    page:
      type: mapping
      label: 'Pages'
      mapping:
        403:
          type: path
          label: 'Default 403 (access denied) page'
        404:
          type: path
          label: 'Default 404 (not found) page'
        front:
          type: path
          label: 'Default front page'
    admin_compact_mode:
      type: boolean
      label: 'Compact mode'
    weight_select_max:
      type: integer
      label: 'Weight element maximum value'
    langcode:
      type: string
      label: 'Default language'
    mail_notification:
      type: string
      label: 'Notification email address'

While that looks like a bit too much, it is pretty simple if you look at the pieces. The top level defines the file as a mapping of key-value pairs and then each key is defined with their type and label. The site name and slogan are both defined using the label type which is defined as translatable in the system. The other elements use string, path, boolean, integer, etc. types. The pages key is in fact defined as a mapping in itself that has three nested keys.

Applying this schema to your configuration, we can tell that the name and slogan keys are translatable and can deal with that for configuration translation. Why have such a complete description of the configuration if we only use a little portion of the information? Well, configuration schema is not only used for translation. For example, when you edit configuration through forms and save the changes, the same schema is used to type-cast values to the right types. That ensures that although forms on the web are all string based, we still end up with the right boolean, integer, etc. values in the configuration file. This is really valuable when it comes to deploying configuration changes, since type changes are not going to happen accidentally. Finally, its not just the types used from the schema. The core configuration translation module in fact generates forms dynamically based on the schema to offer translation forms.

This means that although configuration schemas are optional, if you implement them, you not only integrate with the configuration translation backend, provide all the necessary elements for configuration translation frontends to be generated but even play nice with the deployment system. That is a lot of value for some static YAML files.

Configuration schema can describe dynamic structures as well, for example it is able to describe field settings which may be type specific or views which contain arbitrary plugins nested within each other. I created a fancy cheat sheet and Drupal.org has some great documentation on configuration schema. There is also the Configuration inspector module which helps to debug schema as applied to configuration.

The final question is why is it a static format, why is this not encoded within the PHP code handling configuration? First its used for so many different things that it needed its own definition. Second, it is also going to be used on localize.drupal.org to read default configuration in projects and make translatable default configuration elements available for community translation. That requires that no code should be necessary to run from your project to be able to tell what parts are translatable.

Integrating with the configuration translation user interface

As I explained in the introduction to content and configuration configuration now covers both global settings like site information (as shown above) and configuration entities, where multiple instances of configuration may be present. Examples of those are input formats, user roles, views, menus, fields, and so on. Integration of either of these with the configuration translation system starts with writing your configuration schema. Providing the schema itself is not enough though, the system needs to be able to integrate the translation tab and generate the form for you at the right path.

Global configuration

If you are providing your own global configuration, then you need to tell the configuration translation system which (administration) pages belong to your configuration. Provide those in a MODULENAME.config_translation.yml file. For example, system module provides an item for the site level settings as follows:

system.site_information_settings:
  title: 'System information'
  base_route_name: system.site_information_settings
  names:
    - system.site

In this file, the key is by convention the same as the base route name, the route belongs to the page where this configuration is edited. (Routing is a new distinct subsystem to map paths to controllers in Drupal 8 that we are not diving into here). Finally, it lists all the configuration keys that are edited on that page. This is enough for configuration translation module to be able to attach a new translation tab to that page and generate all the rest of the pages from there, the translation overview, the translation forms, saving of translations, and so on.

It is worth reiterating, the only thing needed to make your global configuration translatable is to describe its structure with schemas and map its configuration key to a page, so the core module can do the rest of the job itself. You only need to add some static YAML files to your module, no code to be written. I think its hard if not impossible to make this easier.

Configuration entities

The situation for configuration entities is just slightly different. You define configuration entities with an annotated class (which we are also not going to dive into here). You are very likely going to provide an edit-form route for your configuration entity type, which is the only thing needed for configuration translation to integrate with the user interface of your configuration entity. An example could be the contact form configuration entity:

<?php
/**
 * Defines the contact form entity.
 *
 * @ConfigEntityType(
 *   id = "contact_form",
 *   label = @Translation("Contact form"),
 *   [...]
 *   links = {
 *     "delete-form" = "/admin/structure/contact/manage/{contact_form}/delete",
 *     "edit-form" = "/admin/structure/contact/manage/{contact_form}",
 *     "collection" = "/admin/structure/contact",
 *   }
 * )
 */
class ContactForm extends ConfigEntityBundleBase implements ContactFormInterface {
}
?>

This ensures the configuration translation module can attach the tab and add its overview and form on the right pages. See the relevant code in config_translation_entity_type_alter(). Once again all the configuration translation management is done transparently, no form or tab or path or anything else needs to be defined other then what you would do anyway for the configuration entity. In this case, you don't even need to ship with a code>MODULENAME.config_translation.yml file.

Structure and placement of translations

As explained in the previous article, translations are not represented by storing the entire configuration file in a different language but instead only the translated pieces. When you export your configuration, translations are located in subdirectories per language under a language directory in the export, so for site settings in three languages, originally English with Hungarian and Spanish as translations, the following files would appear on export:

system.site.yml
language/es/system.site.yml
language/hu/system.site.yml

Both translation files would only contain the name and slogan keys and no other information, because those were the only ones set up for translation. The Spanish file may contain:

name: 'Noticias de última hora'
slogan: 'Información fresca'

It is also possible to ship default configuration translation with your project. Just place your translation files the same way into your modulename/config/install directory in subdirectories. This may be a way to ship some translations with custom projects you built for a client. Note that shipping with translations is absolutely discouraged for drupal.org projects, because for those, translation is happening on localize.drupal.org and the Interface translation module downloads and manages those translations already, giving control to translators.

Accessing translated configuration

Drupal by default always uses the language selected for the page to load configuration with. So if you are viewing a Spanish page, all configuration is loaded with Spanish translations merged in. If you are viewing the same page in Hungarian, the same code will now receive Hungarian translated configuration. This means normally you don't need to do anything special to access configuration in the language needed.

However, there are cases, when you want to load the original copy of the configuration or ask for a specific language. Such as when sending emails to users, you will need configuration values in the right language. The following code is a slightly adapted excerpt from user_mail() to illustrate loading configuration in the preferred language of $account to compose an email:

<?php
  $language_manager
= \Drupal::languageManager();
 
$language = $language_manager->getLanguage($account->getPreferredLangcode());
 
$original_language = $language_manager->getConfigOverrideLanguage();
 
$language_manager->setConfigOverrideLanguage($language);
 
$mail_config = \Drupal::config('user.mail');
 
// ...
  // ...Compose email here...
  // ...
 
$language_manager->setConfigOverrideLanguage($original_language);
?>

Note that you are settings values on the language manager and not the configuration system directly. The configuration system is override-agnostic and can support overrides of different kinds with various conditions. It is the job of the overrides to manage their conditions, in this case to allow changing the language used. The same pattern can be used to load configuration entities in specific languages as well.

Just loading the original copy of the configuration without any overrides (include skipping non-language overrides) is a bit simpler:

<?php
  $config_with_overrides
= \Drupal::config('system.site');
 
$config_without_overrides = \Drupal::configFactory()->getEditable('system.site');
?>

This code example points at another clear difference. While the configuration system applies overrides appropriate for the current context when you get configuration, that means that saving back to that may lead to overrides ending up in your original configuration. That could lead to various problems including security concerns. Therefore only configuration accessed as editable (which does not have overrides) supports modifying values. If you try to modify values on non-modifiable configuration, you will immediately get an exception.

Overrides are not only used for language but also for global value enforcement and may be dependent on group, domain or any other conditions. While these code examples don't look very trivial, good news is you very rarely need to use these patterns. Drupal attempts to do its best to make good assumptions for you in the right environment. For example admin pages set up to edit configuration entities get the entity loaded in an editable form without overrides.

Read more about the override system in the drupal.org documentation.

Issues to work on

  • PARTLY DONE: We still need to make localize.drupal.org expose default configuration for translation. That is unfortunately a non-trivial problem, but we need to solve before translators are asked to translate for the release. See and help at https://drupal.org/node/1933988
  • DONE! You may have noticed overrides can entirely be disabled by code. The enforced application of global overrides especially when important for security is still debated in https://drupal.org/node/1934152.
  • DONE! The configuration schema system is in theory extensible, but not all applications are currently implemented in a generic enough way. See https://drupal.org/node/1928868 for making it return to and fully embrace its original Typed data roots.
  • DONE! While schemas are not required, Drupal core attempts to provide complete schemas for all its module settings. There are still some small gaps and bugs, see https://drupal.org/project/issues/search?projects=&project_issue_followe...
May 19 2014
May 19

After a long 8 months break in the article series, we are back to talk about configuration translation basics. Why the long break? Well, both the configuration and content system was in heavy development with changes and I did not want to get you content that would be quickly outdated. In the meantime Alex Pott also posted a great set of articles titled Principles of Configuration Management Part 1 and Part 2 which serves as great introductions. We’ll cover configuration translation first because that is more baked.

The Drupal 8 configuration system is a boon for language

As I wrote in the previous article in the series, configuration is now encompassing lots of settings that were variables or used custom settings storage in Drupal 7. The biggest value for non-English and multilingual sites in Drupal 8 of the configuration changes is that now a common system is used to manage your site name, email text settings through to views, field settings, entity form displays, etc. We can introduce language and translation support in a way that modules will need to plan with. It is not just an optional contributed add-on but a core feature.

How we know about the language of your configuration?

The Drupal 8 configuration system uses YAML files to transfer settings. (Active configuration is actually stored in the database). These are really simple text based files that have an internal nested-tree data format to them. On the Drupal side, they are just a simple way to store a nested array of data. For language support, the granularity of these files is of importance because we store/support language on the file level. For example, your site settings by default is the following:

uuid: ''
name: Drupal
mail: ''
slogan: ''
page:
  403: ''
  404: ''
  front: user/login
admin_compact_mode: false
weight_select_max: 100
langcode: en

This file is located at core/modules/system/config/install/system.site.yml and as the file location indicates only used at installation time to set up the default site settings. The installer itself fills in the missing pieces, such as default mail address, the site name and uuid and copies this over to the active configuration storage in the database. This file is not used anymore after that and is not changed at its original location.

The langcode key is a “reserved key” we use on the file level to identify the language of the file itself. The “en” code in the file above means that all textual content in this file is considered English for the system. The langcode on this one specific configuration is even more special. This is how we identify the default language of the site as well. When you install Drupal, it sets the language of file to the language used in the installer. The langcode on all other configuration files is just limited to meaning of the language of the data in that respective file.

This sets up the boundaries of language assignment granularity for Drupal 8. A view is stored in one file, so we can store the language of the view overall and we’ll need to assume all strings in the view are in that language, including display names, pager text, empty result information, etc. All user emails sent by the built-in user module are stored in one file, so all those mails are supposed to be in the same language originally.

The system.site configuration gets language assigned in the installer and can be changed by changing the site default language. In the fifth article of my series I covered language assignment across Drupal. When you edit a view, its language can be assigned in the main metadata popup. You can assign language to menus, contact categories, etc. using a simple language selector on them.

So that ends up in a mess of configuration in all kinds of languages?

Sort of. When Drupal and its modules are installed, all of those configuration items are created in English (except system.site that is as explained above). If you did install in a foreign language though, English is not even a configured language on the site. All the new configuration you create will then be in your site’s language. If you add back English or add any number of other languages, you will be able to create configuration in those languages too.

So your site may have configuration in a mix of languages. In fact unless your site is English only, your site will very likely have configuration in a mix of languages. All default installed configuration will be in English while your created configuration may be in any other language. The configuration system is fully equipped to deal with this situation and be able to translate from all source languages to all other target languages. English can even be a source language if it is not configured on the site.

I made this figure to illustrate how a Spanish site might look like even if it does not have English configured on the site. The default installed configuration would be maintained in English with translations available and site specific configuration added in Spanish.

The ultimate result is all the configuration is available in Spanish which was the goal. A multilingual site is a more complex version of the same thing, where configuration may be available in all kinds of languages. The translation is transparent to the rest of the system so when a view is used for example, the right language variant is loaded as needed.

We don’t have copies of the files though for each language

We keep track of configuration language information on each file, which lets us translate them to other languages. The way we translate them is not to duplicate the file but instead to use overrides which are loaded on top of the base file when needed. So our configuration translation system uses the same base configuration files but allows to replace textual elements in the configuration to ones in other languages.

There is nothing stopping contributed modules to make a configuration copy-based translation system available where one menu would be a translation of another menu and both would be high level configuration items for example. But that is not what Drupal core is doing. The configuration translation system in core instead is similarly setup like the new Drupal 8 content translation system that translates values inside the object as opposed to creating copies and relating them to each other.

How are the translations created?

All the originally shipped configuration is considered part of the software and is going to be translated with the community translation system on localize.drupal.org. It is not currently integrated, see the bottom of the article. The system is already integrated however with the translation download and update system in core, so all community translations for shipped views, user emails, contact categories, etc. are going to be downloaded in the installer and translation updates to them will be available and updated on the site.

To translate built-in configuration like user mail settings, you can go to the regular interface translation screen and search for the settings text. That is not the most convenient way to translate all the built-in configuration, but this showcases the integration with configuration translation well.

Drupal core now also includes a Configuration translation module which exposes a user interface to translate all the configuration (with overlapping support for shipped configuration with interface translation as well). The same account email settings translation is accessible with this module enabled on a very convenient “Translate” tab from account settings.

This lists all available languages to translate to, allowing us to add the Hungarian translation. These screens are a lot more convenient but more spread out compared to translating on the interface translation UI.

Configuration across Drupal core has a local tab to allow for translate and/or their summary lists have translation operations listed. For example you can also translate views with operations in the views administration listing.

Of course you may not want to let your site translators access to edit views on your site. That may be dangerous to allow. No problem! We also have a listing of all configuration that can be translated under regional settings that can be used to access the translation screens for all of configuration. This is very useful if you are not a super-admin on the site and may not have access to original configuration pages. This overview offers ways to access all those translation pages.

That is good and all but if you are a developer, how do you make your configuration to support translation? We will cover exactly that in the next article.

Issues to work on

  • DONE! Work is ongoing to make the help text for configuration translation be easier to understand, so people can understand it without lengthy articles like this one. Help at https://drupal.org/node/2161801
  • DONE! The storage solution of translation overrides is changing to support translation deployments better. See https://drupal.org/node/2224887 for that discussion.
  • PARTLY DONE: We still need to make localize.drupal.org expose default configuration for translation. That is unfortunately a non-trivial problem, but we need to solve before translators are asked to translate for the release. See and help at https://drupal.org/node/1933988
May 07 2014
May 07

Drupal is right in the middle of web technology, an ideal integrator of all kinds of things. Just like PHP itself it may be clunky here and there but it is a very efficient tool to build great experiences. And even if you are a great JS developer or a pro PHP person, maybe you have mad debugging skills, you always have something to learn. Now there are great books, sometimes even better videos, but nothing beats hands-on learning. When you get together with other people working on the same thing you learn so much about how they work and even if you gain no new knowledge about programming per say, you learn new tricks and ways to achieve things:

Two days working with the Drupal Community in #DrupalDevDays help you to learn more than one week working alone at home :)

— Javi Santos (@javisr) March 27, 2014

Not only that but you can also look into how the tools you work with are made and that all of us are human:

Greatest thing about code sprints: Hearing respected core devs say "I don't understand that." "Me neither." We all learn. #drupaldevdays

— Marc van Gend (@marcvangend) March 28, 2014

Finally, by helping to improve the tools you use, you gain much better knowledge about them. Close to the start of my web involvement I worked a lot on translating the PHP documentation to Hungarian and I got into Drupal fixing core issues for translations. By becoming one of the thousands building the system you use you also gain more credibility when you are looking for help in your weak areas as well:

after almost a decade of #drupal love i just became a real drupalist at #drupaldevdays - i am now a core contributor! http://t.co/VziKa9y7h9

— aboros (@hunaboros) March 27, 2014

But not everyone can do this right? You need to be a professional programmer and pay expensive fees to get into events? Wrong! So wrong! There are always sprints around the globe and more and more local Drupal events are announced every day. Starting out with a simple issue on a one day sprint is a great start. Drupal can always be improved in all kinds of ways whether that is accessibility testing, documentation, perfecting button colors and radiuses or finding and documenting bugs. All of those are great contributions.


Drupal Dev Days sprint photo by Amazee Labs

The best places to immerse yourself in contribution are multi-day sprints though. If you have any opportunity to go to those, I would definitely suggest you join one. Why? It takes a fair bit of time to get set up, understand the issue, start providing a solution and even though at the end of the one day sprint, you will promise to get back to it a week later from home, it is almost certainly not going to happen. There is nothing wrong with you, you just have other priorities when you get out of sprint-mode. So for ideal involvement pick a multi-day sprint. It is not only that you have more time to work on things, you can get to know the people better as well on the social events. Some conferences, especially DrupalCons include extended sprints before/after the event. If you just go to the main conference days, you have much less chance to interact with people who shape the future of Drupal, while at the extended sprints, you can get involved and work with them real time. How is that for growing your potential?

Here are some examples of events with multi-day sprints where my friends from the multilingual initiative will be sprinting, feel free to add more in the comments:

DrupalCamp Spain in Valencia is coming up in a little over a week on May 16-18th. All three days have sprinting opportunities and some of the leaders from multilingual, frontend and migrations will be there!

Some people may only be aware of the Friday sprint at DrupalCons. Get a lot more out of the event by being involved with the pre and post sprints as well. DrupalCon Austin sprints last from as early as May 30 to as late as June 8th. Likewise DrupalCon Amsterdam sprints will be on from the 27th of September to as late as 5th of October. But DrupalCons are expensive, no? Well, there is no ticket needed to attend on the days when there is only sprinting, that is the pre/post sprint days and the Friday sprint. So if you join at the end, there is 3 days of consecutive working with the Drupal community for no charge where the people you interact with have no conflicting schedules to do or see sessions. How is that for hands-on experience?

Hope to see you at one of these events and other sprints in 2014 and onwards! Learn Drupal by getting involved, it is for the benefit of us all!

Apr 22 2014
Apr 22


The organizer team is still energized after our experience putting together Drupal Dev Days Europe 2014 in Szeged, Hungary between 24 and 30 March.

Several people asked about details and we wanted to document the event for future event organizers to share what worked best for us. We prepared a report for you so if you experienced Drupal Dev Days Szeged, you can look behind the curtain a bit, or if you heard about it, you can see what we did to pull off an event like this. If you were not there and did not hear about it, we included several feedback references as well to give you an idea.

Do you want to see tweets and articles like those about your event? Read the report for our tips!

We definitely did not do everything right but we hope we can help people learn from the things we did right.

Excuse us if the report is a bit too long, we attempted to pack useful information to every single sentence to make reading it worth your time. Send questions and comments to the team.

Apr 13 2014
Apr 13

Did you expect to see how Drupal 8 improves multilingual tasks at NYC Camp? Well, bad luck! I'd like to apologise in place of the NYC Camp team for their messing up the schedule yesterday and their lack of communication following. I was told to set up for my presentation in a room that was not even meant to be a presentation room, let alone my presentation room, even though it was confirmed by several volunteers coming to the room. Later on yesterday, several people asked me why I did not show up for my session. I did.

The good news is that I delivered this talk before, and although the latest recorded copy is definitely not as up to date as the one I worked on for NYC Camp, you can watch it here (fast forward to 12:04 to the start of the presentation itself):

[embedded content]

I would have loved to talk to you, bring you all the good news, answer your questions and hopefully inspire you to join our efforts. I did not get a chance this time. Hope to catch up with you sometime later at other events!

Jan 03 2014
Jan 03

You can also find me passionate about singing, music and amateur acting, especially when these are all combined.

Head to the contact page to send a mail.

Oct 28 2013
Oct 28

Drupal 8 is coming up! Check out https://drupal.org/drupal-8.0 for all the goodness that is coming! While on the surface, a freshly installed Drupal 8 does not look all that different than Drupal 7 (the backend looks and the menus are similar), the underlying system is way more powerful and flexible. We turned flexibility up a notch in countless areas.

How best to get to know this new version? Well, there are blog posts and videos plenty as we get closer to Drupal 8 being more stable. I am writing an article series on multilingual improvements myself. We do know that some people learn great on a focused day of training though, so some of us decided to offer community training (read: really cheap!) before DrupalCamp Vienna on several Drupal 8 topics.

I'm really looking forward this training because I have seven years of experience commercially training people on HTML, Perl, PHP, XML, XSLT, Drupal, etc. and really love the work that is going into assembling materials and setting up a coherent system to teach capabilities and possibilities. I am also looking for this opportunity to find more Drupal 8 bugs (which I will both in preparation and we will as part of the training, I'm sure - Drupal 8 is not yet ready after all). This is a great opportunity to do some deep-dives into what changed in Drupal 8 and learn the new best practices.

I did presentations about Drupal 8 multilingual improvements at various camps and DrupalCons and the feedback was amazing. Someone wrote in Prague: Good session, maybe it would need a little bit more time or limit the content. Presentator felt little overconfident, but that may be just me. He certainly knows what he is talking about.. I cannot help the strong confidence, as I really think Drupal 8 is going to rock your socks off in terms of multilingual improvements. But this is the opportunity to have plenty time to explore all the changes and get to know in a way you would not have a chance for a long while at least.

Oct 11 2013
Oct 11

You can also find me passionate about singing, music and amateur acting, especially when these are all combined.

Head to the contact page to send a mail.

Sep 27 2013
Sep 27

You can also find me passionate about singing, music and amateur acting, especially when these are all combined.

Head to the contact page to send a mail.

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web