Jan 17 2014
Jan 17

[embedded content]

The Internationalization Tag Set 2.0 has now the status of an official W3C-recommendation. Cocomore participated in the development of this standard for encoding information that increases the quality and efficiency of translation and internationalization on the web. Within the EU sponsored LT-Web project we did not only contribute to the standard. We also created a number of reference implementations that put it into practice and demonstrate its benefits. A lot more detail on what we did in the project can be found in the official project deliverables that are now online (we were responsible for D3.1.1, D3.1.5, D5.1.1). Below are short summaries.

Drupal Modules

Within D3.1.1 of the MultilingualWeb-LT project Cocomore implemented modules for translation and ITS2.0 handling within the open-source CMS Drupal. The implementations are based on the translation management (TMGMT) module available for Drupal as a community module (https://drupal.org/project/tmgmt). The implementations provide the following functionality:
  • Base TMGMT module models translation workflow with external LSPs in Drupal
  • Cocomore’s extensions added the following abilities:
    • Handle ITS 2.0 throughout the whole workflow
    • Apply global ITS 2.0 metadata at content node level
    • Handle ITS 2.0 annotation in Drupal WYSIWYG-editors (where content is produced). Annotation via menu bar, context menu, keyboard shortcuts.
    • Standalone ITS 2.0 editor (jquery Plugin) to support annotation in a separate process step, without modifying the actual content. Annotation via menu bar, context menu, keyboard shortcuts.
    • Localization chain interface: Round-tripping of data to/from LSP‘s TMS, including automatic data export and re-import
    • Interface with Enrycher for automatic annotation
These functionalities are embodied in the following modules:
  • Drupal TMGMT Workflow (TMGMT-module extension) to allow workflows with ITS 2.0 annotation
  • Drupal WYSIWYG editor: Plugin for ITS 2.0 annotation
  • Drupal TMGMT Translator Linguaserve: Localization chain interface (see also D3.2.2 and D4.1.3)
  • JQuery plugin for ITS 2.0 annotation in a separate step (new implementation)
  • Drupal Enrycher Integration (see also D3.1.3)
The modules are released under GNU General Public License 2 and can be downloaded and modified. They are available at the following URLs:

ITS 2.0+CMS: Best Practices

One important application for ITS 2.0 is the preparation of web content within a CMS for optimized localization/translation. This is best done by implementing ITS 2.0 directly inside a CMS. The experiences gathered in this context within the MultilingualWeb-LT project are summarized in a best practice documentation published as D3.1.5. It discusses topics that occur when using ITS 2.0 in connection with a CMS, and suggests ways to deal with these topics. The document is informed by the experiences gathered in the MultilingualWeb-LT project, where an ITS 2.0 aware translation workflow was implemented within the open source CMS Drupal. Its aim however is to provide guidance independent of the CMS as far as possible. An important aspect are therefore the characteristics of the CMS that interact with ITS usage and handling. However, not all internationalization-related issues can be resolved by the special markup described in ITS 2.0. The best practices in this document therefore go beyond application of ITS markup to address a number of problems that can be avoided by correctly designing the XML format, and by applying a few additional guidelines when developing content. This document and Internationalization Tag Set (ITS) Version 2.0 implement requirements formulated in the W3C Working Draft Requirements for Internationalization Tag Set (ITS) 2.0.

Drupal Machine Translation Training Module

Within D5.1.1 of the MultilingualWeb-LT project Cocomore implemented a module to send aligned original and translated data with ITS 2.0 markup to a machine translation (MT) provider for data driven creation or optimization of machine translation engines or models. The most common use case will be to train or tune a statistical MT model based on the aligned data and give special consideration on top of the standard techniques to the knowledge that is encoded in the ITS 2.0 markup. But other use cases, like the systematic identification of problematic cases for manual adjustment of a rule based MT system are also conceivable.. While ITS aware MT training was explored in more detail in D 5.2, the scope of this deliverable is the extraction of annotated and aligned bilingual data from the Drupal CMS. This process is based on the ITS 2.0 capabilities added to Drupal as described in the deliverables for WP 3. It was successfully tested in the context of the business case described in these deliverables (translation of VDMA press releases). Based on 141 press releases that were translated from German to French and Chinese. we could provide a three-way parallel annotated corpus of some 12.000 sentences.
Jul 31 2013
Jul 31

On Saturday 20th, Jesús and I visited Santander for attending the Drupal Day. The Drupal Day is an itinerant event organized by the Spanish Drupal Association with a local Drupal community.

Around 80 drupalistas were there, and we had very interesting session, mostly centered around the new things that are coming with Drupal 8. Is a great thing that more and more people in the Spanish community is getting involved in core contributions and attending international events, and IMHO this is making Spanish events more interesting every time.

Drupal Day Spain Santander Group Photo

For trying to attract more contributors, we celebrated a short sprint the evening before, and some new people were introduced about core development workflows, but could have been better if we could have spent more than just two or three hours. We should iterate on improving that for next events, but was nice anyway.

Sessions were recorded, so videos of every session should be available soon in the Spanish Association video channel on Vimeo.

Drupal Day Santander Logo

We want to thank to the local community in Santander, they did a gorgeous job organizing the event and innovating with Drujitos (a blue version of mojito) for the party. And of course, thanks Cocomore for sponsoring our assistance there!

Sep 07 2012
Sep 07

Drupal 8 Multilingual Initiative Code Sprint weekend

I took a train from Frankfurt (Germany) down to Munich the Saturday before the DrupalCon. When I joined the Multilingual Sprint on Sunday morning, many of them had already been sprinting for a full day and a number of issues were ready for review, so I dived in, observing the behavior of Drupal 8 before and after applying patches, proof-reading the patches for anything odd (e.g. typos in the documentation), discussing the issues in comments and in IRC with people who were sitting just across the room (other times actually speaking in person). By the end of the day, instead of the dozen or so people that Gábor Hojtsy, the Multilingual Initiative team lead, had expected, there were close to 50 people at the location, some joining us in the work on Multilingual issues, some working on other Drupal 8 tasks, and some who were just arriving in Munich and followed the Tweets to where we were. Luckily, the location rented for the Saturdays and Sundays before and after the DrupalCon week was big enough to accommodate all the extra arrivals.

While on the topic of the venue we used for those weekends, I’d like to personally thank Stephan Luckow and Florian (“Floh”) Klare of the Drupal-Initiative e.V. for all that they did to find a nice place that would still leave us with a budget for food and for their valiant work on stretching the food budget while still serving up excellent fare, in keeping with the fantastic meals we enjoyed the rest of the week. Instead of ordering delivery, they prepared almost everything themselves, including beautiful open-face sandwiches, fruit platters, and lovely grilled specialties at a club we went to where you can barbecue in the Biergarten.

…thanks for the huge help to the local organizers, especially Florian Klare and Stephan Luckow. They helped us manage collecting and spending sponsor money wisely with the Drupal Initiative e.V, prepared great sandwiches and fruit plates for us and even organized a sprinter party night with grill food. It was amazing to work with such helpful and flexible local organizers.
Gábor Hojtsy, September 5, 2012

Luckow and SirFiChi of the Drupal Initiative, organized the location and made us great food!

Since people were “fresh”, I think a lot of work got done on the first weekend and the Monday before the conference (more than 50 people joined us and worked on various core initiatives on Monday in the room we later used for core most conversations at the Sheraton), which also meant that issues were still fresh in our minds while we had days of sessions and conversations, so when we started sprinting again on Friday we had lots of new ideas for the tasks we were still working on. Friday’s sprints were at the Westin Grand, where there was great attendance both upstairs in the main room as well as a large room downstairs from it, where Drupalize.me hosted a core contribution workshop to ease people into the process of contributing to core. I decided to go to that workshop since I’m still pretty new to it all and found a few people sitting nearby who were I was also able to interest in some Multilingual tasks, so while the main group sprinted upstairs, we also worked downstairs. Later on, I came upstairs, and since there were not a lot of simpler tasks for “core newbies”, like myself, I took some time to sprint on a module I contributed some time back, before there was much of anything for Drupal 7 in the area of “multilingual”… and tried to make my module more multilingual-friendly. I got a few good commits and a new release out for Internal Links and also recruited a colleague to look at the code with me, provide some ideas, and become another maintainer. So I personally found Friday quite productive.

*/ First off, a sprint on this scale would not be possible without sponsors and significant on-site help. DrupalCon provided us with space on Monday and Friday, and some great food on Friday. The rest of the days would not have been doable without comm-press, dotProjects.be, Open8.se, OSINet and Acquia. The [ … ] financial sponsorships they provided paid for our weekend venue [ … ].

I continued sprinting with the Multilingual initiative at the Film Coop Saturday and Sunday, leaving mid-afternoon on Sunday to get back to the train station. When I left the other sprinters, Webchick was only finally getting some rest after her trip home and we had about 20 issues that were marked “RTBC”. In all, there were dozens of issues tackled over the weekend. For a complete overview of all the issues we made progress on, see Gábor’s post about the sprints, where you can also check out his excellent DrupalCon core conversation presentation, “Drupal 8’s Multilingual Wonderland”. There is still a lot to do in the time between now and the “feature freeze” deadline, but we made good progress in the DrupalCon sprints, so hopefully we can push on and get the rest of the critical tasks done in the time remaining.

One of the less trivial tasks I took on during the final sprint weekend was documenting the new language_select field type, which involved checking out the Drupal API (documentation) project, updating the Form API table to include a new Element column (language_select) and Property row (#languages), as well as information about these (below the table) and linking them in all the appropriate places. Currently, updating this page is a bit of a pain, but hopefully we will move to a better system for maintaining this information, perhaps even automated generation. While I’d worked on other Drupal documentation pages before, this was the first time I’d actually contributed patches to update the API, so it was a good learning experience.

If you’d like to help out with the Multilingual initiative or other core contribution, you might first want to take a look at the Drupal 8 Initiatives page, where announcements about coming IRC meeting can be seen. This page also has links to the news, roadmaps, filtered issues, and other pertinent information. Drupalladder.org is also a great place to go for lessons to help you work through the steps of being ready to contribute to Drupal core.

I look forward to seeing you all in IRC and in coming code sprints.

Jun 19 2012
Jun 19

It’s been a busy past several days in Barcelona (for the Drupal Developer Days) and most of us who’d been sprinting during the week before seemed to be in the same condition by Sunday—rapidly running out of energy from progressive sleep deprivation from an increasingly later return to our hotels. But it’s been an exciting week for Drupal core (and contrib) development and significant work has been completed on the Drupal core (mostly building up Drupal 8, but also some for added features in Drupal 7) while a lot of important decisions have been made which will likely shape development in a number of initiatives for the coming months until the sprints at DrupalCon Munich.

In addition to the Sprint I was primarily involved in (I was just trying to get my feet wet with assisting the Drupal 8 core development process by joining the multilingual sprint, but I did write my first committed core patch—admittedly this was a very basic patch), there were also sprints running for “Views in core”, Entity API, Media initiative, Mapping in Drupal 7, configuration management, abstracting social networking, search-related sprints, the Drupal.org upgrade… and possibly more still. I’ll cover some of the highlights of the week that I’m most knowledgeable about.

Multilingual Initiative

The multilingual initiative sprinted all week before the Developer Days sessions, and even continued through the weekend. And a lot of key decisions were made and important code changes committed and pushed to the central Drupal 8.x repository.

New user interface translation improvements in Drupal 8

This is something I got to do a bit with, but Swiss developer, Michael Schmid (Schnitzel on d.o), of Amazee Labs, was the primary developer working on this task during the Sprint. He and his colleague, Vasi Chindris, were among the stars of the week. It was a real privilege to get to look over their shoulders and to get Michael’s support when it came to using Git to manage code in the sandbox we were using for the issue. (Thank you, once again, Michael!) Once everyone was happy with the work, it got committed to core. This new sandbox workflow, used for larger issues, helps avoid a lot of bugs creeping into the main branch, as has happened during previous periods of intense core development. Of course the tests and test bots catch a lot of issues which could otherwise be major headaches for all concerned (automated testing was also a part of Drupal 7 development). If you recall, the long wait for Drupal 7’s release was due to hundreds of critical bugs. Now this should be a thing of the past since we have an established threshold for critical issues; and the core team only commit new patches to the central repository when we are below that threshold (15 “critical” bugs, 100 “major” bugs… among other thresholds specified).

New system for translating Drupal’s user interface

The new user interface translation system allows you to keep imported (community contributed) translations separate from customized translations and search for a particular translation within either or both categories as well as filter by translated strings, untranslated strings, or both. If you have any unsaved translations, they are highlighted to help remind you not to leave the page without saving them and there discussion about providing a dialogue to prevent a site admin from accidentally leaving the page with unsaved changes, too. There is also an issue to allow the string search to be non-case-sensitive (checkbox) to find more strings that contain a particular word or phrase, regardless of text case. Since this feature came up in discussion after the rest of the user-interface changes had already been made, we elected to put the discussion about adding this feature in a separate issue. If you have ideas for what might further improve the Drupal 8 user-interface translation workflow, your input is valued.Customized and imported (community) translations are stored separately

*/

New content language options

Drupal 8 has new language settings per content typeYou can enable translation for a particular content type and also choose to hide the language selector (automatically selecting the language for a new piece of content by any of a number of contextual rules). The automatically selected language for a new piece of content can be any particular language enabled on your site, “not specified”, “not applicable”, “multiple”, the “site’s default language”, the “current interface language”, or the “author’s preferred language”. While all these settings might arguably be a bit confusing for new users, they should help smooth the content creation and translation workflow for most sites. Of course the option to “enable translation” is hidden if the default language for the content type cannot be resolved to a single language (i.e. for “not specified”, “not applicable”, or “multiple”), since translation does not make sense here.

Translate the English UI to… English!

Drupal 8 — Enable English UI translationIn the edit preferences for the English language, you can enable translation to English and then it’s easy to change, for instance, the “Log out” link to “Sign out” (or “Disembark”, “Abandon ship”, “Terminate session” or anything else you might want on a particular site). Of course this could also be useful for fixing any oddities you find in the UI strings provided by contributed modules if you find a mistake in a field description, for instance, you don’t need to wait for a module developer to commit your patch or add a “site English” custom language just to modify a few strings.

Configuration Management related to Multilingual sites

Drupal core team leads and other sprinters discussed multilanguage configuration

One of the biggest issues of the week was determining how multilingual configuration would be handled in Drupal 8. The core team knew that they wanted to store language configuration in files rather than in the database, so that it’s easy to “push” new language configurations to an established site that already has content, among other benefits of this approach. But this brought with it a number of challenges which the Multilingual Initiative team, Configuration Management Initiative team, and other interested parties discussed in several sprint discussions through the week. Many of the standard configurations for a site might also differ, depending on the language: you might, for example, want a different site name or site slogan or logo for each language. There were three different proposals for how to handle multilingual configuration, and to keep a long story short, the final decision was to go with “Plan B” (or a minor variant, thereof). You can still lend your voice to the “review” process in the main issue for the language configuration system in Drupal 8. If you would like an overview of the plans, there is a nice graphic by Gábor Hojtsy (the Multilingual Team lead) which outlines the differences between the three proposals and some variants.

Drupal 8 Configuration Management

Greg Dunlap (“heyrocker” on drupal.org) presented the new configuration management

Angie Byron, aka “webchick” gave a quick overview of the configuration management initiatives goals, tooOne great session from the weekend was the Introduction to the Drupal 8 Configuration Managment System by Greg Dunlap (“heyrocker” on Drupal.org), the Configuration Management Initiative team lead. There has been some good progress in determining what this is going to look like, some of which took place during the sprints in Barcelona. Basically, this will be a bunch of smaller files stored within a logical directory structure in the sites/[…]/files directory. The new configuration system is currently planned to be YAML-based (rather than PHP or XML, which were used in earlier visualizations of the system). And the goal, as described by a slide in Angie Byron’s Sunday-morning keynote, “Drupal 8: What you need to know” is to be like “Features in core, only better”. The aim is to help us remove the complications involved in pushing configuration changes, modified in a development or staging environment, to a site that already has user-created content that we don’t want to lose. The main problem with the current system is that there is no consistent system: configuration settings are scattered across multiple tables, variables, files, and other locations and there is no consistent structure in any case. The idea is now to have a contexts, which Drupal responds to, when determining which configurations files to use.

Angela Byron (“webchick”) talks about the problems the new configuration management system aims to solve

What it should look like when loading a configuration from module code, is something like this:

  $config = config('image.style.large.yml';
  $config->get('effects.image_scale_480_480_1.data');

And when setting and saving configuration data:

  $config = config('system.performance');
  $config->set('cache', $form_state['values']['cache']);
  $config->save();

The YAML code for the image example, which saves configuration for the “large” image style would look something like this:

  name: large
  effects:
    image_scale_480_480_1:
      name: image_scale
      data:
          width: '480'
          height: '480'
          upscale: '1'
      weight: '0'
      ieid: image_scale_480_480_1

This should be pretty easy for developers and site builders to learn to work with and of course an interface is planned which should automatically build the configuration files, when edited by site builders. Configurations will be loaded into the “active store”. Changes are saved back to the active store and back to the YAML files so they can easily be moved between sites (staging and production sites, or completely different sites if they should have some settings in common). Building up an ideal import/export system for configurations is one of the major remaining hurdles. Update: heyrocker’s presentation slides are now available for download, so you can see other examples of Drupal 8 configuration.

Other Drupal 8 news

Twig library committed to core!

Drupal 8 now has Twig in the core/vendor directoryOne of the new developments which has received some press is that Twig, the templating system designed by Fabien Potencier, the innovator behind Symfony, which also bundles Twig, has now been added to the Drupal core repository.

However, the fact that the Twig library is in the repository does not mean that it’s ready for any kind of use yet, except for those who are working to build a new templating engine for Drupal, which uses it. How this works is still open to discussion; according to webchick, it may be that we keep both PHP-based and Twig-based templating engines to ease the pain of this change. On the other hand, while there is a learning curve involved, there are many advantages to Twig, especially in terms of security (removing PHP vulnerabilities from themes, altogether), and the saying that “the drop is always moving” applies here. It may be that Twig is the only templating engine which will be supported by Drupal 8, but if you feel strongly about this or have ideas for how to do this “right”, it’s a good time to get involved.Twig vs PHP template syntax

Context-based layout and blocks

Angela Byron lays out the plan for Drupal 8 layout with contexts

Everything in Drupal 8 will be a block or a layout area and blocks can have multiple contexts which determine their behavior (and whether or not they are displayed). This is going to be a major change which should produce much more flexible layouts and site designs. Of course this will touch on every major Drupal initiative: configuration, HTML5, mobile, multilingual… all are involved.

Drupal 8 will have clean, semantic HTML5 (and will abandon IE)!

Say goodbye to the messy nested div hell! Drupal 8 code is going to be much smaller and cleaner which will make designer/themer types love Drupal and make it possible to produce code that renders nicely, regardless of display size. Oh, and don’t worry about trying to support older versions of Internet Explorer; the community has decided it’s time to put that tiresome task to rest. Yay!

Drupal 8 development needs you!

Webchick, heyrocker, Gábor Hojtsy… all made the same point: As a community effort that’s still underway, the Drupal 8 effort needs more of the community at large to get involved and find ways to help out. There is a lot of complexity, but there will be smaller tasks that anyone could work on, so there’s going to be something for everyone. Even non-coders can help by testing, filing bug reports, helping manage the issue queues, making suggestions, documenting finished features and APIs. There are several places where you can get involved:

  • The core initiatives overview page provides information about when the different teams meet in IRC and in which channels among other information which can help people who want to find ways to get involved.
  • Drupal Ladder is a project aimed at helping more people learn how to contribute to Drupal
  • [ … ] (Comment below if you have other tips for where to get involved)

Big thanks to the organizers, sprint leads, and session speakers

The Drupal Developer Days in Barcelona were a big success because of all of you pulling together to make things happen. The local organizers made us all feel welcome and provided a lovely venue and took us out on the town just about every night. The sprint leaders helped find ways for everyone to play a part in building Drupal 8 or contributing in other ways, and the sessions were awesome.

Jun 14 2012
Jun 14
Morning stand-up meeting at the Drupal 8 Multilingual Sprint

I was supposed to get into Barcelona at 10:30PM on Tuesday evening, but with delays in my flight, it wasn’t till after midnight that our plane landed; it was after 1 a.m. by the time I reached my hotel. Normally travel, when it runs late and long, makes me feel exhausted, but I was excited to be joining my first Drupal core sprint. I’ve been wanting to do a bit more to help build Drupal and it’s great to not only be somewhat aware of what’s coming in Drupal 8, but to also know that I’ve at least played a small part in making it happen.

I wasn’t sure I would attend the Drupal Dev Days in Barcelona till a couple of weeks ago, but I’m glad I’m here. We have a fairly sizable group of developers here at the Citilab helping work on cutting through the issues for Drupal 8 Multilingual Initiative (D8MI). I’ve been helping with some user interface quirks and since it had been long enough since I’d actually done string translations of the user interface, I started out yesterday as a “tester”… at least trying to look at the problem of translating the interface (e.g. translating “Add content” to German) as if I had never done anything like that before. And we did find some issues and, even better, we were able to address and correct those issues during yesterday’s coding. Others have been working on multilingual issues related to the new configuration management system, and a number of other issues which you, too, can help with, if you’d like to join us remotely (or in person, if you happen to already be in Barcelona — the Sprints continue through Friday, too). There are currently about 40 of us in the IRC channel for i18n and I'd say that at least half of those are working on the Sprint. There are about a dozen (give or take, since people are working on other sprints, too) who are here in Barcelona working on D8MI.

You can help make Drupal 8 better, too!

Jump on IRC (#drupal-i18n) and look at the focus issues for Drupal 8 Multilingual Initiative if you’d like to join use remotely. There is a lot going on right now and it’s not all on Multilingual issues, so if you have some time, I’d like to encourage others to join me in helping ease the burden on the few who do so much and at least do a small bit to make Drupal 8 as awesome as it can be.

Nov 14 2011
Nov 14

Alex, Tavish and I had a great time in the T Dot over the weekend at DrupalCamp Toronto. The camp was well-attended, there were lots of great presentations and BoFs, and we even came back with some very hardcore Druplicon touques (hats for you non-Canadians).

Here's a recap of the four sessions we presented at the camp:

Drupal 7 Higher Education Case Study

Alex did a case study on our recent work in using Drupal 7 to build an admissions portal for a major Canadian university, emphasizing how we were able to leverage Drupal 7's new Fields API to allow non-technical administrators to easily customize the online application form.

Responsive Design and Drupal Theming

I presented my approach to implementing responsive design in Drupal, including how to handling elements like Panels, tables, forms, and images. This session expanded on a blog post about responsive design and Drupal that I wrote last month.

Multilingual Site Building in Drupal 7

I also did a presentation on building multilingual sites with Drupal. My focus was on methods for content translation, which is also covered by this blog post.

Poutine Maker: A Tasty Introduction to the Field API in Drupal 7

Tavish, our superstar intern, introduced attendees to the Drupal 7 Field API with his popular Poutine-themed presentation. The presentation covered creating custom fields, including validation, saving, and formatting field data.

A huge thanks to all the organizers and sponsors for putting on such a great event, which gave us a chance to connect with many friends old and new. We're looking forward to heading back to Toronto for the Drupal Business Summit 2011 Toronto, on December 2nd, where Alex will be co-chairing a breakout session on Drupal in Higher Education with Jennifer Hols from ImageX Media.

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web