Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Jun 03 2020
Jun 03

Today, Drupal 9 will be released and we are throwing a celebration party at 1xINTERNET!

On a day like this, we should take this opportunity and embrace what the Drupal community has achieved, it's important for the future of Drupal and then of course it's important to have fun!

So please join our celebration party today on GOOGLE MEET from 16-18pm CET


Check our celebration video

Celebrate Drupal 9  ( built in Drupal 9 of course ) was created for the launch and shows so nicely the great energy in the Drupal community. 

Christoph, our co - founder sat down at the piano and created a karaoke version of the famous Hallelujah song, encouraging people to take part and sing it with him. Allison Manley reacted quickly and here below you can see the results here!

And by the way! It’s not too late to take part! 

We at 1xINTERNET also created a celebration video, where you can see the majority of our employees, based all over the world!  

[embedded content]

Jun 03 2020
Jun 03

For our team at the Drupal Association, the events of the past week have once again emphasized the severity and violence of racism that communities experience in the United States and across the world every day.

Amidst the COVID-19 pandemic, where black and brown people are disproportionately affected, seeing racial turmoil unfold is particularly painful and difficult to witness. We stand with people across the globe in condemning racism, racist behavior and all abuses of power. We grieve for the black community, which has endured another unspeakable tragedy in a long history of injustice.

Drupal is global in scale, reach and opportunity.  It takes many contributions, from many diverse contributors to thrive. Drupal is about standing up for what could be and for promoting hope through access.  As such, our community adheres to a set of values and principles. Of particular importance is our value to “treat each other with dignity and respect” emphasizing that we do not tolerate intolerance toward others.  It is time to live our values out loud seeking first to understand, then to be understood. 

The Drupal Association values equity, diversity and inclusion, and we recognize we still have work to do to create meaningful change.  Here are the ways in which we are encouraging our team to take action.  We are sharing in hopes that you will take action, too.

If you have resources to share or organizations to highlight, please post them on social media using the hashtag #DrupalBetterTogether so that we may compile a comprehensive list for our community.

In the coming weeks and months, you will learn more about the programs we are putting into place to take action.  Our Drupal Community is truly Better Together, and together we will accelerate our ability to affect positive change.  Now is the time to raise our voices, not to silence them.

Jun 02 2020
Jun 02

*/

Return to site

· Drupal,Webform,Share

The Webform module provides all the features expected from an enterprise proprietary form builder combined with the flexibility and openness of Drupal...now available on any website, webpage, and application

The Webform for Drupal 8 is built upon Drupal's Form API (FAPI); the building, validating, and submission handling of a form relies on Drupal's backend code. Drupal, and most content management systems, are moving toward a decoupled approach where the backend content authoring tool is separate from the front-end presentation layer. Until now, there was no easy way to decouple a webform from Drupal's presentation layer.

Decoupled Drupal and sharing Webforms

A decoupled approach for the Webform module means that Drupal would handle the backend form builder and the submission processing. In contrast, the user-facing input form would be rendered using a front-end framework like Angular, React, and Vue.js. In the Drupal community, people are experimenting with decoupling webforms by exposing webform elements using a JSON schema or REST API, and then rendering the input form using Gatsby with React. The biggest limitation of this approach is recreating 80+ webforms elements with complex business logic as React native components. For me, this feels like an overwhelming challenge; it requires rebuilding Drupal's Form API and the Webform module's presentation layer from scratch. Stepping back from this challenge/problem, the goal which everyone is working towards is being about to use a webform outside of a Drupal website.

People want to embed a Drupal webform within a non-Drupal website or application.

Embedding a Webform using an iFrame

A widespread solution to embedding external content within another website is to use an Inline Frame element (

May 29 2020
May 29
Getting Ready for Drupal 9 katherine.druckman Fri, 05/29/2020 - 01:38

With the imminent release of Drupal 9, it’s a great time to give serious consideration to your update strategy. There is much to look forward to this time around, whether you are a Drupal veteran or new to the family.

The good news is that this will be a much easier process than the most recent major version updates, so if you’ve been using Drupal a while, this should come as a relief. However, It is important to note that while it is not a painful migration process, it’s also more than just a few clicks away. Drupal 9 changes a few assumptions many of us are used to as Drupal site builders, but preparing now will take you far.

Drupal 9 is basically just Drupal 8 with updated dependencies and deprecated code removed, which makes for a fairly straightforward update process, but the devil is in the details.

Getting Started

First, there are some prerequisites you’ll want to keep in mind.

  • Your current site must be running on at least drupal 8.8 to upgrade. This means a fully up to date install.
  • You must use PHP 7.3 or higher. Drupal 9 requires php 7.3
  • In some cases, Drupal 9 needs a newer version of whatever database you're on now, so you may need to upgrade your database. See Environment requirements of Drupal 9 for details.

As you start looking for Drupal 9 compatible modules, you may find yourself a little perplexed as you browse modules on Drupal.org and notice that none seem to have a Drupal 9 compatible release.

They do, but it's a bit hidden.

(Note that you do not have to look up every module you are using on Drupal.org. We'll get to a handy reporting and scanning tool shortly, but it's important to note these changes to how modules will work with Drupal 9.)

You will find Drupal 9 compatibility in the small text under "Project Information."

The concept of modules being tied to a major release is old Drupal, and we're living in the future now. Modules can work with more than one major version of Drupal! So, you won't see a big highlighted box for a Drupal 9 release on module pages.

In the near future, you'll start seeing more module releases using semantic versioning on Drupal.org. A few are already there. The major version of core will no longer be part of the module's version number. You can see this in action on the semver_example module and on the Lightning project page.

Lightning version 5.x.x uses semantic versioning.

Evaluate Your Site

Once you have met the requirements to run Drupal 9, the next step is to evaluate your site. The first course of action is to update all modules to the latest version. There's a good chance they're already D9 compatible. Then, check for deprecations in your themes, contrib modules, and custom modules.

Deprecations are old Drupal 8 functions which have been removed from Drupal 9, and you’ll need to find them to replace them with up-to-date code. To do this, I recommend starting with the Upgrade Status module. This module will scan your codebase for deprecations and provide you with a handy report. Install it in your development environment, and note that it must be installed with composer.

$ composer install --dev
$ composer require 'drupal/upgrade_status:^2.0'

Once you've enabled Upgrade Status and navigate to its report at admin/reports/upgrade-status, you'll see some important information about your environment that will indicate your ability to run Drupal 9.

The top of the Upgrade Status report.

 

Upgrade Status will scan your custom modules and themes and all your contrib projects and give you this lovely output.

 

One warning per custom project is great news, especially when the fix is as easy as adding core_version_requirement to the project's info file. Further down the report, you'll see the a list of your contrib modules. Even though we've updated our modules, we may still see that they are not all D9 compatible, and Upgrade Status will show those errors.

You may see some text under the module that indicates its status. Module maintainers may optionally provide this very helpful information. Sometimes it might be as simple as "This module is ready for Drupal 9".

If you see something like "Version 1.x-dev is ready for Drupal 9", "dev" means the latest version isn't D9 compatible. They may also indicate which future release will be compatible, and even include a link to an issue on drupal.org with further information about the module's status, and we’ll see an example of that when we dive deeper into evaluating contributed modules.

But wait, you are probably wondering how Admin Toolbar is compatible when it shows 12 warnings!

You'll errors like this sometimes when using Upgrade Status. There are two main reasons why you'd see this:

  1. The problem might be in a test class, like in this example. Tests can't always be scanned, but you can safely ignore them because you won't run test code in production.
  2. The module might have a third party dependency that you don't have installed. That's totally okay -- those dependencies may not even be Drupal modules, so they don't matter to Upgrade Status.

Generally, you can trust the maintainers that the module is indeed "ready for Drupal 9."

You'll get lucky with some modules, and they'll have errors that are pretty easy to fix. Some may be a little more challenging or involve some refactoring, but many will have  a handy link to the API documentation, as below.

If you click one of the nice error messages, you get to the api documentation page for the deprecated function, which may also contain a helpful link to the change record with thorough examples to help you change your code the best way: 

Change records document changes to Drupal core. They will describe the change and the impact of the change. It should include any interface changes, function or hook changes, and it's helpful for understanding how to address these changes with your own code.

Note that not all errors will have a link to further documentation, but that's okay because a little API searching will do the trick.

The deprecation notice.

If you look at the top ten most popular Drupal 8 compatible modules, all are compatible in their dev branch, and 9 of the 10 have full compatible releases. This is incredible progress.

But, as we get further down the popularity list, we're likely to find some incompatible modules.

As you continue your discovery process, it's a great time to evaluate where your pain points will be. Get involved and try to help modules reach their Drupal 9 compatibility goals, or if a module seems stalled or the last commit was 3 years ago, have a backup plan to include a patch or re-evaluate the need for the module at all.

Acquia's Drupal 9 Deprecation Status page is also a great source of information. This shows the status of all contrib projects, and gives you an idea of where the entire Drupal ecosystem stands with regard to Drupal 9 readiness.

Fix Your Errors

Once you've equipped yourself with the prerequisite information, it's time to move on to fixing any errors you have found. I'd start with the easy ones and work your way up.

Now is a great time to introduce the Upgrade Rector module. Upgrade Rector is a UI that uses drupal-rector to help you automatically generate patches. If you install the Upgrade Rector module, since it integrates nicely with Upgrade Status, you'll be able to generate an automatic patch for some modules as you scan them. Rector is limited at this point though, so you may end up with results similar to the following screenshot, but even a couple patches could get you a little ahead of the game and save a bit of time. And, like everything in the Drupal world, development is moving fast. By the time you install Upgrade Rector and scan your site, it's likely you'll be able to automate several of your patches.

Drupal Rector output visible on the Upgrade Status report (admin/reports/upgrade-status). Click the "patch available" link to see the generated patch. Drupal Rector displays the patch in a modal window.

Below, you see the changed lines highlighted, and your custom code is ready to go. There is a caveat, however. If you've found a contrib module that has not yet been patched in the issue queue and you'd like to add one, I can tell you from experience that if you post a patch like this for a contrib module, an experienced maintainer is likely to ask you to make changes to it. In this case, someone could point out that these replacements use the global Drupal class, which is not always the ideal approach in contrib modules, but it's safe to use in your custom projects.

This example used a contrib project, but it's useful to remember that with contrib projects, you aren't alone, so save yourself some work and check the module issue queues to see if the deprecations have already been patched, or even committed. Some maintainers will also have an explicit link to the main Drupal 9 issue in their queue, and this will be visible on the Upgrade Status Report, on the module page, and on the Acquia Deprecation Status tool.

If they haven't, then you can make the changes yourself and submit a patch to the issue queue and do your good deed for the day. With any luck, you'll just have to focus on your custom code.

In short:

  1. Avoid duplicating work.
  2. Check the issue queue first.
  3. Then submit a patch.

Once you have gone through your list and fixed and rescanned until it's all green, or you have gotten as far as you can, next make sure all your automated tests are passing. There is a pretty good chance the Update Status results screen will never be completely green, and that's okay because static analysis is not perfect, and neither are automated tests.

Once you feel like you've exhausted all your options with static analysis, you can try actually testing with Drupal 9 core.

You can test your current status by requiring Drupal 9 like so:

$ composer require 'drupal/core:~9.0'

Now, you may have to do some composer wrestling to get this going, but you'll get there!

You may see errors like this:

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Conclusion: remove drupal/cookieconsent 1.4.0
    - Conclusion: don't install drupal/cookieconsent 1.4.0
    - Conclusion: don't install drupal/core 9.0.0-beta2
    - Conclusion: don't install drupal/core 9.0.0-beta1
    - Conclusion: don't install drupal/core 9.0.0-alpha2
    - Conclusion: don't install drupal/core 9.0.0-alpha1
    - Conclusion: don't install drupal/core 9.1.x-dev
    - Installation request for drupal/cookieconsent ^1.4 -> satisfiable by drupal/cookieconsent[1.x-dev, 1.4.0].
    - Conclusion: remove drupal/core 9.0.x-dev
    - drupal/cookieconsent 1.x-dev requires drupal/core ~8.0 -> satisfiable by drupal/core[8.0.x-dev, 8.1.x-dev, 8.2.x-dev, 8.3.x-dev, 8.4.x-dev, 8.5.x-dev, 8.7.x-dev, 8.8.x-dev, 8.9.x-dev].
    - Can only install one of: drupal/core[9.0.x-dev, 8.8.x-dev].
    - Can only install one of: drupal/core[9.0.x-dev, 8.9.x-dev].
    - Can only install one of: drupal/core[9.0.x-dev, 8.0.x-dev].
    - Can only install one of: drupal/core[9.0.x-dev, 8.1.x-dev].
    - Can only install one of: drupal/core[9.0.x-dev, 8.2.x-dev].
    - Can only install one of: drupal/core[9.0.x-dev, 8.3.x-dev].
    - Can only install one of: drupal/core[9.0.x-dev, 8.4.x-dev].
    - Can only install one of: drupal/core[9.0.x-dev, 8.5.x-dev].
    - Can only install one of: drupal/core[9.0.x-dev, 8.7.x-dev].
    - Installation request for drupal/core ~9.0 -> satisfiable by drupal/core[9.0.0-alpha1, 9.0.0-alpha2, 9.0.0-beta1, 9.0.0-beta2, 9.0.x-dev, 9.1.x-dev].

You may see errors like this. Any modules that don't declare compatibility with Drupal 9 will not install. They need to do that with core_version_requirement in their info file. As a workaround, you could temporarily remove them until they do, and test the rest, or you could bring them in without using composer, and check them that way. That will allow you to identify any missing core_version_requirement keys in sub modules, or other issues that may have slipped through.

Once you have a Drupal 9 codebase, you are not necessarily home free, as static code analysis doesn't catch everything. At this point, you may face errors when trying to run your site. In fact, it's likely. A good way to check for errors is to run drush cr once you've upgraded. In my experience, references to deprecated services may have slipped through, and they'll pop up once you've switched to Drupal 9.

At this point, you may face errors when trying to run your site. In fact, it's likely. A good way to check for errors is to run drush cr once you've upgraded. In my experience, references to deprecated services may have slipped through, and they'll pop up once you've switched to Drupal 9.

For example:

You have requested a non-existent service "entity.manager".
 
 
You have requested a non-existent service "path.alias_manager".

Next Steps

If errors are coming from a contrib module, your first stop should be the module's issue queue. Report the error if it has not already been reported. Active maintainers and other contributors are likely to respond to the error quickly, especially in popular modules. If you need to explore further, run the module's tests.

If the error is in your custom code, as in contrib modules, some of these changes will be easier, like simply replacing the old service with the new, and sometimes you'll have to rethink your code a little bit. You may find some guidance in the change records: https://drupal.org/list-changes

Once you have your fixes in place, you should again be ready to make sure any automated tests are passing with Drupal 9. Then, give things a manual test for good measure.

But what if it hasn't gone so smoothly? What other challenges might we face?

The differences between getting custom projects ready for D9, versus contrib projects, are pretty clear. You are in charge of your custom code, and you can define your own timeline for a Drupal 9 compatible release. With contrib projects, the maintainers are the gatekeepers, and you may find yourself temporarily blocked while waiting for patches to be committed. The upside is that once ready, it's an easy update away!

Almost all Drupalers are kind and generous people, as you know, but some are a little busy. Ultimately, it's going to take the efforts of many to get the contrib modules you need ready. And in the meantime, you probably have enough custom code to tackle.

Making Sense of Contrib

As someone who used to spend a lot more time using Drupal than looking at issue queues, I can understand how the process of evaluating a module's readiness, or progress toward readiness can be less straightforward. But I'll show you some key places where I go digging for this information.

Search API is a great example of a helpful status message. You'll find the status when using Upgrade Status and at Acquia's deprecation status page.

You'll see that this module has included a link to an issue which will tell you what you need to know about the status of its Drupal 9 compatibility. This is very convenient.

But now, let's look at reCAPTCHA. You'll notice there is no indication of Drupal 9 compatibility on the module page.

Next, we can check it on Acquia's Deprecation Status page.

This suggests reCAPTCHA is ready to go, but let's try and verify. Note that no Drupal 9 plan is provided to indicate the status of its compatibility.

It's always a good idea to check the issue queue for Drupal 9 progress when it's not linked from the status message. We got lucky, and this post is right near the top. 

There are some good signs in this issue. There's a patch and it was committed.

There are also some less good signs. What are these D9 composer require failures? We don't really know the status based on this. And remember, just because this test isn't running does not necessarily mean this test won't work for you. The only way to know for sure will be when you try it.

So, let's go back to the module page. 

The modules automated tests are another important source of truth.

This page looks less promising. The module's tests currently fail with Drupal 9. If we click the failure notice, we'll see the test failures in detail. 

Here we can see the problem. reCAPTCHA can't be Drupal 9 ready until its depedency, CAPTCHA is. So now's a good time to head over to the CAPTCHA module's page and check its status.

Maybe you could help them move it along!

In this post, I've laid out the major steps toward getting your site ready for Drupal 9.

  1. Update core and all your modules, and satisfy Drupal 9 minimum requirements.
  2. Remove all deprecations in your custom code, and make sure all your contrib modules are ready for Drupal 9.
  3. Upgrade to Drupal 9 and test manually.

In summary, if you start this process now, you'll be a little ahead of the game. If your projects are ready, it's a pretty seamless update, and if not, now is the time to get them there. Keeping your current Drupal 8 site up to date will get you far.

Resources

Drupal 9 resources

 

Drupal 9 Overview Page

Upgrade Status Module

Upgrade Rector Module

Acquia's Drupal 9 Deprecation Status Tool

May 29 2020
May 29

This month’s SC DUG meeting featured Will Jackson from Kanopi Studios talking about his virtual background and office.

Before everyone was learning to use Zoom virtual backgrounds, Will had built out a full 3D room for his background, including family pictures and other fun details. He talked about what he built and may inspire you to try some more personalized than swaying palm tree and night skies.

[embedded content]

If you would like to join us please check out our up coming events on MeetUp for meeting times, locations, and remote connection information.

We frequently use these presentations to practice new presentations, try out heavily revised versions, and test out new ideas with a friendly audience. So if some of the content of these videos seems a bit rough please understand we are all learning all the time and we are open to constructive feedback. If you want to see a polished version checkout our group members’ talks at camps and cons.

If you are interested in giving a practice talk, leave me a comment here, contact me through Drupal.org, or find me on Drupal Slack. We’re excited to hear new voices and ideas. We want to support the community, and that means you.

May 27 2020
May 27

Last weekend all of our employees were encouraged to take part in the Drupal 9 porting weekend. The event was an organized to activate as many developers as possible to contribute to the open source project Drupal.

For us at 1xINTERNET it turned out to be a great experience. Feeling the team spirit both within the team and the Drupal community, deeply affected us as a group and on a personal level as well. We paused all client projects on Friday and started with a kickoff meeting, where we went over the days ahead and how we would proceed in updating all of our Drupal projects, supporting each other and anyone that needed help. Our goal, which we achieved, was to ensure that all modules maintained by our colleagues had a stable release. Also, our goal was that the modules we use most often, would also have the Drupal 9 readiness patches become ‘Reviewed and Tested by the Community’.


Everyone can be Makers of Open Source

In our kick-off meeting, we provided some mentoring to make sure that all our colleagues had the necessary knowledge to fully participate in this initiative. This included how to use the drupal.org issue queues, how to create patches, and how to efficiently start an environment capable of performing the necessary testing. Some of our colleagues opted to collaborate with their skills in other areas, helping to prepare for the upcoming launch of Drupal 9 in little more than a week.

We have a mix of people at 1xINTERNET, some of us were contributing for the first time and others are experienced experts, simply a great mix! 

The weekend highlights: 

  • We worked on 46 different projects 
  • We released Drupal 9-ready stable versions for 15 Drupal projects
  • We enabled a couple of new contributors that contributed for the first time to Drupal
  • Our non-coder employees worked on Celebrate Drupal 9 launch and content issues on drupal.org

João Ventura had previously agreed to help with the global contribution in Drupal’s #d9readiness Slack channel, so that issues could be better worked on by the Drupal community. His job was made a lot easier with the extensive documentation prepared in advance by Gábor and Kristen Pol.

We had a lot of fun and our team members loved it. Therefore we have decided to do this more often. We want to plan regular Contribution events at 1xINTERNET and also make sure that all employees participate in making Drupal better.

So, how was your weekend? 

May 27 2020
May 27

We live in the age of the digital, with digital experiences an intrinsic part of our everyday lives. This means that now more than ever there’s a need for incredible amounts of people who have the skills to craft compelling experiences in the digital. A large portion of them is represented by software developers.

However, one of the main characteristics of the digital is its unbelievably fast pace. There are new trends and technologies emerging constantly, and it’s very difficult to keep up, especially for larger businesses whose digital endeavors are broader and encompass multiple different channels. 

What this means is that all these highly skilled developers need to at once be familiar and experienced enough with existing technologies and prepared for any future trends that might emerge further on. It’s definitely no easy task finding and retaining this perfect blend, especially with the unprecedentedly high demand for developers today. 

In this post, we’ll dive deeper into the importance of a development team that’s future-ready, explain what future readiness even means, and look at some tried and tested methods for acquiring a team of future-ready developers. 

What does it mean to be future-ready?

Future readiness means different things for developers and businesses, so we’ll define each separately. Still, both of them ultimately tie together: due to the need for digital experiences, business future-readiness depends majorly upon developer future-readiness.

Developers

If a developer is future-ready, it means they are familiar with and follow best practices, know and effectively implement accessibility guidelines, and are up-to-date with a range of technologies as well as trends (e.g. web components, lazy loading, etc).

They don’t necessarily have to be experts at obscure emerging frameworks, but they do know enough about the state of the software development landscape that they’re able to adopt a new technology if it turns out to offer a significant business advantage, or a greatly improved developer experience.

When it comes to a future-ready development team, one of the most important characteristics is the developers’ ability to cooperate internally. They are able to complement each other’s potential skill gaps to deliver a cohesive final product. 

Businesses

A business that’s future-ready is not locked into a system that doesn’t allow integrations, or that’s dependent on a lot of other technologies that are outdated. 

It’s built on a platform that can scale and that is owned by the business, not a third party. It has the capability of integrating new technologies, and is optimized for mobile and multichannel digital experiences.

By investing in their employees’ growth, as well as by following industry standards and agile methodologies for swift iteration, it is resistant to disruption and always able to use its familiarity with the digital to its advantage.

Why do you need a future-ready development team?

This is more or less a no-brainer; if you want your digital business to be future ready, the baseline of that business has to be future ready. 

Plus, the future is uncertain - that means, while you can never be fully prepared for it, you have to do what you can to at least be somewhat prepared. There are new technology trends emerging all the time, and if you want to be on the cutting edge, you need to be able to leverage them when you or your clients need them. 

Also, considering the current disruption, the nature and importance of digital experiences are themselves changing - right now, for example, we’re seeing a major rise in e-commerce and video conferencing solutions. Those that respond quickly without having to change course have an obvious advantage in navigating such crises. 

This is where a future-ready development team comes into play. It’s even more convenient if you leverage the expertise of a skilled development agency, as you don’t have to invest a lot of resources into vetting and re-/up-skilling your in-house employees, but rather get a readymade team of developers possessing the exact skill-sets that you need.

If you are, for example, a startup building your groundbreaking tech product, you’ll definitely want to make use of the most innovative technology available to you, as well as make sure you’re working with vetted experts who follow best industry practices. Read here how proven Agiledrop engineers can help you build high-end digital products. 

The future is uncertain quote

What’s the best way to secure a team of future-ready developers?

As with any developer, there are several ways, e.g. outsourcing (to an agency or a freelancer) or in-house. For a future-ready team, however, it’s even more important that you’re able to get exactly what you need without too much additional overhead. 

With everything going on recently, it’s become incredibly difficult to attract in-house talent that fits your needs, let alone vet them and/or invest into up-skilling your existing talent. As stated, your best bet right now would be to partner with a company that’s focused exclusively on development, as you can be sure they’ll invest their energies into being up-to-date. 

For complex projects that require various technologies all functioning together, you’d typically need a full team, not just a single developer. Cherry-picking the team with the exact needed skill-sets from a pool of available freelancers would likely be a very time consuming and costly process - plus, you get no guarantee that these individual developers will work well together.

A development company such as Agiledrop can provide a full team that is used to working together and collaborating on complex problems to deliver smooth and efficient solutions. Our engineers are encouraged to learn about any new technologies that interest them, and to share what they’ve learned with the whole team during monthly AgileTalks. 

As is also obvious from our name, we follow agile methodologies in all our projects, but we ultimately always adapt to our clients’ workflow, adopting their tools and processes. This ensures that, while our clients benefit from our expertise in the latest trends, this benefit never comes at the expense of internal consistency. 

So, if you’re currently in the process of searching for a future-ready development team, you’re in luck - get in touch with us and find out how our skilled engineers can help you deliver just the product you need. 

Conclusion

Man in cloak on a rooftop with orange sunset in the background

The faster the pace of the digital, the more important it is to be future-ready. And, as drivers of digital experiences, developers and engineers are key in guaranteeing digital-based future readiness. 

Future-ready businesses have an obvious competitive edge, but it is not always possible to invest in an in-house team of future-ready developers. In those cases, finding and partnering with a development company that’s able to provide the right skills for your needs is definitely the best bet. 

Ideally, you’d also want that partnership to be long-lasting, so that you don’t have to search for the right partner again during every big project. If you’re able to secure a partner that can accommodate your digital requirements when you need them, you’ll never again have to worry about future disruption - you’ll be future-ready.

May 27 2020
May 27

The ecosystem health is vast, diverse and highly complex, dealing with a large and variable amount of data leads to challenges in terms of storage and security. Cloud computing (CC) has emerged as a response to this challenge, allowing access to data anywhere and on any platform, thus introducing computational gains, especially scalability, agility, and elasticity.

Cloud computing allows doctors, patients and even hospitals, to share data and cross reference information from different diseases and treatments anytime and anywhere, as long as there is an Internet connection. Thus providing opportunities to improve health services from the management, technological, safety and legal perspective. Known examples in healthcare are Microsoft Health Vault, Dossia, World Medical Card and Google Health platform (canceled in 2011).

Currently, the most common cloud computing services can be divided into 3 different models, according to the services they offer:

  1. Infrastructure as a Service (IaaS): this layer provides networking, computing resources;
  2. Platform as a Service (PaaS): allows hosting and the implementation of hardware and software to use in applications;
  3. Software as a Service (SaaS): provides software and does not need to acquire licenses (but the user pays as a "service").

Tipos de cloud computing

Image from: Wikipedia

Structural models of cloud computing refers to the delivery of CC models for the users, which can be public, private or hybrid.

  1. In public clouds, applications, storage, and other resources are made available to the general public. Regardless of the service being free or paid, they are similar for all users, each of which can have an instance of this service.
  2. Private clouds are developed exclusively for an organization, having as main characteristic the individuality of use. Of the three types, a private cloud is the most expensive, because it requires technical and specialized hardware, usually from behind corporate firewalls, and can be acquired and managed by the organization itself or by third parties.
  3. The hybrid model is a mix of public and private clouds. This type of cloud offers customers greater control and security, and also facilitates the reduction of the service according to demand.

However, all models have advantages and disadvantages – below we present a summary of the various models.

MISSING TABLE

Threats

The Cloud Security Alliance published, in 2010, the main threats associated with cloud computing: abuse and nefarious use of the cloud; unsafe programming interfaces; malicious insiders; shared technology vulnerabilities; data loss; account theft; risk of unknown safety profile.

Abuse and nefarious use of the cloud is when hackers take advantage of questions, such as free trials, to access and use the technology for criminal activities, utilizing resources from the cloud along the way. An unsafe programming interface can expose users to risks, which may lead to data theft. Malicious insiders generally take advantage of flaws in the authorization levels to access information. The vulnerability is associated with shared technologies in cloud programming. Data loss is related to the risk of data being erased accidentally or the existence of a problem in the physical servers without any user data backup. Account theft is characterized by a hacker/attacker that unlawfully seizes access to a specific server in the cloud. The unknown safety profile is characterized by the loss or incorrect migration of a client’s vital security procedure at the time of migration of services/data to the cloud.

Cloud security and privacy

The health information and data are confidential and may have a major impact on people’s lives, therefore, some questions about the use of genetic markers in recruitment decisions have been raised.

Thus, in order to prevent threats, providers of cloud services should have policies, principles (Cloud Governance) and security and privacy mechanisms. To assure this, the cloud computing systems should follow the ISO 7498-2 standard, PCI DSS Level1 certification, SAS70 Type II, ISO 27001, as well as one of the most important protocols to ensure transparency within the cloud – the SLA (service-level agreement).The use of HTTPS in conjunction with WS-security should be the bare minimum when data is accessed using the cloud.

Mechanisms that maintain data security, such as identification and authentication, authorization, confidentiality and integrity, should also be considered.

Each professional, whether doctor, nurse, or service provider must have an identification code or a number to access the information stored in the cloud, so that patient data privacy is guaranteed.

Authorization is achieved using roles on process flows within the cloud, in order to set privileges, and is very important to ensure integrity. Confidentiality ensures control over the organizations’ information that is distributed in different databases.

Integrity refers to the accuracy and consistency of stored data. ACID (atomicity, consistency, isolation and durability) are cloud properties that should be imposed. This feature is achieved by digital signature. Non-repudiation is achieved through security protocols and the use of tokens, key or card for data transmission within the cloud applications, such as digital signatures, timestamps and receipt confirmation services.

Availability relates to the guarantees that the service provider can give the customer that the system will work when needed, without failure or access problems, and with the resources requested by the client. However, one must consider that external problems may occur, for example, with internet access, which can be totally unpredictable.

Below is a presentation of the intersection of service implementation and cloud models compared to the 6 safety requirements, where a check mark (√) means a mandatory requirement in the specific service model and in the underlying implementation model, while an asterisk (*) means optional.

MISSING TABLE

Benefits and challenges

A survey developed by the International Data Corporation (IDC), during the third quarter of 2009, identified the following as the main benefits of using the cloud: payment of what is used only, fast/easy implementation for end users, encourages the use of standard systems, simple system sharing with partners. In terms of challenges, security, availability and performance were the major concerns raised.

Issues not mentioned in the survey, but vitally important, are the legal and audit regulations associated with the cloud, such as physical location of the data, and a transparent data generation for the client and the law. Each country has its own legislation concerning data and procedures. For example, EU rules are substantially more restrictive compared to other countries (particularly the United States). In Europe, Directive 95/46/EC protects individuals with regard to the processing of personal data and the free circulation of such data.

May 22 2020
May 22

Today we participated with our full team in the  Drupal 9 porting weekend. You can read about our commitment to Drupal 9 here.

During this efforts we updated the contributed module Image Edit. The module allows to edit images in place on Drupal 8 and Drupal 9 websites. The module was originally developed for a client. Initial development was by our colleague Melinda, who is currently on a maternity leave.

While making the module ready for Drupal 9, we fixed some issues and released the first stable version.

When I tested the patches provided by different colleagues, I realized how useful this module is.

Therefore I created some examples and a short screen cast about it.

With this module you can transform an image like the two images below.

May 20 2020
May 20

Read our roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community. You can also review the Drupal project roadmap.

Project NewsDrupal 9.0

Drupal 9 will be released on June 3rd, 2020

Despite the disruption of a global pandemic, the Drupal community has beaten the odds and Drupal 9.0 is on time for it's scheduled release window on June 3rd, 2020.

Drupal 9 represents the culmination of all of the features developed over the course of the Drupal 8 lifecycle, as well as the realization of the project's commitment to easy upgrades.

Drupal 9 beta 3 is available now, and the first release candidate will be available soon. We encourage you to participate in the Drupal beta test program, to help us ensure a smooth release.

Drupal 9 porting weekend from May 22-23

With the release of Drupal 9 only a couple weeks away, the community is coming together to support the effort to get modules ready for Drupal 9. After a successful Drupal 9 porting on April 28th, nearly 75% of the top 200 most used modules are already Drupal 9 compatible. Never before has so much of the contributed module ecosystem been ready even before the new release.

If you'd like to join in on the Drupal 9 module porting weekend, community member @kristen_pol has written a guide to the event.

New Drupal Brand assets available

The Drupal Association was very pleased to announce a new evergreen Drupal brand in time for the release of Drupal 9.

What does 'evergreen' mean?

The new branding is evergreen in the sense that it is no longer tied to a specific major version of Drupal. Whether for Drupal 9, Drupal 10, and beyond this new brand can remain the consistent identity for Drupal. This parallels the Drupal project's own development philosophy, where major version upgrades should no longer be a difficult process for end users.

With these new brand materials we hope to be able to unify the presentation of Drupal throughout the ecosystem, and help reintroduce Drupal to the world when the project inevitably gains more attention during Drupal 9's release.

We encourage you to begin using the new brand within your own materials as well - to support this effort.

Automated Deprecation Patches to port your module to Drupal 9

The Drupal Association is working together with the Drupal Rector team from Palantir and contributor Tedbow from Acquia to provide automatically generated Drupal Rector patches for all the projects on Drupal.org.

As of April, these patches are already available through DrupalCI. In May we hope to begin having a bot automatically post these fix patches to Drupal.org issues

More Drupal 9 Readiness tools

Drupal.org Updates

Events listing feature on Drupal.org

As we spoke about in last month's update, we've been working on a Event Listing Content Type on Drupal.org to help replace the aging Groups.Drupal.org subsite, and to support our global community by providing a central repository of events.

EDITED: The new event listings are being beta tested by the Event Organizers Working Group before a roll-out to initial community users.

Respectful Advertising

The COVID pandemic and its impact on DrupalCon has only emphasized the need for the Drupal Association to further diversify its revenue sources. We've made significant strides over the last several years, but as we've heard from many of you, that work must accelerate.

We've made a few changes over the course of April to start accelerating this revenue diversification:

In the most noticeable change, we've partnered with CarbonAds, a network that focuses on advertising to technical audiences, to create placements on Drupal.org.

These placements are hidden for Drupal Association members, so this program also helps promote DA membership as well, and does not put advertising in the workspace of our committed supporters.

Enhanced Membership Options

Speaking of membership - we also made some major overhauls to the Drupal Association membership program during the #DrupalCares campaign. Many members of the community reached out to us asking for more options for supporting the Drupal Association through membership, and we were happy to accommodate those requests.

There are now new membership levels for annual renewal, and we've also added a monthly membership option, which is now the default. We hope to continue to expand the membership program and its benefits to further support our fiscal stability in the future. 

#DrupalCares

A special thanks

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank:

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

May 20 2020
May 20

The jQuery project released version 3.5.0, and as part of that, disclosed two security vulnerabilities that affect all prior versions. As mentioned in the jQuery blog, both are

[...] security issues in jQuery’s DOM manipulation methods, as in .html(), .append(), and the others. Security advisories for both of these issues have been published on GitHub.

Those advisories are:

These vulnerabilities may be exploitable on some Drupal sites. This Drupal security release backports the fixes to the relevant jQuery functions, without making any other changes to the jQuery version that is included in Drupal core or running on the site via some other module such as jQuery Update. It is not necessary to update jquery_update on Drupal 7 sites that have the module installed.

Backwards-compatibility code has also been added to minimize regressions to Drupal sites that might rely on jQuery's prior behavior. With jQuery 3.5, incorrect self-closing HTML tags in JavaScript for elements where end tags are normally required will encounter a change in what jQuery returns or inserts. To minimize that disruption in 8.8.x and earlier, this security release retains jQuery's prior behavior for most safe tags. There may still be regressions for edge cases, including invalidly self-closed custom elements on Internet Explorer.

(Note: the backwards compatibility layer will not be included in the upcoming Drupal 8.9 and 9.0 releases, so Drupal 8 and 9 modules, themes, and sites should correct tags in JavaScript to properly use closing tags.)

If you find a regression caused by the jQuery changes, please report it in Drupal core's issue queue (or that of the relevant contrib project). However, if you believe you have found a security issue, please report it privately to the Drupal Security Team.

May 20 2020
May 20

Low-code platforms have been slowly gaining popularity in the Marketing stack of organisations. Low-code essentially means being able to assemble UI of your Digital experience visually and often without the need for a dedicated Development team. Webflow for Websites and Shopify for e-commerce are among the list of rising low-code platforms. These platforms help organisations become more agile and help enable various functions like Design, Marketing to come on board as Makers on the Digital experience. Today we will talk about Cohesion now renamed as Site Studio, a low code offering for Drupal websites by Acquia, we will discuss benefits and some of the paradigms that make it an effective low code option in Drupal stack.

Drupal has witnessed a healthy growth over these years as a pillar for Enterprise Digital stack, Acquia's Site Studio brings the DIY spirit to this stack. What more and more Marketers are looking for is quicker and cheaper ways to assemble landing pages and work out of a component library of Digital elements that are performant and consistent to the Design systems of the organisation. A low code approach to Drupal will only work if its aware of its capabilities & flexibility, that is what Cohesion brings on the table. Ability to apply Design systems on your Digital Experience built on Drupal, DIY!

Benefits of Acquia's Site Studio

  1. Low code 
  2. Highly configurable
  3. Easy maintainable
  4. Easily modifiable 
  5. Highly scalable
  6. Component-based driven
  7. Less developer dependent

Acquia Site Studio

Acquia's Site Studio provides a contributed module and theme which needs to be installed in Drupal, quite similar to the way we install other modules and themes. However, to use this service, it is required to purchase a cohesion package that provides us with an API Key, Agency Key, Site ID, and an API Server URL which needs to be inserted in the Account Settings form.

Cohesion DX8

Here are the settings and key features of Acquia's Site Studio:

  1. Website Settings
  2. Style Settings
  3. Components
  4. Templates

Site Studio provides an in-site configuration form which takes care of your branding requirements. You can add your own branding elements like fonts, colours, icons, grid settings, SCSS variables, etc. directly onto the site. And there is absolutely no need to make any changes in the theme folder as well as the theme settings. Doesn’t this sound cool! Your basic frontend settings would be handy to you and you can modify it directly from the site without asking for help from a developer.

Website Settings

Cohesion DX8

Style Settings

It’s not over yet, with Acquia Site Studio you can create/modify styles of a site. Acquia Site Studio provides settings to update the base style or to create custom styles. 

Cohesion DX8

And this is how the default preview and configuration pane looks like.

Now let’s understand what comes under Base styling and Custom styling

Base styling - Styling which is going to be consistent throughout the site, e.g., font-size of body. Font formatting of Headings, Style of a button, style of link, etc.

Custom styling - Styling which will be different for each instance or as a variation of base style. E.g., Big button, Small button, Read More Link, Consistent layout related styles (Padding Top & Bottom Large, Padding Top & Bottom Small), Social icons theming, etc.

Let’s look at the example of CTA link styling.

  • CTA style structure
    • Link styling
    • After pseudo-element styling
    • On hover pseudo-element styling
  • Style properties required for CTA link and those properties are added in the config form through properties button on the top right corner of the styling pane
  • Styling properties required for pseudo-element styling
  • Styling properties required for pseudo-element after hover

COHESION DX8

COHESION DX8

cohesion DX8

Component Builder

Now let’s introduce the coolest feature of the Acquia's Site Studio -  a Component Builder. Acquia Site Studio follows a complete component-based development approach. So, what is the component-based development approach? 

Acquia Site Studio provides a list of elements. By using these elements even simple and complex components can be created and be made configurable by creating a component form. The amazing part is that everything can be built by mere drag and drop. Doesn’t it sound super user friendly?

Let’s look at an example of a Hero Component and figure out which elements could be part of this component?  Let’s create a list of elements(atoms).

  • Hero Image
  • Hero Title
  • Hero Description
  • CTA button

Let’s consider the below-displayed hero component that we are trying to achieve.

Cohesion dx8

Based on the Hero component design, the hero component structure would look similar as described in the below image.

Cohesion DX8

A component will be created which will have the above-mentioned elements. But what about styling? This is the real beauty of Acquia Site Studio. It allows us to create a base style as well as a custom style and those can be appended to the component. Doesn’t it sound fascinating?

Here would like to take you back to the styling part of the section where we have styled CTA link. Now, here in the above component CTA link element can directly assign the style which has been created under custom style. And as per image, styling can be appended to CTA Link element.

COHESION DX8

Layout

There are multiple ways to create different layouts. Here, with Acquia Site Studio, the layout can be defined by a Layout Category component and also component dropzone element can be provided to add other components. 

Does it mean? Woah! Yes, it means nesting use of the component is also possible. I won’t provide the screenshot for this example right now.

Template

There are four main templates:

  • Master Template
  • Content Template
  • Views Template
  • Menu Template

The template names clearly state the use of each template, but, I am sure you must have some questions related to master templates. 

What will it hold? The thumb rule is, this template should be least modifiable, means, it should not be updated frequently. We can ensure this by using only consistent section/components/regions added to it.

Acquia Site Studio Demo 

Explore how Site Studio enables you to apply design systems on your Digital Experience built on Drupal!

I hope this blog was helpful to understand the basic outline of Acquia Site Studio. Stay tuned! We have an upcoming blog that will talk about the advanced usage of templates. 

P.S - If you want to learn more about Site Studio, Acquia has a training and a full support guide - https://cohesiondocs.acquia.com/6.1 and get access to the free sandbox here - https://www.acquia.com/products-services/acquia-cohesion#create-sandbox.

Interested to read more about Site Studio? Here are some articles for you:

May 20 2020
May 20

I recently finished porting this website from a static site generator to Drupal 8, meaning that this site has now been powered by three different major versions of Drupal (6, 7 and 8) as well as by two static site generators since it was first launched in early 2010.

The majority of the content was imported using migrations from JSON feeds that I created. This included:

  • Blog tags
  • Blog posts
  • Talks
  • Redirects

In some follow-up posts, I'll be looking at each migration separately, describing any issues and look at how it was used to import its respective content.

I'll update this post with the links to the follow-up posts, and they are also available from the blog series' page.

May 12 2020
May 12

When your website users are able to quickly find what they are looking for, you naturally get more conversions. A good UX consultant will always recommend you to add a search function to a website.

Read on to discover more details about why search is important. If you are using Drupal, this post will be of special interest to you because we will describe how to make the website search functionality really fast and user-friendly with Lunr.js.

Why is search important on a website?

Many customers ask “Does my website need a search?". On large and complex websites, an internal website search is a must-have in order to improve website navigation. The decision to add a search box to a website is especially vital for content-rich websites, e-commerce stores, knowledge bases, and so on.

However, it is also very helpful for small and medium-sized websites. Here we will discuss the ways search functionality on any website, when it is fast and user-friendly enough, can benefit its owner:

  • If a user finds what they need through the search box quickly, they are likely to purchase it.
  • Search results give you new ideas about products or services users are interested in.
  • You get information about user behavior for your marketing strategies.
  • A search feature on your website provides you with new SEO keywords.
  • Adding search functionality to a website makes your design more customer-centric.
  • A search box increases your session duration and reduces the bounce rate.

You can always reach out to our web development team for a free consultation and further creation of a search feature on your website at very affordable prices. We respect your budget and will recommend a fast, easy-to-use, and efficient search solution that will not require extra costs. Meanwhile, let’s discuss how search functionality is created on websites.

How to add a search function to a website built with Drupal?

The answer to the question about how to set up search functionality on a website, depends on the CMS you are using. Each CMS has its own special extensions that provide for the search feature.

In Drupal, to create basic search options for smaller sites, the built-in Search module is enough. It allows your users to search for full words in Drupal entities (content nodes, users, etc.). You can also specify the indexing settings and choose the ranking factors like:

  • publication time
  • activity in comments
  • keyword relevance

Through the use of extra modules, Drupal websites can also get more complex search features like:

  • faceted search
  • search by alternate spellings
  • similar content suggestions
  • result highlighting
  • search through attachments
  • multisite search
  • and much more

To achieve this, they connect to robust search platforms like Apache Solr or Elasticsearch through contrib modules like Search API Solr Search and Elasticsearch Connector, as well as using modules like Search API, Facets, and many more.

Superfast and easy-to-use internal website search with Lunr.js

In addition to the ones described above, there are other interesting, JavaScript-based options to add a search function to a website built with Drupal. They offer specifically fast client-side search. One of these we are sharing with you right now — your Drupal can have a search functionality based on Lunr.js.

What is Lunr?

Lunr.js is a full-text search library to use in the browser. It is a small but full-featured library that provides great search experiences. Lunr.js offers a simple search interface for finding the content that best matches the search queries. It requires no server-side search services and needs no external dependencies.

You might have noticed its “lunar” brand design. Lunr.js is an alternative to the famous search engine Solr. “A bit like Solr, but much smaller and not as bright,” is the official slogan of Lunr.

Lunr.js

All JavaScript solutions are known for exceptional speed, and Lunr is no exception. Lunr.js search functionality is especially good for cases when an instant search in Drupal is needed.

A sum-up of Lunr.js search features

  • instant search results with suggestions
  • client-side search without overloading Drupal
  • keyword-based search
  • partial and fuzzy search options
  • a scoring system for showing relevant results
  • paging
  • location history
  • lazy-loading search results
  • and more

Modules for the integration between Lunr.js and Drupal

  • Lunr search

The Lunr search module integrates Lunr.js with Drupal. It uses Drupal Views to pre-build a search index and the search result pages. These are delivered to the client using JavaScript. Its features include no-configuration multilingual support, custom field/facet searches, and more.

Lunr search Drupal module
  • Search API Lunr

The Search API Lunr module provides a Search API backend Lunr search with configurable fields. Instead of adding content to a search back-end, JSON files are loaded directly into the browser.

Add a search feature to your website with our experts!

Ready to give your website a new boost through adding a search functionality? Our website support agency experts are ready to do it at affordable prices.

We have both developers who specialize in Drupal and those who have mastered WordPress. JavaScript frameworks are another passion of our team, because we know they can enrich your search box, like any other website elements, with special speed and interactivity.

Contact us and let’s discuss your best price!

May 06 2020
May 06

Drupal 9 Is Almost Here

Drupal 9.0.0 will officially drop on June 3, 2020. The good news (which we’ve already covered in detail here, and here) is that Drupal 9 will not be a significant upgrade from Drupal 8. 

To recap, it is effectively a version bump, with two caveats:

  • Symfony 4 will replace Symfony 3 as a core dependency. This will improve Drupal’s underlying core functionality, stability, and security.
  • Some Drupal core code that has already been marked as deprecated will be removed entirely. This code still exists in Drupal 8 to support previous upgrades from Symfony 2 in the past, and all deprecated code has Drupal 9 ready alternatives which should be used instead. There are automated tests that can assure any organization if the project is using any such deprecated code, and will even offer suggestions to replace it.

However, just because you’re on Drupal 8 now, don’t assume you’ll be ready to make the leap to 9 when June rolls around.

It’s More Than Code

While the code part is relatively easy to prepare for, the long-term viability of your Drupal 8 AND 9 CMS depends on you following Drupal best practices as well. An ideal Drupal site (i.e. one that will last the next two years and /or successfully make the leap to Drupal 9) will be one that leverages core functionality as much as possible.

Here are some of the more recent additions to Drupal core that all Drupal 8 sites should consider using in order to be ready for Drupal 9.

1) Paragraphs vs Panels vs Layout Builder

The ability for editors to layout content has been a long road. Panels was king, but Layout Builder has since replaced Panels. Layout Builder also has the added benefit of being part of Core, so it will work more closely with other core functionality.

The Paragraphs module is still very useful for content layout. However, the days of abusing Paragraphs for controlling layout are over. Start using it for what it was originally intended for, managing complex, tightly-bound bundles of content bits, but certainly not for actual page layout. There should be no more content-specific paragraphs within layout-specific paragraphs. 

This is an area where Layout Builder can help with content placement, while still using Paragraphs for content components.

2) Media Management

The Media module is now part of core, and most asset management in Drupal will begin there. Managing images, documents, and other digital assets as content types or custom entities should no longer be the pattern. Community support of core Media is ever-growing, and this is where the community’s focus will remain in the future.

3) Workflow: Workbench Moderation vs. Content Moderation

Editorial workflow is extremely important for many sites. Drupal 8 didn’t offer anything for this at first, and the community rallied around the ubiquity of Workbench Moderation. With the addition of Content Moderation in core, which is a very close facsimile of Workbench Moderation, this is where future attention will lie.

The Lightning install profile includes some tools to help transition from Workbench Moderation to to Content Moderation, so organizations don’t have to struggle alone in silence.

4) Modern Front-End Patterns

This is not strictly a Drupal 9 readiness point, but following good component-based design patterns is essential to maintaining a modern web platform. Using baseline tools such as Pattern Lab or Storybook is a great place to start to manage the components of a website. Drupal themes play nicely with these design implementation systems, often ingesting the same templates directly.

This sets up any Drupal site nicely for using component-based layout with Layout Builder and even Paragraphs-based content components.

Conclusion

Ensuring any Drupal project is not using deprecated code is the first step in Drupal 9 readiness. Even more important is ensuring that it is taking advantage of the latest core functionality and features, as these will be where the focus of the Drupal Contrib Community will be. Taking advantage of Core as much as possible will help with cases where once-common contrib modules are eventually abandoned over time.

Apr 22 2020
Apr 22

Some time ago, I announced that I was planning on writing a book on automated testing and test driven development with Drupal. I created a landing page and set up a mailing list, but I wasn't sure at that point what I was going to cover or create as part of the book.

I'm going to write a book on automated testing in Drupal. Join the mailing list for updates, and I'm happy to take suggestions on what to cover. https://t.co/YXNpe6f8Ft #drupal

— Oliver Davies (@opdavies) May 15, 2018

Being a meetup and DrupalCamp conference organiser, after some thought I decided to build a website for an example conference, and that some of this code would then be included in the book as example content. This seemed to cover most of what I originally wanted, through features like a call for papers for potential speakers to propose sessions, allowing organisers to administer and moderate those proposals, automatically sending notification emails to submitters and displaying the accepted sessions.

I've started building it with Drupal 8.8 and it is now available on GitStore to purchase access to, including all future updates as I continue building the application - adding new features and upgrading to Drupal 9 once it is released. There are some other interesting things there too, such as using feature flags to enable or disable functionality, and using GitHub Actions to run the tests automatically.

The book itself I've added a page for on Leanpub, and I'll be continuing to add content to it in parallel to building the example codebase. Once there is enough content, I will release the first draft for purchase.

Any purchases that are made via Gitstore or Leanpub, an amount will be donated to the Drupal Association and the #DrupalCares campaign to help sustain the Association during COVID-19.

Apr 17 2020
Apr 17

Those of us who have experience traveling the long dark roads of migration projects between Drupal’s previous major version releases may, understandably, be feeling some trepidation at the impending release of Drupal 9 (potentially as soon as June 3, 2020). 

The journey from Drupal 7 to Drupal 8 in particular was a major challenge for many developers, with massive API changes, a completely new theme layer, and a fundamental OOP architecture. 

The investment in these massive changes, however, is paying big dividends as we approach D9. This journey is going to be a lot easier in comparison. In fact, for many of us, the journey is nearly over before it begins.

D9 is D8 (mostly)

Here is the big takeaway: if you are on Drupal 8.8+, you are already fully API compatible with Drupal 9. Drupal 8’s OOP structure and its core Symfony foundation have made it possible for the Drupal API to be more stable and continuous between major versions. In fact, the major differences between Drupal 8 and Drupal 9 are the removal of deprecated code and the update of core dependencies so that they remain on supported versions.

If D9 is D8 (mostly), then why should I go through the trouble?

Drupal depends on third party libraries and frameworks, and those libraries and frameworks must be updated over time to maintain support and security updates.  Sometimes, it’s impossible to maintain backward compatibility with older versions of these dependencies. Drupal 8 adopted semantic versioning, which made it possible to introduce major new features and backward compatible third-party dependency updates in minor releases, while also making it easier to adopt backward compatibility breaking changes in a clear and predictable way.

Drupal 8’s central core dependency, Symfony 3.4, will stop receiving bug fixes in November 2020, and stop receiving security updates in November 2021. Drupal 9 will depend on Symfony 4.4, which is not fully backward compatible with Symfony 3.4. 

Your site needs to remain secure, so you need to adopt Drupal 9.  The good news is that even though Drupal 9 is planned for release June 3, 2020, Drupal 8.9x will remain supported until November 2021. This time around, we have a much lower level of effort to transition, and a long window of time to make it happen.

Dealing with deprecated code

In most cases, removing instances of deprecated code from your custom code is a simple process. Matt Glaman’s drupal-check tool can scan your code and return a report of any deprecated code in use. Many changes will be as simple as changing a call to: file_unmanaged_copy() to \Drupal::service(‘file_system’)->copy(). Some changes may require a little more work, but that work is being done within the same Drupal 8 API you’ve grown accustomed to.

For contributed code, identify the modules you rely on that use deprecated code, and use this as a great opportunity to contribute back to the community by testing and submitting patches.

The bottom line, however, is that you must deal with your deprecated code. It will not work in D9.

Dealing with core dependency updates

Here are some of the most important core dependency changes that may impact you.

  • Symfony 3 to Symfony 4.4: If you are making direct use of any Symfony code, you may need to update that code to its Symfony 4.4 equivalent

  • Twig 1 to Twig 2: There are a few changes and deprecations in Twig 2 that may require code updates in your templates. See Preparing for use of Twig 2 in Drupal 9

  • jQuery UI: Most of the world has moved on from jQuery UI, and in Drupal 9, so follows Drupal. Most jQuery UI components are being removed from core, with a few exceptions. If your project requires jQuery UI components, you will need to bring them in as contributed code

There are of course other core dependency changes. See Drupal.org’s Drupal 9 documentation for more information.

Are contributed modules ready for Drupal 9?

The state of contributed modules is much better compared with the transition from Drupal 7 to Drupal 8.  

Most Drupal 8 compatible modules are either fully code compatible with Drupal 9, or require a few deprecated code updates (and a very minor yml file change). The burden on module maintainers is much lighter, and the low level of effort presents an excellent opportunity for the Drupal community to contribute back deprecation changes and test patches.

The transition to Drupal 8 was a tough one, but the architecture and strategy changes that the core development team adopted are paying off in a big way as we look toward Drupal 9 and beyond. The stable, gradual, and predictable advancement of the API creates an environment that promotes best practices, and provides teams with a more predictable landscape with less risk.

Apr 16 2020
Apr 16

Mega menus are not a design trend anymore, but an essential part of most of the websites related to news or eCommerce.

The Menu Item Extras module for Drupal 8 improves the default menu system in Drupal, by allowing the site builder to add fields to the menu items. That way, it is possible to create a mega menu with a couple of simple steps.

Follow along to learn how to use this module. Let’s start!

Step #1: Install the Required Modules

In addition to the Menu Item Extras module, you will need to install Viewfield. Viewfield provides a field that holds a reference to a View and renders it whenever the entity containing the field is displayed.

  • Open the terminal application of your PC.
  • Type:
    • composer require drupal/viewfield
    • composer require drupal/menu_item_extras

200414 menu item extras 001200414 menu item extras 001

Enable both modules after downloading.

  • Click Extend.
  • Check both modules.
  • Click Install.

200414 menu item extras 003

Step #2: Create a Taxonomy System

Perhaps the word “system” is not very appropriate since we will only create one vocabulary with ten associated terms. However, this example will explain the basics, so you can work with more complicated taxonomy systems in the future.

  • Click Structure > Taxonomy > Add vocabulary.
  • Enter the name “Topics” and click Save.
  • Click Add term and add all terms detailed below one by one.
    • Topics
      • D8
      • Symfony
      • WP
      • Magento
      • CSS
      • HTML
      • JS
      • PHP
      • Python
      • DevOps

200414 menu item extras 004

Step #3: Create the Content Types

For the purpose of this tutorial, we are going to use 2 content types with the following fields:

  • Services
    • Service Image / Image / Allowed number of values: 1
    • Intro text / Text (formatted) / Allowed number of values: 1
    • Description / Default field

200414 menu item extras 005

  • Blog
    • Category / Taxonomy term / Allowed number of values: unlimited
    • Description / Default field

200414 menu item extras 006

Make sure you select the Reference type (Topics) when creating the Taxonomy field.

200414 menu item extras 007

Step #4: Create Content

We are going to create 3 nodes of type Service:

  • Tutorials
  • Videos
  • Books

Each one of these nodes will be associated with a second-level element in the mega menu. On the other hand, we are going to create about 10 nodes of type Blog with different terms associated with them. Make sure that you include 2 of the terms (ex.: D8, WP) with more frequency than the others.

200414 menu item extras 008

The Content screen should look at the end of this process more or less like the image below.

200414 menu item extras 009

Step #5: Create the Menu Items

The first level of the main menu will have the following menu items:

  • Home
  • About
  • Resources
  • Contact

Each one of the menu items will have an additional (extra) field. This field will be set at the second level of the menu, that is, each one of the card items in the mega menu. This field will allow us to present each one of the view blocks.

  • Click Structure > Menus.
  • Click Edit menu, to edit the Main navigation.

200414 menu item extras 010

  • Click Add link 3 times to add the missing first-level menu items.
  • Click Save each time you enter a menu item name.

Use the special tag to display only the link text.

200414 menu item extras 011

200414 menu item extras 012

Once you have created and rearranged the links of your menu,

  • Click Manage fields > Add field, to add a field to the menu items.
  • Select a field of type Viewfield.
  • Give it a proper name and label.
  • Click Save and continue.

200414 menu item extras 013

  • Set the Allowed number of values to Unlimited.
  • Click Save field settings

200414 menu item extras 014

You will have the option to link a default view to this particular field. Leave these fields empty.

  • Scroll down and click Save settings
  • Click Manage display.

200414 menu item extras 015

  • Hide the menu card label.
  • Click Save.

Step #6: Create the Views

  • Click Structure > Views > Add view
  • Show Content of type Services.
  • Create a block with an unformatted list of fields.
  • Choose to show 3 items per block.
  • Click Save and edit.

200414 menu item extras 016

  • Add the Service image field
  • Set the image style to Thumbnail and link the image to the Content.
  • Click Apply

200414 menu item extras 017

  • Add the Intro text field as well.
  • Rearrange the fields.
  • Click Save.

200414 menu item extras 018

  • Click Structure > Views > Add view
  • Show Content of type Blog.
  • Create a block with an unformatted list of fields.
  • Choose to show 1 item per block.
  • Click Save and edit.

How to Use the Menu Item Extras Module for Drupal 8

  • Add the field Category.
  • Click Add and configure fields.
  • Click the Multiple field settings.
  • Select Unordered list.
  • Set the number of values to display to 3.
  • Click Apply.

200414 menu item extras 020

  • Remove the Sort criterion
  • Click Apply.
  • Click Save to save the view.

Step #7: Add the View Blocks to the Menu Items

  • Click Structure > Menus
  • Edit the Main navigation.
  • Edit the menu item Resources.
  • Add the 2 blocks you created in step # 5.
  • Click Save.

200414 menu item extras 021

Step #8: The CSS Styles

I am not going to explain this code in detail since it is out of the scope of this tutorial. However, you can research more about Drupal Views and CSS Grid in this article. If you want to practice more with mega menus, take also a look at this article.

#block-bartik-main-menu a,
#block-bartik-main-menu li span {
background: transparent;
text-shadow: unset;
color: whitesmoke;
}

#block-bartik-main-menu .menu-level-0 {
display: grid;
grid-template-columns: repeat(4, 1fr);
}

#block-bartik-main-menu .field--name-field-menu-card .field__items {
display: flex;
}

#block-bartik-main-menu .menu-dropdown-0 {
position: absolute;
left: 0;
top: 166px;
width: 100%;
z-index: 10;
}

#block-bartik-main-menu a,
#block-bartik-main-menu li span {
background: transparent;
text-shadow: unset;
color: whitesmoke;
}

#block-bartik-main-menu .menu-item {
display: grid;
align-content: center;
justify-content: center;
}

#block-bartik-main-menu .menu-level-0 {
display: grid;
grid-template-columns: repeat(4, 1fr);
}

#block-bartik-main-menu .menu-dropdown-0 {
position: absolute;
left: 0;
width: 100%;
padding-top: 16px;
background-color: rgba(141,139,132, 0.9);
}

.view-tutorials .view-content {
display: grid;
grid-template-columns: repeat(3, 1fr);
}

.view-tutorials .views-row {
display: grid;
justify-items: center;
}

.view-tutorials .views-row span > a {
font-size: 1.3rem
}

.view-tutorials .views-row .views-field-field-intro-text {
font-size: 1rem;
}

#block-bartik-main-menu a,
#block-bartik-main-menu li span {
background: transparent;
text-shadow: unset;
color: whitesmoke;
}

#block-bartik-main-menu .menu-level-0 {
display: grid;
grid-template-columns: repeat(4, 1fr);
}

#block-bartik-main-menu .field--name-field-menu-card .field__items {
display: flex;
}

#block-bartik-main-menu .menu-dropdown-0 {
position: absolute;
left: 0;
top: 166px;
width: 100%;
z-index: 10;
}

#block-bartik-main-menu a,
#block-bartik-main-menu li span {
background: transparent;
text-shadow: unset;
color: whitesmoke;
}

#block-bartik-main-menu .menu-item {
display: grid;
align-content: center;
justify-content: center;
}

#block-bartik-main-menu .menu-level-0 {
display: grid;
grid-template-columns: repeat(4, 1fr);
}

#block-bartik-main-menu .menu-dropdown-0 {
position: absolute;
left: 0;
width: 100%;
padding-top: 16px;
background-color: rgba(141,139,132, 0.9);
}

.view-tutorials .view-content {
display: grid;
grid-template-columns: repeat(3, 1fr);
}

.view-tutorials .views-row {
display: grid;
justify-items: center;
}

.view-tutorials .views-row span > a {
font-size: 1.3rem
}

.view-tutorials .views-row .views-field-field-intro-text {
font-size: 1rem;
}

200414 menu item extras 022

I hope you liked this tutorial. Thanks for reading!


About the author

Jorge lived in Ecuador and Germany. Now he is back to his homeland Colombia. He spends his time translating from English and German to Spanish. He enjoys playing with Drupal and other Open Source Content Management Systems and technologies.
Apr 16 2020
Apr 16

This month’s SC DUG was a round table discussion on working remotely during the Covid-19 lock down. We had actually planned this topic before the crisis emerged in full, but found ourselves having to pivot our talking points a fair bit.

[embedded content]

The discussion centered on things that people are dealing with, even those of us who work remotely on a regular basis. A few resources were shared by people on the call, including Pantheon’s Donut Slack Bot.

If you would like to join us please check out our up coming events on MeetUp for meeting times, locations, and remote connection information.

Apr 15 2020
Apr 15

Our normally scheduled call to chat about all things Drupal and nonprofits will happen TOMORROW, Thursday, April 16, at 1pm ET / 10am PT. (Convert to your local time zone.)

No set agenda this month -- we can discuss whatever Drupal related thoughts are on your mind. If you would like to contribute to the conversation, please join us. 

All nonprofit Drupal devs and users, regardless of experience level, are always welcome on this call.

Feel free to share your thoughts and discussion points ahead of time in our collaborative Google doc: https://nten.org/drupal/notes

This free call is sponsored by NTEN.org but open to everyone.

REMINDER: New call-in information -- we're on Zoom now!

  • Join the call: zoom.us/j/7423234165
    • Meeting ID: 742 323 4165
    • One tap mobile
      • +16699006833,,7423234165# US (San Jose)
      • +13462487799,,7423234165# US (New York)
    • Dial by your location
      • +1 669 900 6833 US (San Jose)
      • +1 929 205 6099 US (New York)
  • Follow along on Google Docs: https://nten.org/drupal/notes
  • Follow along on Twitter: #npdrupal

View notes of previous months' calls.

Apr 14 2020
Apr 14

Read our roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community. You can also review the Drupal project roadmap.

...what a changed world we live in compared to just a month ago. There are so many people across the world who make Drupal what it is, and Drupal.org is where we all come together.

Tim Lehnen (hestenet) | CTO - Drupal Association

Project News

Drupal 9 release window confirmed

The Drupal Core release managers and the Drupal Association team successfully released Drupal 9.0.0-beta in March, which means that we are on track for the June release window of Drupal 9. The target release date for Drupal 9 is June 3, 2020. 

There are several things you can do to help: 

  • Test the latest Drupal 9 beta. At the time of writing, Drupal 9.0.0-beta2 is the latest available version.
  • Test the Drupal 8.9.0-beta. This will be the last minor release of Drupal 8, to come out simultaneously with Drupal 9. 
  • If your organization would like to formally join the beta test program, you are welcome to do that as well. 

It's also time to get your own sites, contributed modules, and custom modules ready for Drupal 9. The community has built a variety of tools to help: 

#DrupalCares Fundraising Campaign

#DrupalCaresAt the end of March we also kicked off the #DrupalCares fundraising campaign. This campaign is specifically targeted at closing the budget gap caused by the impact of COVID-19. 

Drupal is used across the globe - but did you know Drupal is also in use by organizations on the front-line of the COVID-19 fight? The National Institutes of Health, the CDC National Prevention Information Network, Oxfam, Unicef and many others are using Drupal to spread key messages about safety and prevention. The Drupal Association supports these organizations with the infrastructure that hosts new releases and security updates. 

As of today, project founder Dries and Vanessa Buytaert have also announced a #DrupalCares matching campaign. They will match all individual donations, membership upgrades, and membership renewals through the end of April - up to $100,000. We appreciate their leadership and support. 

Drupal.org Updates

Semantic Versioning Available for All Projects

Semantic versioning is now available for all contributed projects on Drupal.org. All versions of Drupal greater than Drupal 8.8.3 are compatible with semantic versioning, and the old version format will continue to be supported until Drupal 10. 

Example of Semver

Using semantic versioning allows contributed project maintainers to follow the same pattern that has been so successful for the core release cycle--enabling core to release significant features every six months. The semver version pattern is: MAJOR.MINOR.PATCH. To summarize how this is used in the Drupal project: PATCH versions should include bug fixes and security releases; MINOR versions can include new features, but should remain backwareds compatible; MAJOR versions can break backwards compatibility and remove deprecated code.

Project maintainers who want to continue to support versions of Drupal at or below 8.8.3 should continue using the old version schema.

Major overhaul to Packaging Pipeline

Drupal PackagingAs we mentioned in last month's update, we've significantly overhauled how Drupal.org's packaging pipeline works, to provide better support for scaffolding Composer installations, to improve introspection, and to make it easier to enhance and maintain as Drupal continues to evolve as a software platform. 

There were several components to this packaging update: 

  • The Packaging Process is now a multi-step Jenkins pipeline, making it easier to observe packaging progress at each phase. 
  • The Packaging Process uses composer create project to ensure that the generated .zip and .tar.gz archives are ready to use Composer. 
  • The subtree splitter step of the packaging process uses a local path repository, so the packaging infrastructure does not depend on GitHub/Packagist
  • Many steps of the pipeline have had performance improvements and reductions in redundant work, to speed up the release process as much as possible. 
  • The use of local path repositories will also help reduce the risk of early embargo break for any security releases. 

Coming soon: Community event listings on Drupal.org

Groups.Drupal.org is well past its prime, but there's still a tremendous need for the community to organize around local events. Especially in light of shelter-in-place orders during COVID-19, we want to support the community in making their virtual events accessible to all.

In collaboration with the Event Organizers Working Group - we've been building a new event listings feature for drupal.org that will allow users to submit their DrupalCamps and other events, include a feed that can be aggregated by tools like Drupical.com, and provide a baseline for future features that will allow us to sunset Groups.Drupal.org

Drupal Association Updates

Contribution Recognition Committee

We know that the community at large is very interested in the future of Drupal.org's Contribution Credit system. The Contribution Recognition Committee has been holding regular meetings since its formation at DrupalCon Amsterdam in 2019, and has been making good progress gathering community input and creating early models for the next generation of the credit system.

The next phase of the committee's work is interviews with key stakeholders in the community, before assembling a list of final recommendations. Please understand:The CRC is on a brief pause as many committee members have found their time constrained for extra child care and other concerns during shelter-in-place orders for COVID-19, but we hope to resume progress as soon as possible. 

You can reach out to the committee here

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank: 

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Apr 10 2020
Apr 10

Drupal 9 is scheduled to be released on June 03, 2020. This is when the final Drupal 8 minor release 8.9 will also be released. Considering the history of previous Drupal major upgrades, Drupal 9 will relatively be smooth. Thanks to the semantic versioning introduced in Drupal 8. The upgrade to Drupal 9 will just be another minor upgrade with deprecated code removed. Drupal 8 has brought a lot of standardization in the Drupal world, thus allowing Drupal as a project to grow incrementally. 

To put it in simple terms, Drupal 9 contains the same code as of 8.9 + deprecated code removed. Here’s a reference image from the Drupal 9 documentation.

Drupal 9

That is if you reduce Drupal 9 from the last Drupal 8 version (8.8.x) the rest is just the deprecated code and dependency upgrades. 

However, upgrading all the underlying dependencies and removing all deprecated API is a challenging task. Contributors around the world are working hard to get this done especially when the world is facing an epidemic. This is a challenging time for the entire world, yet Drupal contributors have shown their love towards Drupal and are helping it advance to the next release.

This article highlights some of the system requirements, the new dependencies and some of the most commonly used APIs which are going to be removed in 9.0.0.

Let us expedite Drupal’s journey to its 9th version!

| Third-party dependency updates

  • Upgraded Symfony from 3 to 4.4

  • Upgraded Twig from 1 to 2

  • Upgraded CKEditor from 4 and 5

  • Upgraded PHPUnit from 6 to 7

  • Upgraded Guzzle from 6.3 to 6.5.2

  • Popper.js updated to version 2.0.6

| System requirement updates

  • Apache, at least version 2.4.7
  • PHP - at least 7.3. (PHP 7.4 also supported)
  • Database: MySQL (5.7.8), MariaDB (10.2.7), SQLite (3.26), PostgreSQL (10)

| Modules being removed in D9

  • Block_place

  • Entityreference

  • Field_layout -> Replaced by Layout Builder.

  • SimpleTest module has been moved to contrib

  • Action renamed to action UI

| New features in Drupal 9

Drupal 9 will have the same new features as of Drupal 8.9. Thus, Drupal 9.0 will not include new features. But Drupal 9.1 will continue to receive new features as it did for the minor D8 releases.

| Breaking APIs

Following is a list of major APIs which were deprecated in Drupal 8 and will be removed in Drupal 9.0.0. That means if your code contains any of these codes, they need to be replaced before upgrading to Drupal 9. This list is not exhaustive and has been curated by scanning Drupal core codebase for deprecated warnings and sorted according to the usage of these APIs in contributed projects as given Drupal 9 Deprecation Status by Acquia.

  • drupal_set_message() removed

change record

drupal_set_message(), drupal_get_message() functions removed.

Recommendation: 

Use messenger service 

Example: 

\Drupal::service('messenger')->addMessage('Hello world');
  • Drupal::entityManager() removed

change record

EntityManager has been split into 11 classes.

Recommendation: 

Services like entity_type.manager, entity_type.repository, entity_display.repository etc to be used instead. See change records for more info.

Example: 

\Drupal::entityTypeManager()->getStorage('node')->load(1);
  • db_*() functions removed

change record

All of the db_* procedural functions part of the Database API layer have been deprecated.

Recommendation: 

Obtain the database connection object and execute operations on it. See change records for more info.

Example: 

$injected_database->query($query, $args, $options);

$injected_database->select($table, $alias, $options);
  • drupal_render() removed

change record

drupal_render() and drupal_render_root()  functions

Recommendation: 

Use renderer service

Example: 

\Drupal::service('renderer')->render($elements,$is_recursive_call);
  • Methods for generating URLs and links deprecated

change record

Drupal::l(), Drupal::url(), Drupal::linkinfo() etc and many methods for generating URL & links are removed

Recommendation: 

See example. 

Example: 

EntityInterface::toLink();

EntityInterface::toUrl();
  • File functions removed

change record

file_unmanaged_copy()
file_unmanaged_prepare()
file_unmanaged_move()
file_unmanaged_delete()
file_unmanaged_delete_recursive()
file_unmanaged_save_data()
file_prepare_directory()
file_destination()
file_create_filename()

Recommendation: 

Use file_system service 

Example: 

\Drupal::service('file_system')->copy($source, $destination, $replace);

\Drupal::service('file_system')->move($source, $destination, $replace);

\Drupal::service('file_system')->delete($path);

  • Loading of entities removed

change record

node_load(), entity_load(), file_load() etc removed.

Recommendation: 

Use entity_type.manager service to get specific entity storage and then use the respective load functions

Example: 

use \Drupal\node\Entity\Node;

$node = Node::load(1);

or

$file = \Drupal::entityTypeManager()->getStorage('file')->load(1);
  • Functions to view entities are deprecated

change record

entity_view(), entity_view_multiple(), comment_view() etc removed.

Recommendation: 

Use entity view builder handler

Example: 

$builder = \Drupal::entityTypeManager()->getViewBuilder('node'); 

$build = $builder->view($node, 'teaser');
  • Date formats API changes

Some of the functions and hooks removed (change record)

format_date()

hook_date_formats()

hook_date_formats_alter()

system_get_date_format()

system_get_date_format() etc.

Recommendation: 

Use date.formatter service 

Example: 

\Drupal::service('date.formatter')->format()
  • Unicode::* methods removed

Following functions removed (change record)

Unicode::strlen()

Unicode::strtoupper()

Unicode::strtolower()

Unicode::substr()

Unicode::strpos()

Recommendation: 

Use mb_* functions
 
Example: 

mb_strlen();
  • Functions retrieving extensions info removed

change record

_system_rebuild_module_data(), system_get_info() etc are removed

Recommendation: 

Use extension.list.module, extension.list.profile and extension.list.theme services

Example: 

\Drupal::service('extension.list.module')->getAllInstalledInfo();

\Drupal::service('extension.list.theme')->getAllInstalledInfo();

  • drupal_get_user_timezone() removed

change record

drupal_get_user_timezone() function removed.

Recommendation: 

Replaced with an event listener which updates the default timezone

Example: 

date_default_timezone_get();
  • SafeMarkup methods are removed

change record

SafeMarkup::checkPlain() SafeMarkup::format() etc removed.

Recommendation: 

See change record for replacements 

Example: 

Html::escape();

| More Changes/Updates

  • Drupal core themes no longer extend Classy. Read more

  • Drupal core themes, Bartik, Claro, Seven, and Umami no longer depend on Stable.

  • New Stable theme for D9, recommended new themes to be built on new D9 stable theme. Old D8 stable to be removed from core and be moved to a contributed project before D10.

  • A new administration theme, Claro (targeted for inclusion in Drupal 9.1)

  • Drupal 9 won't be able to run updates from 8.7.x or earlier databases anymore, it is necessary for all new updates added to the code base to be tested from a Drupal 8.8.x starting point. Read more

  • Changes to how HTML Elements are inserted via AJAX commands. Read more

  • ZendFramework/* packages have been updated to their Laminas equivalents. Read more

  • PhantonJS based testing removed. Read more

  • The jQuery UI asset libraries not in use by Drupal core have been marked deprecated and have been removed from core in Drupal 9.

  • Drupal 9 will continue to depend on CKEditor 4 and jQuery 3.4.

  • Modules to be compatible with Drupal 8 and 9 at the same time and to support semantic versioning for contributed projects

  • jquery.cookie has been replaced with js-cookie version 2. 

| Conclusion

At this point, 9.0.0-beta2 is released, which means the code for 9.0.0 is stable and is ready for testing by end-users. Now is a good time to test upgrade your existing D8 sites to the latest version of 9.  If you have contributed a project in drupal.org, it is also a good time to check your extensions for D9 readiness. There are several tools which can speed up this process of making your extensions compatible with D9.

Have questions about how Drupal 9 will impact your site? We are here to help. Check out our Drupal Services or send us an email at [email protected]

| Important References

If you see any discrepancies in the information provided above, please let us know.

Thanks!

Apr 06 2020
Apr 06

If you have a site that has content populated using a file/image type field and want to convert it to use a Media field type you would need to create a new media type field and re-populate all of the image content. This is not practical for a website with lots of content so we came up with a method to use an update script to automatically populate the new media field from the old file/image field type.

Preparation

If you haven't already, the first step is to create your Media entity. Then create the new media reference field on your existing content type.

Convert

Now place the following code (modified for your field and content type values) in a custom module and run Drupal's update script (/update.php or "drush updb")

use Drupal\media\Entity\Media; /**
 * Create media entities from existing file fields.
 */
function MY_MODULE_update_8101() {
 
// Load all of the article and page nodes.
 
$nodes = \Drupal::entityTypeManager()
    ->
getStorage('node')
    ->
loadByProperties(['type' => ['article']]);
  foreach (
$nodes as $node) {
   
// Verify that the file field has a value.
   
if (!empty($node->field_OLD_FIELD_NAME->entity)) { // * use your existing file/image field name here
      // Create the new media entity and assign it to the new field.
      //dpm($node->field_OLD_FIELD_NAME->entity); // * use your existing file/image field name here
     
\Drupal::logger('update_image')->notice(
       
sprintf('preparing image for node "%s".', $node->getTitle())
      );
     
$node->field_NEW_MEDIA_FIELD_NAME->entity = MY_MODULE_create_media_image_entity
       
$node->field_OLD_FIELD_NAME->entity,
       
$node->field_OLD_FIELD_NAME->alt,
      );
     
$node->save();
      \
Drupal::logger('update_image')->notice(
       
sprintf('Updated image for node "%s".', $node->getTitle())
      );
    }
  }
}
/**
 * Creates a media image entity from a file entity.
 *
 * @param $file
 *   The existing file object.
 * @param string $alt
 *   The image alt text.
 *
 * @return \Drupal\media_entity\Entity\Media
 *   The media entity.
 */
function MY_MODULE_create_media_image_entity($file, $alt = NULL) {

 

$media_entity = Media::create([
   
'bundle' => 'teaser_image', // * use your bundle name here
   
'uid' => '1',
   
'name' => $file->alt,
   
'status' => TRUE,
   
'field_media_image_1' => [
     
'target_id' => $file->id(),
     
'alt' => $alt,
    ],
  ]);
 
$media_entity->save();
  return
$media_entity;
}
?>
Apr 03 2020
Apr 03

Previously I wrote about the hidden power that resides in the hands of a designer. Here are 10 questions you can apply to a supplied design, and the answers to them, or even the process of getting those answers, can bring a good design through to being a great design for your project. Remember, a design is just a picture until it is implemented, and it is important that the technical implementation is considered at the design stage.

1) Image Sizes

How many image sizes are there across the site? How many different aspect ratios? Is there any commonality between images? Where can we reuse an image? In Drupal terms, we're asking: Can I use image styles? How many do I need? In which area of the site can I use each image style?

If there seems to be no obvious pattern, it can be worth articulating the benefits of using standardised image styles to the designer, such as performance and improved editor experience.

Remember that with a responsive design, images will usually stretch to fill a space, so image styles will generally set an image to its maximum required width. The single most important thing to consider is aspect ratio. If all the images have the same aspect ratio, then the implementation of a design becomes much, much more straightforward.

2) Content Order

In a responsive site, as screen size diminishes, chunks of content flow around each other and jump below other content to accommodate the smaller screen width. It is important to realise that designs for smaller screens are sometimes supplied without due regard to this flow of content. If you see content order on a mobile design that doesn't match up with that in the desktop design, ask about it. Either it is a mistake that is easy to rectify at the outset of the project, or it will turn into a monster that will eat your budget.

3) Horizontal Alignment

People love horizontal lines. They particularly love columns of content, with line breaks, each of which line up with its neighbour in the next column. Unfortunately, the web, and responsive design in particular does not make such a design easy to implement.

It is bad practice to set absolute heights to elements in responsive design, to allow elements to expand as necessary, but without control of heights, any content change or width change will break the layout. As widths decrease, the height of content in a restricted space increases, which means that either the content will overflow its bounds, the content will be clipped, or the container expands, breaking the nicely arranged horizontal alignment.

There's no easy solution to this, so it is important to bring it to the attention of all concerned early on, lest it become an issue late in the project when deadlines are tight.

Often this issue only comes to light at launch time, when a client is putting real content, and replacing the nice, consistent three lines of dummy text in the design.

Ask how much content should be in the space. Ask whether the lines need to line up. Ask whether you want to rely on brittle Javascript to make heights equal. Ask early.

4) Long Titles

In a typical design, in my experience, titles and teaser text will be short, bordering on terse, lending themselves to smart, ordered layouts, and small containers in which to fit these elements.

When a site is live in the wild, titles can be really, really long. Editors are given content, control over which they may not have. The design has to accommodate this situation. Try swapping out
'Lorem Ipsum Dolor Sit Amet.'
with
'Many editors struggle to fit their content into the cramped confines of a small title block.'

Then see if the design still works. Ask about maximum title length and the ebb and flow of text on the web.

What happens with a long title on a small screen?

5) Large, Full Screen Background Images

Do you really need it? How big is it in kilobytes? Is it worth the extra page weight? Can the weight of it be reduced by blanking out where the content will live? What happens on small screens? Do they need to download it?

Think of performance, think of the value of that background image, and think of mobile users.

6) Fonts

What fonts are in use? Are they special? Is there a cost associated with them? Do they have all the necessary characters in the font? (e.g. I recently had to use a font that had no ampersand character.) Does the client know about any cost? Are they happy with that? Where are webfonts in use? Are they just for headings? Or are they for body text? Do they work on cheap, low-resolution screens? (If they don't, it's probably a poor choice.) Is the 'Flash Of Unstyled Text' before the webfont loads acceptable? How much does the font add to the page weight? Can you use a subset of the font? Do you need bold, italic and other variations?

7) View Modes

How many different ways is the same content (e.g. a node or a bean) viewed on the site? Where on the site are these places? Each different representation directly corresponds to a View mode. If there are many variations, can these be rationalised? View modes are immensely useful, but there comes a point where YAVM (Yet Another View Mode) becomes painful. Less is more.

Another consideration here is seeing if view modes can be shared across content types. For example, is the listing page for news posts the same as the listing page for events? If so, we can use the same view mode for both? This will cut down on the time needed to style these view modes with CSS.

8) Configuration

Does the client need to configure anything? What does the site editor want to be able to change? Should footer blocks be editable? Or any of the site chrome? Should only the main content area be editable? Should the site editor be able to modify listings? Panel Pages? Forms?

The answers to these questions are relevant because they directly affect how you approach a build.

(Aside: read our article The Drupal Content Editor deserves and easy Life.)

9) Browsers

What browsers need to be supported? Can the design even be properly implemented in older browsers? What does 'graceful degredation', or 'progressive enhancement' look like in practice with this design? Which design elements, e.g. funky CSS3 effects or killer Javascript libraries, won't work in poorer browsers? Is there a fallback? Should there be? Ask about analytics. What are the actual site visitors using?

The older the supported browser, the more it will eat into your budget, and the older you go, the more it will eat.

10) Colours

How many shades (e.g., of grey) are there in the design? Can you tell the difference between them? For lighter shades, can you even see them on a poor screen? You can bet that the design was put together on a high quality, high resolution, bright screen. Does it work on a low budget, 10 year old 15" LCD screen? If colours cannot readily be differentiated on such a device, then they may be surplus to requirements.

Further Reading

Josh Riggs' excellent presentation from Drupalcon LA on creating great design without a huge budget.

Would you like to create something beautiful?

Ask us about it.

Mar 13 2020
Mar 13

Read our roadmap to understand how this work falls into priorities set by the Drupal Association with direction and collaboration from the Board and community. You can also review the Drupal project roadmap.

Project News

Drupal 9 beta is closer than ever!

At the time of writing this post, there are fewer than three beta-blockers for Drupal 9. This hopefully means that we'll be seeing a beta release of Drupal 9 very soon. 

What does this mean for you?

Now's the time to get familiar with what's coming in Drupal 9, and to check your contributed or custom modules to see if you're ready to go. The community has put together a number of tools that you can use: the upgrade status module, the Drupal Check command line tool, and Drupal Rector.

We also need your help! We're looking for more individuals and organizations to participate in the Drupal Beta Test program. It's a great way to contribute to Drupal.

Call for Sponsors & Contributors: Automatic Updates

We're really proud of the work we accomplished in the first phase of the automatic updates initiative; in Drupal 7 and Drupal 8, sites that don't depend on Composer workflows now have complete support for securely and automatically updating Drupal Core. In the second phase of this work we want to extend that support to contributed projects, and to support Composer-based site installations. 

We need your help to make the second phase happen. Will you contribute?

Learn more on our call for sponsors & contributors post.

Drupal.org Updates

DrupalCon Minneapolis Program Update

In preparation for releasing the full DrupalCon Minneapolis speaker schedule, we've made some updates to the accepted sessions page. 

The newly redesigned page now highlights our excellent keynote speakers (to include Mitchell Baker from Mozilla!) as well as other featured speakers for this year's event. On top of that you can filter the list of sessions by track, to get a jumpstart on finding your favorite sessions, before the full schedule is released. 

Ready to enable Semantic Versioning for Contributed Projects

We've rearchitected the version management for contributed projects, so that they can begin using Semantic Versioning as we enter the Drupal 9 era. You can see an example of this in practice on this sample project: semver_example. 

We're coordinating with the Drupal core maintainers to select a window for enabling the new semver functionality across all projects. We want to ensure that Drupal end-users will still be able to find and easily understand which projects they can use once projects are able to be compatible with both D8 and D9, and are using semver version numbering. 

Not familiar with semantic versioning

The three digit numbering scheme (MAJOR.MINOR.PATCH) is designed to provide guide rails around API breaking changes. In Drupal core for example, patch releases are incremented whenever there are bug fixes or security releases. Minor releases indicate that new features have been introduced. And the Major version only changes when deprecated APIs are removed and fundamental architectural changes have been introduced.  Contributed project maintainers are encouraged to adopt the same pattern.

Updated display of releases

Speaking of releases - we've recently updated the display of releases to provide a cleaner view of release metadata. This should make it much easier to understand the history of recent releases, and to see at a glance which ones were bug fixes vs. feature releases vs. security releases. 

New Release Meta Data

You can see a detailed example by looking at the release history for Drupal core

Drupal usage stats by branch

Because of the six-month minor release cycle, it's become much more important to have more granular insight into what minor versions of Drupal are in use in the wild. 

Usage stats by branch

As you can see above, we've updated the usage stats for Drupal to display usage by branch. This is mostly useful for Drupal Core, but may be valuable for contrib maintainers as well as they look to understand which versions of their projects are in highest demand. 

Coming soon: An updated UX for project browsing

With the release of Drupal 9, it will be possible for contributed projects to be compatible with both major versions of Drupal. Perhaps more interestingly, because of the release of new features with minor versions, there are some projects that may only be compatible with a certain range of minor versions (e.g: 8.6.x - 9.2.x). 

This is a powerful improvement in ensuring that key modules are ready to use with Drupal 9 on day one, but it also has the potential to be confusing for Drupal site owners and evaluators who are trying to discover what projects they can use. We're looking to update the project browsing on Drupal.org to make sure discoverability doesn't suffer with this change. If you have good ideas about this user experience, please feel free to share them on the issue!

Drupal 9 Readiness

Packaging enhancements

Beginning with Drupal 8.8.0, Drupal needed to be packaged from the output of Composer create project, rather than as the simple contents of a git clone. These changes to packaging have additional ramifications for how we manage tagged releases for Drupal core, and in particular for how we manage security releases. We've been making a variety of updates to the Packaging pipeline since Drupal 8.8 to make the process more transparent, resilient, and performant, and that work continues. 

DrupalCI

DrupalCI: Support for new Postgres environments

Because minimum requirements are changing with Drupal 9, we've added new test environments for both Postgres 10 and Postgres 12

DrupalCI: Updated SQLite version

SQLite has also been updated within the DrupalCI test environment to version 3.26, to support testing on the correctly supported version. 

DrupalCI: Support for MariaDB environments

MariaDB forked from MySQL after the acquisition by Oracle, but at first had remained fairly consistent. However, with recent versions MariaDB has had to diverge, and so we are now providing explicit testing support for MariaDB, with test environments for versions 10.2.7 and 10.3.22. 

———

As always, we’d like to say thanks to all the volunteers who work with us, and to the Drupal Association Supporters, who make it possible for us to work on these projects. In particular, we want to thank: 

If you would like to support our work as an individual or an organization, consider becoming a member of the Drupal Association

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

Mar 13 2020
Mar 13

This month’s SC DUG featured Chris from MindGrub and Kaylan from Pantheon talking about Load Testing.

Launching a website can be a nerve-wracking experience, often times with developers working up until the wire trying to finish that one last feature. If only there was a crystal ball that would show you a vision of how your site would fare when the masses were set loose upon it.

Good news for you, there is! Load testing.

[embedded content]

View the slides from this talk.

We frequently use these presentations to practice new presentations, try out heavily revised versions, and test out new ideas with a friendly audience. If you want to see a polished version checkout our group members’ talks at camps and cons. So if some of the content of these videos seems a bit rough please understand we are all learning all the time and we are open to constructive feedback.

If you would like to join us please check out our up coming events on MeetUp for meeting times, locations, and remote connection information.

Mar 12 2020
Mar 12

Category 1: Web development

Category 2: Drupal

Category 3: Data architecture and engineering

Category 4: Data analytics

Category 5: Design, research, and content strategy

Category 6: Operations

Mar 09 2020
Mar 09

Since tickets are no longer required, sign up for our mailing list to get all the latest updates. Links to join sessions will be on the session pages on the day of the event.

Due to the worldwide Coronavirus/COVID-19 response and in consideration of the health of our community, we have reconsidered the implications of moving forward with business as usual for MidCamp. While there are currently no travel or gathering warnings for Chicago, we need to consider that historically, >40% of our attendees travel to the event. Additionally, current advice and historical evidence suggest that proactive steps are essential in containing the network effects of the virus.

Fortunately, we have the tools and ability to share our information in a remote manner, and as such we’re moving MidCamp to a 100% remote format this year. There’re many details to work out, but our aim is to deliver the same quality content on the same schedule as we would have done in person.

What does this mean for attendees?

  • In-person social events will be cancelled and replaced with virtual events where possible

  • All Wednesday trainings will be cancelled.

  • Thursday & Friday sessions will be presented and attended via videoconference (details to come). We will have no in-person events at DePaul.

  • All tickets will be refunded.

    • We are still accepting individual sponsorship donations. Donations will go toward deferring cancellation costs, toward costs of the virtual event, and towards our 2021 event.

  • If you’ve booked accommodations with the Midcamp 2020 group at Hotel Versey, call (773) 525-7010 to cancel. 

Our goal is to maintain the high-quality content you've come to expect from MidCamp without adding any risk to the community we care so deeply about. We appreciate your patience and flexibility with this change of format. If you have any questions at all, feel free to contact us at [email protected], or hop into our Slack https://mid.camp/slack.

Thanks again for your support!

Mar 02 2020
Mar 02

As of Drupal 8.7, the Media and Media Library modules can be enabled and used out-of-box. Below, you'll find a quick tutorial on enabling and using these features.

out-of-box before media and media library

In the past there were two different ways to add an image to a page.

  1. An image could be added via a field, with the developer given control over its size and placement:
     

    Image field before media library
  2. An image could be added via the WYSIWYG editor, with the editor given some control over its size and placement:
     

    Image field upload choices screen

A very straightforward process, but these images could not be reused, as they were not part of a reusable media library.

reusing uploaded media Before Drupal 8.7

Overcoming image placement limitations in prior versions of Drupal required the use of several modules, a lot of configuration, and time. Sites could be set up to reference a media library that allowed editors to select and reuse images that had previously been uploaded, which we explained here.

This was a great time to be alive.

What is available with Media Library

Enabling the Media and Media Library modules extends a site's image functionality. First, ensure that the Media and Media Library core modules are enabled. 

Enable media library in drupal

A media entity reference field must be used with the Media Library. It will not work with a regular image field out-of-box.

Image field on manage display page

On the Manage form display page, select "Media library" widget. 

Media library widget on manage display page

On the "Node Add" and "Node Edit" forms, you’ll see the below difference between a regular image field and a field connected to the media library.

Media library field on node edit

Click on “Add media” and you’ll see a popup with the ability to add a new image to the library or to select an image that is already in the library.

Media field grid

With a simple configuration of the field, if multiple media types are allowed in the field, you’ll see vertical tabs for each media type.

Media grid with multiple media types

WYSIWYG configuration

The WYSIWYG editor requires a few steps when configuring the media library for a specific text format. First, a new icon will appear with a musical note overlapping the image icon. This should be added to the active toolbar and the regular image icon should be moved to the available buttons.

wysiwyg toolbar configuration

Under “Enabled filters,” enable “Embed media."  Under the filter settings, vertical tab settings can be chosen for media types and view modes. Once that configuration is saved, you’ll see on a WYSIWYG editor that you have the same popup dialog for adding a new image to the media library, or selecting an already-uploaded image.

wysiwyg media configuration

Once you are on a "Node Add or "Node Edit" page with a WYSIWYG element, you’ll see the media button (image icon plus musical note).

Media button on wysiwyg editor

Clicking on the media button brings up the same, familiar popup that we saw earlier from the image field:

media library grid

This article is an update to a previous explainer from last year. 

Feb 23 2020
Feb 23

Every site has a lot of weekly / daily attempts to abuse your forms to submit their bad content, create users, login... Here we will try to ban some of those IPs directly in the server firewall, so they can't even reach Drupal.

At first, this method might not be applicable for you. If you have a professional hosting / PaaS, your provider could (should ?) already take care of this. At least globally, maybe not specifically for Drupal attacks. For this method, you must have be able to log on your server with a user that has admin rights. This post is especially written for Debian Buster, but it should be more or less the same for other Debian versions. For other distributions, you might have to tweak some things.

Then, as you will see in the conclusion, I tend to think that this method is not sustainable in the long term. This really has to be taken as an experiment.

I also want to mention that this blog post is heavily inspired by Integrate Honeypot with Fail2ban. Basically it was my starting point and I'm mostly complementing it here.

Step 1: Drupal configuration

Let's start on the Drupal side of things. You will need to enable the Syslog module (Core). Like the name suggests, this module will write the Drupal system logs into the linux syslog. In Debian, this log is /var/log/syslog. Now, among other logs, each time a login attempt fails, it will be recorded in the syslog.

Then, if you want you can also protect the login form with Honeypot as well. You can also protect other forms such as the user registration form and the global contact form. The important part is to be sure to have the option Log blocked form submissions checked on Honeypot configuration.

Step 2: Fail2Ban configuration on your server

Fail2Ban is available in the Debian repositories:

sudo apt install fail2ban

It's advised to never edit the Debian configuration files, so we will create only .local files. Let's start with the global jails configuration:

sudo nano /etc/fail2ban/jail.d/global.local

[DEFAULT]
ignoreip = 127.0.0.1 1.2.3.4
findtime = 3600
bantime = 31536000
maxretry = 2

Replace 1.2.3.4 by your own IP(s) before anything else! Otherwise you could loose access to your remote server.

Then you can set global variables that will override the too low default values.

  • findtime (in seconds) will parse the logs for the last specified seconds. Do not enter a huge value here because this could impact the server performance.
  • bantime (in seconds) is the time for which the IPs will be banned. Don't hesitate here to have a big number. Here it's 1 year.
  • maxretry is the number of times an attempt can be retried.

Now let's define a configuration for Drupal jails:

sudo nano /etc/fail2ban/jail.d/drupal.local

#[drupal-auth]
#enabled = true
#port = http,https
#filter = drupal-auth
#logpath = /var/log/syslog

[drupal-auth-custom]
enabled = true
port = http,https
filter = drupal-auth-custom
logpath = /var/log/syslog

[drupal-honeypot-auth]
enabled = true
port = http,https
filter = drupal-honeypot-auth
logpath = /var/log/syslog

[drupal-honeypot-contact]
enabled = true
port = http,https
filter = drupal-honeypot-contact
logpath = /var/log/syslog

The first jail and filter, drupal-auth, already exists by default in Debian when installing Fail2Ban. However, I found that the regex was not working for me everytime. I still enabled it in the beginning. I added a custom version that works better, drupal-auth-custom, and I deactivated the first one. Finally I added 2 jails and filters with Honeypot.

Finally, we have to create the filters that we declared in those jails. The first filter also exists after Fail2Ban installation, so let's create the 3 others:

sudo nano /etc/fail2ban/filter.d/drupal-auth-custom.local

# Fail2Ban configuration file for Drupal failed login attempts
# Author: Tipi Koivisto
# taken from fail2ban drupal module https://www.drupal.org/project/fail2ban
[Definition]
# Option: failregex Notes.: regex to match the password failure messages in the
# logfile. The host must be matched by a group named "host". The tag ""
# can be used for standard IP/hostname matching and is only an alias for
# (?:::f{4,6}:)?(?P[\w\-.^_]+) Values: TEXT
failregex = \|\d*\|user\|<HOST>\|.*Login attempt failed for \s?
# Option: ignoreregex Notes.: regex to ignore. If this regex matches, the line is
# ignored. Values: TEXT
#ignoreregex =

sudo nano /etc/fail2ban/filter.d/drupal-honeypot-auth.local

# Fail2Ban configuration file for Drupal Honeypot blocked form submissions
# copied from Author: Tipi Koivisto
[Definition]
# Option: failregex Notes.: regex to match the password failure messages in the
# logfile. The host must be matched by a group named "host". The tag ""
# can be used for standard IP/hostname matching and is only an alias for
# (?:::f{4,6}:)?(?P[\w\-.^_]+) Values: TEXT
failregex = \|\d*\|honeypot\|<HOST>\|.*\|Soumission bloquée pour user_.*
# Option: ignoreregex Notes.: regex to ignore. If this regex matches, the line is
# ignored. Values: TEXT
#ignoreregex =

sudo nano /etc/fail2ban/filter.d/drupal-honeypot-contact.local

# Fail2Ban configuration file for Drupal Honeypot blocked form submissions
# copied from Author: Tipi Koivisto
[Definition]
# Option: failregex Notes.: regex to match the password failure messages in the
# logfile. The host must be matched by a group named "host". The tag ""
# can be used for standard IP/hostname matching and is only an alias for
# (?:::f{4,6}:)?(?P[\w\-.^_]+) Values: TEXT
failregex = \|\d*\|honeypot\|<HOST>\|.*\|Blocked submission of contact_message_contact_form.*
# Option: ignoreregex Notes.: regex to ignore. If this regex matches, the line is
# ignored. Values: TEXT
#ignoreregex =

Notice that in /etc/fail2ban/filter.d/drupal-honeypot-auth.local, I had to use the french translation of the message because that's how Honeypot logs it for me. Also, you might have another form ID in /etc/fail2ban/filter.d/drupal-honeypot-contact.local

At last, restart the Fail2Ban service:

sudo systemctl restart fail2ban

Testing it

Careful if you try to pretend to be a hacker to see if your configuration works, you could be banned from your own server! I would just recommend to wait a little bit for real cases.

See if the jails are all active (you should have 5 with the default sshd jail):

sudo fail2ban-client status

See the status of a jail and if it has banned IPs:

sudo fail2ban-client status drupal-auth-custom

Test a jail filter:

sudo fail2ban-regex /var/log/syslog /etc/fail2ban/filter.d/drupal-auth-custom.local

One last word: when you test fail2ban-regex, if you find matches, those will not be banned. Same thing when you restart the server: already processed logs will not be taken into account. One would have to reset Fail2Ban's database or do other hacky things but it's not really worth it. After all, you've lived with those unbanned IPs for a while now, right ?

Results after 48 hours

After 48 hours I have:

  • 3 IPs banned by drupal-auth
  • 9 IPs banned by drupal-auth-custom which include the 3 above. That's why I deactivated drupal-auth and use only the custom version.
  • 0 IPs banned by drupal-honeypot-auth (I will have to check that it works).
  • 10 IPs banned by drupal-honeypot-contact

So a total of 19 IPs and they are all different.

Conclusion

For now I'm happy with this method but it should not be sustainable in the long term. With already 19 different IPs in 48 hours, if it goes on at the same rate for 1 year I will have around 3500 Iptables rules. This without counting the bans with the standard sshd jail. This should slow down too much my server.

Most importantly, I'm only banning on the basis of 3 Drupal specific rules, which miss the huge part of other attack attempts.

I'll keep the experiment going on for some time, just to see. In the end, I think that it might not be worth the effort. Configuring a more robust system would probably imply to use IPset which can handle large amounts of IPs. Also, it might be interesting to import or sync IP blacklists collected by a service and to contribute to this service data with your Fail2Ban. At this point, work is starting to add up and either you are really passionate about this, or you just take a very good hosting provider.

Update 2020-08-25: now 258 banned IPs for drupal-honeypot-contact, 199 for drupal-auth-custom and the rest is negligible. I've set up 1 year bans. Seems pretty much sustainable form my website, finally.

Feb 19 2020
Feb 19

Our normally scheduled call to chat about all things Drupal and nonprofits will happen TOMORROW, Thursday, February 20, at 1pm ET / 10am PT. (Convert to your local time zone.)

This month, in addition to our usual free-for-all, we'll be talking about hosting on Pantheon. There has been a lot of discussion in the community and on the Drupal Slack #nonprofits channel about some of the pricing changes they have implemented. If you would like to discuss and contribute to the conversation, please join us.

We will also have an update on our community's plans for the upcoming Nonprofit Technology Conference (20NTC).

All nonprofit Drupal devs and users, regardless of experience level, are always welcome on this call.

Feel free to share your thoughts and discussion points ahead of time in our collaborative Google doc: https://nten.org/drupal/notes

This free call is sponsored by NTEN.org but open to everyone.

REMINDER: New call-in information -- we're on Zoom now!

  • Join the call: https://zoom.us/j/308614035
    • Meeting ID: 308 614 035
    • One tap mobile
      • +16699006833,,308614035# US (San Jose)
      • +16465588656,,308614035# US (New York)
    • Dial by your location
      • +1 669 900 6833 US (San Jose)
      • +1 646 558 8656 US (New York)
  • Follow along on Google Docs: https://nten.org/drupal/notes
  • Follow along on Twitter: #npdrupal

View notes of previous months' calls.

Feb 19 2020
Feb 19

One of Drupal’s defining features is its open source nature, and the fact that it invites contributions from its vast community. Having been in the Drupal world for a few years now, I found myself wanting to make my own contributions, but unclear on how best to do so. 

The Drupal docs are sparse in some places, overly verbose in others, and they don’t take into account some of the specific tools we at Phase2 rely on, like Docker and Xdebug.

This guide is intended to help remove all of the barriers for getting patches up to Drupal.org.

Find an issue

If you’re following this guide with an issue already in mind, feel free to skip this section. 

Otherwise, here are some steps to help you find an issue to solve: 

  • Project Work: Any bug, UI problem, error message, or other Drupal issue you run into while working on a project is a good opportunity to contribute
  • Needs Testing: If you know your way around automated testing, then consider searching for the tag ‘needs tests’ in the issue queue. There are often dozens of patches languishing in limbo and waiting for a couple of tests so that they can be finally committed
  • Composer Patches: In all likelihood your project has patches listed in its composer.json that either you, or one of your teammates decided was necessary. This is a perfect place to contribute because you’ll already have some context about what the issue is, which makes it easier to jump in and add whatever is necessary to finally get it committed (which could be as simple as a small test or some documentation)
  • Comment history: Check the post history on your dashboard for issues that you’ve already engaged with. Maybe they could still use some help.
  • Novice Tag: Some issues have been marked with the “novice” tag to indicate that they’re particularly good for people just getting started in the contrib space

Once you find an issue, consider following it by clicking the green star icon. That way you’ll receive notifications on comments. 

Regardless of the approach you use to find an issue, keep in mind that it being either Core or Contrib will have a significant impact on your work. Core patches have a lot more visibility and the potential to help a lot more people, but since they’re core they’re going to have a lot more barriers to entry and will almost certainly require automated testing.

Contrib patches will likely have a smaller impact, but are usually easier to get in because they only need approval from the module’s developers. Once you’ve settled on an issue, you’re ready for the next step. 

Dreditor

Before moving any further I would highly recommend you install Dreditor in your browser. This tool adds a number of tools to Drupal.org, the most important being the “review” button that it will add to patch links.

This button displays patch files in an easy-to-read format that makes full use of javascript and color-coding.

Image

Screenshot of Dreditor
On the left is the default patch display, the right is the same patch appearing in Dreditor. Not only does this make patches from other contributors easier to understand, it also gives a nice UI for commenting on individual lines and code review.
 

Set up your environment

You should only have to do the set up steps once. If you already have your environment going  feel free to skip this.

Docker

Like other projects, we should have a virtual environment to work in. At Phase2 we usually rely on Docker/Docksal for that task. Fortunately this repo makes it easy to set up a D8 environment using Docksal.

  • Clone the repo: 
git clone https://github.com/docksal/drupal8-contrib.git
  • Get to the root:
cd drupal8-contrib
  • Initialize the site:
fin init

Xdebug


Xdebug is a crucial tool for debugging PHP code, but it can be a challenge to set up. Use the following steps to get it running on your Drupal8 local site.

  1. Go into Preferences > Languages and Frameworks > PHP > Debug and make sure your settings match the following screenshot.

Image

Xdebug Settings Screenshot

2. Preferences > Languages and Frameworks > PHP > Servers and make sure your settings match the following screenshot.

Image

Server Settings

3. Go to your docksal.env file and change the value of XDEBUG_ENABLED to 1.

Image

Xdebug Enabled Setting

4. Restart your containers with fin restart to make sure your change to the environment takes effect.

5. If Xdebug still isn’t working, try the following steps:

  • Make sure Xdebug is not muted
  • Make sure your listener (the phone icon in PHPStorm) is enabled
  • Put a breakpoint in index.php, and if Xdebug fires successfully then your code isn’t doing what you think it is
  • Check the docs https://docs.docksal.io/tools/xdebug/

Be aware that Xdebug, while helpful, can also slow down your system a lot, so you don’t always want to have it enabled. Here are two ways to address that issue:

  • Make a Bash alias to toggle xdebug and restart containers fin config set XDEBUG_ENABLED=1 && fin restart
  • Use the existing fin php/xdebug on and fin php/xdebug off commands to toggle xdebug. This shortcut runs faster and doesn’t require restarting containers, but the site will still have some slowdown compared to the previous method.

Git setup

Follow these steps to get Git set up.

  • Are you patching Core or a module?
    • Core
      • cd docroot
      • git checkout -b 12345-short-description\
    • Module
      • cd modules
      • git clone [email protected]:project/machine_name
      • cd module_name`
      • git checkout -b 12345-short-description
  • 12345 is the issue number, and short-description is just something to remind you of the issue.
  • Are you working on top of an existing patch?
    • Make sure you’re in the project’s root directory
    • Download the patch file curl -O https://www.drupal.org/files/[patch-name].patch
    • Apply the patch git apply [patch-name].patch
    • Confirm the changes by running git diff
       

Code your fix

Now you’re finally able to write the code that will address your issue, and there are a few things to keep in mind while doing so.

Testing

First, make sure whatever you add is reinforced by automated tests, either by updating existing tests or making your own. You can run tests using the following commands:

Test an entire file: fin phpunit path/to/file
Run an individual tfin init est: fin phpunit path/to/file --filter=testFunctionName

You also probably want to turn off Xdebug before running tests as it will make them take much longer. 

Functional tests are particularly guilty of this problem and as such can be hard to create. However, if you pass the --printer="\Drupal\Tests\Listeners\HtmlOutputPrinter" tag into your test, your terminal will print out something like the following.

Image

HtmlOutputPrinter Result Screenshot

Each url represents a different step in running the functional test, such as logging in or creating a user. Assuming this test was using the aforementioned docksal setup, you could see the generated html by going to: fin http://drupal8-contrib.docksal/sites/simpletest/browser_output/Drupal_Tests_user_Functional_UserCancelTest-313-38450349.html

Writing good tests will likely save you time in the long run, not only by catching regressions, but also because tested code is far more likely to be merged in.

Commit Strategy

Feel free to follow whatever commit strategy makes sense to you, since the patch file you’ll be creating won’t have an associated git history. That being said, it’s recommended that if you’re working on top of multiple patches that you make a commit between each patch to make jumping between them easier.

Finalizing

Once you’re confident about your work, make sure to add comments where appropriate and run php codesniffer to ensure that your code is up to Drupal 8 standards:
fin composer phpcs path/to/file
 E.g. fin composer phpcs core/modules/field/tests/src/Kernel/FieldStorageCrudTest.php

Once your code is complete, it's time to generate a patch file. Depending on how long you’ve been working on the patch, you’ll probably want to pull down the most recent version of the Drupal master branch and rebase your new branch off of it to make sure that your patch file only contains your changes. The following command will generate your patch file.
git diff master_branchname > [patch_name.patch]
E.g git diff 8.8.x> my_patch.patch

Finally, if you’re adding to an existing patch its best practice to generate an interdiff file to post with your patch. An interdiff is a text file in patch format that makes it easier for reviewers to understand the changes between patches. 
git diff HEAD~1 > ISSUE_ID-COMMENT_ID-interdiff.txt

Shortcuts

It should be clear by now that this process requires a lot of detailed and error-prone steps, fortunately there are a number of shortcuts and tools that can make this process easier.

Gitconfig

You can put a number of incredibly useful shortcuts into your local gitconfig file.  For example, the following lets you download and apply a patch very easily:

a = apply --index
ac = "!f() { curl [email protected] | git a; }; f"


E.G. git ac https://www.drupal.org/files/issues/2019-04-23/core-field-storage-error-message-3050264-1.patch

This code lets you easily download contrib modules from Drupal.org:

[url "[email protected]:project/"]
 insteadOf = dm:


E.G. (from the modules directory) git clone dm:token

With a little Bash knowledge you can put all kinds of useful shortcuts in the gitconfig file.

Drupal CLI

I also highly recommend installing the Drupal CLI tool which you can find here. There are a host of useful things in this CLI, and here are some of the highlights.

  • List test results for an issue drupalorg drupalci:list [issue_number] 
  • Apply the latest patch from the issue drupalorg issue:apply [issue_number]
  • Generate a patch for the issue from committed local changes drupalorg issue:patch [issue_number]
  • Create a branch for the issue drupalorg issue:branch [issue_number]

Dorgflow also has some helpful command line tools, for instance after all your work is finished you can type dorgflow and it will create a patch and an interdiff file for you.

Make sure you fully understand the process that is being run before using these shortcuts. While they’re helpful, they can break, and you do not want to be fully dependent on them.

Post your patch

Once you’re done with your patch and interdiff, post it to the issue with a comment about what you changed and set its status to ‘needs review,’ this will run all tests and tell other developers that your work is ready for feedback. If your patch isn’t getting a lot of attention, feel free to reach out to people on the official Drupal Slack Channel and ask for reviews. Also, keep in mind that while writing your own patches is great, contribution also involves commenting on and reviewing other people’s work. In general, the more you review, the more likely you are to get your work reviewed.
 

Feb 15 2020
Feb 15

This month for SC DUG I gave a talk on the importance of self-directed learning for professional development as a developer — or really any other modern career. It was an extension and revision of my December blog post on the same topic. The presentation runs a hair over 30 minutes, and parts of the discussion are included as well.

[embedded content]

We frequently use these presentations to practice new presentations, try out heavily revised versions, and test out new ideas with a friendly audience. If you want to see a polished version checkout our group members’ talks at camps and cons. So if some of the content of these videos seems a bit rough please understand we are all learning all the time and we are open to constructive feedback.

If you would like to join us please check out our up coming events on MeetUp for meeting times, locations, and remote connection information.

Jan 30 2020
Jan 30

It all started as a simple question to our Drupal developers:

"If you have a Published node, then create a new Draft, what version do you get from Node::load?"

There were two answers:

  1. "Node::load will always give you the Published revision of the node."
  2. "Node::load will give you the latest revision of the node."

Take a moment to think about this and select an answer before you continue reading.

Plain Drupal 8 without any Content Moderation

When you install a fresh Drupal 8 site, content moderation is disabled. In this situation, life is very simple:

2. Node::load will give you the latest revision of the node

Regardless of whether it is published or not. Going to the node/view page will show the latest revision (and will have a pink background in the standard Bartik theme if it isn't published).  The node/edit page will show the latest revision.

Basically, in this scenario, it doesn't matter whether the node is published or not. The latest revision is always the "default" revision.

Adding Content Moderation

Content Moderation was added to Drupal core in version 8.5.0. When you enable the content_moderation, a basic "Draft, Published, Archived" workflow is created for you.

Using the default workflow, enable it for one of your content types (such as Articles).  Then create an Article and set the workflow state to Published. Then edit the same article and set the workflow state back to Draft.  In this case, the answer is:

1. Node::load will give you the Published revision of the node (not the latest draft).

Now Publish the article again, and then edit the article and set the workflow state to Archived.  Now:

2. Node::load will give you the latest revision of the node (the Archived version, which is not published).

So how does this all work? Why does it seem, at first glance, to be inconsistent?

Understanding the Default workflow state

If you look at the default workflow configuration and edit a State, you will see a checkbox called "Default revision".  In the Drupal node_revision database table, there is a column for each revision of a node called revision_default.

When you save a new revision, Drupal will set the database revision_default column based on the "Default revision" checkbox for the state that you selected.

In the case of the default workflow installed by content_moderation, the Published state has "Default revision" checked (because all Published states must also be set to default). But the Archived state also has "Default revision" checked! Only the Draft state has the "Default revision" unchecked.

The Node::load function, and the node/view page will always show the latest "default" revision of a node.  The node/edit page will always show the latest revision (highest revision id) regardless of the "default" value.  When the highest revision id does not have the revision_default value set to True, Drupal will add a "Latest  Version" tab next to View and Edit that allows you to view the latest revision rather than the current default/published version.

This behavior holds true even at the database layer. The node and node_field_data tables always contain the data for the default revision, not necessarily the published revision.

In code, you can get the latest (highest) revision id for content entities via:

$vid = \Drupal::entityTypeManager()
  ->getStorage('node')
  ->getLatestRevisionId($nid);

and then load that revision via:

$node = \Drupal::entityTypeManager()
  ->getStorage('node')
  ->loadRevision($vid);

Custom Workflow States

When creating your own Workflow and States, be sure to understand how this "Default Revision" option works in Drupal 8. The "Published" checkbox for the State is controlling the status property of the node, and the "Default Revision" checkbox is controlling the revision_default property. Any number of states can be set to be Published or Default.

If you are creating an API or other backend code that needs to load either the published revision of a node, or the latest revision (to be edited), be sure you are not just calling Node::load without understanding the default revision. When an API makes changes to content, it should always be loading the latest revision, not just using Node::load.

For example, if you add a state called "Revised" to indicate a node that was previously published and now has a forward "draft" revision, you don't want this marked as a "Default Revision" otherwise users might see unpublished content (or get an "access denied" error if they don't have permission for unpublished content).

Summary

In the future it might be nice to have a direct getStorage()->loadLatestRevision() API in Drupal code to make this a bit easier. The storage query has a latestRevision tag method that can be added (for example, creating a loadProperties helper that returns the latest revision of the matching query). But for now, the "default" terminology remains somewhat confusing. As more sites start using the core Content Moderation function I expect to see this continue to improve in core.

Jan 07 2020
Jan 07

Happy New Year!!! Our normally scheduled call to chat about all things Drupal and nonprofits will happen Thursday, January 16, at 1pm ET / 10am PT. (Convert to your local time zone.)

This month, in addition to our usual free-for-all, we'll be talking about Drupal and CiviCRM.  Have you got it up and working your Drupal 8 site? For those of us still working in Drupal 7, what can or should we doing to prepare for the inevitable upgrade? What are your favorite resources for working with these two systems?  Come share your experiences!

Feel free to share your thoughts and discussion points ahead of time in our collaborative Google doc: https://nten.org/drupal/notes

We have an hour to chat so bring your best Drupal topics and let's do this thing!

This free call is sponsored by NTEN.org but open to everyone.

REMINDER: New call-in information -- we're on Zoom now!

  • Join the call: https://zoom.us/j/308614035
    • Meeting ID: 308 614 035
    • One tap mobile
      • +16699006833,,308614035# US (San Jose)
      • +16465588656,,308614035# US (New York)
    • Dial by your location
      • +1 669 900 6833 US (San Jose)
      • +1 646 558 8656 US (New York)
  • Follow along on Google Docs: https://nten.org/drupal/notes
  • Follow along on Twitter: #npdrupal

View notes of previous months' calls.

Jan 03 2020
Jan 03

Finally, we are one more step closer to Drupal 9. In order to make your Drupal 8 website ready for Drupal 9, you will need to update your existing website to Drupal 8.8, the latest stable release in 8.8.x branch. This branch removes all the incompatible APIs and prepares your website for Drupal 9. And which is why, It is highly recommended that you update to 8.8.

Before you start with the update, make sure you take backup of your website (code base & DB) or preferably, carry out this process on a staging site or on your local Drupal 8 set up. For this, it is assumed that you have composer installed for your Drupal 8 environment.

Modify composer.json

The first step is to modify composer.json manually.

1. Unset / empty “replace”: {},

2. Remove "merge-plugin" entirely.

3. Append “App\\”: “app/”, “”: ”src/” to “autoload”

Once you are done with modifying the composer.json, it is time to execute a few commands with composer on CLI.

Execute composer commands

composer remove webflo/drupal-core-strict --no-update

composer remove drupal/core --no-update

composer require 'composer/installers:^1.7' --no-update

rm composer.lock

rm -rf core

rm -rf vendor

composer require drupal/core-recommended:^8.8 --update-with-dependencies

Now, sit back and relax while composer performs the update. Once the updates are downloaded, it is time to perform the db updates using drush updb. Congratulations! You have successfully updated your existing Drupal 8 website to 8.8 and you are now ready for Drupal 9!

Jan 01 2020
Jan 01

The Problem

Replacing files uploaded to your Drupal site can be very frustrating. In most cases, when an editor wants to replace a document, they want to keep the exact same filename and filepath and just overwrite the contents of the file. This is important in cases where the file is linked to elsewhere throughout the website or on other websites outside of the editors control. If you use the media module to manage documents on your site, you'll quickly discover that it's not possible to upload a replacement file for a document and keep the same filename.

Why is this? The core file field allows you to remove and upload a replacement file. But when you upload a replacement file with the same filename, Drupal appends a number to the end of it, like _0, _1, etc. This is because the old file is not actually removed from the filesystem immediately. Drupal will mark it as a temporary file and it will be removed during a cron run sometime in the future (* see exception to this below). Drupal won't overwrite that file with the new one, so it appends a number to the end to make it unique. Drupal also does this to support revisions properly. If the file field is attached to an entity that has revisions enabled, then Drupal keeps that old file around because previous revisions may require it.

* Actually, as of Drupal 8.4, old unused files are never deleted unless you have a specific config override set that enables this cleanup.

Existing Contrib Solutions

There is a 4 year old Drupal core issue where the community has tried to resolve this problem. Probably the most popular solution has been to use the Media Entity Download module. It works by creating a special route for each media entity that will directly return the file associated with that entity as a download. So if you had a document media entity with ID 59, editors can create links to /media/59/download and Drupal will serve the file directly from that path. This solves the problem because it hides the actual file system path from visitors. It doesn't matter what that path is or how many numbers Drupal has appended to the end of replacement files, the visitor never sees it and you'll never have broken links.

The main problem I have with Media Entity Download is that it gets Drupal involved in serving up the files which are normally not handled by Drupal at all. After all, the files are just sitting there in the public filesystem. The web server (apache, nginx) typically serves them to visitors and leaves Drupal out of it. With this module, every time a file is requested, Drupal is bootstrapped and a PHP process is tied up until the download completes to the visitor. This is a bit scary if you get a large surge of traffic with many users requesting large downloads.

The New Solution

I just completed work on the Media Entity File Replace module to provide another solution to this problem. This module is intended for site builders that are using the Media module to manage documents on their site. It works by adding a replacement file widget to the media edit form for media types that use a file source (Image, Document, etc). You can control which specific media types it's enabled for by visiting the form display configuration and showing/hiding it. When replacement files are selected and the form is saved, the file is first uploaded to temporary storage and then copied over the existing file. The filepath and filename are kept in tact. Just the contents of the file and the metadata describing the file are updated - which is what editors want most of the time.

Edit form for a document media entity showing the "replace file" form widget

Note that the module still supports the previous behavior of NOT overwriting the original file. Editors just need to uncheck the "Keep original filename" checkbox and the replacement file will be uploaded and the media entity updated to reference it.

This module should fill a functionality gap in document management in Drupal - I hope others find it useful!

Dec 23 2019
Dec 23

From time to time conversations come up among developers, and other fellow travelers, about being self-taught vs getting formal training. Over time I’ve come to realize that the further and further you get into your career, the less the distinction means anything; eventually we are all mostly self-taught.

I’ve written before about the value of my liberal arts education and I stand by my assertion that what I learned in that setting was, and is, valuable to my life and work. But just because something was useful to life does not mean it was the only way to acquire the skills. It’s a good way for many people, but far from the only way.

For anyone in a technical field, and most professional fields really, to succeed over time you need to learn new tools, skills, and techniques. The tools I knew when I graduated college are all largely outmoded or significantly upgraded, and I’ve had to learn a variety of technologies that didn’t exist in 2001.

Within the Drupal community lots of people talk about being self-taught, sometimes with pride sometimes with embarrassment, but in truth very few people were formally trained on the platform. Lots of very successful developers in the Drupal community (and beyond) have degrees in fields like religion and art history, not computer science, and have taught themselves how to do awesome things. In fact, I’ll argue that just about every Drupaler taught themselves most of what they know about Drupal. How they did that can vary widely, but we are a community with few formal training programs and lots of people who stumbled into Drupal trying to solve a non-technical problem. Even advanced workshops at conferences dig deep into one small area and expect you to generalize that knowledge to your projects, which I count as self-teaching. For example, I had a friend ask the other day about how to control the PDO connection settings in Drupal 7 — which I didn’t know how to do, but knew they were similar to Drupal 8 — so I sent him my Drupal 8 instructions and he figured it out how from there. He’s now taught himself how to do what he needed for that project and in the process generalized the approach for whatever he may need next time.

So then it is important for all of us to find, and hopefully share, techniques for self-teaching — even for those who have some kind of formal training. Here are my suggestions for people who are starting out and haven’t yet found the pattern that works for them:

  1. Assume you first solution is wrong. Most of us have, or will, stumble our way through a project where we don’t really know what we’re doing without a lot of support. We usually learn a great deal in the process, and launching those projects can feel pretty good cause you’ve succeeded at something hard. It is easy to get into the habit of assuming the solutions from that project were correct because they worked. In truth those projects are really rough around the edges, and just because we got it to work does not mean the solution was good. Assuming the first solution is good enough forever is how you become an expert beginner which then takes a lot of effort to undo. Once you have a working solution, step back and see if you can think of a better one, or see if you now can guess better search terms to see if someone else wrote up a different solution to the same problem. Admit your work could be better and try to improve it.
  2. Learn a few more programming languages. Most people who are self-taught from the start, and even some who have a BA/BS in Computer Science, only know 2 or 3 programming languages (PHP, JS, and CSS+HTML are often the only languages new people learn at first). One of the courses I took by chance in college forced me to learn 8 in 16 weeks. It was grueling, miserable, and darned useful. I can still learn a new language in just a couple weeks and rarely do I hit a language construct I don’t recognize. You don’t need to go that far. When I first started out a mentor told me you should learn a new language every year, and for several I did. Some of those, not the languages I learned in college, are the ones I use most day-to-day. All told I’ve spent time writing code in more than twenty different languages. That many isn’t terribly useful but the more languages you learn, the more you learn to understand the elements of your primary language.
  3. Learn basic algorithms and to measure complexity. The kind of thinking that goes into formal algorithms will help you be a better developer overall; badly thought through processes is the place I tend to see the largest gaps between developers with and without formal training. Any college-level CS program will put you through an algorithms course that teaches a variety of specific algorithms and force you to understand their structures. If you didn’t go through one of those programs, this is probably the course that will help you the most. On the one hand most of us rarely rewrite these algorithms as on modern platforms some library or another will provide a better version than we are likely to craft for our project. But learning what they are, when they are used, and how to understand their performance is useful for any project that involves lots of data or processing. MIT has a version of their algorithms course from 2011 online, or find one through another provider. Even if you just watch the lectures (really watching, not just vaguely have them on while cooking and cleaning), you can learn a great deal of useful information. I learned a lot watching those lectures as it refreshed and updated my understanding of the topics.
  4. Find and learn from mentors. Notice I used a plural there; you should try to find a few people willing to help you learn your profession, and more generally help you learn to advance in your field. Most of us benefit from learning from the experiences of multiple people, and who we need to learn from changes over time. I had the great experience of having a few wonderful mentors when I was first starting out, and much of the advice they gave me still serves me well. Some of it contradicted, and resolving those contradictions forced me to learn to do things my own way and find my own solutions.
  5. Learn other platforms. This is both a protection against future shifts in the market, and also a way to see how things work from outside your current professional bubble. Drupal developers can learn a lot from writing a WordPress plugin, or better yet an add-on for a platform in another language (think about Plone, Gatsby, or Hugo). Or try to learn to work with a platform like Salesforce or AWS. Other platforms have different communities, different learning styles, and different patterns. Like understanding additional languages, different platforms help you broaden your understanding and provide insights you can bring back to your main work.
  6. Learn to give and take criticism. Part of learning is getting feedback on your work, and part of being on a team is sharing feedback with others. If you took art or music classes in high school or college you probably learned some of the basic lessons you need here, but if you didn’t, consider taking one now at your local community college or art center. The arts are wonderful for getting experience with criticism. For all art is often open to interpretation, it also requires specific skills. If you play off-key, it sounds wrong. If your sculpture collapses under its own weight, the project failed. If your picture’s subject is out of focus, you need to re-shoot it. Sure there are brilliant artists who can violate all the rules, but if you have never experienced an art critique you are not one of those artists. The experience of getting direct, blunt, and honest feedback will help you understand its value and how to give that feedback yourself.
  7. Share what you think you know. We learn a great deal with we teach others. Both because it forces us to refine our thinking and understanding so we can explain it, and because learners ask questions we cannot answer off the top of our heads. This can be user group or conference presentations, internal trainings for your team, mentoring junior developers, writing a blog, or anything else that gets your from learning to teaching. It’s okay if you’re not 100% right, that’s part of how we learn. A few years ago I was doing a joint project with a junior developer who asked me a lot of questions, and pushed hard when she thought I was making mistakes. When she asked why I was selecting a solution or setting a pattern, she was never satisfied with “because that’s the best way to do it.” She wanted me to explain why that was the best way. If I couldn’t walk her through it right away, I went back and hunted for reference material to explain it or if that failed I tested her counter ideas against my plans to see if I was missing something. While I was usually right, not always and we did make changes based on her feedback. More importantly it forced me to show my work in fine detail which was a good exercise for me and gave her insights to help her do better work.
  8. Find your own patterns. At the start I said this list was for people who didn’t have their own patterns yet. In the long-run of your career you need to figure out what you need to know to get to where you want to go next. Eventually you will need to find a pattern that works for you and the life you are living. No one can tell you what that is, nor how to learn it all yourself. Experiment with learning styles, areas of work, roles, and types of projects as much as you are able until you feel your way to the right solutions for you.
Dec 23 2019
Dec 23

Making multilingual sites is hard. I’ve been using Drupal since version 5 and I can say a few things about the evolution of Drupal multilingual capabilities:

  • First, Drupal 8 is – in my opinion – the first version of Drupal where someone could say that multilingual works, pretty much out of the box.
  • Second, the documentation about how to deal with different scenarios is quite good.
  • And third, from a user experience perspective, translating the user interface of a site is really hard.

In this post we will talk about the third point and what we did to manage that complexity.

Our Current Scenario

We are building a really complex site, and the main challenges we faced regarding multilingual are:

  • The site is a multisite architecture, one database using Organic Groups.
  • Each group represents a country, and each country needs its site in one or more languages.
  • We have several variations of the same language depending on the region this language is spoken in.
  • We don’t want to let content editors translate the site directly from the UI.
  • We don’t speak all the languages the site is in.

The last item is quite relevant, when you don’t speak a language, you cannot even be sure if the string you are copying into a textbox says what it should.

The First Attempt

We started with a translation matrix to store all the translations. A simple Google drive spreadsheet to track each string translation in each language.

Each column uses the language code as a header.

Using a tool to convert Spreadsheets into po files we get each translation file fr.po, es.po, pt.po.

We used wichert/po-xls to achieve this with good results.

Not So Fast

This initial, somewhat naive, approach had a few problems.

  • Drupal string translations are case sensitive. This means that if you made a typo and wrote Photo instead of photo the translation will fail.
  • Some strings are the result of a calculation. For example. Downloads: 3 is actually managed by Drupal as Downloads: @count.

But the more complex item is that Drupal 8 has two ways to translate strings. The first one is inherited from Drupal 7. The one that makes use of the well known t function for example t('Contact us.').

The other one is a new way that allows site builders to translate configuration entities.

The two scenarios that allow translation of a Drupal site.

Translating Configuration Entities is Really Hard

To translate configuration entities, you need to identify which configuration needs translation, and find the exact part relevant to you. For complex configuration entities like views, this could be quite hard to understand.

Even for an experienced site admin this can be hard to understand.

Another problem that we had to solve was the vast amount of configuration alternatives you have when dealing with a medium-size Drupal site.

Each category has a lot of items to translate.

It was clear to us that in order to translate all those items we needed to find another way.

More problems&mldr; Identifying Which Strings to Translate is Hard

One thing to consider when dealing with Drupal translations is that it’s not easy to identify if a string is displayed somewhere in the frontend or if it is only a backend string.

Translating the entire codebase may not be a viable option if you want to keep a short list of translations reviewed by a group of people. In our case, it was important to make sure that translations are accurate, and that translators do not feel overwhelmed.

We don’t have a great solution to this problem yet. One of the strategies we used was to search for all the strings in twig templates and custom modules code using a grep search.

egrep -hro "[\>, ]t\('.*'\)" . | cut -c 5-   # Get strings inside ->t(...) and t(...)
egrep -hro "{{ '.*'\|\t" .                   # Get twig strings '....'|t
egrep -hro " trans .*" .                     # Get twig strings inside trans

However, as we figured out later by reading the documentation, twig strings cannot be used as a source for translations. Internally, Drupal maps those strings back to the regular use of t('strings').

This means that strings like:

{% trans >}}Copyright {{ year }}{% endtrans >}}

Are actually converted to

t('Copyright @year')

And that last string is the one you should use as source of the translation.

At the end, we cleaned up the spreadsheet list using visual inspect, and so far it has been working fine.

How We Solved the Problems?

To recap the problems we had:

  • We did not want to translate all the available strings.
  • We did not know all the languages, therefore copy and pasting was a risk.
  • Translators were expecting to have a reduced number of strings to translate.
  • Configuration translations are quite complex to track.

As we mentioned before using the xls-to-po tool, we were able to obtain the PO files to translate one part of the strings that we needed to translate.

We also used drush_language to automate the process.

drush language-import --langcode=fr path/to/po_files/fr.po

This little snippet iterates over all of the po files in the po_files directory and imports the language using the drush command mentioned above.

find po_files -type f -name *.po | xargs basename --suffix=.po | \
xargs [email protected] drush language-import --langcode=@ @.po

The xls spreadsheet has in the first column the Message Id, and the language codes of the system

By using conditional cell colors, we can quickly identify which translations are pending.

Solving the Configuration Translation Problem

The second part of our problem was a bit more tricky to fix.

We used a custom script to get all the config entity strings that were relevant to us.

Here is a simplified version of the script.

$prefix = 'views.view.custom_view';
$key = 'display.default.display_options.exposed_form.options.reset_button_label';

$configFactory = \Drupal::service('config.factory');
$list = $configFactory->listAll($prefix);

$rows = [];

foreach ($list as $config_name) {
  $columns = [];
  // Add the unique identifier for this field.
  $columns[] = $config_name . ':' . $key;

  // Get the untranslated value from the config.
  $base_config = $configFactory->getEditable($name);
  $columns[] = $base_config->get($key);

  $rows[] = $columns;
}

If you wonder how to get the $prefix and $key, they are obtained by inspecting the name of the field we want to translate in the Configuration Translation UI.

You need to inspect the HTML of the page, see the name attribute.

We print the result of the script to obtain a new CSV file that looks like this

The first column is a unique id that combines the prefix and the key.

Then, we copy and paste this CSV file as a new tab in the general translation matrix, and complete the header with the rest of the languages translations.

Finally we use a spreadsheet formula to find the translation we want for the languages we are interested in.

=IFERROR(VLOOKUP($B2,$Strings!$A$2:Y299,COLUMN()-1,0);"")

This will search for a match in the Strings matrix, and provide a translation.

Spreadsheet magic.

Final step: Importing the Configuration Strings Translation Back to Drupal

Once we have all the translations we need. We export the CSV file again and use this other script (simplified version) to do the inverse process:

use Symfony\Component\Serializer\Serializer;
use Symfony\Component\Serializer\Encoder\CsvEncoder;
use Symfony\Component\Serializer\Normalizer\ObjectNormalizer;

$filename = 'path/to/config_translations.csv';

$serializer = new Serializer([new ObjectNormalizer()], [new CsvEncoder()]);
$configFactory = \Drupal::service('config.factory');
$languageManager = \Drupal::service('language_manager');

$serializer->encode($data, 'csv');
$data = $serializer->decode(file_get_contents($filename), 'csv');

foreach ($data as $row) {
  $name_key = array_values($row)[0];
  list($name, $key) = explode(':', $name_key);

  // The languages we care start after the second column.
  $languages = array_filter(array_slice($row, 2));

  foreach ($languages as $langcode => $translation) {
    $config_translation = $languageManager
                            ->getLanguageConfigOverride($langcode, $name);
    $saved_config = $config_translation->get();
    $config_translation->set($key, $translation);
    $config_translation->save();
  }
}

Some Other Interesting Problems We Had

Before finishing the article, we would like to share something interesting regarding translations with contexts. As you may know, context allows you to have variations of the same translation depending on, well&mldr; context.

In our case, we needed context to display different variations of a French translation. In particular, this is the string in English that we needed to translate to French:

Our organization in {Group Name}

In France, this translates into Notre organisation en France. But if you want to say the same for Canada, due to French grammatical rules you need to say Notre organisation au Canada (note the change en for au).

We decided to create a context variation for this particular string using context with twig templating.

{% trans with {'context': group_iso2_code} >}}
Our organization in { group_name }
{% endtrans >}}

This worked ok-ish, until we realized that this affected all the other languages. So we need to specify the same translation for each group even if the language was not French

This is not what we want...

After some research we found the translation_fallback module but unfortunately it was a Drupal 7 solution.

Long story short, we ended up with this solution.

{% if group_uses_language_context >}}
  {% trans with {'context': country_iso2_code} >}}
    Our organization in { group_name }
  {% endtrans >}}
{% else >}}
  {% trans >}}Our organization in { group_name }{% endtrans >}}
{% endif >}}

Which basically provides two versions of the same string. But if the group needs some special treatment, we have the change to override it. Lucky for us, xls-to-po has support for strings with context. This is how we structured the translations for strings that require context:

CA, in this case, is the ISO code for Canada

Conclusion

For us, this is still a work in progress. We will have to manage around 20 or more languages at some point in the project. By that point, having everything in a single spreadsheet may not be maintainable anymore. There are other tools that could help us to organize source strings. But so far a shared Google Sheet worked.

We still use configuration management to sync the strings in production. The snippets provided in this post are run against a backup database so we can translate all the entities with more confidence. Once we ran the script we use drush config:export to save all the translations to the filesystem.

Dec 18 2019
Dec 18
  • If you are using Drupal 8.7.x, you should upgrade to Drupal 8.7.11.
  • If you are using Drupal 8.8.x, you should upgrade to Drupal 8.8.1.

Versions of Drupal 8 prior to 8.7.x are end-of-life and do not receive security coverage.

Alternatively, you may mitigate this vulnerability by unchecking the "Enable advanced UI" checkbox on /admin/config/media/media-library. (This mitigation is not available in 8.7.x.)

Additional information

All advisories released today:

Updating to the latest Drupal core release will apply the fixes for all the above advisories.

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web