Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Apr 01 2021
Apr 01

The last year has highlighted to us all how important it is for the global community to come together and solve problems. We rate ourselves highly at ComputerMinds, and figure it's time to share and stretch our abilities to the full. So I'm here on this special day to announce that we are branching out beyond just resolving bugs on websites, to fixing any kind of bugs in any problem space. There's so much market potential, we're really quite excited at the possibilities for bringing innovative solutions to the world!

Medical bugs

The ugliest bug of them all, COVID-19, has been such a terrible challenge for us all. We've been inspired by the countless heroes across the world who have stood up in the face of it, so we want to help too by putting our services to more significant tests than just Drupal websites. We have continued to serve our clients during the pandemic as well as we can so far, but now it's time for us to help with other kinds of bugs. We'll start with the common cold and flu, and work our way up to the bigger beasts. We believe in our approach and that our experience will propel us to find solutions. To help us with this, we'll team up with the best in the business with offices in Bristol and Coventry.

Pest control

Inspired by the wonderful pest control hawk that flies around our Coventry office, we will help fix your bug problem. Our Drupal experience has taught us to search for the root problems and to stop at nothing to go down debugging rabbit holes - so we are perfectly suited to this industry too. Unwanted animals and insects, beware! But we've also got a strong ethical heartbeat too - we always want to do things the right way, after all. We'll continue to work with existing partners to campaign for sustainable pest control, and against unnecessary culling.

Lifestyle bugs

Life coaching is a blossoming market. We believe too many rush ahead to give advice about making lifestyle changes, before pausing to eliminate 'bugs' in people's lives that hold them back. Too many web projects need rescuing because of the issues that hold them back - and these are often much deeper than mere software issues, but go down to 'people' problems. We're ready to bring our experience from these situations to help people become the best versions of themselves. We recognise that in some scenarios, people need to give themselves more slack in their expectations, whilst others need to the right pressures applied to improve performance. Sometimes proper recovery from major trauma must be prioritised; for others we are well-placed to encourage physical exercise for all the benefits and widened perspective it brings; for others laziness is the 'bug' we will identify and help clients overcome!

Engineering

We've already been solving problems in electrical engineering and patching up domestic engineering horror stories. We've worked with some genuine motor engineering history beneath us. So now we're opening up to offer our services to the great people of Bristol and Coventry. Our cities are well known for engineering feats, so we are keen to partner with the local firms that are facing bugs in their work. Our contribution towards the Coventry Motofest event demonstrates our passion for great engineering - the traditional kind, not just software engineering.

Software problems - not just Drupal

Drupal will always be our specialism, but the knowledge gained over the years from those projects gives us plenty of wisdom for any software project. We can already offer consulting for commerce websites built on other platforms. You probably already knew that we have great experience in building websites with GatsbyJS, and native mobile apps in other technologies. We are ready for you with all sorts of advice that can be applied to nearly any web project, whether that be advice on analytics, A/B testing solutions or development methodologies. But we're not limiting ourselves to the internet either any more - bugs will always be found in all kinds of software!

So what are you waiting for? Find out how we can help fix your bugs!

Jan 12 2021
Jan 12

The vast majority of community-contributed Drupal 8 modules now have releases that are compatible with Drupal 9, but what can you do if you need to use a module that doesn’t? Well, you’re likely to find a compatibility patch in the project's issue queue. But the tool most of us use with composer to apply patches, cweagans/composer-patches, is a plugin that can only make its changes after composer reads a package's metadata about its compatibility (and dependencies). So for contrib modules that haven't yet committed their patches, attempting to apply the patch in the usual way doesn't help. For example, before commerce_migrate was compatible with Drupal 9 (update: it is now!), when I tried applying the necessary patch this way, composer just gave me an enormous and confusing error:

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Conclusion: don't install drupal/commerce_migrate 2.0.0-rc3
    - Conclusion: don't install drupal/commerce_migrate 2.0.0-rc2
    - Conclusion: remove drupal/core 9.0.9
    - Installation request for drupal/commerce_migrate ^[email protected] -> satisfiable by drupal/commerce_migrate[2.0.0-rc1, 2.0.0-rc2, 2.0.0-rc3].
    - Conclusion: don't install drupal/core 9.0.9
    - drupal/commerce_migrate 2.0.0-rc1 requires drupal/core ^8.7 -> satisfiable by drupal/core[8.7.0, 8.7.0-alpha1, 8.7.0-alpha2, 8.7.0-beta1, 8.7.0-beta2, 8.7.0-rc1, 8.7.1, 8.7.10, 8.7.11, 8.7.12, 8.7.13, 8.7.14, 8.7.2, 8.7.3, 8.7.4, 8.7.5, 8.7.6, 8.7.7, 8.7.8, 8.7.9, 8.8.0, 8.8.0-alpha1, 8.8.0-beta1, 8.8.0-rc1, 8.8.1, 8.8.10, 8.8.11, 8.8.12, 8.8.2, 8.8.3, 8.8.4, 8.8.5, 8.8.6, 8.8.7, 8.8.8, 8.8.9, 8.9.0, 8.9.0-beta1, 8.9.0-beta2, 8.9.0-beta3, 8.9.0-rc1, 8.9.1, 8.9.10, 8.9.11, 8.9.2, 8.9.3, 8.9.4, 8.9.5, 8.9.6, 8.9.7, 8.9.8, 8.9.9].
    - Can only install one of: drupal/core[8.7.0, 9.0.9].
    - Can only install one of: drupal/core[8.7.0-alpha1, 9.0.9].
...
    - Installation request for drupal/core (locked at 9.0.9) -> satisfiable by drupal/core[9.0.9].


Installation failed, reverting ./composer.json to its original content.

The error in situations like this goes on and on, including scary messages like 'Conclusion: remove drupal/core', and sometimes about all sorts of seemingly unrelated packages! Ultimately, composer is trying to bend over backwards to use versions of Drupal and its dependencies that would work with the module, but it won't find any. Or if it does, you might find yourself downgraded to an old version of Drupal, which you certainly don't want!

So what's the solution?

We need to override the compatibility metadata for the contrib module.

Drupal sites use https://packages.drupal.org/8 as the metadata provider, which you'll find listed in your project's root composer.json file under a repositories section. This section instructs composer where to get the metadata from ... so if you add your own 'package' repository to be used first, composer will happily use that, ignoring what drupal.org tells it:

"repositories": [
    {
      "type": "package",
      "package": {
        "name": "drupal/commerce_migrate",
        "type": "drupal-module",
        "version": "dev-8.x-2.x",
        "source": {
          "type": "git",
          "url": "https://git.drupalcode.org/project/commerce_migrate.git",
          "reference": "9ac26262b3443d20e69cea69652abc2dee39fee5"
        }
      }
    },
    {
      "type": "composer",
      "url": "https://packages.drupal.org/8"
    }
  ],

In this example, for the commerce_migrate module, we specify the latest commit (at the time of writing) from its development branch (8.x-2.x), as that's what the patch is intended to get ultimately applied to. That branch also matches our requirement in our composer.json file: "drupal/commerce_migrate": "dev-8.x-2.x". (Note that composer needs the dev- prefix when using git branches.)

So this section redefines the module's dev-8.x-2.x version for composer. As we require that version, we get the snapshot of the codebase at that specific commit - skipping the metadata that drupal.org would have added which would restrict its compatibility.

Now, the patch can be added in the usual patches section, and then running composer update drupal/commerce_migrate will apply it successfully!

This technique can also be used to override other package metadata for composer, such as a change in dependencies. But beware - it means you're no longer using the real releases for the package, and in this example, locks to a specific commit. So you will need to keep an eye out for when the patch gets committed to the module, and for any other work in its codebase that should be incorporated over time. It's a little like taking a snapshot of the project, and applying just the specific changes you need - at the cost of losing out on any goodness that the maintainers may add over time. This approach does avoid forking the codebase away at least. As this replaces the module’s metadata entirely, it’s worth checking for any dependencies or other compatibility metadata that its own (potentially now patched) composer.json file might have, and copying them to your project’s root composer.json file.

Given how quickly the community & its leaders have rallied to make modules D9-ready, I wouldn't be surprised if this trick can be removed in a few months' time, as maintainers continue to commit these (often trivial) patches. Any projects that don't, may even get covered automatically some day - plus, if their maintainers aren't doing this, they're also unlikely to be adding much of note in the meantime anyway, so it's pretty safe. I would just keep an eye out for any new releases - especially security ones - to those projects.

Get notified

I suggest finding the issue that currently tracks making the module compatible with Drupal 9. This may be linked to from the project page, or it's usually one of the more recent/active ones in its issue queue. If you're logged into drupal.org (and you should be - sign up if you haven't!), then you can press a 'Follow' link that has a nice green star next to it, to get email notifications of updates. The 'View all releases' link from a project page takes you to a page that has an RSS feed, which can be used to subscribe to notifications of new releases too. That's well worth doing for a number of reasons! If you didn't know it, you can also subscribe to notifications of security releases - head to https://www.drupal.org/security where you'll find instructions at the top of the sidebar (which is below the list of posts if you're on a mobile).

Thanks to heddn, dpi and larowlan for this solution. Photo by Josh Carter on Unsplash.

Nov 11 2020
Nov 11

TLDR : Check your cookie popup!

It seems everyone is talking about core web vitals at the moment, spurred on by Googles’ recent announcement that page experience (which includes core web vitals) will start influencing search ranking in May 2021.

We won’t go into detail on web vitals in this post - there is plenty of information already on the web, including from Google themselves on the excellent web.dev site.

Instead, this post will look specifically at cumulative layout shift - or CLS as it’s known. I like to think of this as the jerkiness measure of your site - the higher the number the worse the jerkiness.

Getting your CLS score is pretty easy - you can use pagespeed or lighthouse (built right into chrome). I will assume (because you are reading this post) that you have done this already AND you are not happy with your score. So - what next - how do you go about diagnosing and fixing the issue?

The reading online makes lots of great suggestions, mostly around helping the browser reserve space for elements that are late loaded (images, video etc). This is all good stuff - however - it didn’t help me and the sites I was working on.

So first off I switched to incognito mode (I wanted to simulate a fresh first-time visit to the site - for various reasons) in chrome, loaded the site and then fired up the performance tab in the chrome dev tools. This is a scary looking tab - there is a lot going on here - and you can safely ignore 98% of it (at least for now). You just need to make sure the ‘Screenshots’ tick box is ticked - then hit the little refresh icon - this will refresh the page and start profiling the page load. It’s worth pointing out that this is running locally so you can use it on your local dev sites.

This takes a few seconds and when complete you should have a lovely looking film strip of screenshots along the top - mouseover and you get a larger version. Start at the left and move rightwards - and keep your eyes open for any obvious content shifts.

I did this for 3 individual and unrelated client sites - and found the cookie popup to be the culprit in 2 of the sites - it was appearing mid-way through the load and pushing ALL the content down - fixing this took around .4 off the CLS score (which is a lot). The other site had the ENTIRE expanded mobile menu briefly rendering at the top of the page - which then disappeared (as it should) and shifted all the page content up by about 800px! Fixing this took about .5 off the CLS score.

So - the takeaway from this is probably before you spend ages fiddling with width and height attributes on your img tags, have a quick look to make sure there isn’t some low hanging fruit massively impacting your CLS.

Oct 27 2020
Oct 27

Over the last year or so, I've got quite engaged with Drupal slack. I've loitered in channels like #d9readiness and #config, discussed issues with members of the security team, and asked questions to module maintainers (and received answers!). But most of all, I've helped people out in the #support channel. This has been an interesting experience in many ways, so I thought I'd share my reflections. The Drupal slack workspace is intended for the community, so if you're reading this - it's probably for you too. Hopefully my thoughts might help prepare you to use it as an effective tool.

Ultimately, I've got stuck in as a way to contribute to the Drupal community in a new way as part of our CM contribution challenge. That has given me motivation to give more than I get, but I've been pleasantly surprised how much I - and ComputerMinds - have benefitted from being part of Drupal slack. When we've been unsure about the status of a module or the way forward with an issue, it's been great to be able to reach out to exactly the right people. Especially in the current climate that limits our ability to meet people in-person (e.g. at Drupalcon). So, thank you to those people that have helped me :-) We've even received sponsorship to work on some module issues after discussion on slack - we ended up becoming responsible for releasing security fixes for the Commerce Ingenico project after it had been shut down due to vulnerabilities. That's a win for us, and it's a win for the community, who can use that module once again.

Slack has benefits, but also drawbacks, as it is yet another system clamouring for attention in a crowded digital world. So it's important to use it appropriately for the good of the community, but also wisely for your own sanity. Drupal's challenging learning curve means that there are a lot of people out there wanting help, and the #support channel is full of them. (I've spotted some of the most experienced drupallers asking for help there too; so it's not just for newbies.)

Learning Curve for Popular CMSSource: Learning Curve for Popular CMS

Sustaining a good level of support is a challenge, especially doing so with a friendly manner. My approach has been to answer questions I know I can help with and avoid those that I know other people would be better placed to answer. Some questions can suggest a shaky foundation of understanding, so unless I'm sure I can help fix that, I tend to stay away from those too. Often a question is a symptom of a deeper issue - either with someone's website, or their understanding of Drupal components. On the whole, I believe I have helped a lot of people - and not just with their immediate questions. Some people come back to me with direct messages weeks or months later, knowing that I might be able to help.

Unfortunately my mental health has suffered a bit since the coronavirus pandemic began, so I've had to keep an eye out for things that may be contributing to that. Slack offers connection to people 24/7 - which can be good, but it's not a truly deep connection, and can be relentless. While my motivation for interacting with Drupal slack was to try and contribute to the community, that's not entirely for it's own sake. ComputerMinds want to help Drupal and its community flourish - but that's partly because we want that for our own benefit! So I decided to focus on supporting people that might be within closer reach, by skipping over most posts from outside UK business hours. (I'll happily reply to DMs/threads asynchronously though if necessary.)

In conclusion, I think Drupal slack is a really handy tool for connecting with the Drupal community. I'm very glad to help people through it, even if that means only making a difference slowly, to one person at a time. There are some incredibly helpful people on there who I see answering questions again and again. I tried to be one of those for a while, though I learnt to take care to make my offering sustainable. If you need help, try out Drupal slack. If you don't .... then you can probably be a help to someone there! Most of us are a mix of those extremes anyway. Try to be kind on slack - to yourself and others. I figure if we all help each other, we might make climbing that Drupal learning cliff a bit more personal, and a bit more pleasant. Win-win!

Photo by Paul Gilmore on Unsplash

Jul 28 2020
Jul 28

Many of us at ComputerMinds have always taken pride on doing Drupally things the right way whenever possible, and then helping the community do so too. One of these things is displaying values from fields on content entities. We wrote before about how to do this in Drupal 7 and Drupal 8. It's now the turn of Drupal 9! Thankfully, this updated version is basically the same as the last one, as D9 is very similar to D8 on the surface, but with old cruft ripped out to allow it to continue improving. So the short answer to "How can I show a field programmatically?" is still:

$entity->field_whatever->view();

Isn't that great? Your existing code for Drupal 8 already works with Drupal 9! That was the aim of the update; to make the upgrade incredibly easy. Whereas upgrading from Drupal 7 can be a mammoth task (you might want to get in touch with us to help!), the jump to D9 is much simpler.

So is anything different? Most changes are buried well within Drupal's innards. The most relevant difference for displaying fields in Drupal 8 as opposed to Drupal 9 is that if you were originally loading the entity object ($entity) using the entity.manager service (e.g. from \Drupal::entityManager()), you now need to use the entity_type.manager service (e.g. \Drupal::entityTypeManager()).

Our previous article on rendering fields in D8 contains much more detail, which is still totally valid for Drupal 9. That will help you tweak the formatter settings to view a field with, or how to get raw values out of the field. For example:

// Render an image field with a specific image style.
$entity->field_my_image->view([
  'type' => 'image',
  'label' => 'hidden',
  'settings' => array(
    'image_link' => 'content',
    'image_style' => 'square_icon',
  ),
]);

// Get the raw value out of a single-value link field.
$link = $entity->field_web_address->uri;

A comment on that article did point out that you can get fatal errors if you use this code too naïvely. That's because magic methods are used here, with the assumption that you are sure the field exists on the entity. If you don't, then just break the chaining down:

// The $field variable will just be null if the entity doesn't have this field.
if ($field = $entity->field_whatever) {
  $to_show = $field->view();
}

Alternatively, you can use get() methods instead of the magic methods. But if you do, you'll probably want to surround your code with Exception handling to catch InvalidArgumentException exceptions, as the magic method getters are more lenient in more scenarios.

Photo by Belle Hunt on Unsplash

Jul 14 2020
Jul 14

I recently released a new contributed module to aid translation on Drupal 7 sites: Entity Translation: Separated Shared Elements Form (ETSSEF). Yes, it has a convoluted name! It finally resolves a suggestion from years ago in an Entity Translation project issue, to allow editing untranslatable fields separately to translatable ones. One of our clients has a multilingual product database site with a few hundred fields on their content, so anything like this that could reduce the size of their editing forms is useful. I figure the best way to demonstrate this is with a recipe that blends it together with some other super (but generally obscure) modules. I hope you can spot parts that may be helpful for your projects!

Screenshot of the top of an edit translation form, including Shared tab and Add fields to form widget

The Recipe
 

Ingredients

Take a look at each of these project pages linked above for a flavour of what each module will bring to the mix.


Recipe Difficulty Rating: Intended for experienced Drupal cooks only; others may prefer to try our takeaway service.
 

Method
  1. Enable each of the modules listed above, and set the admin theme to be used.
     
  2. Configure the various Field storage modules. Try to understand what each of these is doing, and adjust appropriately for you if necessary:
    // Turn off storage of revision info for your content type that has many fields.
    $bundle = 'farmer';
    variable_set('field_sql_norevisions_entities', array(
      'node' => array(
        $bundle => 1,
      ),
    ));
    // Default to using Blob storage (1 table instead of 1000s).
    variable_set('field_storage_default', 'field_sql_blob_storage');
    // Relabel the options in the UI to make the distinction clear.
    variable_set('field_storage_ui_relabel_options', array(
      'field_sql_blob_storage' => 'Retrievable only',
      'field_sql_storage' => 'Sortable & Filterable',
    ));
    // Load default-sql-storage fields in batches of 20 (instead of 1 at a time).
    variable_set('field_sql_storage_group_load_max_fields', 20);
    
     
  3. Set up your content type with many many fields. Choose 'Retrievable only' for the storage type for any fields that don't really need to be used for querying against, or sorting/filtering in lists. This will improve performance, as all the field data for those is stored together in a 'blob' column in the database so can be loaded (& unserialized) in a single go, rather than requiring select queries from so many different individual database tables.
     
  4. Configure nodes of this type to use entity translation (field translation) and head to the entity translation settings at /admin/config/regional/entity-translation. Set their 'Shared elements on translation forms' setting to 'Only display on their own separate forms':
    Entity translation settings showing Shared elements configured to show on their own forms
    This ensures that untranslatable fields are just edited on the initial Edit tab (in a 'Shared' secondary tab); with just translatable fields in the translation forms. When there are so many fields, it's worth slimming down forms as much as possible! This also has the advantage that untranslatable data can be edited without needing to touch any specific translation.
     
  5. Override the edit form for your node type in a custom module, to use the Field Attach Form Selective module, so that fields are only shown on the form as they are filled in. This vastly reduces the amount of stuff on the form. I've written a gist that demonstrates this, and includes wiring it up to work nicely with ETSSEF. You must copy the entire contents of node_form() from node.pages.inc in Drupal core into the farmer_node_form() function, but replace the call to field_attach_form() at the bottom, with a call to field_attach_form_selective(). Use the same arguments.
     
  6. I then added some classes and CSS to the secondary tabs on the page to show the flag icons next to each language, as well as repeating the current tab name in the page title. I then added CSS to fix the page header in place so that editors easily retain that contextual information as they scroll down the giant forms. Otherwise it's too easy for them to forget which language they are editing!
Season to taste

Now when you edit your content type, your forms will be much slimmer and your site will run far smoother with hundreds of fields. As with any recipe, take the bits of this that are to your taste, ignore others, or blend it into your own creations! This was only for D7, so bringing the ideas over to Drupal 8/9 in some form would be an obvious thing to do. I’d love to hear of other ingredients you use to help when editing content forms with enormous amounts of fields, translatable or not.

Photo by Maarten van den Heuvel on Unsplash

Jul 07 2020
Jul 07

Drupal 7 introduced the brilliant feature of letting users cancel their own account and with it various options for what to do with content they've created when they are cancelled. One of these options is to:

Delete the account and its content.

Which can prove somewhat problematic if used incorrectly.

You see, Drupal is very good at the latter part: deleting all the content created by the user. It's not very good at warning someone that they are about to delete potentially a lot of important content.

The scenario

Let me set the scene for you. Someone had an account on a Drupal site and did a lot of work, making pages etc. Then they left the organisation. Someone else comes along and after a while thinks: I should clean up all these old user accounts and delete them, we don't need them any more.
Unfortunately they use the aforementioned Delete the account and its content option.

A few days pass and then they notice that the cookie policy page has gone missing. And they are sure that the FAQ section had more than 3 questions in it.
Oh dear.

They now face a serious problem. They have two 'easy' options to resolve it:

  1. Restore a database backup from before they deleted the user to recover all the lost content.
  2. Attempt to manually re-create all the content that was deleted.

However, they've been using the site in the interim and have changed lots of content. So have other users of the site. They can't simply restore a database backup from before all the content was deleted because they'd lose all the changes since then. But they also size up the volumes of missing content, and they simply aren't sure what content has gone missing, but know that it's hundreds of pages. Also the references between content have been broken, content that still exists on the site is trying to reference content that isn't there. So now not only do they need to re-create content but they have to go around fixing all the other site content that references that content. Oh my.

The third option

There is another way:

  1. Automatically re-create all the content that was deleted.

But how?

If you've got a decent backup from before the deletion happened then you contact your friendly ComputerMinds and we'll help you out by following something along the lines of the below. If you don't have a decent backup, then you're toast: Learn your lesson and start making backups of your data that you can restore from!

But you've got that backup, right? Ideally from as close as possible to, but not after, the account and content being deleted. So let's see what you/we do with it:

We're going to repeat the deletion and work out how to put it all back.

Begin by restoring the code, files and database from your backup to a development machine.
Load up the site in your browser and get ready to perform the exact same operation that caused the problem in the first place, but don't perform it yet!

Now, identify tables that contain changes that you don't really care about, the more the merrier. I'm thinking the watchdog table, any cache_* tables etc. You might need expert knowledge of the site to make this list as long as possible, it'll help later because you can really cut down the amount of noise and work you'll have to do later.

Once you've done that you want to make a 'pre-delete' database dump. Something like this:

drush sql-dump --structure-tables-list='sessions,cache,watchdog' > pre-delete.sql

Now, go back to your browser and cancel the account in the same way that was done before, so: Delete the account and its content.

Once the deletion has happened we want to run the same drush command as before, but save the results to another file.

drush sql-dump --structure-tables-list='sessions,cache,watchdog' > post-delete.sql

Now we essentially have two database snapshots, the difference between the two is all the content that was deleted. So we'll aim to produce a set of SQL queries to restore all that to the production database.

I had very mixed results with trying to get two MySQL dump files that would diff easily in a way that would leave the correct INSERT/UPDATE statements to put all the content back. Comparing the two dump files pre-delete.sql and post-delete.sql directly just didn't seem to work.

Percona to the rescue!

There's a tool in the Percona suite called pt-table-sync that will diff two databases and produce a set of SQL statements that would make the data consistent between the two, i.e. the SQL 'diff'.

There's a final wrinkle that means that you actually need another database server at this point, because pt-table-sync can only sync from one server to another, not between two databases on the same server. However, in the age of Vagrant or Docker getting multiple MySQL servers running on your machine is no big issue. I'm going to suppose you have two database servers running on ports 3306 and 3307 on your local machine.

Restore each of the SQL dump files from before to an identically named database on the servers respectively. Then you can get pt-table-sync to produce the magic:

pt-table-sync --print --databases=db_name h=127.0.0.1,P=3306 h=127.0.0.1,P=3307 > content-restore.sql

To make the diff go the right 'way' make sure the server with the post-delete.sql file is listed first in the command line. And you may need to adjust the command to get it to connect to your servers correctly.

Once you've done that content-restore.sql should contain a set of SQL commands that you could run on the production server to restore all the deleted content. However, I'd recommend doing one final manual look through the file and making sure that nothing is going to run against tables that don't really matter or that can't be recovered in other ways.
It's a text file so review it line by line and understand what each line is going to do and make sure they are the expected changes!

Once you've done all that you can execute the content-restore.sql file on your production server and that should restore everything that was deleted from the database!

Wrap up

So we've done this twice now, for different clients. We were happy that we were able to recover their content and not force them to either lose all other changes made or have to re-create a lot of pages.
We learnt so much the first time we did this, that the second time it was actually a fairly smooth process that didn't take very long at all despite having to restore thousands of pieces of content. We've also taken steps to stop people from using this particularly dangerous option when cancelling a users account.

Obviously all of the above relies on having backups of your database, and being able to retrieve a point-in-time, not just the 'latest' one. If you don't have this in place already, go now and get that sorted!
If you have backups, maybe bookmark this page so that if you ever need to recover a large amount of accidentally deleted content you'll know a (fairly) easy way that works well.

Jun 23 2020
Jun 23

So we challenged ourselves to contribute back to the Drupal community this year. How are we doing? Here's a simple update on what each of us has done so far. Hopefully we'll see other ComputerMinds team members join this list by the end of the year.
 

Christian Sanders

You may have noticed Christian's recent article on updating jQuery. That work included producing patches to Drupal core itself and the jquery_update project. Christian has also pushed for user interface improvements in paragraphs (which I hope to implement later this year).

James Silver

James updated a patch for using tokens in webform components. He continues to own a whole 17 (yes, 17!) sandbox projects on drupal.org.

James Williams (me!)

I've already written in more detail on my recent contribution to the XML sitemap project and to recommend sponsoring contributions. The Drupal 8 release of the Language Hierarchy project was my first sponsored contribution, and since then I've made it available for Drupal 9 as well. I've started answering support requests in Drupal slack on a semi-regular basis, but otherwise most of my contributions are still around code. Here's a summary:

Mike Dixon

Even our head 'mind still gets in on the act. He has reported and solved a bug in the schema.org metadata module (used for SEO) and is helpful when he can to respond to support requests for modules that we use. And as the boss, we should be grateful that he encourages us all to use some of our company time to give back to the Drupal world!

Nathan Page

We kicked off this challenge with Nathan's dive down the rabbit hole, and before long Nathan had even dabbled in reviewing and producing patches for Drupal core. I'd say that was the aim of the challenge: to grow as developers and encourage others to help the Drupal project - so I think we can safely say, 'mission accomplished'. Nathan has since moved onto pastures new (Hi Nathan if you're reading this!), but we're delighted to see his contributions have continued.

Steven Jones

Perhaps our biggest all-time contributor to Drupal, Steven has continued to provide brilliant contributions. I consider him my mentor in this field, and I'm not the only one. Some of his contributions are to the infrastructure that some of us use with Drupal. For example, making Valet+ support Drupal sites better. Here's a summary of how he's helped out on drupal.org so far this year:

Stephen Tweeddale

Steve's chief contribution this year is actually outside of the Drupal ecosystem - he has continued to maintain a plugin for GatsbyJS sites: gatsby-source-git. This uses a git repository as a source for pages to go into a static Gatsby site, just like the regular one for Markdown files within a site. (Did you know ComputerMinds do GatsbyJS, not just Drupal?!) The two systems are being used together on more and more projects nowadays, so I think it's only fair on Steve to include this here ;-)

~

We're proud of all this! We've been involved with sorting a couple of security issues in contrib modules too, which are rightly kept secret. Of course there's plenty of scope to do more, especially by giving more of our time to help out beyond code.

Why not join us in taking on this challenge, to make a contribution to Drupal every month this year? Good luck! Let us know how you get on in the comments below.

Jun 16 2020
Jun 16

There are some key files like robots.txt and .htaccess which are often tweaked for Drupal websites. These can be considered part of the 'scaffolding' of a site - they control the way the site works, rather than its content or design. Any new release of Drupal core that includes changes to them specifically mentions that they need updating, as those changes may have to be merged with any customisations made on your site. For example, there was a security release that added rules to .htaccess, which were essential for any site to incorporate and the template settings file, default.settings.php, also gets regular updates which are easy to miss out on. The new Drupal Scaffold composer plugin can now ensure that these files are always up-to-date by default. But that can mean it's now too easy to lose customisations, as those files are taken out of our direct control. (They now behave like files from external dependencies, which are usually excluded from version control.)

It's not a good idea to 'hack' (i.e. make changes to) core files. Drupal developers even dissuade each other from doing this by joking about bad things happening to kittens! But while these scaffolding files may come from core, they all live outside of Drupal 8's /core directory. (A full list of these files is near the bottom of this article.) This leaves them vulnerable to the forgetful developer coming along and tweaking them without thinking. To be fair, it's quite right to expect to be able to tailor them for SEO, specific business requirements, performance gains, debugging needs or whatever.

So the Scaffold composer plugin provides some ways to customise these files in a 'nice' way, all of which require some little edits to your project's root composer.json file.
 

  1. Simply append or prepend some lines

    Create a file containing the lines that you want to add, and reference it within the 'extra' section:

      "extra": {
        "drupal-scaffold": {
          "file-mapping": {
            "[web-root]/robots.txt": {
              "append": "assets/my-robots-additions.txt"
            }
          },
          ...
        }
      }

    Replace 'append' with 'prepend' as the key if needed. This is great for robots.txt, which usually just wants some additions beyond what Drupal normally provides. I've used it for default.settings.php to suggest some useful project-specific config overrides for developers.

  2. Override a file entirely

    Create the file you want to use instead of core's version, and reference it within the 'extra' section:

      "extra": {
        "drupal-scaffold": {
          "file-mapping": {
            "[web-root]/robots.txt": "assets/robots-override.txt"
          },
          ...
        }
      }

    This loses out on any improvements that Drupal may add over time, but is handy if you want to take back control of the file entirely. For example, some SEO agencies like to determine the contents of robots.txt entirely (although the RobotsTxt module may be more useful for that). To entirely exclude a file, map it to false (the RobotsTxt module requires that). 

  3. Patch a file

    Create a patch of changes that you want to make, and use the post-drupal-scaffold-cmd script event hook:

      "scripts": {
        "post-drupal-scaffold-cmd": [
          "cd docroot && git apply -v ../patches/my-htaccess-tweaks.patch"
        ]
      }

    This is really useful if you have specific changes to merge into a specific place of a scaffolded file, like in .htaccess. This ensures you get the benefit of updates made by core to the file.

    Pro tip: run composer install; git diff -R .htaccess > patches/my-htaccess-tweaks.patch to produce the patch if .htaccess is still under version control!

Once these are in place, you can then ensure to remove and exclude all the scaffolded files from version control, if you haven't already. Here's example commands you could run to remove them. Make sure to replace 'docroot' with your webroot subdirectory.

# Commands to remove scaffolded files
git rm .editorconfig .gitattributes --ignore-unmatch;
cd docroot;
git rm .csslintrc .eslintignore .eslintrc.json .ht.router.php .htaccess index.php robots.txt update.php web.config modules/README.txt profiles/README.txt themes/README.txt example.gitignore INSTALL.txt README.txt sites/README.txt sites/development.services.yml sites/example.settings.local.php sites/example.sites.php sites/default/default.services.yml sites/default/default.settings.php --ignore-unmatch

...and a snippet you could paste into your project's .gitignore file. (Again, replace 'docroot' if necessary.) This should then be committed for this to all work out.

# Lines to add to your project's .gitignore file.
# Files from the Drupal scaffold for composer.
/.editorconfig
/.gitattributes
docroot/.csslintrc
docroot/.eslintignore
docroot/.eslintrc.json
docroot/.ht.router.php
docroot/.htaccess
docroot/example.gitignore
docroot/index.php
docroot/INSTALL.txt
docroot/README.txt
docroot/robots.txt
docroot/update.php
docroot/web.config
docroot/sites/README.txt
docroot/sites/development.services.yml
docroot/sites/example.settings.local.php
docroot/sites/example.sites.php
docroot/sites/default/default.services.yml
docroot/sites/default/default.settings.php
docroot/modules/README.txt
docroot/profiles/README.txt
docroot/themes/README.txt

A current list of the files can be found in core's composer.json file.

Good luck - you can now rest assured that the Drupal kittens will rest in peace ?

Photo by Dan Diza on Unsplash

Jun 04 2020
Jun 04
Drupal 9 logo

Update: Since writing this article the EOL of Drupal 7 has been extended from November 2021 until November 2022.

It’s here! June 3rd 2020 marks the official release date of the first production ready version of Drupal 9. It feels like Drupal 8 was only released a short while ago but it turns out it’s been 4.5 years already! The release of any new major version of Drupal is an exciting milestone in the project’s history and with a shiny new brand logo in place, Drupal 9 is ready to hit the ground running.

We are already well underway doing upgrade builds for our clients that are still on Drupal 7, and for those that are on Drupal 8 already, we will be shortly getting those running on Drupal 9 too.

With the release of Drupal 9 now out in the wild, you only have until November 2021 November 2022 until Drupal 7 reaches end of life (EOL) with Drupal 8 still reaching EOL November 2021. By that time those versions will no longer receive any further security updates. So it’s imperative that any Drupal 7 or 8 sites out there have plans to complete the upgrade to Drupal 9 in time before then because the work required for Drupal 7 sites to upgrade may be pretty significant. One of our developers Christian wrote an excellent article on upgrading your existing Drupal 7 site in May last year, so be sure to check that out if you haven’t already.

So what new features can you expect when upgrading a Drupal 8 site to Drupal 9? Well, nothing immediately obvious. The first release of Drupal 9 (9.0.0) contains the exact same features as the final minor release of Drupal (8.9.0), just with updated dependencies that Drupal 9 relies upon and there’s been a clear out of deprecated code from the 8.x codebase. This means Drupal can continue to improve in future with less baggage holding it back.

olivero d9 themeWork in progress 'Olivero' theme for Drupal 9

This is an exciting new front end theme currently in the works called ‘Olivero’ - which aims to give Drupal 9 a fresh modern default theme - and a new default admin theme called ‘Claro’. New and exciting features of Drupal 9 will start arriving with the release of Drupal 9.1 onwards...

At ComputerMinds we still believe strongly in Drupal as the number one platform of choice to build our sites upon and it’s evident that others share that view as there are now over 1 million websites powered by Drupal. It remains a very powerful platform which allows us to build exactly the kind of experience that a client wants, tailored to their specific needs, whilst being able to scale up to potentially handle millions of users and pieces of content. (If your site gets that big!)

Being open source itself and using other open source technologies like Symfony, Composer, Twig and PHPUnit to power it means it’s now more widely accessible than ever to other PHP developers that may have experience with other frameworks using some of the same underlying technology.

Whether it’s powering nonprofit websites such as the Rainforest Alliance, Government websites such as the Colorado General Assembly, providing a superior CMS editing experience for a global magazine like The Economist, helping boost Tourism numbers to Fiji, promoting your favourite music artist like Bruno Mars or helping you to find your favourite recipe on BBC Good Food, Drupal really can do it all.

Drupal 9 is available to download right now from drupal.org so what are you waiting for? Go check it out! (Of course those of you with a properly managed workflow will be upgrading from 8 to 9 using composer, so get that composer file updated!)

If you’ve liked what you’ve read and feel like you’re ready - or in a position - to start thinking about a site upgrade, why not start a conversation with us today by using our contact form. We’d love to hear from you and look forward to seeing what benefits we can bring to your site.

May 05 2020
May 05

We've been busy recently, but that doesn't stop us at ComputerMinds contributing back to the Drupal community! For our latest multilingual website, we needed an XML sitemap with alternate links and hreflang attributes. This site uses separate domains for each language - for example, www.example.se (??) and www.example.no (??). Search engines need these alternate links to help them understand how to match up each translation of a page, which are distributed across these different domains. But this site is built on our existing Drupal 7 e-commerce platform that uses the XML sitemap project, which has no support for alternate links (nor entity translation).

Sometimes contributing is glamourous (think of getting new features into Drupal core, or creating new modules for the community), but other times it just involves stepping back to gain some perspective, and do some menial tasks. This was the latter! There have been two long-running issues on drupal.org about getting the functionality I needed, both created way back in 2012:

So my challenge was to wade into these two, and the 281 comments on them, to figure out how to make progress. It turned out that I'd actually dipped into the first one nearly 2 years ago, and my colleagues had used work from them before too. But a lot changes in that time! The patches needed updating ('re-rolling') to work with the most recent code of the XML sitemap project itself and to work with the latest versions of entity_translation. I particularly enjoyed spotting a comment from our very own Mike Dixon, who threw a spanner in the works with a patch that confused everyone!

Eventually I created updated patches, resolved some bugs, and incorporated some additional valuable work that hadn't yet been reviewed by anyone. These patches ensured our client would be satisfied, and hopefully someone else will come along to review & approve them some day too.

Perhaps the most interesting thing to have come out of the work was a snippet of PHP I relied on to process links that needed adding to the sitemap. The XML sitemap project provides a UI to rebuild things in a batch, but recent changes to respect access mean that even that does not quite build the sitemap entirely. Instead, the work is done via cron, including a queue. Queues are pretty brilliant for ensuring background processes happen at some point, without hitting timeouts, or having to code up boilerplate code in hook_cron() implementations. But there's no way to limit what runs on cron (without additional modules) - I didn't want other modules to go doing other things when I was working on this, I just wanted my sitemaps to be built! So I came up with this, which is otherwise almost entirely pinched from drupal_cron_run():

function limited_cron($cron_modules, $queue_keys = array()) {
  $queues = module_invoke_all('cron_queue_info');
  drupal_alter('cron_queue_info', $queues);

  // Limit to the queue(s) that we specifically want.
  $queues = array_intersect_key($queues, array_flip($queue_keys));

  foreach ($queues as $queue_name => $info) {
    DrupalQueue::get($queue_name)->createQueue();
  }

  $implementations = module_implements('cron');
  $implementations = array_intersect($implementations, $cron_modules);
  foreach ($implementations as $module) {
    // Do not let an exception thrown by one module disturb another.
    try {
      module_invoke($module, 'cron');
    }
    catch (Exception $e) {
      watchdog_exception('cron', $e);
    }
  }

  foreach ($queues as $queue_name => $info) {
    if (!empty($info['skip on cron'])) {
      // Do not run if queue wants to skip.
      continue;
    }
    $callback = $info['worker callback'];
    $end = time() + (isset($info['time']) ? $info['time'] : 15);
    $queue = DrupalQueue::get($queue_name);
    while (time() < $end && ($item = $queue->claimItem())) {
      try {
        call_user_func($callback, $item->data);
        $queue->deleteItem($item);
      }
      catch (Exception $e) {
        // In case of exception log it and leave the item in the queue
        // to be processed again later.
        watchdog_exception('cron', $e);
      }
    }
  }
}

// Only run the cron implementations and queues that the
// XML sitemap project provides.
limited_cron(array('xmlsitemap', 'xmlsitemap_node'), array('xmlsitemap_link_process'));

This allowed me to very easily get my sitemap built up quickly, by repeatedly calling limited_cron() with just the things that I needed to run. Note that it doesn't act exactly the same as the normal Drupal cron, which runs as the anonymous user and avoids updating the session. But I have found myself returning to use this on other projects for other modules' cron queues. Hopefully you might find it useful too :-)

Photo by Ian on Unsplash

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web