Dec 12 2018
Dec 12

After reading this from Ars Technica, which describes how a developer offered to 'help' the maintainer of an NPM module - and then slowly introduced malicious code to it - I can't help but wonder if the Drupal community is vulnerable to the exact same issue. Let's discuss!

Please, don't touch my package

NPM modules have been hacked at before, and it's not pretty when it happens. Because of the way we use packages, it's a lot easier for nasty code to get sucked in to a LOT of applications before anyone notices. Attacks on the code 'supply chain', therefore, have tended to be high-profile and high-damage.

NPM is used as a source for a huge number of code projects, many of which use other bits of code from other NPM packages. Even moderate applications for PC, mobile or web can have hundreds or thousands of NPM packages pulled in. It's common for packages to depend on other packages, which depend on other packages, which need other packages, which require... you get the picture? There are so many fragments, layers and extra bits that NPM is used for, that the developers of the applications don't necessarily know all the packages that are being pulled in to their application. It's so easy to just type "npm require somefancypackageineed" without thinking and without vetting. NPM will just go and get everything for you, and you don't need to care.

That's how it should be, right? We should be able to just add code and know that it's safe, right? In a perfect world, that would be fine. But in reality there's an increasingly large amount of trust being given when you add a package to your application, and developers don't realise it. It's events like this that are making people aware again that they are including code in their projects that they either do not scrutinise or do not know exists.

Drupal's moment will come

Fortunately, Drupal is a little different to NPM. Whilst modules are often dependent on other modules, we tend to have a lot less layers going on. It's much easier to know what modules and dependencies you're adding in when you include a new module. But that doesn't mean we're immune.

This particular incident came about when a tired, busy module maintainer was approached and offered help. It's a classic social engineering hack.

"Sure, I'll help you! [mwahaha]"

What struck me was that Drupal probably has hundreds of module maintainers in similar circumstances. Put yourself in those shoes, for a moment:
- You maintain an old Drupal 7 module
- It has a few thousand sites using it still
- You're busy, don't have time for it anymore

If somebody offered to sort it all out for you, what would you say? I'm pretty sure most would be ecstatic! Hurrah! But how would you vet your new favourite person in the whole world, before making them a co-maintainer and giving them the keys to the kingdom?

Alternatively, what of this:
- There is an old module, officially unmaintained
- It still has users
- The maintainer cannot be contacted

Drupal has a system for allowing people to be made maintainers of modules, when the original maintainer cannot be contacted. How are these people vetted? I'm sure there's some sort of check, but what if it's not enough?

In particular, I want to point out that as Drupal 7 ages, there will be more and more old, unmaintained and unloved modules still used by thousands of sites. If we forget them and fail to offer them sufficient protection, they will become vulnerable to attacks just like this. Drupal's moment will come.

This is an open source issue

It would be rather very easy to run away screaming right now, having decided that open source technologies sound too dangerous. So I'll put in some positive notes!

That Drupal should be increasingly exposed to the possibility of social engineering and malevolent maintainers is no new issue. There are millions of open source projects out there, all exposed to exactly these issues. As the internet grows and matures and ages, these issues will become more and more common; how many projects out there have tired and busy maintainers?!

For now, though, it must be said that the open source communities of the world have done what few thought possible. We have millions of projects and developers around the world successfully holding onto their trusty foundations, Drupal included. Many governments, enterprises and organisations have embraced the open source way of working on the premise that although there is risk in working differently, there is great merit in the reward. To this day, open source projects continue to thrive and to challenge the closed-source world. It is the scrutiny and the care of the open source community that keeps it clear and safe. As long as we continue to support and love and use our open source communities and contributions, they will stay in good repair and good stead.

If you were thinking of building a Drupal site and are suddenly now questioning that decision, then a read of Drupal's security statement is probably worthwhile.

Know your cattle by name

The key mitigation for this risk, it should be said, is for developers to know what code is in their application. It's our job to care and so it's our job to be paranoid. But it's not always easy. How many times have you installed a module without checking every line of code? How many times have you updated a module without checking the diff in Git? It's not always practicable to scan thousands and thousands of lines of code, just in case - and you'd hope that it's not necessary - but that doesn't mean it's not a good idea.

Using Composer with Drupal 8 makes installing new modules as easy as using NPM, and exposes the same problems to some extent. Add in a build pipeline, and it's very easy to never even see a single line of the new code that you've added to your project. Am I poking a paranoia nerve, yet? ;)

For further fun, think back to other attacks in the last year where sources for external JS dependencies were poisoned, resulting in compromised sites that didn't have a single shred of compromised code committed - it was all in the browser. How's THAT for scary!

In short, you are at risk if:
- You install a module without checking every line of code
- You update a module without checking every line of code / the diff
- You use a DEV release of a module
- You use composer
- Your application pulls in external dependencies

These actions, these ways of working all create dark corners in which evil code can lie undetected.

The light shall save you

Fortunately, it can easily be argued that Drupal Core is pretty safe from these sorts of issues. Phew. Thanks to the wide community of people contributing and keeping keen eyes on watch, Core code can be considered as well-protected. Under constant scrutiny, there's little that can go wrong. The light keeps the dark corners away.

Contrib land, however, is a little different. The most popular modules not only have maintainers (well done, guys!), but many supporting developers and regular release cycles and even official 'Security Coverage' status. We have brought light and trust to the contrib world, and that's a really important thing.

But what does 'Security Coverage' really provide? Can it fail? What happens if there is a malicious maintainer? I wonder.

When the light goes out

Many modules are starting to see the sun set. As dust gathers on old Drupal 7 modules and abandoned D8 alpha modules, the dark corners will start to appear. 'Security Coverage' status will eventually be dropped, or simply forgotten about, and issue lists will pile up. Away from the safety of strong community, keen eyes and dedicated maintainers, what used to be the pride of the Drupal community will one day become a relic. We must take care to keep pride in our heritage, and not allow it to become a source of danger.

Should a Drupal module maintainer be caught out by a trickster and have their work hacked, what would actually happen? Well, for most old D7 modules we'd probably see a few thousand sites pull in the code without looking, and it would likely take some time for the vulnerability to be noticed, let alone fixed.

Fortunately, most developers need a good reason to upgrade modules, so they won't just pull in a new malicious release straight away. But there's always a way, right? What if the hacker nicely bundled all those issues in the queue into a nice release? Or simply committed some new work to the DEV branch to see who would pull it in? There are loads of old modules still running on dev without an official release. How many of us have used them without pinning to a specific commit?

Vigilance is my middle name!

I have tried to ask a lot of questions, rather than simply doom-mongering. There's not an obvious resolution to all of these questions, and that's OK. Many may argue that, since Drupal has never had an issue like this before, we must already have sufficient measures in place to prevent such a thing happening - and I disagree. As the toolkit used by the world's hackers gets ever larger and ever more complex, we cannot afford to be lax in our perspective on security. We must be vigilant!

Module maintainers, remain vigilant. Ask good questions of new co-maintainers. Check their history. See what they've contributed. Find out who they really are.

Developers, remain vigilant. Know your cattle. Be familiar with what goes in and out of your code. Know where it comes from. Know who wrote it.

Drupalers, ask questions. How can we help module maintainers make good decisions? How can we support good developers and keep out the bad?

Some security tips!
- Always know what code you're adding to your project and whether you choose to trust it
- Drupal projects not covered by the Security Team should be carefully reviewed before use
- Know what changes are being made when performing module updates and upgrades
- If using a DEV version of a module in combination with a build process, always pin to a specific git commit (rather than HEAD), so that you don't pull in new code unknowingly

May 30 2018
May 30

The new GDPR laws are here, hurrah!

Having a number of developers handling databases from a number of client sites could easily be a nightmare, but we at ComputerMinds spent quite some time thinking about how to get and keep everybody safe and squeaky clean on the personal data front.

Here's a quick run-down of the key things to be aware of - and a pretty poster to help you keep it all in mind :)

Remove personal data from your system

  1. Review all databases on your computer, making sure to consider also those .sql dump files still sat in your downloads directory or your Recycle bin/trash.
  2. If there are databases that you need to keep on your system, then you must sanitize them by encrypting, anonymizing or removing personal data.
  3. Review all testing / UAT environments and ensure they're running off sanitized databases where possible.

Stay clean by using sanitized databases

Set up some _drush_sql_sync_sanitize() hooks to deal with personal data stored on your site. Then either have your Jenkins server use it to provide sanitized dumps, or ensure that your developers use it to sanitize databases immediately after importing.

When setting up your hook, make sure to consider things like:

  • User table - clear out email addresses, usernames etc.
  • Custom fields on users - names, telephone numbers etc. that you've added.
  • Webform / contact form submissions - make sure that your Webform / contact form data gets cleared out. Webform 7.12 and above has these hooks included, but it's good to double-check.
  • Commerce order table - you'll need to remove personal data from the commerce orders.
  • Commerce profile tables - make sure that the personal data in the profiles gets anonymized or removed.
  • Commerce payment gateway callback tables - these will have detailed payment transaction data, and absolutely must be cleared out.
  • URL aliases & redirects - by default Drupal sets up aliases for users' usernames, so you'll need to review those tables.
  • Comments - these usually have name, email and website fields that will need clearing out. But their body content may also have personal data in too, so you might be better off just binning the lot.
  • Watchdog / logging tables - these take up lots of space, so you probably don't want to export them off the live site anyway, but think seriously about the personal data inside if you do decide you want to import them elsewhere. Truncate recommended.
  • Cache tables - these can be huge, so you probably don't want to export them off the live site anyway, but think seriously about the personal data inside if you do decide you want to import them elsewhere. Truncate recommended.

This is certainly not a complete list, but we can't tell you what custom fun you've implemented on your site - so its' down to you to go check your tables!

Stay vigilant

  • Ensure future development environments and UAT/test environments are built using sanitized databases.
  • If you receive user data via email, immediately delete the email and attachments and reprimand the sender!
  • Talk to your clients about changes that need to be made to their sites.

The "GDPR and You" poster for developers, by ComputerMindsPDF download link below!

Jan 12 2016
Jan 12

The preview functionality in action - everything works nicely inside our iFrame.It's a well known fact that a large proportion of sites turn off the node preview button. It just doesn't work well. Your unsaved changes don't show and the admin theme is used in place of your site theme - not helpful.

The default Drupal node preview.

Workflows like that provided by Workbench Moderation give previews that work nicely, but that involves a lot of complexity with revisions and workflow. It really changes the way you (and your nodes, for that matter) work. For the sites out there that don't have something like Workbench Moderation in place, this usually means that there is no accurate preview functionality. End of.

So, when a couple of weeks ago the marketing team for Coventry Motofest asked if we could make the page preview work, we were surprised to find that we'd NOT turned off node preview for pages!

Rather than shy away, we took some time to see what we could do. What we've ended up with is a really neat working preview (complete with unsaved changes), that shows in the actual site theme (not the admin theme) and even can show at different (admin defined) viewport sizes. There are still some tweaks that could be made, but for a first go with *no additional custom code required* (just a cheeky patch - details coming) it's pretty darn good!

How it works

The mainstay of the functionality - the preview - is provided by [Page Preview](https://www.drupal.org/project/pagepreview). It's a recent addition to the Drupal world and is still in Alpha, but a quick review of the code doesn't raise any alarms. It essentially provides an iFrame, which, complete with changes, shows the node perfectly in your site theme. Amazing.

One thing better than seeing your node, though, is seeing your node as it will be seen by your site visitors. With almost all sites now embracing mobile-first and responsive methodologies in their design practices, it would be really helpful to be able to preview the node at a variety of sizes. Cue Responsive Theme Preview!

Responsive Theme Preview is actually a D8 module with a D7 backport. With all its snazzy complexities we couldn't get its preview to work as we'd like. *However* we were able to take some of the best bits and integrate them with Page Preview. RTP has a nice little UI for creating, managing and selecting viewport sizes for use in the preview, and a neat set of buttons for choosing the viewport size whilst previewing. If you use the D8 Navbar you'll get an even nicer UI via the navbar. We made a patch for Page Preview which pulls these tools into place.

What you get is this - a beautiful node preview, complete with breakpoint buttons :)

Your turn - how to do it

So to summarise: Install pagepreview, with THIS PATCH. Install responsive_preview.  Done.

You can then customise the breakpoints via your new admin page at /admin/config/content/responsive-preview.

Oh, and you'll probably need to  turn preview back on  ;) We'd almost forgotten how - go to /admin/structure/types/manage/ and for each content type hit Edit > Submission form settings > Preview before submitting > Optional / Required. If you've disabled via custom code, then we'll leave that to you.

Improvements

Looking forward, it would be really nice to also have a little grabber, so you can just take your mouse and manually resize the preview to any width you'd like. Then, you could see how your content shows at any width. Watch this space... or if you've a spare moment then beat us to it!The preview functionality in action - everything works nicely inside our iFrame.

Jun 02 2015
Jun 02

We've done a fair bit of reading around lately, thinking through different ideals on content construction. There's a fine balance to be struck between atomic units of content, vs. an enormous amorphous blob of formatted HTML, and articles like this by Wim Leers and sessions like this provide great food for thought.

ComputerMinds recently sponsored the site for Coventry Motofest, and I was lucky enough to be part of the team. We've done a few sites lately with our bespoke block-based system, but we wanted this site to be a bit simpler. Different content types would still be needed, as well as control for content sizing at different breakpoints, but we wanted more simplicity throughout - our block system can get a bit fiddly at times. We decided to take a foray with the Paragraphs module, and what an adventure it's been!

What's a Paragraph?

The Paragraphs module works on the principle of breaking content into different chunks, named paragraphs. Paragraphs don't have to be your conventional text paragraphs as you would normally know them, however - they can be text, images, slideshows, tabs… anything you want! This means that putting pages together is as simple as clicking the button for the new paragraph type you want, and then putting in your text / images / options.

Paragraphs are essentially field collections, which makes them a fairly familiar platform to work with. You create different Paragraph Bundles (types of paragraph), which can be individually templated, and add exactly the fields you need. Once you've added a Paragraphs field to your content type(s), you can then define which Paragraph Bundles can be used on that content type.

Paragraphs in action

We created paragraphs for media (text and images), maps, partner ads, views, social media, events and Webforms, and these power all the content on the site. Take a look at the site - each chunk of content is a paragraph.

image

I've highlighted some here - There are two media paragraphs (orange, blue), a paragraph that shows items from a view (red) and a MailChimp signup form paragraph.

Paragraphs go Mobile

You can't develop a site in 2015 and not think very carefully about how it works on mobile. Following our mobile-first strategy, we needed to be able to choose how wide the paragraphs were to be at different viewport breakpoints. We added a field on each paragraph bundle, which defines sizing instructions for the paragraph to be passed through the theme layer.

Our defaults provided full-width paragraphs at the 'mobile' breakpoint upwards - but more complex defaults are possible. With this field in action, we're able to define widths of 1/4, 1/3, 1/2, 2/3, 3/4 and full-width, enabling some really nice layouts.

It's even possible to target content at a particular breakpoint - over the MotoFest weekend, we were able to add content to the home page which targeted only mobile users. By adding a map to the home page, we made the site just a little bit quicker and easier for mobile viewers - just what you need during a city-wide event with 100,000 people in attendance!

image

Making the editorial role easier

One of our goals was to keep the editorial role as simple as possible - if somebody just wants to change some text, that should be really really easy. A great way to enable fast edits is to have an 'Edit' contextual link on each paragraph, which means you don't need to return to the rather large node edit form. Instead, you get an edit page just for that paragraph. James Williams made a patch which made this possible, and hopefully this'll make its way into a future module update.

image

Because they're rectangular and have grey in between, we've called our paragraphs Bricks

The ultimate standard in editorial ease would be same-page editing. We had a quick play with QuickEdit, but it only works with basic fields at the moment and it looks like a fair bit of work to make it play nicely with Paragraphs right now.

What we love about Paragraphs

Paragraphs makes content creation really nice. You can add lots of different chunks of content to a page, lay them out how you want to, reorder and reposition them, and easily update the template files to get everything exactly how you want to. Changing the look and feel of a page takes just a few quick changes.

This fits in nicely with this site's model, where different pages have radically different content, and the size of the site doesn't make it worth restricting control of layout and formatting. Our editors can make style and formatting choices, and that works well for this scenario.

We've enjoyed this first adventure with Paragraphs, and have had fun trying out different layouts and formats. Even better, it'll be easy enough to copy this functionality on future sites (we're already porting some of these elements to a number of our clients' sites, and they love it too) and continue developing it.

Developer

Nathan is a recent Masters of Computer Science graduate from the University of Warwick, and has lots of little chunks of experience in a variety of languages and arenas. He is currently the main developer for our automated testing ambitions.

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web