Aug 23 2018
Aug 23

[embedded content]

Drupal Console and Drush are two command line (CLI) tools built for Drupal. For a long time Drush was the only CLI tool and it was very useful for managing Drupal sites. Common tasks you’d do with Drush are rebuild caching, installing sites, import/export configuration and so much more.

Then Drupal Console came onto the scene and offered other goodies such as the ability to generate boilerplate code, which Drush 9 can now do as well. People often ask “Can you run Drush and Drupal Console together” and the answer is yes, I personally use both. If you install Drupal using drupal-composer/drupal-project then you get both Drush and Drupal

In the video above, you’ll learn how to use Drush and Drupal Console.

Drupal Console

Drupal Console is a CLI tool for Drupal built using the Symfony Console library. One of its many strengths is the ability to generate boilerplate code. With a few simple commands you can create a module or a custom entity type.

How to Install Drupal Console

Drupal Console can be installed into your Drupal site using Composer, follow the instructions over on the Drupal Console documentation page.

Use the links below to jump to a section of the video:

  • Drupal Console intro: 02:31
  • How to install Drupal Console: 04:29
  • Using Drupal Console launcher: 06:27
  • Overview of Drupal Console commands: 07:41
  • How to use debug commands: 08:33
  • Using debug:router command: 10:11
  • Using debug:container command: 11:49
  • Using debug:cron command: 15:08
  • Using debug:user command: 15:08
  • How to use generate commands: 17:15
  • Using generate:module command: 18:07
  • View module code generated by Drupal Console: 20:42
  • Using generate:entity:content command: 20:59
  • View code generated by generate:entity:content command: 24:43
  • Install module generated by Drupal Console: 27:21
  • How to download modules using Drupal Console: 31:10
  • Viewer question: What’s composer?: 34:45

Drush

Drush is the original CLI tool for Drupal. You can use it to interact with a Drupal site and generate boilerplate code.

How to Install Drush

Drush is just a PHP library and can be installed using Composer. For details read the Drush install documentation page.

Use the links below to jump to a section of the video:

  • Drush intro: 37:22
  • How to install Drush: 40:56
  • Using Drush: 42:11
  • Overview of Drush Commands: 43:00
  • Using Drush generate command: 44:11
  • Using generate module-content-entity command: 45:12
  • View module code generated by Drush: 46:23
  • Install module generated by Drush: 47:13
  • Overview of common Drush commands: 52:00
Ivan Zugec

About Ivan Zugec

Ivan is the founder of Web Wash and spends most of his time consulting and writing about Drupal. He's been working with Drupal for 10 years and has successfully completed several large Drupal projects in Australia.

Oct 13 2017
Oct 13
October 13th, 2017

Welcome to the second episode in our new video series for Emulsify. Emulsify 2.x is a new release that embodies our commitment to component-driven design within Drupal. We’ve added Composer and Drush support, as well as open-source Twig functions and many other changes to increase ease-of-use.

In this video, we’re going to teach you how to create an Emulsify 2.0 starter kit with Drush. This blog post follows the video closely, so you can skip ahead or repeat sections in the video by referring to the timestamps for each section.

PURPOSE [00:15]

This screencast will specifically cover the Emulsify Drush command. The command’s purpose is to setup a new copy of the Emulsify theme.

Note: I used the word “copy” here and not “subtheme” intentionally. This is because the subtheme of your new copy is Drupal Core’s Stable theme, NOT Emulsify.

This new copy of Emulsify will use the human-readable name that your provide, and will build the necessary structure to get you on your way to developing a custom theme.

REQUIREMENTS [00:45]

Before we dig in too deep I recommend that you have the following installed first:

  • a Drupal 8 Core installation
  • the Drush CLI command at least major version 8
  • Node.js preferably the latest stable version
  • a working copy of the Emulsify demo theme 2.X or greater

If you haven’t already watched the Emulsify 2.0 composer install presentation, please stop this video and go watch that one.

Note: If you aren’t already using Drush 9 you should consider upgrading as soon as possible because the next minor version release of Drupal Core 8.4.0 is only going to work with Drush 9 or greater.

RECOMMENDATIONS [01:33]

We recommend that you use PHP7 or greater as you get some massive performance improvements for a very little amount of work.

We also recommend that you use composer to install Drupal and Emulsify. In fact, if you didn’t use Composer to install Emulsify—or at least run Composer install inside of Emulsify—you will get errors. You will also notice errors if npm install failed on the Emulsify demo theme installation.

AGENDA [02:06]

Now that we have everything setup and ready to go, this presentation will first discuss the theory behind the Drush script. Then we will show what you should expect if the installation was successful. After that I will give you some links to additional resources.

BACKGROUND [02:25]

The general idea of the command is that it creates a new theme from Emulsify’s files but is actually based on Drupal Core’s Stable theme. Once you have run the command, the demo Emulsify theme is no longer required and you can uninstall it from your Drupal codebase.

WHEN, WHERE, and WHY? [02:44]

WHEN: You should run this command before writing any custom code but after your Drupal 8 site is working and Emulsify has been installed (via Composer).

WHERE: You should run the command from the Drupal root or use a Drush alias.

WHY: Why you should NOT edit the Emulsify theme’s files. If you installed Emulsify the recommended way (via Composer), next time you run composer update ALL of your custom code changes will be wiped out. If this happens I really hope you are using version control.

HOW TO USE THE COMMAND? [03:24]

Arguments:

Well first it requires a single argument, the human-readable name. This name can contain spaces and capital letters.

Options:

The command has defaults set for options that you can override.

This first is the theme description which will appear within Drupal and in your .info file.

The second is the machine-name; this is the option that allows you to pick the directory name and the machine name as it appears within Drupal.

The third option is the path; this is the path that your theme will be installed to, it defaults to “themes/custom” but if you don’t like that you can change it to any directory relative to your web root.

Fourth and final option is the slim option. This allows advanced users who don’t need demo content or don’t want anything but the bare minimum required to create a new theme.

Note:

Only the human_readable_name is required, options don’t have to appear in any order, don’t have to appear at all, or you can only pass one if you just want to change one of the defaults.

SUCCESS [04:52]

If your new theme was successfully created you should see the successful output message. In the example below I used the slim option because it is a bit faster to run but again this is an option and is NOT required.

The success message contains information you may find helpful, including the name of the theme that was created, the path where it was installed, and the next required step for setup.

THEME SETUP [05:25]

Setting up your custom theme. Navigate to your custom theme on the command line. Type the yarn and watch as pattern lab is downloaded and installed. If the installation was successful you should see a pattern lab successful message and your theme should now be visible within Drupal.

COMPILING YOUR STYLE GUIDE [05:51]

Now that we have pattern lab successfully installed and you committed it to you version control system, you are probably eager to use it. Emulsify uses npm scripts to setup a local pattern lab instance for display of your style guide.

The script you are interested in is yarn start. Run this command for all of your local development. You do NOT have to have a working Drupal installation at this point to do development on your components.

If you need a designer who isn’t familiar with Drupal to make some tweaks, you will only have to give them your code base, have them use yarn to install, and yarn start to see your style guide.

It is however recommended the initial setup of your components is done by someone with background knowledge of Drupal templates and themes as the variables passed to each component will be different for each Drupal template.

For more information on components and templates keep an eye out for our soon to come demo components and screencasts on building components.

VIEWING YOUR STYLE GUIDE [07:05]

Now that you have run yarn start you can open your browser and navigate to the localhost URL that appears in your console. If you get an error here you might already have something running on port 3000. If you need to cancel this script hit control + c.

ADDITIONAL RESOURCES [07:24]

Thank you for watching today’s screencast, we hope you found this presentation informative and enjoy working with Emulsify 2.0. If you would like to search for some additional resources you can go to emulsify.info or github.com/fourkitchens/emulsify.

[embedded content]

Thanks for following our Emulsify 2.x tutorials. Miss a post? Read the full series is here.

Pt 1: Installing Emulsify | Pt 2: Creating your Emulsify 2.0 Starter Kit with Drush | Pt 3: BEM Twig Function | Pt 4: DRY Twig Approach | Pt 5: Building a Full Site Header in Drupal

Just need the videos? Watch them all on our channel.

Download Emulsify

Web Chef Chris Martin
Chris Martin

Chris Martin is a support engineer at Four Kitchens. When not maintaining websites he can be found building drones, computers, robots, and occasionally traveling to China.

Aug 02 2017
ao2
Aug 02

Drupal 8 is quite an improvement over previous versions of Drupal with regard to the reproducibility of a site from a minimal source repository.

The recommended way to set up a new Drupal 8 site is to use composer, and the most convenient way to do that is via drupal-composer/drupal-project, which brings in Drush and drupal-console as per-site dependencies; from there the site can be installed and managed.

However the fact that Drush and drupal-console are per-site dependencies poses a problem: they are not available until the site dependencies have been downloaded, this means that it's not really possible to add “bootstrapping” commands to them to make it easier to set up new projects.

This is what motivated me to put together drupal-init-tools: a minimalistic set of wrapper scripts which can be used when Drush and drupal-console are not available yet, or when they have to be combined together to perform a task in a better way.

drupal-init-tools has been developed mainly to speed up prototyping, and it is particularly useful to set up sites under ~/public_html when the web server is using something like the Apache userdir module; however I feel it could be useful in production too after some cleanups.

drupal-init-tools standardizes the setup and installation of new Drupal projects: for example the drin new command makes sure that the original drupal-composer/drupal-project repository is still accessible from an upstream git remote so that the developer can easily track changes in drupal-composer/drupal-project and keep up if appropriate for their particular project.

Some drin commands also provide a nicer interface for frequent and important tasks, like the actual site-install step, or the creation of an installation profile that could replicate the installed site.

Here are some examples of use taken from the drin manual.

Create and install a new Drupal project:

cd ~/public_html
drin new drupal_test_site
cd drupal_test_site
$EDITOR bootstrap.conf
drin bootstrap --devel

Create an installation profile from the currently installed project:

drin create-profile "Test Profile" test_profile

Clean and rebuild the whole project to verify that installing from scratch works:

drin clean
drin bootstrap

A quick way to test drupal-init-tools is this:

git clone git://git.ao2.it/drupal-init-tools.git
cd drupal-init-tools/
make local
./drin --help

Give it a go and let me know what you think.

If it proves useful to others I could have it packaged and uploaded to Debian to make it more “official”, IMHO it makes sense to have such a small meta-tool like drin as a system global command.

Jun 12 2017
ao2
Jun 12

Most of the information I have come across about migrating from Drupal 6 to Drupal 8 is about migrating content, however before tackling this problem another one must be solved, maybe it is obvious and hence understated, so let's spell it out loud: preserving the site functionality.

That means checking if the contrib modules need to be ported to Drupal 8, and also checking if the solution used in the previous version of the site can be replaced with a completely different approach in Drupal 8.

Let's take ao2.it as a study case.

When I set up ao2.it back in 2009 I was new to Drupal, I choose it mainly to have a peek at the state of Open Source web platforms.

Bottom line, I ended up using many quick and dirty hacks just to get the blog up and running: local core patches, theme hacks to solve functional problems, and so on.

Moving to Drupal 8 is an opportunity to do things properly and finally pay some technical debt.

For a moment I had even thought about moving away from Drupal completely and use a solution more suited to my usual technical taste (I have a background in C libraries and linux kernel programming) like having the content in git and generate static web pages, but once again I didn't want to miss out on what web frameworks are up to these days, so here I am again getting my hands dirty with this little over-engineered personal Drupal blog, hoping that this time I can at least make it a reproducible little over-engineered personal Drupal blog.

In this series of blog posts I'll try to explain the choices I made when I set up the Drupal 6 blog and how I am re-evaluating them for the migration to Drupal 8.

The front page view

ao2.it was also an experiment about a multi-language blog, but I never intended to translate every content, so it was always a place where some articles would be in English, some in Italian, and the general pages would be actually multi-language.

This posed a problem about what to show on the front page:

  • If every node was shown, there would be duplicates for translated nodes, which can be confusing.
  • If only nodes in the current interface language were shown, the front page would list completely different content across languages, which does not represent the timeline of the blog content.

So a criterion for a front page of a partially multi-lingual site could be something like the following:

  • If a node has a translation in the current interface language, show that;
  • if not, show the original translation.

The “Select translation” module

In Drupal 6 I used the Select translation module which worked fine, but It was not available for Drupal 8.

So I asked the maintainers if they could give me the permission to commit changes to the git repository and I started working on the port myself.

The major problem I had to deal with was that Drupal 6 approached the multi-language problem using by default the mechanism called "Content translations" where separate nodes represented different translations (i.e. different rows in the node table each with its own nid), tied together by a tid field (translation id): different nodes with the same tid are translations of the same content.

Drupal 8 instead works with "Entity translations", so one single node represents all of its translations and is listed only once in the node table, and actual translations are handled at the entity field level in the node_filed_data table.

So the SQL query in Select translation needed to be adjusted to work on the node_filed_data rather than of the node table, as it can be seen in commit 12f70c9bb37c.

While at it I also took the chance to refactor and clean up the code, adding a drush command to test the functionality from the command line.

The code looks better structured thanks to the Plugin infrastructure and now I trust it a little more.

Preserve language

On ao2.it I also played with the conceptual difference between the “Interface language” and the “Content language” but Drupal 6 did not have a clean mechanism to differentiate between the two.

So I used the Preserve language module to be able to only switch the interface language when the language prefix in the URL changed.

It turns out that an external module is not needed anymore for that because in Drupal 8 there can be separate language switchers, one for the interface language and one for the content language.

However there are still some issues about the interaction between them, like reported in Issue #2864055: LanguageNegotiationContentEntity: don't break interface language switcher links, feel free to take a look and comment on possible solutions.

More details about the content language selection in a future blog post.

Jan 29 2017
Jan 29
Drush stands for "Drupal shell" which means a powerful tool for managing Drupal installation from command line interface. Drush provides a lot of useful commands for dealing with a cache, modules, cron, database etc. But some of contrib modules also provide some extra drush commands for specific functionality (like features module comes with commands for managing features). Here's a bunch of a useful drush commands which I use every day.


Cache

# Clear all caches.
drush cc all

# Clear menu cache.
drush cc menu

# Select cache bin for clearing.
drush cc

Modules

# Download module by name. You can grab module
# name from module's page on drupal.org.
drush dl module_name

# Download module by name.
# into sites/default/modules/custom
# directory.
drush dl module_name --destination=sites/default/modules/custom

# Enable module by name.
drush en module_name

# Disable module by name.
drush dis module_name

# Uninstall module by name.
# This commands will call hook_name_uninstall().
drush pmu module_name

# Get list of available modules
# in your Drupal installation.
drush pml

# Check if module module_name
# is available in your Drupal
# installation.
drush pml | grep module_name

User

# Get an one-time login link
# for user by his/her uid.
drush uli uid

# Get an one-time login link
# for user by his/her email.
drush uli [email protected]

# Block user by his/her username.
drush ublk test_user

# Unblock user by his/her username.
drush uublk test_user

# Change password for user by his/her name.
drush upwd --password="password" user_name

Pending database updates

# Run all available database updates
# implemented in hook_update_N() in
# *.install files. 
drush updb

Cron

# Run cron.php. It will call all hook_cron()
# defined in modules and process cron queues.
drush cron

Variables

# Get variable value by name.
drush vget variable_name

# Set variable value by name.
drush vset variable_name "Test value"

# Delete variable by name.
drush vdel variable_name

Registry

# Install registry_rebuild drush extension.
drush dl registry_rebuild

# Rebuild Drupal registry. Usefull when some of
# modules was moved into another folder.
drush rr

Features

# Get list of available features.
drush fl

# Get list of overridden features. Please
# note if you have multilingual site and
# site active language is not english then
# you have to replace word "Overridden" by
# it's translation.
drush fl | grep Overridden

# Show diff of overridden feature by name.
drush fd feature_name

# Update feature by name. Put changes from
# database into a code.
drush fu feature_name

# Revert feature by name. Restore database
# state from a code.
drush fr feature_name

Sql interface

# Show information about current database connection
# that defined in settings.php file.
drush sql-connect

# Login into sql console interface.
drush sqlc

# Execute a query.
drush sqlq "SELECT * from node;"

# Restore database dump from dump_name.sql file.
drush sqlc < dump_name.sql

# Save database dump into dump_name.sql.gz file.
drush sql-dump --gzip --result-file=dump_name.sql.gz

Other

# Run custom php code.
drush eval "variable_set('site_name', 'Test site name');"

# Get all dblog messages in realtime (tail style).
# Useful when you want to debug a remote Drupal
# installation where you have ssh access.
drush ws --tail

Key notes:

Jan 07 2017
Jan 07
If you have redirect functionality implemented with drupal_goto() function on hook_init() probably you aren't able to run drush commands because they crashe with a message:
Drush command terminated abnormally due to an unrecoverable error.  [error]
It happens because drush bootstraps Drupal application but can't perform redirect. To avoid this you need to check whether script runs from cli or not:
<?php

/**
 * Implements hook_init().
 */
function module_init() {
  if (!drupal_is_cli()) {
    drupal_goto('some/path');
  }
}

Key notes:

Aug 23 2016
Aug 23

Now we've got the experience of a number of production D8 sites under our belt we took the time to consolidate our CMI workflow into some useful drush commands.

And naturally we've open sourced them.

Read on to find out more about our drush CMI tools.

Use case

Say you're working on a local development environment for a project where the client is adding and editing configuration. For example, the project might be using contact-forms or Yaml forms for user interaction. Each of these and their associated fields, form and view displays are a config object. As the client is editing these, you don't want that configuration tracked in your source control.

So you start working on a new feature, the first thing you do is sync down a QA or production database and then you run your config import so that your local environment is in a clean state.

You work on some features and the time has come to export those changes to your config export folder, in order to check in the new work into git ready for deployment.

Enter drush cexy

drush cexy

Its like drush cex but with some powersauce.

So the normal drush cex command comes with a --skip-modules option that prevents configuration from say devel module from being exported. But let's go back to our original use case.

We want to export all configuration, but we want to exclude certain patterns.

This is where the --ignore-list option of drush cexy comes in.

In our project we have a ./drush folder, so we stick a file in their called config-ignore.yml with contents as follows.

ignore:
  - field.field.contact_message.*
  - field.storage.contact_message.*
  - contact.form.*
  - core.entity_form_display.contact_message*
  - core.entity_form_display.contact_form*
  - core.entity_view_display.contact_message*
  - core.entity_view_display.contact_form*
  - system.site
  - workbench_email.workbench_email_template.*

You'll note there are some wildcards there. We're ignoring all contact message fields and forms as well as any form or view display configuration. Additionally we're ignoring Workbench Email templates and the system site settings.

So now we run drush cexy like so

drush cexy --destination=/path/to/config-export --ignore-list=/path/to/drush/config-ignore.yml

So what this does is export the active configuration, and then apply the ignore list to remove unwanted configuration.

So now when you run git status you should only see changes you want to commit.

Single install configuration

So lets assume you're working on a feature branch that requires installation of the Google Analytics module.

You download and enable the module

drush dl google_analytics
drush en -y google_analytics

And then you export your configuration with drush cexy using your build tool of choice (make in this case - cause remembering all the flags is bound to go wrong)

make export

After running that you find you have a new file in your exported configuration folder:

google_analytics.settings

Now you know that you want this configuration to be editable by the client, as they'll have different GA urchin codes on different environments.

So you don't want to check this into git. But, you do need it to be deployed at least once. And that's where drush cexy's sibling comes in drush cimy.

drush cimy

drush cimy is the import equivalent of drush cexy. We've found it significantly increases our CMI workflow productivity.

So returning to our single install of the google analytics settings. You'd just exported your config using drush cexy and found yourself with a new google_analytics.settings.yml file that you needed to deploy, but only once.

drush cimy combines the following features

  • The power of drush cim --partial
  • The ability to perform config deletes
  • The ability to perform one-time installs

The format is as follows

drush cimy --source=/path/to/config-export --install=/path/to/config-install --delete-list=/path/to/config-delete.yml

So we move the google_analytics.settings.yml out of our config-export folder and into our config-install folder. And then we add it to our drush/config-ignore.yml file, so it doesn't get exported in the future.

Partial imports

So as alluded above, drush cimy is similar to drush cim --partial in that it does partial imports.

The way drush cim --partial works is equivalent to the following

  • firstly it creates a temporary folder
  • then it exports all active configuration
  • then it copies your nominated config-export folder (the one under source control) over the top (in the temporary folder)
  • then it imports from the temporary folder

So what you get imported is all active config plus and new config from the config export, with changes in the exported config taking precedence over the active config.

The main pain point with using --partial is you don't get config deletes.

e.g. if you delete a config file from git (it is no longer in your config-export folder) because it is still present in the active configuration, it still remains after import.

So why is this a problem. So let's consider a scenario where someone enabled dblog module on QA, and saved the settings so that dblog.settings.yml is in the active configuration.

Your core.extensions.yml that is tracked in git does not contain dblog module. But dblog.settings.yml depends on dblog module.

So you work away on your feature and go to deploy to QA. But the import step of your deployment automation fails, because --partial places a copy of dblog.settings.yml in the temporary folder and tries to import it, but because dblog module is going to be disabled by the import, you have an unmet config dependency.

This is where the --delete-list flag kicks in. Let's look at a sample delete list file

delete:
  - dblog.settings

So this is where drush cimy varies from drush cim, its (equivalent) logic is as follows

  • As with drush cim --partial, first it creates a temporary folder
  • Move one-time install configuration into the folder first - so that active configuration takes precendence over initial state
  • export all active configuration
  • delete any configuration found in active configuration that is listed in the delete list
  • copy the nominated config-export (tracked in source control) over the top, taking final precendence
  • import the result

So this means you get the good bits of partial imports, without the dependency dramas that can result. It also allows you to perform valid deletes, something that isn't possible with drush cim --partial - for example you might want to delete a field from active configuration. Previously you'd have to write an update hook to do that before you performed your config import. Now you just list it in the config-delete.yml file

Installation

cd ~/.drush
wget https://raw.githubusercontent.com/previousnext/drush_cmi_tools/8.x-1.x/drush_cmi_tools.drush.inc
drush cc drush
drush CMI Drupal 8
Aug 04 2016
Aug 04

When you use Drush, especially in crontabs, you may sometimes be bitten by RAM or duration limits. Of course, running Drush with the "-d" option will provide this information, but it will only do so at the end of an annoyingly noisy output debugging the whole command run.

On the other hand, just running the Drush command within a time command won't provide fine memory reporting. Luckily Drush implements hooks to make acquiring this information easily, so here is a small gist you can use as a standalone Drush plugin or add to a module of your own:

Aug 04 2016
Aug 04

When you use Drush, especially in crontabs, you may sometimes be bitten by RAM or duration limits. Of course, running Drush with the "-d" option will provide this information, but it will only do so at the end of an annoyingly noisy output debugging the whole command run.

On the other hand, just running the Drush command within a time command won't provide fine memory reporting. Luckily Drush implements hooks to make acquiring this information easily, so here is a small gist you can use as a standalone Drush plugin or add to a module of your own:

Jan 31 2016
Jan 31

Here is a complete guide to get your drush working OS X El Capitan.

1) Download latest stable release using the code below or browse to github.com/drush-ops/drush/releases.

wget http://files.drush.org/drush.phar

(Or use our upcoming release: wget http://files.drush.org/drush-unstable.phar)

2) Test your install.

php drush.phar core-status

3) Rename to `drush` instead of `php drush.phar`. Destination can be anywhere on $PATH.

chmod +x drush.phar
sudo mv drush.phar /usr/local/bin/drush

4) Enrich the bash startup file with completion and aliases.

drush init

5) Add the following lines to .bashrc. (Check which PHP version you are using!)

export MAMP_PHP=/Applications/MAMP/bin/php/php5.6.10/bin
export PATH="$MAMP_PHP:$PATH"
export PATH=$PATH:/Applications/MAMP/Library/bin
export PHP_OPTIONS='-d memory_limit="512M"'

6) Add the following line to .bash_profie

if [ -f ~/.bashrc ]; then . ~/.bashrc; fi

That’s it, you will have a fully functional drush on your Macintosh.

Jan 04 2016
Jan 04

Introduction

What if we had a Drupal theme that does exactly what we want? No more divs, in divs, in divs. No more weird css. Out of the box responsiveness, both adaptive as fluid. Debuggers, good javascript additions, Gulp. Maybe even a little styleguide generator.

That’s what we thought when we started working on our own custom Drupal starter theme in July 2013. We called it “FortyTwo”.

A brief history

Before we we started working with FortyTwo, we did theming, like many others, based on the Zen or Omega base themes. It worked, it got us where we wanted, but it had its downsides. Besides Zen we also looked at Omega and other major base themes. They all had their pros but also a lot of cons.

During a talk I had with one of the frontenders at Finalist about this we started to think: Why not create our own theme. Let’s combine all of the pros from all those themes to one, and maybe add some other nifty features.

The start

The first ever site we developed with FortyTwo, and the site that actually developed a big part of FortyTwo was the Finalist website. Back then it was a good starting point for us to develop website frontends the way we wanted to. Combined with good backend developers we could start working on good frontends.

Finalist.nl websiteThe Finalist website as built with FortyTwo

Because we developed FortyTwo as a base theme for Drupal we could use it to create all kinds of different websites, no matter the shape or size.

Features

Let me enumerate some of the best features FortyTwo has right now:

  • SASS; FortyTwo has completely been made using the SASS css preprocessor.
  • Drush integration; Easily create your sub theme using our custom drush command.
  • Out of the box responsiveness; Without no effort themes developed using FortyTwo are responsive. Both adaptive or fluid. Extending the responsiveness is easy by just copying some files. Want to change the breakpoints? They are defined in the sass settings file.
  • Gulp; We have added gulp support for some extra features.
    • SASS watch and compiling with auto prefixes for IE8 and IE9.
    • JSLint integration, while watching the theme folder the console will output errors or warnings from JSLint if there are any.
    • Uglify of JavaScript files.
    • Clearing of Drupal cache on changes in PHP, include or info files in the theme.
    • And last, but not least automatic reloading of the browser on file changes.
  • Icomoon integration; For easy development we have created a starter set of icomoon icons and added them to the theme. You can easily extend those icons and use theme using our custom icomoon SASS mixin.
  • Layout styles; It is possible to choose different styles of layout, whether you want your content column on the left, right or the middle, it is just a setting;
  • Debuggers; We have added some handy debuggers in FortyTwo:
    • It is possible to show the grid on the background of the page. The grid uses the settings set in the sass settings file.
    • It is possible to show a responsive identifier. This makes it easier to create responsive websites. A bar at the bottom of the website shows you on what breakpoint you currently are viewing.

Debuggers in actionThe above image shows the mentioned debuggers in action.

There is more, but you have to see for yourselves what FortyTwo can offer you!

Drupal 8

Since August 2015 we have also started developing the Drupal 8 version of the theme. Since Drupal 8 is becoming the new standard we wanted to be ready for it when it came out. On the date Drupal 8 was officially released FortyTwo was already finished and is now in active development.

I will write a separate blog about the porting of FortyTwo from Drupal 7 to Drupal 8 on a later date.

How about the future?

We are still in active development. Since the release of FortyTwo-8.x-1.0 Only bugfixes and really nice features are added to the Drupal 7 version of FortyTwo. Other new features are only added for Drupal 8.

On the roadmap are:

  • KSS; Knyle Style Sheets integration. Using KSS you can automatically create style guides from CSS and SASS.
  • FortyTwo admin theme; Admin themes are always difficult to pick. Because we work with a lot of clients, different kinds of content and content managers we believe that we can create a admin theme (of course based on FortyTwo) that suits the needs of everybody.
  • FortyTwo example theme; Since the start of development we always wanted to create a example theme that everybody can use, out-of-the-box.

Get involved!

Yes, we would love to see you around on the project page and the issue queue ! We heavily depend on your input, testing and feedback, but also on your commitment as a community developer, by helping us implementing new features and fixes.

Oct 23 2015
Oct 23

One of the big trends during the Drupal 8 creation has been the replacement of info hooks by two main mechanisms: annotations, and YAML files. In that light, hook_drush_command(), as the CLI equivalent of the late hook_menu, replaced by various YAML files, looks just like a perfect candidate for replacement by a commands section in some mymodule.drush.yml configuration file. Turns out it is incredibly easy to achieve. Let's see how to kill some hundred lines of code real fast !

The goal

Just like its hook_menu() web counterpart, hook_drush_command() defines controllers, the main difference being that it maps them to CLI commands instead of web paths. Its structure is that of a very basic info hook: a nested array of strings and numbers, which maps neatly to a YAML structure. Consider the canonical sandwich example from Drush docs.

hook_drush_command mymodule.drush.yml <?php
function sandwich_drush_command() {
 
$items = array(); // The 'make-me-a-sandwich' command.
 
$items['make-me-a-sandwich'] = array(
   
'description' => "Makes a delicious sandwich.",
   
'arguments' => array(
     
'filling' => 'The type of the sandwich (turkey, cheese, etc.). Defaults to ascii.',
    ),
   
'options' => array(
     
'spreads' => array(
       
'description' => 'Comma delimited list of spreads.',
       
'example-value' => 'mayonnaise,mustard',
      ),
    ),
   
'examples' => array(
     
'drush mmas turkey --spreads=ketchup,mustard' => 'Make a terrible-tasting sandwich that is lacking in pickles.',
    ),
   
'aliases' => array('mmas'),
   
// No bootstrap at all.
   
'bootstrap' => DRUSH_BOOTSTRAP_NONE,
  );
   
?>
commands:
  # The 'make-me-a-sandwich' command.
  make-me-a-sandwich:
    description' => "Makes a delicious sandwich."
    arguments:
      filling: 'The type of the sandwich (turkey, cheese, etc.). Defaults to ascii.'
    options:
      spreads:
        description: 'Comma delimited list of spreads.'
        example-value: 'mayonnaise,mustard'
    examples:
      'drush mmas turkey --spreads=ketchup,mustard': 'Make a terrible-tasting sandwich that is lacking in pickles.'
    aliases: ['mmas']
    # No bootstrap at all.
    bootstrap: -1
    

The trick

There is no trick ! Well, almost... assuming that simple format for the mymodule.drush.yml file, all it takes is loading it. This can even be entirely generic, like this : <?php
use Symfony\Component\Yaml\Yaml;/**
* Implements hook_drush_command().
*/
function beanstalkd_drush_command() {
 
$file = preg_replace('/(inc|php)$/', 'yml', __FILE__);
 
$config = Yaml::parse(file_get_contents($file));
 
$items = $config['commands'];
  return
$items;
}
?>

Since your drush plugin for module mymodule is named mymodule.drush.inc (or mymodule.drush.php if you write them like me), the name of the Yaml file can be deduced from the plugin name. And then, the Symfony Yaml component parses it to a plain array matching the expected hook_drush_command() structure.

This is the mechanism used by the Drupal 8 version of the Beanstalkd module.

The limitations

You may have noticed a small limitation : hook_drush_command() implementations may need to use constants like DRUSH_BOOTSTRAP_NONE. With such a limited implementation, you will need to convert the constants to their values from drush/includes/bootstrap.inc. Should the mechanism come to be included in Drush at some point, a more evolved mechanism will likely provide translation for these constants, too.

Oct 05 2015
Oct 05

With Drupal, it’s possible to build very intricate solutions by gluing the right combination of contrib modules together. Of course, there is a downside to relying only on contrib modules, particularly for one-off or highly custom tasks. Bringing in a contrib module means you assume responsibility for feeding and caring for it. If it’s a mature module, like Views, that’s one thing. But if you find yourself looking at a little-used module that hasn’t seen a stable release yet—or worse hasn’t been updated in months—you may be better off rolling your own solution.

Drupal 8 development has shown that PHP itself, and the wider PHP community, already provides ways to solve common tasks. In this post, I’ll show you some core PHP functionality that you may not be aware of; pulling in packages and libraries via Composer is a topic for another day.

In this particular case, I need to export a CSV file showing all nodes added to a site after a specific date. This report may get created just a handful of times. True, I could easily build a View and use Views Data Export to download it as CSV but doing it purely in code has these benefits:

  • No module dependencies. It’s at least one less thing to update and maintain, especially if security issues are found. If data exports in multiple formats were a more import feature across this site, the Views Data Export module would definitely make sense.
  • Easy to deploy. If I make it a view, I have to remember to export it and make sure it’s deployed to all my environments. If someone edits the view on one of them and doesn’t export it, I could lose improvements or fixes. Granted Features can really streamline the process, but in this case it’s not critical, and as you’ll see everything we do will be in code anyway.
  • Integration with Drush. I created a Drush command to run this at the command line. This made it quicker for me to develop since it was easy to run without involving a browser. PHPStorm’s built-in terminal was perfect for this task. Drush commands are also useful for automation and scheduling. If I need, this script can be cron’d to run daily and pipe the output somewhere.
  • Efficient resource usage. Depending on how PHP is configured, if you’re running a command line script you may not have to worry about memory_limit or max_execution_time settings. These can cause your script to terminate unexpectedly when you’re processing a lot of data. In this case, we could be exporting hundreds or thousands of nodes depending on how far back in time we go.

A Minimal module.info File

For this example, I created a simple exporters module and prefixed it with my initials to prevent unexpected naming clashes. Below are the contents of my om_exporters.info file, free of any other module dependencies. I also created an om_exporters.module file just in case but it ended up empty.

name = OM Exporters
description = Exports newest content to CSV
core = 7.x
php = 5.5
version = 7.x-1.0

Creating a Drush Command

Creating a custom Drush command is straightforward. Drush will look for new commands in a file called <module>.drush.inc. In that file, you should have a function named <module>_drush_command(), which implements a hook_Drush_command(). When you add or change commands, you’ll need to clear caches for Drush to know about the changes.

In this case, the function is simple:

function om_exporter_drush_command() {
    $items['export-newest'] = array(
        description' => 'Export Newest Content to CSV',
        'aliases' => ['ex-new'],
        'callback' => 'om_export_newest',
        'arguments' => [
            'date' => '',
        ]
    );

    return $items;
}

As you can see, the key to $items becomes our Drush command export-newest. We can give it a friendly description, a shorter alias (ex-new), specify the function to run in callback, and list any arguments we require.

Now, when you run the Drush command as shown below, whatever function listed as the callback will be invoked and passed the arguments needed.

Drush export-newest 2015-08-01

Validating Dates

The first thing we’ll do is validate the date argument using PHP 5’s DateTime class. If you’re not familiar with it, this class makes it easy to work with Dates and Timezones without a lot of fuss in an object-oriented manner. It’s easier to understand than using older functions like time(), date(), and strtotime().

The code below takes the $date_arg passed in from Drush and makes sure its parsable as a date. If validation fails, our function does not continue.

function om_export_newest($date_arg) {
    // get the default timezone
    $tz = variable_get('date_default_timezone', 'UTC');
    $tz = new \DateTimeZone($tz);
    // First validate date, we assume $date_arg is
    // a string that can be parsed by \DateTime so
    // you have the flexibility to pass
    // in '-1 months' or '28 days ago'
    try {
        $since = new \DateTime($date_arg, $tz);
    } catch (\Exception $ex) {
        watchdog('export-newest',
            Could not parse date:' . $ex->getMessage(),
            null,
            WATCHDOG_CRITICAL);
        return;
    }
    // ..

Output CSV with SPL’s File Object

The Standard PHP Library is a collection of useful classes that’s not as well known as it should be. It’s intended to solve common problems. In this task, I used the \SplFileObject class to work with files as objects. I find it a lot easier than remembering and looking up the different file_* functions, since I get autocompletion in my IDE. For this, we create a file object that writes to STDOUT so that our command will output everything to the terminal or screen. First, we create our SPLFileObject:

// we will use SPL to send our data to STDOUT formatted as CSV
$fout = new \SplFileObject("php://stdout");

Scripts write to STDOUT by default anyway, but to use the built-in fputcsv method, we need to specify it explicitly.

Typically the first line of a CSV file is the header describing the columns that follow. Now, here’s where we use fputcsv(). This method takes in an array and then writes it as a CSV file. The method automatically handles using commas to separate fields, enclosing text fields with quotes, and so on. You can even configure how all that is handled; for example, if you need to use ‘;’ as the separator. See the online documentation for fputcsv for details.

// write our headers
$fout->fputcsv([
    'nid', 'type', 'title', 'date_created', 'path_alias'
]);

Finding Nodes with EntityFieldQuery

Drupal’s native EntityFieldQuery API is a powerful alternative to always relying on Views to create and filter some collection of nodes, users, taxonomy terms, etc. It’s also object-oriented (I keep saying that) and provides a very readable interface for querying Drupal’s databases. It abstracts away the underlying data store for you, so you don’t need to know exactly what tables or fields everything is in. For that same reason, it’s much safer to use than doing a direct db_query().

One thing that is tricky at first is wrapping your head around the terms it uses. Entities have properties that are common to all the entities of that kind. For nodes, these are things like the title, created date, the bundle (content type), the status, and more. If it’s fieldable, it can have fields specific to a bundle. If you need to query on property values, you use propertyCondition. For fields, use fieldCondition. The same pattern holds if you need to sort them by one or the other. To dig deeper, see How to Use EntityFieldQuery.

The code below shows how get all the nodes with a created timestamp greater than the one we pass to our Drush script.

// query for nodes newer than the specified date
$query = $query = new EntityFieldQuery();
$query->entityCondition('entity_type', 'node')
            ->addMetaData('account', user_load(1)) // Run the query as user 1
            ->propertyCondition('created', $created->format('U'), '>')
            ->propertyOrderBy('created', 'ASC');

Iterating Entities with Generators

Generators were introduced in PHP 5.5. They’re a special kind of function that uses the yield keyword to return an item that’s part of some collection. In practice, they’re a simple way to build a function that iterates over a collection without having to build the whole collection in memory first.

In the code excerpt below you’ll see how we loop through the nodes returned by our EntityFieldQuery. We do this in a generator—notice the yield keyword.

$result = $query->execute();
if (!empty($result)) {
    // $result is an array with node ids
    foreach ($result['node'] as $nid => $row) {
        $count++;
        
        // TRUE will reset static cache to
        // keep memory usage low
        $node = node_load($row->nid, null, TRUE);
        if ($count < 100000) {
            yield $node;
        }
    }
}

The generator function is called in a foreach loop which terminates once the generator stops yielding values.

    // use a generator to loop through nodes
    foreach (nodes_generator($since) as $node) {
        $fout->fputcsv([
            $node->nid,
            $node->type,
            $node->title,
            $node->created,
            url('node/' . $node->nid)
        ]);
    }

Usage

With the module complete and enabled, we have a new Drush command out our disposal. We can invoke it in a terminal shell with the command below:

drush export-newest 2015-06-01

That command will output our nodes as a stream of CSV lines. To save the output in a file, just redirect it as shown below. Once you download the file, you can send it to your client and/or project manager to peruse in Excel.

drush export-newest 2015-06-01 > newest-nodes.csv

Conclusion

There you have it, a straightforward independent Drush command that uses built-in PHP libraries to export a collection of nodes to CSV. Of course, we could have done it with Views and some other modules, but this module is easy to version and quick to deploy. It only depends on Drupal core, so it’s super low-maintenance.

For the full script, check out the gist on github

Mar 24 2015
Mar 24

Drush for Developers (Second Edition) book cover Calling this book a "second edition" is more than a little bit curious, since there is no Drush for Developers (First Edition). I can only assume that the publisher considers Juampy's excellent Drush User's Guide (reviewed here) as the "first edition" of Drush for Developers (Second Edition), but that really minimzes ths book, as it is so much more than an updated version of Drush User's Guide. It would be like saying The Godfather, Part 2 is an updated version of The Godfather - which is just crazy talk. Drush Developer's Guide is more of a sequel - and (like The Godfather, Part 2) a darn good one at that.

This is not a book about learning all the various super-cool Drush commands available and how awesome Drush is and how it can save you all sorts of time (see Drush User's Guide). This book is for people who already know what Drush does, already have it installed, and probably already use it every day. It doesn't lay the groundwork like a guide for newbies, it takes your average Drush user and "levels them up."

Granted, the first chapter has some necessary introductory material: what is Drush, how do you install it, how can you use it to save the world. Been there, done that. But, it is in the first few pages of the second chapter that Juampy tips his hand - this is really going to be a book about using Drush to create an efficient local-dev-stage-production workflow.

He introduces the topic of the "update path"; a sequence of drush commands that can be used when pushing code (with features) between environments to ensure that the "destination" environment is properly updated to take advantage of the updated code. Turns out this book is actually about creating an efficient workflow - it just happens that Drush is an excellent tool to accomplish the task. For the most part, he starts with a very basic "update path" and then spends the rest of the book iterating on it and improving it in virtually every section.

Along the way, Juampy does a great job of introducing us to a number of intermediate-to-advanced topics. He covers Drupal's registry for module and theme paths, the Features module, cron, Jenkins, Drush and Drupal bootstrapping, and Drupal's Batch API - to name a few. Each of these new topics never seem to be forced - there is always a compelling reason to spend a few pages with each one in order to improve the "update path".

Juampy also does a nice job of covering the basics of creating a custom Drush command, including an entire chapter devoted to error handling and debugging. I especially appreciated his examples of how to create a custom Drush command that calls a number of other Drush commands. This makes it possible to design a custom workflow that can be executed with a minimum of commands.

The book reaches its crescendo in the second-to-last chapter, "Managing Local and Remote Environments". Juampy deftly covers site aliases then quickly integrates them with the "update path" he started in chapter 2, to end up with a super-solid workflow that anyone can adapt for their own purposes.

He wraps things up with a chapter on setting up a development workflow, but by the time I got to this chapter, my mind was already racing with how I could improve my current workflow with the ideas already presented. The information in the last chapter seems incremental in comparison to the rest of the book, but there are great ideas nonetheless. Having a site's docroot at the second level in the Git repo and fine tuning database dumps from production was the icing on the cake.

Personally, I can't wait until I finish writing this review (I'm almost there!) so that I can implement some of Juampy's ideas in some of my client projects. If you feel like you're only using 10% of Drush and want to take it to the next level, buy this book.

I did exchange a few emails with Juampy while I was writing this review. (He did confirm that there is no "Drush for Developers (First Edition.)" I can only hope that the "Second Edition" in the title doesn't dissuade anyone from picking up this book - it is a worthy addition to any Drupal professional's library.

Trackback URL for this post:

http://drupaleasy.com/trackback/714

Attachment Size drushfordevelopers_bookcover.jpg 21.42 KB
Mar 19 2015
Mar 19

After a very successful drush code-sprint at BADCamp 2014, drush make now supports YAML format!

Instead of the old INI format

api = 2 ; Set contrib directory. defaults[projects][subdir] = "contrib" core = "7.x" projects[drupal][type] = "core" projects[drupal][version] = "7.32" ; Remove scary ajax error when autocomplete terminates: https://www.drupal.org/node/1232416#comment-8748879 projects[drupal][patch][] = "https://www.drupal.org/files/issues/D7-fix_autocomplete_terminated_error..." ; Ensure plain text fields evaluate line breaks. projects[drupal][patch][] = "http://drupal.org/files/text-plain-1152216-24.patch" projects[addressfield][version] = "1.0-beta5" projects[addressfield_tokens][version] = "1.4" projects[admin_views][version] = "1.3" projects[field_collection][version] = "1.0-beta7" ; Field collections are ownerless https://drupal.org/node/1954124 projects[field_collection][patch][] = "https://drupal.org/files/issues/field_collection-ownerless_fields-195412..." ; Fixes fatal error in migrate code: https://www.drupal.org/node/2315921#comment-9028779 projects[field_collection][patch][] = "https://www.drupal.org/files/issues/migrate-fatal-error-2315921-01.patch"

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

api = 2

; Set contrib directory.

defaults[projects][subdir] = "contrib"

core = "7.x"

projects[drupal][type] = "core"

projects[drupal][version] = "7.32"

; Remove scary ajax error when autocomplete terminates: https://www.drupal.org/node/1232416#comment-8748879

projects[drupal][patch][] = "https://www.drupal.org/files/issues/D7-fix_autocomplete_terminated_error..."

; Ensure plain text fields evaluate line breaks.

projects[drupal][patch][] = "http://drupal.org/files/text-plain-1152216-24.patch"

projects[addressfield][version] = "1.0-beta5"

projects[addressfield_tokens][version] = "1.4"

projects[admin_views][version] = "1.3"

projects[field_collection][version] = "1.0-beta7"

; Field collections are ownerless https://drupal.org/node/1954124

projects[field_collection][patch][] = "https://drupal.org/files/issues/field_collection-ownerless_fields-195412..."

; Fixes fatal error in migrate code: https://www.drupal.org/node/2315921#comment-9028779

projects[field_collection][patch][] = "https://www.drupal.org/files/issues/migrate-fatal-error-2315921-01.patch"

YAML can be used with the latest version of Drush 7:

api: 2 # Set contrib directory. defaults: projects: subdir: "contrib" core: "7.x" projects: drupal: type: "core" version: "7.33" patch: # Remove scary ajax error when autocomplete terminates: https://www.drupal.org/node/1232416#comment-8748879 - "https://www.drupal.org/files/issues/D7-fix_autocomplete_terminated_error..." # Ensure plain text fields evaluate line breaks. - "http://drupal.org/files/text-plain-1152216-24.patch" addressfield: "1.0-beta5" addressfield_tokens: "1.4" admin_views: "1.3" field_collection: version: "1.0-beta7" patch: # Field collections are ownerless https://drupal.org/node/1954124 - "https://drupal.org/files/issues/field_collection-ownerless_fields-195412..." # Fixes fatal error in migrate code: https://www.drupal.org/node/2315921#comment-9028779 - "https://www.drupal.org/files/issues/migrate-fatal-error-2315921-01.patch"

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

api: 2

# Set contrib directory.

defaults:

  projects:

    subdir: "contrib"

core: "7.x"

projects:

  drupal:

    type: "core"

    version: "7.33"

    patch:

        # Remove scary ajax error when autocomplete terminates: https://www.drupal.org/node/1232416#comment-8748879

      - "https://www.drupal.org/files/issues/D7-fix_autocomplete_terminated_error..."

        # Ensure plain text fields evaluate line breaks.

      - "http://drupal.org/files/text-plain-1152216-24.patch"

  addressfield: "1.0-beta5"

  addressfield_tokens: "1.4"

  admin_views: "1.3"

  field_collection:

    version: "1.0-beta7"

    patch:

        # Field collections are ownerless https://drupal.org/node/1954124

      - "https://drupal.org/files/issues/field_collection-ownerless_fields-195412..."

        # Fixes fatal error in migrate code: https://www.drupal.org/node/2315921#comment-9028779

      - "https://www.drupal.org/files/issues/migrate-fatal-error-2315921-01.patch"

Included .make files whether local, or discovered recursively within downloaded projects, can be in either YAML of INI format.

In order to use the newly-supported YAML format, simply name files with a .yml extension, such as my_project.make.yml.

The best part? This can be used now! Even though YAML files are mostly a new concept for Drupal 8, drush make will parse YAML make files for Drupal 7, and even Drupal 6.

Mar 18 2015
Mar 18

After a very successful drush code-sprint at BADCamp 2014, drush make now supports YAML format!

Instead of the old INI format

api = 2 ; Set contrib directory. defaults[projects][subdir] = "contrib" core = "7.x" projects[drupal][type] = "core" projects[drupal][version] = "7.32" ; Remove scary ajax error when autocomplete terminates: https://www.drupal.org/node/1232416#comment-8748879 projects[drupal][patch][] = "https://www.drupal.org/files/issues/D7-fix_autocomplete_terminated_error..." ; Ensure plain text fields evaluate line breaks. projects[drupal][patch][] = "http://drupal.org/files/text-plain-1152216-24.patch" projects[addressfield][version] = "1.0-beta5" projects[addressfield_tokens][version] = "1.4" projects[admin_views][version] = "1.3" projects[field_collection][version] = "1.0-beta7" ; Field collections are ownerless https://drupal.org/node/1954124 projects[field_collection][patch][] = "https://drupal.org/files/issues/field_collection-ownerless_fields-195412..." ; Fixes fatal error in migrate code: https://www.drupal.org/node/2315921#comment-9028779 projects[field_collection][patch][] = "https://www.drupal.org/files/issues/migrate-fatal-error-2315921-01.patch"

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

api = 2

; Set contrib directory.

defaults[projects][subdir] = "contrib"

core = "7.x"

projects[drupal][type] = "core"

projects[drupal][version] = "7.32"

; Remove scary ajax error when autocomplete terminates: https://www.drupal.org/node/1232416#comment-8748879

projects[drupal][patch][] = "https://www.drupal.org/files/issues/D7-fix_autocomplete_terminated_error..."

; Ensure plain text fields evaluate line breaks.

projects[drupal][patch][] = "http://drupal.org/files/text-plain-1152216-24.patch"

projects[addressfield][version] = "1.0-beta5"

projects[addressfield_tokens][version] = "1.4"

projects[admin_views][version] = "1.3"

projects[field_collection][version] = "1.0-beta7"

; Field collections are ownerless https://drupal.org/node/1954124

projects[field_collection][patch][] = "https://drupal.org/files/issues/field_collection-ownerless_fields-195412..."

; Fixes fatal error in migrate code: https://www.drupal.org/node/2315921#comment-9028779

projects[field_collection][patch][] = "https://www.drupal.org/files/issues/migrate-fatal-error-2315921-01.patch"

YAML can be used with the latest version of Drush 7:

api: 2 # Set contrib directory. defaults: projects: subdir: "contrib" core: "7.x" projects: drupal: type: "core" version: "7.33" patch: # Remove scary ajax error when autocomplete terminates: https://www.drupal.org/node/1232416#comment-8748879 - "https://www.drupal.org/files/issues/D7-fix_autocomplete_terminated_error..." # Ensure plain text fields evaluate line breaks. - "http://drupal.org/files/text-plain-1152216-24.patch" addressfield: "1.0-beta5" addressfield_tokens: "1.4" admin_views: "1.3" field_collection: version: "1.0-beta7" patch: # Field collections are ownerless https://drupal.org/node/1954124 - "https://drupal.org/files/issues/field_collection-ownerless_fields-195412..." # Fixes fatal error in migrate code: https://www.drupal.org/node/2315921#comment-9028779 - "https://www.drupal.org/files/issues/migrate-fatal-error-2315921-01.patch"

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

api: 2

# Set contrib directory.

defaults:

  projects:

    subdir: "contrib"

core: "7.x"

projects:

  drupal:

    type: "core"

    version: "7.33"

    patch:

        # Remove scary ajax error when autocomplete terminates: https://www.drupal.org/node/1232416#comment-8748879

      - "https://www.drupal.org/files/issues/D7-fix_autocomplete_terminated_error..."

        # Ensure plain text fields evaluate line breaks.

      - "http://drupal.org/files/text-plain-1152216-24.patch"

  addressfield: "1.0-beta5"

  addressfield_tokens: "1.4"

  admin_views: "1.3"

  field_collection:

    version: "1.0-beta7"

    patch:

        # Field collections are ownerless https://drupal.org/node/1954124

      - "https://drupal.org/files/issues/field_collection-ownerless_fields-195412..."

        # Fixes fatal error in migrate code: https://www.drupal.org/node/2315921#comment-9028779

      - "https://www.drupal.org/files/issues/migrate-fatal-error-2315921-01.patch"

Included .make files whether local, or discovered recursively within downloaded projects, can be in either YAML of INI format.

In order to use the newly-supported YAML format, simply name files with a .yml extension, such as my_project.make.yml.

The best part? This can be used now! Even though YAML files are mostly a new concept for Drupal 8, drush make will parse YAML make files for Drupal 7, and even Drupal 6.

Nov 24 2014
Nov 24

What is Drush?

If you’re asking that question right now then congratulations! You are one of the lucky people who will have your life changed today! Cancel everything and read up on Drush, the command line bridge to Drupal.

Everybody knows about Drush, ya Dingus!

That’s more like it. Who doesn’t love Drush, right? Right!

But more and more, I find myself seeing people reinventing things that Drush already handles because they just don’t know all that Drush can do. It’s getting frustrating, and I want to fix that.

First, The Basics

Stuff everybody knows

Here are a few Drush commands that most people know and love, just to get them out of the way:

  • drush updb: run pending database updates
  • drush cc all: clear all caches
  • drush dl <something>: download the <something> module
  • drush en <something>: enable the <something> module

Stuff everybody knows (Features Edition™)

And if you’re using Features, you’re probably familiar with:

  • drush fra: revert all Features
  • drush fe: export a new or updated Feature with a new component
  • drush fu <featurename>: update the <featurename> Feature with updated site config
  • drush fr <featurename>: revert the site’s config to the current state of the <featurename> Feature

Aliases

For a lot of the fun stuff, you’ll have to understand Drush aliases. If you don’t, here’s the gist: Drush aliases give you an easy way to run Drush commands on remote Drupal sites, as opposed to only being able to use it on your local sites. If you’re constantly SSH’ing into different environments just to run a couple quick commands, you need to stop doing that.

There’s lots of documentation about Drush aliases and how to create your own, but most of the docs lack notes on some of the lesser known awesome things you can do with aliases. Keep reading, sailor.

Well, one more thing. This is probably a good time to mention a couple quick commands.

Firstly, let’s run an arbitrary shell command on our dev environment.

drush @foo-dev exec echo $SOMETHING 1 drush@foo-devexececho$SOMETHING

Or maybe we should just go ahead and SSH in to do something a little more complex.

drush @foo-dev ssh 1 drush@foo-devssh

Or maybe we need to do a bunch of aliased commands, but we want to do it without SSH’ing in (because the commands require local files or something). We can make a Drush alias persist until we tell it to stop using:

drush site-set @foo-dev 1 drushsite-set@foo-dev

And then when we’re done doing what we do, we can just run it again without the “@foo-dev” argument to unset it.

Now, keep reading, sailor.

Syncing Ship

(Warning: these headlines are going to get worse and worse)

One of the most common things to do with Drush aliases is to sync stuff from one alias to another.

For example, want to sync the dev site database down into your local?

drush sql-sync @foo-dev @foo-local 1 drushsql-sync@foo-dev@foo-local

How about files? Sync ‘em!

drush rsync @foo-dev:%files @foo-local:%files 1 drushrsync@foo-dev:%files@foo-local:%files

Or maybe some unwashed sent you a DB dump and you have to import it the old fashioned way?

cat ~/path/to/file.sql | drush sql-cli 1 cat~/path/to/file.sql|drushsql-cli

Sometimes you want to drop your entire database before importing, to make sure you don’t get any tables left behind from your old install that aren’t supposed to be there. That’s as easy as:

drush sql-drop 1 drushsql-drop

Sometimes, it’s useful to be able to automate running arbitrary SQL commands on multiple environments, and that’s pretty easy too. Say for example that you quickly want to get the username for uid 1 on the prod environment (the “drush user-information” command would be much better for this, but shut up).

drush @foo-prod sqlq 'select name from users where uid = 1' 1 drush@foo-prodsqlq'select name from users where uid = 1'

That one is also good for automation, like if you want to write a quick script that changes the username for uid 1 on all environments.

Drupal Without Drupal

It’s often useful to run one-off arbitrary code within the context of Drupal, without having to actually put it in the codebase somewhere. This is typically done one of two ways:

If it’s just a short one-liner, then there’s the ever-useful “php-eval” (aka “ev”) command. For example, let’s inspect a node object.

drush @foo-dev php-eval 'print_r(node_load(123));' 1 drush@foo-devphp-eval'print_r(node_load(123));'

Or if it’s a longer one, then we can just throw our code into a PHP file, and run it using:

drush php-script filename.php 1 drushphp-scriptfilename.php

Reports Cohorts

Drush is really good at getting us information from Drupal without waiting for a full page load.

How many times have you navigated to the Watchdog page and sat through page load after page load while you went through the pagination and added filtering and blah blah blah to find an error message? Stop doing that! Do this instead:

drush watchdog-show 1 drushwatchdog-show

There are a lot of useful options for watchdog-show, such as:

  • –tail (continuously show new messages)
  • –full (show the full output, with all of the fields, instead of the summarized version)
  • –severity (such as “–severity=error”)
  • –count (show more than the default 10, such as “–count=100?)

And I’d bet you’re familiar with “drush vget” to get variables, but did you know you can pass “–format=whatever” to get the results formatted as JSON or CSV or YAML or a bunch of other things, for easy scripting?

Another one of my favorites is this charm, which basically prints out the stuff you see on the Status Report page in Drupal. It’s nice for sanity checking before pushing releases live.

drush status-report 1 drushstatus-report

And then there’s this guy, which prints out a bunch of useful info about the current installation, such as DB info, path to PHP executable and .ini file, Drupal version, Drupal root, etc. It’s a nice first step when debugging a broken install.

drush status 1 drushstatus

And for those times when you need to edit a config file (php.ini, or settings.php, or an alias file, or .htaccess, etc.), you can run this to let you choose which of those files to edit and it’ll open it up in an editor for you:

drush config 1 drushconfig

Using Users

Drush is nothing short of a miracle when it comes to user management.

First of all, there’s the ever-annoying task of logging in as this user or that user. You usually don’t know the password, or maybe you’re just too lazy to type it. Run this to open up your browser with a one-time login link so you can skip all of that malarky:

drush user-login name-or-uid 1 drushuser-loginname-or-uid

Or, if you’re slightly less lazy, and just want to change the password to something so that you can log in the old fashioned way:

drush user-password name-or-uid --password=test1234 1 drushuser-passwordname-or-uid--password=test1234

Then there’s the “fun” process of adding a user and filling out the form. Skip that:

drush user-create person123 --mail="[email protected]" --password="letmein" 1 drushuser-createperson123--mail="[email protected]"--password="letmein"

Once that’s done, you probably want to give that new user some roles. For role stuff, you have this:

drush user-add-role "user editor" person123 drush user-remove-role "user editor" person123 1 2 drushuser-add-role"user editor"person123drushuser-remove-role"user editor"person123

But watch out! The role you need to add doesn’t exist yet! Let’s add it, and give it some permissions.

drush role-create 'user editor' drush role-add-perm 'user editor' 'administer users' 1 2 drushrole-create'user editor'drushrole-add-perm'user editor''administer users'

If you just need to show information about a user, such as email address, roles, UID, etc., try this. I’m embarrassed to say that I’ve been using raw SQL for this for years.

drush user-information name-or-uid 1 drushuser-informationname-or-uid

Fields of Dreams

One of the most under-used things that Drush gives you is field management tools. I’m going to be lame here and just copy and paste the docs, since they’re pretty self explanatory.

Field commands: (field) field-clone Clone a field and all its instances. field-create Create fields and instances. Returns urls for field editing. field-delete Delete a field and its instances. field-info View information about fields, field_types, and widgets. field-update Return URL for field editing web page. 1 2 3 4 5 6 Fieldcommands:(field)field-cloneCloneafieldandallitsinstances.field-createCreatefieldsandinstances.Returnsurlsforfieldediting.field-deleteDeleteafieldanditsinstances.field-infoViewinformationaboutfields,field_types,andwidgets.field-updateReturnURLforfieldeditingwebpage.

Other Schtuff

Here are some great commands that don’t really fit into any clear-cut categories.

It has some neat archiving tools:

drush archive-dump #backup code, files, an DB to a single package drush archive-restore #expand one of those archives into a full Drupal site 1 2 drusharchive-dump#backup code, files, an DB to a single packagedrusharchive-restore#expand one of those archives into a full Drupal site

Somewhat similar to those is this one, which will download and install Drupal, serve it using a little built-in server, and log you in, all in one command. Note that this one includes about a bazillion options and is super duper powerful.

drush core-quick-drupal 1 drushcore-quick-drupal

Drush also lets you play with the cache, which can save a lot of time when debugging a caching issue:

drush cache-get your-cid your-bin drush cache-set your-cid your-data your-bin your-expire 1 2 drushcache-getyour-cidyour-bindrushcache-setyour-cidyour-datayour-binyour-expire

There are a couple unknown commands for working with contrib:

drush pm-info modulename #display included files, permissions, configure link, version, dependencies, etc. drush pm-releasenotes modulename #show the release notes for the version of module that you're using 1 2 drushpm-infomodulename#display included files, permissions, configure link, version, dependencies, etc.drushpm-releasenotesmodulename#show the release notes for the version of module that you're using

Run cron! Not exciting, but super useful.

drush cron 1 drushcron

Have you ever been in the middle of debugging and you know that something is happening in a form_alter (or some other hook) but you’re not sure in which module? Try this command, which will tell you all of the implementations of a given hook, and let you choose one to view the source code of it.

drush fn-hook form_alter 1 drushfn-hookform_alter

And finally, this bad boy is basically “Drush docs in a box” and has a TON of useful info. Seriously, try it now.

drush topic 1 drushtopic

Drushful Thinking

There’s a giant heap of useful Drush commands, some of which you hopefully hadn’t seen before. So what, right?

The “so what” is that it’s useful to start thinking in terms of “how can Drush do this for me?” and you’ll often find that the answer is “pretty easily.”

Play a game with yourself. Next time you’re working on site building or anything that involves a lot of clicky clicky in the Drupal UI, give yourself a jellybean or a chip or something every time you do something in Drush instead of in the UI.

But why? Well for one, before you know it, you’ll be spending  much less time waiting on page loads. But secondly, Drush lends itself to automation, and thinking in terms of Drush naturally leads you to think in terms of automating and scripting things, which is a great place to be.

Practice some Drushful Thinking! And let me know any of your favorite Drush tips and tricks in the comments. Check out drushcommands.com for some more inspiration.

Nov 12 2014
Nov 12

After a very successful drush code-sprint at BADCamp 2014, drush make now supports YAML format!

Instead of the old INI format

INI api = 2 ; Set contrib directory. defaults[projects][subdir] = "contrib" core = "7.x" projects[drupal][type] = "core" projects[drupal][version] = "7.32" ; Remove scary ajax error when autocomplete terminates: https://www.drupal.org/node/1232416#comment-8748879 projects[drupal][patch][] = "https://www.drupal.org/files/issues/D7-fix_autocomplete_terminated_error..." ; Ensure plain text fields evaluate line breaks. projects[drupal][patch][] = "http://drupal.org/files/text-plain-1152216-24.patch" projects[addressfield][version] = "1.0-beta5" projects[addressfield_tokens][version] = "1.4" projects[admin_views][version] = "1.3" projects[field_collection][version] = "1.0-beta7" ; Field collections are ownerless https://drupal.org/node/1954124 projects[field_collection][patch][] = "https://drupal.org/files/issues/field_collection-ownerless_fields-195412..." ; Fixes fatal error in migrate code: https://www.drupal.org/node/2315921#comment-9028779 projects[field_collection][patch][] = "https://www.drupal.org/files/issues/migrate-fatal-error-2315921-01.patch" 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 api=2   ; Set contrib directory.defaults[projects][subdir]="contrib"   core="7.x"projects[drupal][type]="core"projects[drupal][version]="7.32"; Remove scary ajax error when autocomplete terminates: https://www.drupal.org/node/1232416#comment-8748879projects[drupal][patch][]="https://www.drupal.org/files/issues/D7-fix_autocomplete_terminated_error..."; Ensure plain text fields evaluate line breaks.projects[drupal][patch][]="http://drupal.org/files/text-plain-1152216-24.patch"   projects[addressfield][version]="1.0-beta5"   projects[addressfield_tokens][version]="1.4"   projects[admin_views][version]="1.3"   projects[field_collection][version]="1.0-beta7"; Field collections are ownerless https://drupal.org/node/1954124projects[field_collection][patch][]="https://drupal.org/files/issues/field_collection-ownerless_fields-195412..."; Fixes fatal error in migrate code: https://www.drupal.org/node/2315921#comment-9028779projects[field_collection][patch][]="https://www.drupal.org/files/issues/migrate-fatal-error-2315921-01.patch"

YAML can be used with the latest version of Drush 7:

YAML api: 2 # Set contrib directory. defaults: projects: subdir: "contrib" core: "7.x" projects: drupal: type: "core" version: "7.33" patch: # Remove scary ajax error when autocomplete terminates: https://www.drupal.org/node/1232416#comment-8748879 - "https://www.drupal.org/files/issues/D7-fix_autocomplete_terminated_error..." # Ensure plain text fields evaluate line breaks. - "http://drupal.org/files/text-plain-1152216-24.patch" addressfield: "1.0-beta5" addressfield_tokens: "1.4" admin_views: "1.3" field_collection: version: "1.0-beta7" patch: # Field collections are ownerless https://drupal.org/node/1954124 - "https://drupal.org/files/issues/field_collection-ownerless_fields-195412..." # Fixes fatal error in migrate code: https://www.drupal.org/node/2315921#comment-9028779 - "https://www.drupal.org/files/issues/migrate-fatal-error-2315921-01.patch" 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 api: 2   # Set contrib directory.defaults:  projects:    subdir: "contrib"core: "7.x"projects:  drupal:    type: "core"    version: "7.33"    patch:        # Remove scary ajax error when autocomplete terminates: https://www.drupal.org/node/1232416#comment-8748879      - "https://www.drupal.org/files/issues/D7-fix_autocomplete_terminated_error..."        # Ensure plain text fields evaluate line breaks.      - "http://drupal.org/files/text-plain-1152216-24.patch"     addressfield: "1.0-beta5"  addressfield_tokens: "1.4"  admin_views: "1.3"  field_collection:    version: "1.0-beta7"    patch:        # Field collections are ownerless https://drupal.org/node/1954124      - "https://drupal.org/files/issues/field_collection-ownerless_fields-195412..."        # Fixes fatal error in migrate code: https://www.drupal.org/node/2315921#comment-9028779      - "https://www.drupal.org/files/issues/migrate-fatal-error-2315921-01.patch"

Included .make files whether local, or discovered recursively within downloaded projects, can be in either YAML of INI format.

In order to use the newly-supported YAML format, simply name files with a .yml extension, such as my_project.make.yml.

The best part? This can be used now! Even though YAML files are mostly a new concept for Drupal 8, drush make will parse YAML make files for Drupal 7, and even Drupal 6. Want to learn more about DRUSH make files? Check out Joe Turgeon’s “Getting Started With Grunt Drupal Tasks

Nov 05 2014
Nov 05

In September, Phase2 released a Grunt-based tool for building and testing Drupal sites. We have been working on the tool since January, and after adopting it as part of our standard approach for new Drupal sites, we wanted to contribute it back to the community. We are happy invite you to get started with Grunt Drupal Tasks.

Grunt is a popular JavaScript-based task runner, meaning it’s a framework for automating tasks. It’s gained traction for automating common development tasks, like compiling CSS from Sass, minifying JavaScript, generating sprites, checking code standards, and more. There are thousands of plugins available that can be implemented out-of-the-box to do these common tasks or integrate with other supporting tools. (I mentioned that this is all free and open source software, right?)

Grunt Drupal Tasks is a Grunt plugin that defines processes that we have identified as best practices for building and testing Drupal sites.

Building Drupal

The cornerstone of Grunt Drupal Tasks is the “build” process, which assembles a runnable Drupal site docroot from a Drush make file and custom code and configuration.

The make file defines the version of Drupal core to use, the contrib modules, themes, and libraries to download, and even patches to apply to any of these components. The make file can include components released on Drupal.org or stored in public or private repositories. For patches, our best practice is to reference patches hosted on Drupal.org and associated with an issue. With these options, the entire set of components for a Drupal site can be declared in a make file and consistently retrieved using Drush.

After the Drush make process assembles all external dependencies for the project, the Grunt Drupal Tasks build process adds custom code and configuration. This includes custom installation profiles, modules, and themes, as well as “sites” directory files, like sites.php and settings.php for one or many subsites, and other “static” files to override, like .htaccess and robots.txt. These custom components are added to the built docroot by symlink, so it is not necessary to rebuild for every update to custom source code.

These steps results in a Drupal docroot assembled from custom source in the following structure:

src/ modules/ profiles/ sites/ default/ settings.php static/ themes/ project.make 1 2 3 4 5 6 7 8 9 10 11 12 13 14 src/   modules/     custom modules>  profiles/     custom installation profiles>   sites/     default/       settings.php     optionally, other subsites or sites.php>   static/     optionally, overrides for .htaccess or other files>   themes/     custom themes>   project.make

Grunt Drupal Tasks includes other optional build steps, which can be enabled as needed for projects. One such task is the “compile theme” step will compile Sass files into CSS.

This build process gives us a reliable way for assembling Drupal core and contrib components, for adding our custom code, and integrating development tools like Sass. By using Grunt to automate this procedure, it becomes a portable script that can be shared among the project’s developers and used in deployment environments.

Testing Drupal

In order to help make best practices the default, Grunt Drupal Tasks includes support for a number of code quality and testing tools.

A “validate” task is provided that includes checking basic PHP syntax and Drupal coding standards using PHPLint and PHP Code Sniffer. We highly recommended that developers use this command while coding, and have included it as part of the default build process.

An “analyze” task is also provided, which adds support for the PHP Mess Detector. This task may be longer-running, so it is better suited to run as part of a continuous integration system, like Jenkins.

Finally, a “behat” task is provided for running test scenarios with Behat and the Drupal Extension. This encourages writing Behat tests for the project and committing them with the project code and build tools, so the tests can be run by other developers and in the integration environment by a continuous integration system.

Scaffolding for Drupal Projects

The old starting point for Drupal projects was a vanilla copy of Drupal core. Grunt Drupal Tasks offers scaffolding for Drupal projects that starts with Drush make, integrates custom code and overrides, and provides consistent support for a variety of developer tools.

This scaffolding is provided through the example included with Grunt Drupal Tasks, which is the recommended starting point for new projects. The scaffold structure adds a layer above the aforementioned “src” directory; this layer includes code and configuration related to Grunt Drupal Tasks (Gruntconfig.json and Gruntfile.js), dependencies for the supporting tools (composer.json), and other resources for the tools (features/, behat.yml, and phpmd.xml).

The example includes the following:

features/ src/ .gitignore Gruntconfig.json Gruntfile.js behat.yml composer.json package.json phpmd.xml 1 2 3 4 5 6 7 8 9 features/src/.gitignoreGruntconfig.jsonGruntfile.jsbehat.ymlcomposer.jsonpackage.jsonphpmd.xml

For full documentation on starting a new project with Grunt Drupal Tasks, see CONFIG.md.

Learning More

Watch the Phase2 blog for more information about Grunt Drupal Tasks. If you are attending the Bay Area Drupal Camp this week, please check out my session on Using Grunt to Manage Drupal Build and Testing Tools.

Oct 23 2014
Oct 23

.Drush aliases allow us to execute commands on a remote site from the local console. It is the perfect tool for the lazy drupal developer. With drush aliases I rarely login to a remote server, I execute all the drush commands from my local console. It is also a great for workflow automation. Here is an example command using an alias: drush cc @dev.site This command will execute the cc (clear caches) on the @dev.site as it is defined in the alias. For this to work, you will have to:

  1. Have passwordless login in the target system
  2. Set up the site alias

Passwordless login

For passwordless login, we need to copy our public key to the remote machine: ssh-copy-id [email protected] Then try to login to the ssh server without a password. Once this is set up, we can set up our aliases.

Set up a drush alias

A drush alias is an array that defines a site. It includes things like the remote site address, the db user etc. Drush reads this array so that it can login to the remote site and execute drush commands via ssh. Here is a sample alias array:

<?php
$aliases
['dev.site'] = array (
 
'remote-host' => 'dev.site.com',
 
'remote-user' => 'devsite',
 
'root' => '/home/devsite/public_html',
 
'uri' => 'http://dev.site.com',
);
?>

Or, use the following snippet if drush is not installed in the target server:

<?php
$aliases
['dev.site'] = array (
 
'remote-host' => 'dev.site.com',
 
'remote-user' => 'devsite',
 
'root' => '/home/devsite/public_html',
 
'uri' => 'http://dev.site.com',
 
'databases' =>
  array (
   
'default' =>
    array (
     
'default' =>
      array (
       
'driver' => 'mysql',
       
'database' => 'dev_db',
       
'username' => 'dev_db_user',
       
'password' => 'dev_db_pass',
       
'host' => 'localhost',
       
'prefix' => '',
       
'collation' => 'utf8_general_ci',
      ),
    ),
  ),
);
?>

This array must be placed in a file with the following name: example.aliases.drushrc.php Put this file in your ~/.drush or in the sites/all/drush folder. You can obtain the alias definition above by logging into the target site and executing the command: drush site-alias --with-db --show-passwords --alias-name=dev.site @self

Have fun

You can perform cool actions like copying the database from the live site to the dev from your local console: drush sql-sync @live.site @dev.site

Oct 08 2014
Oct 08

This is a simple trick which (unless my googlefu simply failed me) I didn't find described anywhere when I had a quick look:

$ drush ev '$file = file_load(21749); var_dump(file_delete($file, TRUE));'
bool(true)

This means all the appropriate hooks are called in file_delete so the Drupal API gods should smile on you, and you should get to see the TRUE/FALSE result reflecting success or otherwise. Note that we're passing $force=TRUE "indicating that the file should be deleted even if the file is reported as in use by the file_usage table." So be careful.

To delete multiple files you could use file_load_multiple but there's not a corresponding file_delete_multiple function, so you'd have to loop over the array of file objects.

That's all there is to this one.

Oct 07 2014
Oct 07

For drupal we have a number of different methods to backup the database. Having a backup strategy is one of the easiest things to do but often overlooked. Perform regular backups to keep your sanity when disaster hits. Implement a backup strategy for daily weekly and monthly backups and look cool to the client. I am going to discuss about 3 different ways to backup your database.

  1. backup and migrate
  2. drush and mysqldump
  3. automysqlbackup

Use the Backup and Migrate drupal module

Backup and migrate module (https://drupal.org/project/backup_migrate) is perhaps the most common way to keep your db safe. It counts more than 290,000 installs and there are many reasons for this. It supports encryption, rotation and the option to create a site archive complete with files + db. Also, it supports multiple destinations:

  • local filesystem backup
  • backup and email
  • backup to ftp
  • backup to cloud (AWS S3, rackspace, dropbox)
  • NodeSquirrel (A backup service by the makers of backup and migrate module)

The downside is that it relies on drupal cron to execute the backup task so it puts some load on apache. For bigger sites this may be an issue.

Backup a drupal database with drush or mysqldump

Drush is the Swiss army knife for drupal. Among other things, it can dump the database of a site with a simple command: drush sql-dump --result-file=site-stg.sql --gzip Execute this command out of the drupal directory, or the backups will be accessible by anyone: drush -r /drupal/root sql-dump --result-file=site-stg.sql --gzip If --result-file is provided with no value, then date based filename will be created under ~/drush-backups directory. If drush is not available in the server, you can still use the mysqldump command to get a db dump. mysqldump -udb_user -pdb_pass drupal_db &gt; drupal.sql Or generate a compressed archive: mysqldump -udb_user -p drupal_db | gzip -9 &gt; drupal.sql.gz Another option is to use drush and get the database directly on a remote server. This requires drush installed on both servers and your drush aliases working. Execute this from the remote server to get the dump: drush @client-site sql-dump &gt; client-site.sql Or perform the same procedure with mysqldump: ssh [email protected] "mysqldump -udb_user -pdb_pass drupal_db &gt; site.sql; gzip site.sql" scp [email protected]:site.sq.gz . Combine any of the above commands with some bash scripting to obtain a dynamic filename, add it to cron and perform daily backups automatically: drush @client-site sql-dump &gt; `date +%Y-%m-%d`.sql

Backup up with automysqlbackup

Automysqlbackup is a sourceforge script that performs automated backups (http://sourceforge.net/projects/automysqlbackup/). Automysqlbackup is my favorite method because it is a set and forget solution. It can automatically backup new databases and will rotate the archives. Automysqlbackup provides:

  • backup rotation with daily, weekly and monthly
  • backup encryption

Automysqlbackup can not backup to a remote location unless you use a network drive. It can however, email you the dump. Are you using some other method to keep your sites safe? I would love to hear in the comments below.

Oct 05 2014
Oct 05

Excerpt from a Drupal 8 menu links treeOne of the interesting aspects of the revamped menu/links system in Drupal 8 is the fact that menu links are now in easily parseable YAML files, the "(module).links.menu.yml" in each module, in which each menu link can be bound to its parent link, hopefully producing a tree-like structure.

This provides an easy way to check consistency of the links graph, by ensuring links newly defined in your shiny custom module are linked to a valid parent link path, to help with the overall navigability of the site. This can be checked using the Tooling module, and its Drush command tolt, like in this example.

  • clone the module to (yoursite)/modules/tooling
  • ensure Drush is working on your site
  • ensure GraphViz (the dot command) is working on your system
  • drush tolt | dot -Tsvg > mysite.svg

That's it: the mysite.svg file should now contain the tree of your menu links, with all orphan links flagged in orange for you to examine, like in the image.

Attachment Size drupal8-menu-links.png 141.34 KB
Aug 26 2014
Aug 26

Features is one of those modules we can not do without. One annoying issue with features is that it is slow. The bigger the drupal site, the slower the features UI pages load. And waiting for the page to load is one of the most disturbing things during drupal development.

Adding a new component to a feature is a multistep process: Go the features page, add the component to the feature, download the feature, remove the old feature and extract the new one. And repeat many times during the day.

Fortunately, we can speed this process quite a bit. Drush to the rescue again! This post will explain how to use drush to add new components to a feature.

Add a component to a feature with drush

We can use drush to add component to a feature using the features-export command.

Here is the command help information:

➜  drush help fe
Export a feature from your site into a module. If called with no arguments, display a list of available components. If called with a single argument, attempt to create
a feature including the given component with the same name. The option '--destination=foo' may be used to specify the path (from Drupal root) where the feature should
be created. The default destination is 'sites/all/modules'. The option '--version-set=foo' may be used to specify a version number for the feature or the option
'--version-increment' may also to increment the feature's version number.

Arguments:
feature                                   Feature name to export.                                                               
components                                Patterns of components to include, see features-components for the format of patterns.

Options:
--destination                             Destination path (from Drupal root) of the exported feature. Defaults to 'sites/all/modules'.
--version-increment                       Increment the feature's version number.                                                      
--version-set                             Specify a version number for the feature.                                                    

Aliases: fe

Assuming you have a view called content_list and a feature called f_registration_views, here is how you add the view to the feature.:
drush fe -y f_registration_views views_view:content_list --version-increment
If the feature above does not exist, it will be created.

That's great. But how do you add a content type, or a rule to a feature? We need to know what components are available and the sources for each component:.

LIst the components avaiable

There is another drush command that lists all the component in a drupal site. :

➜  drush fc node          
Available sources                                            
node:page              Provided by:f_registrations_content_types
node:article

The above says that there are 2 content types in the system: article and page. Additionally, the page content type is already contained in a the f_registrations_content_types feature. To add the article content type the the same feature use drush fe egain:

drush fe -y  f_registrations_content_typesnode:article --version-increment

Use drush fc to get a list of all the available components:

When working with features do not forget about the other commonly used features commands:

drush fra
drush fua
drush fl

Aug 17 2014
Aug 17

So I have a folder for drush scripts _above_ several doc root folders on a dev user's server. And I want to run status or whatever and my own custom drush scripts on _different_ Drupal web app instances. Drush has alias capability for different site instances, so you can do:

$ drush @site1 status

So, how to set up an aliases file?

(I'm on Ubuntu with Drush 6.2.0 installed with PEAR as per this great d.o. doc page Installing Drush on Any Linux Server Out There (Kalamuna people, wouldn't you know it?)).

Careful reading of the excellent drush documentation points you to a Drush Shell Aliases doc page, and from there to the actual example aliases file that comes with every drush installation.

So to be able to run drush commands for a few of my local Drupal instances, I did this:

  • In my Linux user directory, I created the file ~/.drush/aliases.drushrc.php
  • Contents:
<?php

$aliases['site1'] = array(
  'root' => '/home/thevictor/site1/drupal-yii',
  'uri' => 'drupal-yii.example.com',
);
$aliases['site2'] = array(
  'root' => '/home/thevictor/site2',
  'uri' => 'site2.example.com',
);

Then I can do, from anywhere as long as I am logged in as that user:

$ cd /tmp
$ drush @site1 status
...
$ drush @site2 status

and lots of other good stuff. Have a nice weekend.

Bookmark/Search this post with

Apr 14 2014
Apr 14

Alias Directory

You can place the aliases.drushrc file either in the 'sites/all/drush' directory or your global environment drush folder (eg. /home/username/.drush) and the naming convention is 'group.aliases.drushrc.php' so I normally use 'project.aliases.drushrc.php' or 'client.aliases.drushrc.php' to group related sites.

Dev (/local)

Create an alias array defining your local development site:

$aliases['dev'] = array(
  'uri' => 'sitename.dev',      // The uri as configured in you apache hosts
  'root' => '/path/to/web/root',
  'path-aliases' => array(
    '%files' => 'sites/default/files',
   ),
);

You can now (if you placed the alias file in your global drush directory) use drush from any directory, using:

drush @project.dev status

or

drush @project.dev cc all

Did you say any directory?!

Yep! Since you have defined you webroot in the global drush aliases file, you don't have to be in your webroot when running drush, and really, you don't even have to be on the same server...

Production (/remote)

To get the alias details for a remote machine, the easiest place to start would be to just ssh into it and run:

drush sa @self --with-db --show-passwords --with-optional

The result looks like this:

$aliases['self'] = array (
  'root' => '/path/to/drupal/root',
  'uri' => 'http://default',
  'path-aliases' => array(
    '%drush' => '/path/to/drush',
    '%site' => 'sites/default/', 
  ),
  'databases' => array(
    'default' => array(
      'default' => array(
        'database' => 'site_db',
        'username' => 'site_user',
        'password' => 'site_pass',
        'host' => 'localhost',
        'port' => '',
        'driver' => 'mysql',
        'prefix' => '',
      ),
    ),
  ),
);

You can just copy this directly into your local drush alias file and add remote details liek this:

$aliases['live'] = array (
...
  'uri' => 'http://mysite.com',
  'remote-host' => 'ip.or.domain',
  'remote-user' => 'ssh_user',
...
  'path-aliases' => array(
    '%files' => 'sites/default/files',
...

The result allows you to run drush commands locally and have them acting on a remote site.

Jiminy Cricket!

  'remote-port' => 3201,

If you have a seperate port for mysql

  'ssh-options' => '-o PasswordAuthentication=yes',

If you can't use an ssh key

Syncing files

You can sync the files directory between sites:

drush rsync -y @project.live:%files @project.dev:sites/default

or

drush -r /path/to/web/root rsync -y @project.live:%files @self:sites/default

This post is mainly snippets and tips for me to remember drushrc tools in my day to day work.
Other (/better) blog posts are as follows:

Mar 28 2014
Mar 28

I clearly remember the moment of excitement and joy couple of years ago when I discovered drush. Interacting with drupal from the command line? Wow! This was definitely something new to me, never seen it before in any cms.
Digging more, I discovered the wp-cli for the wordpress, but it seems that this in no way as mature as drush.

I use drush on a daily basis. For simple tasks such as clearing the caches or updating a feature to more complex stuff like in a deployment process or as an interactive php shell.

As with drupal, drush can also be extended. It is just a matter of writing the proper hook in the proper file. It is really simple and everyone can write their own drush commands.

As an example, I am going to show you how you to trigger a node save.

First, we will need a drush command file.
The drush command file must end with .drush.inc It can be placed in a number of locations. One common place is the drush folder in your home directory: ~/.drush If you put the drush command file in that directory, your commands will be available for all your projects. Another option is to create a module and add your command file to the module folder. This is more preferable if the command functionality is specific for a project.

In this file, we are going to implement the hook_drush_command() hook. This hook defines the drush command
We will replace hook with the module name and use the same name for the first part of the command file:
File: mymodule.drush.inc
Hook function: mymodule_drush_command()

The hook_drush_command() hook contains a description of the drush command:

<?php
/**
* Implements hook_drush_command().
*/
function mymodule_drush_command(){
$items['custom-save-node'] = array(
'description' => "Triggers a node-save",
'arguments' => array(
'nid' => 'The nid',
),
'aliases' => array('cns'),
'examples' => array(
'drush save-node 12' => 'Triggers a asve for the node with nid=12',
),
);
return
$items;
}
?>

This is self explanatory. Now we need to write the php code to be executed when the above command is run:

<?php
function drush_mymodule_custom_save_node($nid){
node_save($nid);
}
?>

Naming is important here. We always need to prefix with drush_ following by the hook name and the command name.

That’s all. Now, when we type drush custom-save-node 12, the node with nid=12 will be saved.

Now, let’s say that you want to perform another action before saving this node. You can implement the validate hook:

<?php
function drush_mymodule_custom_save_node_validate($nid){
//check that this node is of certain type and return error if not:
drush_set_error('You can not save this node.');
}
?>

There are a few hooks that are executed before and after your command, here are some:

<?php
drush_mymodule_custom_save_node_validate
()
drush_mymodule_custom_save_node_pre_validate()
drush_mymodule_post_custom_save_node()
drush_mymodule_pre_custom_save_node()
?>

To see all hooks, run drush custom_save_node --show-invoke

Similarly, implementing those those hooks for other commands, we can alter their functionality. For example, prevent the drush dis command from disabling the webform module:

<?php
function drush_maintenance_module_pre_pm_disable($module){
if(
$module==’webform’){
drush_set_error('You can not uninstall webform module);
}
}
?>

Did you like this post? Drop me a line in the comments below

Oct 21 2013
Oct 21
Automating new dev sites for new branches

Over the past few years we've moved away from using Subversion (SVN) for version control and we're now using Git for all of our projects.  Git brings us a lot more power, but because of its different approach there are some challenges as well. 

Git has powerful branching and this opens up new opportunities to start a new branch for each new ticket/feature/client-request/bug-fix. There are several different branching strategies: Git Flow is common for large ongoing projects, or we use a more streamlined workflow.  This is great for client flexibility — a new feature can be released to production immediately after it's been approved, or you can choose to bundle several features together in an effort to reduce the time spent running deployments.  Regardless of what branching model you choose you will run into the issue where stakeholders need to review and approve a branch (and maybe send it back to developers for refinement) before it gets merged in.  If you've got several branches open at once that means you need several different dev sites for this review process to happen.  For simple tasks on simple sites you might be able to get away with just one dev site and manually check out different branches at different times, but for any new feature that requires database additions or changes that won't work. 

Another trend in web development over the past few years has been to automate as many of the boring and repetitive tasks as possible. So we've created a Drush command called Site Clone that can do it all with just a few keystrokes:

  1. Copies the codebase (excluding the files directory) with rsync to a new location.
  2. Creates a new git branch (optional).
  3. Creates a new /sites directory and settings.php file.
  4. Creates a new files directory.
  5. Copies the database.
  6. Writes database connection info to a global config file.

It also does thorough validation on the input parameters (about 20 different validations for everything from checking that the destination directory is writable, to ensuring that the name of the new database is valid, to ensuring that the new domain can be resolved).

Here's an example of how it's run:

drush advo-site-clone --destination-domain=test.cf.local --destination-db-name=test --git-branch=test
---VALIDATION---
no errors

---SUMMARY---
Source path         : /Users/dave/Sites/cf
Source site         : sites/cf.local
Source DB name      : cf
Destination path    : /Users/dave/Sites/test.cf.local
Destination site    : sites/test.cf.local
Destination DB name : test
New Git branch      : test

Do you really want to continue? (y/n): y
Starting rsync...                                             [status]
Rsync was successful.                                         [success]
Creating Git branch...                                        [status]
Switched to a new branch 'test'
Git branch created.                                           [success]
Created sites directory.                                      [success]
Updated settings.php.                                         [success]
Created files directory.                                      [success]
Starting DB copy...                                           [status]
Copied DB.                                                    [success]
Complete                                                      [success]

There are a few other things that we needed to put in place in order to get this working smoothly. We've set up DNS wildcards so that requests to a third-level subdomain end up where we want them to.  We've configured Apache with a VirtualDocumentRoot so that requests to new subdomains get routed to the appropriate webroot.  Finally we've also made some changes to our project management tool so that everyone knows which dev site to look at for each ticket.

Once you've got all the pieces of the puzzle you'll be able to have a workflow something like:

  1. Stakeholder requests a new feature (let's call it foo) for their site (let's call it bar.com).
  2. Developer clones an existing dev site (bar.advomatic.com) into a new dev site (foo.bar.advomatic.com) and creates a new branch (foo).
  3. Developer implements the request.
  4. Stakeholder reviews on the branch dev site (foo.bar.advomatic.com). Return to #3 if necessary.
  5. Merge branch foo + deploy.
  6. @todo decommission the branch site (foo.bar.advomatic.com).

Currently that last step has to be done manually.  But we should create a corresponding script to clean-up the codebase/files/database/branches.

Automate all the things!

Aug 19 2013
Aug 19

Drupal.org uses a patch based workflow in order to improve its software. Today I present a concrete instance where this significantly slowed us down. Views performance work is blocked when it didn’t have to be.

Today, Dries committed a fine patch by Sun which moved a lot of test files into a happier place. After that commit, Dries reviewed Daniel’s Views patch which changed a couple of those same test files. Daniel’s patch failed to apply so Dries changed the issue to Needs Work. This doesn’t need to happen anymore. Git knows about file moves and can overcome this problem when we use it fully. If Daniel’s change was a merge instead of a patch, git uses its ancestry to apply changes to the moved test files. Try for yourself - the commands I ran are below.

We could all increase our velocity by using a merge workflow. Think of all the patches that need reroll currently which would not need reroll. Think of all the patches considered disruptive when they don’t need to be. I know that we can’t avoid all ‘Needs reroll’ events, but we can avoid a lot.

For now, I encourage folks to use d.o. sandboxes or Github to host their forks, and to post git merge commands into the issue queue instead of uploaded patches. Discussion of these changes should remain in the main issue queue. I think we should document this workflow at Drupal.org Git tutorials.

Feedback welcome.

Start from a clean 8.0.x:

# Create  branch whose tip is the commit before Sun's
$ git checkout -b daniel ebace9e
#Apply Daniel's patch
$ wget -O tmp.patch https://www.drupal.org/files/issues/1849822-view_render-19.patch; git apply tmp.patch;
$git commit -am "Fix #1849822. Convert (HTML) view rendering to a render array"
#Switch to main branch
$ git co 8.0.x
#Merge Daniel's patch and watch git's work its magic
$ git merge daniel
Auto-merging core/modules/views/tests/src/Unit/Routing/ViewPageControllerTest.php
Auto-merging core/modules/views/tests/src/Unit/Plugin/Block/ViewsBlockTest.php
Merge made by the 'recursive' strategy.
 core/modules/rest/src/Plugin/views/display/RestExport.php 
 core/modules/user/src/Tests/UserBlocksTest.php
 core/modules/views/src/Annotation/ViewsDisplay.php
 core/modules/views/src/Plugin/Block/ViewsBlock.php
 core/modules/views/src/Plugin/views/display/Attachment.php
 core/modules/views/src/Plugin/views/display/DisplayPluginBase.php
 core/modules/views/src/Plugin/views/display/Feed.php
 core/modules/views/src/Routing/ViewPageController.php
 core/modules/views/src/ViewExecutable.php
 core/modules/views/tests/src/Unit/Plugin/Block/ViewsBlockTest.php
 core/modules/views/tests/src/Unit/Routing/ViewPageControllerTest.php
 core/modules/views/views.module
 core/modules/views/views.theme.inc
 13 files changed, 147 insertions(+), 32 deletions(-)
Jul 16 2013
joe
Jul 16

It's been a really long time since I've worked on a Drupal build that didn't make use of the features module in some way or another. For better or worse it's turned into one of those modules like Views that's simply a part of the expected toolkit for many projects. Yet I still occasionally meet people who haven't heard of it yet. It's always fun to see the lightbulb over their head when you explain it to them and you can see the gears start churning as they immediately start imagining all the tedious box checking they can eliminate from their daily workflow.

The 2.x version of Features came out not too long ago and we are starting to make use of it in our work. It's a great update that's added a few new elements and greatly improved the usability of the the module. It reminds me again how important this module has become to solving so many different problems for us. So I want to take this opportunity to remind you that it exists. And for those of you who haven't used it before, hopefully this post will get the gears churning a little bit.

What exactly does Features do?

James Sansbury wrote an excellent retrospective on the Features module that does a good job of explaining the problem space and how we ended up with the Features module. In summary, one of the things that makes Drupal awesome is the ability to do so much site building without ever having to write any code. You can create content types, click together views, and change the default behavior of the user registration process all through your browser. The challenge in this is deploying those changes to other team members or from a development instance on your laptop to a live site.

Because content is likely continuing to be generated on the live site whilst you work away on your flashy new view you can't (???) simply replace the database on production with a copy from your laptop. We need a way to remember the configuration changes that we made on our local environment, so we can replicate them in the production environment. Enter Features. I often think of features as way of automating the things that I used to record in a checklist. Deploying meant going through my checklist and configuring those things on the live site really fast; hoping no one saw the site when things were broken for a few minutes. This presentation from DiWD 2008, while a little dated, does a good job of explaining the problem space.

Tell me more!

Want to learn more about using Features in your own workflow? Here are some good resources to help get you started:

In addition to the blog post linked above, James also has a recorded presentation that covers a lot of the high-level who, what, and why of features.

We have a complete series on creating features and deploying them with Drush that covers the majority of things you're likely to do with Features in your day-to-day workflow.

While Drush isn't required to use Features it certainly makes it more pleasant. If you haven't used Drush before or need some help we've got a series of videos covering Drush basics that will help you on that front.

I would also recommend taking a look at this blog post about naming things, since it's important to have a good naming convention when using Features.

Sometimes you're going to run into things within Drupal that can't be exported with the Features module. In this case, we still fall back on old reliable hook_update_N() and performing direct database queries. It's not always pretty, but it's a whole lot better than the checklist approach. If you find yourself needing to go this route, review this video on Altering the Database. It covers the process of creating your own update hooks, with a focus on altering the schema of an existing table. But you'll learn how to execute an update function and you can start adding your custom queries there.

Finally, what about writing your own modules and making the configuration data exportable via Features? We've got a short video about making things exportable via CTools. And in Drupal 7 you can make your custom entities built (using the Entity API)[] exportable with just a few extra tweaks to your controller.

Jun 04 2013
Jun 04

Features is a must have tool for deploying drupal sites. Features can be used for storing the website configuration in files, instead of the database. It can for example allow you to store a view in a file. You can then commit this code to your repo as usual.
Must of the site's configuration is stored in variables. Features, when used with strongarm, can be used to also export those variables from your staging env to your production. You can always find the name of the variable, by inspecting the html code. I use another trick to easily locate the variable name:

You can use

drush vget

to get the value of variable. For example,

drush vget cache

will return 1 or 0, depending on the value of the caching parameter set in admin/config/development/performance. If you dont know exactly the name of the variable, you can for example do a

drush vget pathauto

This will return all the variables that contain the word pathauto in the name.

Lets say that you need to change some variables of the pathauto module. Here is how to identify your variables:

drush vget pathauto > temp1

Change the module's options

drush vget pathauto > temp2

Then do a

diff temp1 temp2

The last command will show you which variables containing the word pathauto were altered by your operations. You can then use that name with strongarm on in a drush script to carry the variables to the desired env. Personally, I prefer using drush script, because it is faster and feels more direct.

To export a variable with drush, just type

drush vset var_name var_value

Did you like this post? Drop me a line in the comments below

Apr 18 2013
Apr 18

I use drush a lot during development. A common task is to restore the database or copy the database from the stg server to my local machine.
A lot of people use Backup and migrate to do this which works fine but is just too many steps for me. With drush, this can be as long as a single command:

drush sql-sync @site-dev @self --sanitize

This command will copy the db from the dev server to the local machine.
The --sanitize switch will sanitize the email addresses and the passwords of the users. This way, you will be sure that there will no email accidentally sent from the devel machine.
Be carefull, not to switch the order of @site-dev and @self or you will copy the local db to the dev server.
It is usually good to do a:

drush sql-drop

before running the sql-sync command. This will drop the local db and it will likely save you some time troubleshooting.

Another method copying the remote db is to use the sql-dump command:

drush @site-dev sql-dump --gzip --result-file=site-dev.sql

You can then decompress the file and import it:

gunzip site-dev.sql.gz;`drush sql-connect` < site-dev.sql

Did you like this post? Drop me a line in the comments below

Mar 14 2013
Mar 14

Episode Number: 

128

The Drupal 7 Panels module allows the creation of customized page layouts using an easy to use drag and drop page builder. Panels can be used to build custom pages, change the layout of node (or content type) pages, and even modify the Drupal user profile or account page.

In this episode you will learn:

  • An overview of what the Drupal Panels module is and how it can be used
  • A quick overview of Drupal 7 Panels options and configurations
  • How to create a custom Drupal 7 Panel page

Thanks to ModuleNotes.com for sponsoring this episode of the Daily Dose of Drupal.

DDoD Video: 

Mar 12 2013
Mar 12

If you're trying to trouble shoot problems on a Drupal site and need to get some information out of Drupal then using the command line can get you answers quick.

Traditionally calling any of the Drupal API functions to diagnose something would require peppering your modules with temporary debug code, or writing a whole new module to call API functions and display results.

The command line tool drush is a swiss army knife of useful Drupal functionality, and to make our life easier in calling Drupal API functions for a specific site we can use the 'php-ev' command, often abbreviated to 'ev'. With this command we remove the need to create new modules or adding temporary code to existing modules.

Because the PHP code being run is executed in the context of the Drupal site who's DocumentRoot you are in, you can pull information from the database about entities, users and the inner workings of Drupal.

Examples of useful things you can do:

  • Print a list of all modules which are reacting to a hook being called. Here we're finding all modules that hook in to the cron system to find one which could be executing a drupal_goto() or something equally naughty.
drush ev "print_r(module_implements('cron'))"
  • determine the alias of a system path.
drush ev "print_r(drupal_get_path_alias('node/3614'))"

If you come up with any handy clever ones share them on DropBucket.

Jan 25 2013
Jan 25

The issue

These last few days, I had noticed a problem with Drush Make and patches: some patches, be they rolled by our team or from elsewhere, would apply without a glitch, but some others, which worked normally according to the test bot on Drupal.org, would fail to apply without any obvious reason.

I had mostly put it out of my list of pressing issues when I really had to use an old version of OpenLayers, 7.x-2.0-alpha2 to be specific, AND apply a patch fixing one of the bugs in that module: behaviors plugin not being located correctly (http://drupal.org/node/1898662 if you want details). So I rolled the patch, tested it locally, the qa.d.o bot applied it and did not report more errors than expected for that old version.... and my Drush Make install refused to apply it.

Here was the relevant excerpt:

projects[domain] = 3.7
projects[domain][patch][] = "http://drupal.org/files/domain-foreach_argument-1879502-1.patch"
; ...snip...
projects[openlayers] = 2.0-alpha2
projects[openlayers][patch][] = "http://drupal.org/files/0001-Fix-the-path-file-declaration-for-behaviors.patch"
The Domain patch applied normally, but the OpenLayers patch would't apply. What could be wrong ?

The diagnostic

After tracing my way into DrushMakeProject::applyPatches(), I got a more explicit message: neither git patch nor trusted old patch could locate the patched file, includes/openlayers.behaviors.inc. Why ?

Comparing a standalone checkout of OpenLayers 7.x-2.0-alpha2 and the one in /tmp/make_tmp_(some id)__build__/sites/all/modules/contrib/openlayers, the problem became more obvious: that file was missing, as well as a good number of others. What ? Download failure ?

Not in the least: checking openlayers.info, I noticed the version downloaded by OpenLayers was no longer the chosen 7.x-2.0-alpha2 specified in the Makefile and which it previously downloaded normally, but the currently latest 7.x-2.0-beta3. Ahah...

The fix

After digging a bit more into Drush, it appeared that whenever you specify an extra info bit about a project download, lines in the short format like projects[openlayers] = 2.0-alpha2 are ignored, so Drush downloads the latest published version. The fix became obvious: use the "extended syntax", like this, for the same excerpt:

projects[domain][type] = module
projects[domain][version] = 3.7
projects[domain][patch][] = "http://drupal.org/files/domain-foreach_argument-1879502-1.patch"
; ... snip ...
projects[openlayers][type] = module
projects[openlayers][version] = 2.0-alpha2
projects[openlayers][patch][] = "http://drupal.org/files/0001-Fix-the-path-file-declaration-for-behaviors.patch"

This also explained why the other patches applied normally: each of them had been rolled against the latest module version, so the specified version was ignored, but the version actually being downloaded ended up being the same, and the patch applied normally.

Jan 15 2013
Jan 15

Average: 5 (2 votes)

Doug Hercules (dhercjr on drupal.org) is a graduate of the 2012 class of the DrupalEasy Career Starter Program (http://drupaleasy.com/dcsp) and currently working as an intern with DrupalEasy.

VirtualBox interfaceThis week I had the opportunity to clone a website from a git repository using Quickstart. Quickstart is a really quick, pre-made PHP Drupal development environment in a VirtualBox, which allows you to install a virtual machine to run Linux on your Windows PC. I've only gone through the process of cloning a site into Quickstart once before, and this time I thought I’d document it in my blog for myself and anyone else who might want to do this in the future. The general idea is this - there’s a site that I need to work on that is stored in a remote git repository. I want to get a copy of the site up-and-running inside Quickstart. To get started, I downloaded Quickstart, installed VirtualBox, then imported the Quickstart file into it.

Once I got Quickstart fired up on my Windows laptop, I set off to work. I used Drush (which comes built-in to Quickstart) to configure my virtual host and create an empty database and directory for the site. The Getting started with Quickstart page is a great resource for all the Quickstart specific Drush commands that it includes. At my command prompt I typed

drush qc apache dns database --domain=localsitename.dev

Important to note, this created an empty “localsitename_dev” database. Remember that underscore when you’re setting the name of your database later during Drupal's installation process.

Next, I wanted to clone (Quickstart also comes with Git pre-installed!) the site into the directory I just made, so I changed the directory to the new localsitename directory and I used the git clone command to copy the site from the remote repository. I added a “ .” ("space-period") at the end of the clone command so it cloned the repository into the current directory.

Next, I went to my browser and installed the site by going to http://localsitename.dev/install.php (remember, the Drush command I used automatically also set up this virtual host), this is where I needed to remember that underscore when I named the database. If you run into any files directory and/or settings.php issues, here’s some commands that might come in handy:

$ cp //to copy my settings.php file
$ mkdir //to make a new files directory
$ ls -al //to show me the permissions to each file
$ chmod //to change file permissions

Cheesy Poofs!

At this point, the site was installed with a fresh database. The final step was to get copy the database from the shared development server. I first enabled the Backup and Migrate module on my new local site, then I went to dev site and performed a quick backup (/admin/config/system/backup_migrate); saving it to my local “downloads” directory. 

Finally, I went back to my new local site and restored  (/admin/config/system/backup_migrate/restore) by uploading the file I just downloaded.

Poof! I have a cloned website!

Trackback URL for this post:

http://drupaleasy.com/trackback/555

Dec 15 2012
Dec 15

So I have just converted all the content on this blog to markdown. It was rather painful. I had really old content ranging as far back as 2005 in here, and I went through about 3 distinct markup filters here, most of which were irregular and changing according to the position of the sun, the drupal.org releases and wind speed. Now it's all markdown. This involved patience, drush and 3 hours of wasted time. Now, the fact that Markdown picked up speed is always a little strange to me. The syntax isn't particularly complete, which leads to non-standard extension like markdown-extra popping up, with the inevitable variations according to the language. Github, for example, has its own flavor of the famous markup. Finally, Drupal's filters are kind of klunky: the usual < url > markup doesn't work. So things are a little weird, but Markdown seems to be here to stay, or anyways it's the only markup I have seen supported reliably across multiple CMS and sites. One has to wonder why we are still stuck with plain old HTML on Drupal.org...

The actual conversion

The conversion was rather annoying. I had to track down all those formats, which meant mostly converting a wiki-like syntax from the freelinking module to markdown. (It's actually more complicated than that, because there was also the simplewiki filter, but let's ignore that because they were few and I just did them by hand.)

In the end, I arrived to the following script:

 2) {
    $mdwn = "[" . $match[2] . "](" . $match[1] . ")";
  } else {
    $mdwn = "[" . $match[1] . "](" . $match[1] . ")"; # hack: drupal fails on 
  }
  print "$orig\t=>\t$mdwn\n";
  return $mdwn;
}

$q = db_query("select node.nid, format, FROM_UNIXTIME(created) AS c, body, teaser, node.title from node_revisions inner join node on node.vid = node_revisions.vid where format = 1 AND ( teaser like '%[[%' OR body like '%[[%' ) order by created LIMIT 1;");

while ($row = db_fetch_object($q)) {
  print $row->nid . " | " . $row->format . " | " . $row->c . " | " . $row->title . "\n";
  $node = null;
  foreach (array('teaser', 'body') as $part) {
    print "checking $part... ";
    $newpart = preg_replace_callback('/\[\[(\|]*)(?:\|(]*))?\]\]/', 'wiki2mdwn', $row->$part);
    if ($newpart != $row->$part) {
      print "replacement... ";
      if (is_null($node)) {
        $node = node_load($row->nid);
        print "node loaded... ";
      }
      $node->$part = $newpart;
    }
  }
  if (!is_null($node)) {
    node_save($node);
    print "node {$node->nid} saved... ";
  }
  print "\n";
}

$q = db_query("SELECT nid, cid,FROM_UNIXTIME(timestamp),format, subject, comment FROM comments WHERE format = 1 AND comment LIKE '%[[%' ORDER BY cid LIMIT 1;");

while ($row = db_fetch_object($q)) {
  print "checking comment {$row->cid} in node {$row->nid} with subject {$row->subject}... ";
  $newcom = preg_replace_callback('/\[\[(\|]*)(?:\|(]*))?\]\]/', 'wiki2mdwn', $row->comment);
  print "\nsaving... ";
  db_query("UPDATE comments SET comment = '%s' WHERE cid = %d", $newcom, $row->cid);
  print "comment {$row->cid} in node {$row->nid} saved.\n";
}

Yes. This is klunky and ugly. But it works. If you have more than... say.. 200 nodes or comments to convert, I would strongly recommend optimizing this into SQL directly, but I was worried I would break stuff so I preferred operating on a preg_replace_callback() than plain SQL.

Oh, and this is a drush snippet, for those who don't know about that (rather old) drush feature, by the way. :) To run this, you basically dump this in a file and run it:

drush @anarcat.koumbit.org wiki2mdwn.php

Notice how I use a drush alias there - this one is automatically created by the Aegir this site lives on. Time saver.

So long and annoying, but at long last done!

Created at midnight, December 15th, 2012. Edited late Friday afternoon, February 6th, 2015.
Dec 10 2012
Dec 10

Average: 5 (3 votes)

Doug Hercules with shuttle3.. 2.. 1.. and lift-off of a new career!!! Here I go off to explore a new world that just 6 months ago I’d never even heard of! The strange blue teardrop world of Drupal!

For the past 20 years, I have been blessed to work my dream job as an engineer in the Space Shuttle Program, what a ride! The retirement of the shuttles meant a new direction in my life, and since there’s not a huge demand for rocket scientists these days, it meant seeking some new doors to open up and see what’s out there.

I happened upon a chance to enroll in a 10-week web development course called DrupalEasy Career Starter Program, so I dove right in not having any computer background, but realizing the web is the future! I successfully completed the class and am now starting an internship and an introduction into the community of Drupal. With all the helpful support in Drupal, it’s not a program you learn - it’s a universe of mutual relationships you join... so here’s my first step towards growing in the community.

Being a visual brained kinda guy, one of the most valuable things I’ve learned is it’s easier to learn a module if I just load it, enable it and see what it’ll do. So it became obvious that I wanted to make quick, disposable sites for testing and I didn’t want to spend a long time generating the site. I’m using the VirtualBox/Quickstart environment so from& Getting started with quickstart I found that from a command line prompt, I could create a site using a Drush make file with one simple command line:

[email protected]:~/websites$ drush quickstart-create all 
--domain=newsitename.dev --makefile=makefilename.make
--profile=standard

My make file includes about a dozen modules that I commonly use in most start-up sites, including admin_menu, module_filter and devel to name a few. For more info on building a make file check out http://drupal.org/node/1476014 . These simple steps get me up and running with a clean site for testing in just a couple minutes. Without the time investment of setting up a site, trying out a new module becomes very easy and eliminates the risk of harming any other site.

On the DrupalEasy.com redesign to Drupal 7 site, I’m working with the social media module and the Widgets module, it helps you put follow and share buttons into your website with sites like Twitter, Facebook and Google+, I’ll cover that more in my next blog post.

Note from Mike: Doug Hercules is a graduate of the 2012 DrupalEasy Career Starter Program. He is interning with DrupalEasy for the next few months and is currently working on the redesign of DrupalEasy.com on Drupal 7. You can follow Doug on Twitter at dhercjr.

Trackback URL for this post:

http://drupaleasy.com/trackback/548

Dec 05 2012
joe
Dec 05

In this lesson, we’re going to talk about what a Drush command is exactly. We're assuming you’re at least somewhat familiar with Drush and running commands, and general interaction with the terminal. In order to demonstrate both how Drush commands work, and to give a better idea of what we’ll be working towards, we're going to take a look at the Drush command we’re going to write for the Databasics module and explain a bit about arguments and options. We’re going to

  • Run a couple of Drush commands and see the difference between arguments and options in action.
  • Talk about the command that we’re going to learn to write in this series
  • See how to get a list of all the commands Drush knows about and the help text for those commands
  • Learn about the path Drush searches for the commands that it uses
Dec 05 2012
joe
Dec 05

Drush is one of the coolest tools available for Drupal developers. It provides all sorts of commands for assisting in Drupal development, automating workflows and making it easier to script various parts of the drupal workflow. But sometimes the commands that come with drush or those available from other modules just don’t quite cut it. Sometimes we need to automate things that are specific to our own site. Drush commands can also be really useful for quick one-off data migration tasks and other things for which writing a whole module might just be overkill. In this series we’re going to learn about:

  • What a drush command is and where drush finds the commands that it can use
  • How to use hook_drush_command in order to tell drush that we want to provide new commands
  • How to output data from our commands using some of drush's built-in helper functions.
  • Writing our own custom drush commands
  • Passing arguments to a command
  • Passing options to a command
  • How to prompt the user for input while your command is executing
  • Invoking other drush commands from our own
  • Creating make files

This series assumes that you’re already familiar with the basics of drush, that you’ve got it installed on your system and that you’re comfortable with running drush commands provided by core and contrib modules. If you’re not familiar with drush, you might want to start with the Introduction to Drush Series which covers all of the prerequisites for this series.

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web