Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Jun 15 2013
Jun 15

This is Part 1 in a series of posts on migrating a site from Squarespace 5 into Drupal 7. This is an overview of what we're doing and why, mostly discussing the format of the Squarespace 5 XML file, which is undocumented. My hope is to make the process of importing Squarespace data easier for everyone else.

The Mission

I'm working on a fairly large site (ApertureExpert.com), currently running on Squarespace 5, migrating to Drupal 7. Squarespace is an excellent platform with many virtues, but it doesn't allow the level of customization that my client is looking for, so we're moving the site to Drupal. Now, of course you lose the built-in, high-availability hosting, and some of the very good polished UI that Squarespace gives you, so this move is not for everyone. But if you want control, and to have your blog, forums, and e-commerce wrapped up in a single package, Drupal is your friend. One example: my client wants users to have all their purchased products, forum posts, and comments available through their user profile. Squarespace can't do that, Drupal can.

This site has been active for several years now, and has a very active community of users, so the first big task was to import all of those items into Drupal. Fortunately, Squarespace 5 will let you export your entire website as XML. Note: it doesn't look like Squarespace 6 (the current version) supports full site export. There is a WordPress-compatible export that you could import into Drupal using WordPress Migrate (or, obviously, into WordPress), but it seems to cover only blog posts. This site is older, running Squarespace 5, so we can get everything.

Now that we have exported data, what can we do with it?

Migrate

Enter Migrate module, maintained by Mike Ryan. My friend and colleague Ashok Modi has talked about Migrate many times at LA Drupal meetups and camps, and this was one of the first projects I've had where I really needed it. To move data using Migrate, you write a fairly simple PHP class for each thing (node, comment, user, taxonomy term, etc) and wrap them in a Drupal module. Migrate provides all the essentials of data migration: importing (duh), rollback (deleting imported items so you can try again, especially important if there's existing content on your Drupal site), updating previously imported items, easy mapping of fields, data massaging functions, dependency management, and more. As I'll now say to anyone who will listen: it's the best. Migrate can import from nearly anything: any SQL database Drupal supports (MySQL, Postgres, Oracle, MSSQL, etc), CSV and XML files, probably more. It has very good documentation and excellent example code. The example code is so good you can probably write many migrations with copy and paste, and a few modifications. It's been a while since I've had to do this kind of work manually, but if there's a framework or CMS out there with a better migration tool than this, I'll eat my hat.

Giant XML File of Doom (GXFD)

It's great Squarespace will export all your data, but unfortunately the file is completely undocumented. There's no schema, no comments, no web page explaining anything about it. You can't even use it to backup a site and import it back into Squarespace! It's a lot better than nothing, and at least it's XML (i.e. not some weird proprietary format), but it's just barely sufficient.

Another problem: it's One. Giant. File. I can't provide an example file from this website, but I can say this: this site that's been in existence since the beginning of 2010 has hundreds of blog entries with thousands of comments, over ten thousand forum posts, all by several thousand users. Its export file is about 50 MB of ugly XML. That's a lot of text, making it difficult to deal with. Without documentation, that meant I had to open the file and inspect it myself to figure out how it was put together. I tried several dedicated XML editors, thinking their outlining tools would help, but none of them was able to work with a file this large efficiently. At least not on my computer (which is just a couple years old, and quite a champ in my opinion). I thought I might finally need to upgrade to an SSD to make this work! BBEdit, my go-to editor, was able to open the file, probably because it doesn't do outlining or structure scanning on most XML files, so I kept searching for something helpful.

XMLstarlet

Enter XMLstarlet. This command-line utility (I installed it with homebrew) is an XML powerhouse. I used it primarily to run XPath queries while figuring out the structure of the GXFD, and to extract its structure. I strongly recommend it. It doesn't support XSLT 2.0, which would have helped me a time or two, but overall it's the bee's knees. Once installed, the command is xml, with lots of options (see its documentation for everything). Here are a couple examples:

To dump the element structure of an XML file (the -u switch gives you all the elements, but cuts out duplicates):

% xml el -u ss-data.xml

The output looks like this, showing attributes with @

squarespace-wireframe
squarespace-wireframe/@websiteId
squarespace-wireframe/@websiteIdentifier
squarespace-wireframe/@createdOn
squarespace-wireframe/audiences
squarespace-wireframe/audiences/audience
squarespace-wireframe/audiences/audience/name
squarespace-wireframe/audiences/audience/id
squarespace-wireframe/audiences/audience/display-name
squarespace-wireframe/audiences/audience/description
squarespace-wireframe/audiences/audience/description/@encoding
...

I usually save these directly into a file using a redirect:

% xml el -u ss-data.xml > filename.txt

Or pipe the output right into BBEdit:

% xml el -u ss-data.xml | bbedit

You can download the full structure of the Squarespace 5 site I'm working on here. It's a zipped text file.

As you can see, there's a lot of stuff in there, and trying to sort out this structure visually would have been basically impossible. I imagine, but don't know, that this is pretty similar across Squarespace 5 sites. Without any data, it's 71 KB. That's a lot of elements!

Segmenting the Export File with XPath

To build the migration, I needed to find the XPath queries that would return the data I wanted. This took a while, but using XMLstarlet on a BBEdit shell worksheet, I was able to experiment easily, and save the commands as I went. Shell worksheets are like a scratchpad for the terminal, and were very helpful in this work. In each of the following cases, I used XMLstarlet to extract the relevant queried item into its own file, preferably one per migration. It means some extra work up-front to produce the files, but that could be automated if needed, and each migration can use less memory by processing a smaller file (or files in some cases).

The commands I used to create these smaller files look like this:

% xml sel -I -t -m "/xpath/query/element" -c . -n squarespace-data-file.xml > squarespace-subset.xml

The command sel tells XMLstarlet to select a portion of the document using XPath. You can see what all the switches mean in the documentation, but basically in this command I'm select /xpath/query/element with XPath against the source document squarespace-data-file.xml, and saving the result to a file called squarespace-subset.xml.

After you've done this, you end up with a file full of elements, like this:

(lots of data)(lots of data)(lots of data)(lots of data)(lots of data)

To make this a valid XML file, you need to add a containing element, and an XML header:

(lots of data)(lots of data)(lots of data)(lots of data)(lots of data)

Having done that prep work, you'll be able to use the file with Migrate.

Finding the Goodies with XPath

Here are the XPath queries I used to extract the relevant data, and samples (trimmed and modified for public consumption) of the results.

User accounts:

/squarespace-wireframe/website-member-accounts/website-member-account

These are fairly straightforward, thankfully.

XXXXXXX[email protected]MisterPersonmisterpersonfalsetruetruecbjRTKkQDuQcHpjdQR2MqQS3JlE=Mister Personality

Everything I need is in these elements. Users will need to reset their passwords, but their email addresses, usernames, and everything else is preserved. Nice!

Blog posts:

/squarespace-wireframe/sections/section/modules/module[id='0000000']/content/journal-entry[not (parent-id)]

This query says "give me all the blog posts in module 0000000 that do not have a parent-id element". parent-id indicates a Followup post, explained below.

The biggest issue here is finding the Module ID. View the HTML source of your blog's main page, and you should see something like this in some initial JavaScript:

Squarespace.Constants.CURRENT_MODULE_ID = "0000000";

Or this in the page's body tag:

Once you have that ID, plug it into that XPath query, and you should be all set.

Sample data:

My first post!2010/1/1/my-first-post.html
  <body encoding="base64">R29vZCB0aW1lcyE=</body>
  11111112010-01-01 18:28:07.02010-01-12 01:05:58.2870000000truetrue9999999

Once again, pretty much everything I need to create a node is here. I can even preserve URL paths using that url-id element. +1 for Squarespace here. -100 for making me dig through XML to find it on my own, but still, thanks. Also note the base64-encoded body, which will be dealt with later.

Blog comments:

/squarespace-wireframe/sections/section/modules/module[id='0000000']/content/comment
/squarespace-wireframe/sections/section/modules/module[id='0000000']/content/journal-entry-comment

Yes, two queries here. Unfortunately, comments do not contain a reference to their parent post. The first query returns something like this:

  <body>I have a comment!</body>
  1234560000000true8675309

The second:

654321000000086753091111111

Life would be a lot easier if that journal-entry-id in the second query were in the first comment element instead of in this separate element. I wasn't able to find a way to modify the XML using XSLT to make that change. If I were going to spend my life migrating Squarespace sites to Drupal, I would write some code to do this transformation, in XSLT or some other language. If you're reading this and know how to do that, let's talk!

Forum posts

/squarespace-wireframe/sections/section/modules/module/modules/module/content/discussion-post[not (parent-id)]

This query says "give me all the forum posts that do not have a parent-id element". (I didn't include a module ID because I wanted all forum posts at once, so I could segment them by container in the migration code.) Unlike blog posts, the content of these posts is not included in the returned data. Example:

false000000077777771111111

As with blog comments, you need a separate query to retrieve the content.

/squarespace-wireframe/sections/section/modules/module/modules/module/content/comment

The returned data looks like the second blog comment query above. In this case, it would be nice to insert the content of the forum post right into that discussion-post element, but as with the comment issues, I didn't spend the time to figure out how to do that.

Forum comments

In Drupal's core Forum module, original forum posts/topics are nodes, and all follow-ups are comments. Each Migrate class creates a single type of object, so I had a separate class for forum comments. Here's the XPath to retrieve those:

/squarespace-wireframe/sections/section/modules/module/modules/module/content/discussion-post[boolean(parent-id)]

This is very similar to the forum posts above, but instead of not, I use boolean, which means "give me all the discussion-post elements that do have contain a parent-id element". So it's just like the content above, but with one extra element:

false0000000777777711111112222222

As with blog comments, I need a separate query to retrieve the content. In this case, I used the same file generated by the second query for forum posts above, and let my migration code pull out the right ones.

Followup posts

Squarespace lets you create separate posts that are displayed above comments, but below the original post's content. These are just like the Blog posts above, but include a parent-id element.

/squarespace-wireframe/sections/section/modules/module[id='000000']/content/journal-entry[boolean(parent-id)]

The data look exactly the same as regular blog posts. The trick here was figuring out what to do with them, since Drupal doesn't offer an equivalent object. For this first pass, we opted to update the bodies of the posts, but you could also create special comments, or even create a custom Drupal entity.

Next steps

Having worked through all this, you'd have a set of smaller, more readable XML files for your migrations. Next up, we'll look at how to write the PHP code that powers these migrations, and how to run them using every Drupalero's friend: drush.

Special thanks to Joseph Linaschke, fine photographer and proprieter of ApertureExpert.com, who hired me to do all this work.

Jun 14 2013
Jun 14

CKEditor 4.x has been out for a while now. Something I really enjoy about the new release is the new skin, for which the people at CKEditor ran a contest. The winner of the contest was Moono, but I also really like the silver skin. So today I want to show you how you can change the skin when using CKEditor 4.x in Drupal. There is an overview of skins on ckeditor.com, but there's not much there yet. Moonocolor is worth a look, but we are going to focus on silver, which you can find on Github.

I'm going to show you how it's done by writing just a few lines of code (which stBorchert wrote for me :)) and I'm also including a feature module which you can just throw into your site to get going (make sure to grab a database dump first, just in case).

Assumptions

Here's the code that goes in your custom module's .module file:

  1. /**

  2.   * Implements hook__wysiwyg_editor_settings_alter().

  3.   */

  4. function MODULENAME_wysiwyg_editor_settings_alter(&$settings, $context) {

  5.  global $base_url;

  6.   if ($context['profile']->editor == 'ckeditor') {

  7.    $skins_path = drupal_get_path('module', 'MODULENAME') . '/ckeditor/skins';

  8.    // For flexibility we use a variable to get the active skin name.

  9.    $active_skin = variable_get('MODULENAME_skin', 'silver');

  10.    // Set custom skin.

  11.     $settings['skin'] = sprintf('%s,' . '%s/%s/%s/', $active_skin, $base_url, $skins_path, $active_skin);
  12.   }

  13. }

I created a WYSIWYG feature that serves as my custom module and contains my text format, its wysiwyg settings and the silver skin. If you have WYSIWYG module and CKEditor where they belong (as stated under Assumptions) then you can just download this feature and activate it like you would any other module. I created my own text format so it doesn't interfere with the text formats you might already have on your site. The settings are identical to the Filtered HTML format. Check the .module file of the feature to see above code in action. I have included a README.txt in the feature. You have to switch to text format undpaul WYSIWYG to see the editor in the silver skin.

Download the feature

Get the feature and try it out!

Have fun and let me know what you think.

Jun 04 2013
Jun 04

The Date Picker widget, when using webforms, defaultly appears in American form (Month, Day, Year). For UK sites this isn't often desirable. Doing a quick google and I stumbled upon 's blog post on how to re-theme the element from earlier this year. I thought it was certainly one approach, but felt there must have been a neater alternative. So here is my approach:


/**
 * Implements hook_webform_component_render_alter().
 */
function THEMENAME_webform_component_render_alter(&$element, &$component) {
  if ($element['#type'] == 'date') {
    $element['#process'][] = 'THEMENAME_webform_expand_date';
  }
}

/**
 * Process function to re-order the elements in the date widget.
 */
function THEMENAME_webform_expand_date($element) {
  $element['day']['#weight'] = 0;
  $element['month']['#weight'] = 1;
  $element['year']['#weight'] = 2;
  return $element;
}

Obviously you can replace THEMENAME with what ever theme or module name you like; in my case I put this in the template.php file.

Jun 02 2013
Jun 02

Back when Drupal 7 was being developed, there was a big inititiative about getting new pretty themes into Drupal core. There were three really good suggestions, one of them being Bartik, which the most people got behind, so it could be finished in time to get into Drupal 7 core. One really good suggestion that couldn't be finished in time was Corolla by Jeff Burnz, which is now a contrib theme. The third suggestion was our theme Busy by eigentor, a theme targeted at corporate websites. Just like Corolla, there wasn't enough time to finish it up for Drupal 7 core. Busy has been a contributed theme ever since, because we didn't want it to get lost only because it didn't make it into Drupal core.

After DrupalCon Portland we got really excited about Drupal 8 so we decided to see how porting Busy would go. Work had to be done to exchange some template variables and rename some files, but it was pretty straightforward and only took a little more than an hour to finish. After twig was moved to core as the new default template engine, some more work had to be done (see "Upgrading a phptemplate theme from Drupal 7 to Drupal 8").

Busy will stay minimally maintained for now, though we might port it to use twig in the future.

If you want a new look for your Drupal 8 site, check out Busy and let us know what you think!

Upgrading a phptemplate theme from Drupal 7 to Drupal 8

So now that twig is in core as the default template engine, how do you port your theme to Drupal 8? We considered converting the theme to twig, but that would take up too much time for now. We just wanted to publish the theme for Drupal 8 as soon as possible. At first of course, we ported the theme to the new structure. These are just a few changes, like renaming the THEME.info file to THEME.info.yml and changing its contents to yml, which is simple. You can check core's bartik.info.yml to see how this needs to be done. Your template.php file is renamed to THEME.theme, the contents of this file didn't have to be changed for Busy. We did have to change some variable names in some of our template files (like the comment template) by referring to core's templates, which are nicely documented within each file. For example, in our comment.tpl.php, we had to replace $picture with $user_picture.

THEME.info.yml file

If you have a phptemplate theme, simply add this line to the theme's .info.yml file:

  1. engine: phptemplate

Without this line you will not even be able to activate the theme.

Menus, breadcrumbs, messages

After adding this line, you might still have problems seeing the theme you're used to. In Busy we got fatal errors due to the way we printed the menus in our page.tpl.php. This is what it looks like in Drupal 7:

[gist:5690836]

And this is what we need to do now to print menus like the main menu in your page.tpl.php:

If you want to add classes or ids to the menu, you can do the following in your THEME.theme file (In Drupal 7 this file is called template.php):

  1. /**

  2.  * Implements hook_preprocess_HOOK() for page.tpl.php

  3.  */

  4. function busy_preprocess_page(&$variables) {

  5.   // Pass the main menu and secondary menu to the template as render arrays.

  6.   if (!empty($variables['main_menu'])) {

  7.     $variables['main_menu']['#attributes']['id'] = 'main-menu';

  8.     $variables['main_menu']['#attributes']['class'] = array('links', 'clearfix');

  9.   }

  10.   if (!empty($variables['secondary_menu'])) {

  11.     $variables['secondary_menu']['#attributes']['id'] = 'secondary-menu';

  12.   }

  13. }

Now if you print variables like $messages just using print $messages, you will not get any output. Look through the code in your page.tpl.php for any print $VARIABLE and make it print render($VARIABLE). We had to do this for $breadcrumb and $messages.

Drupal 8's theming system is still under heavy construction and I'm sure there will be more stuff we'll have to change in our template files very soon, but this way works well for now if you are just interested in porting your phptemplate theme to Drupal 8.

Apr 18 2013
Apr 18
Oh The Huge Manatee! Author Image

Thoughts and technical notes from the field. I'm Campbell Vertesi, a principal software engineering lead at Microsoft, opera singer, and parent. These are the things that go through my mind.

Apr 04 2013
Apr 04

The Views module is the most installed Drupal module and gives great power to developers for building websites. Because of its rich feature set, Views will be integrated in core as part of Drupal 8. Developers may extend the Views functionality by writing custom plugins using the complex plugin system. This helps implementing functionality for special use cases in Views.

In this blog post I would like to explain how to write a custom "Default Argument Handler" for Views and how to develop a simple context system using the Flag module by providing a sample use case as example.

Hint: The complete code of the module can be accessed for testing purposes at github.

The Use Case

Our client produces organic pet food as a family business and wants to increase sales using e-commerce. Furthermore, the daughter of our client will publish a monthly magazine with tips and tricks for each dogs and cats. The magazine should be advertised on the new website of the customer. Older magazine arcticles will be published on the website later on.

A key feature of the new website will be a context system which delivers products and articles to the user based on his favorite pet. We use the Flag module to flag the pets for each user.

Hint: To keep it simple we flag the pets by hand.

Preparations

Creating flags

First of all we create a vocabulary "flag context" with some terms which will be flagged later.

Afterwards we create a new flag at admin/structure/flags with the flag type "taxonomy term" choosing "flag context" as bundle. 

Extending the content type

By adding an "Entity Reference" field to a content type (i.e. article of the standard profile) we make sure that we can link content to the terms which will be flagged by the user. After adding the field we create at least one node for each term. For this use case we would also need to add this field to our products but this will be sufficient for our example.

Next, we have to flag a term for testing the functionality later. Just visit a term page and flag the term using the link on the page.

Creating Views

The user should see what their favorite pet is. We make this happen by creating a View.

The View filters terms by our created vocabulary "flag context" and has a relationship "Flags: Taxonomy Term flag" on "Flag Context" and  "Current User".  We use "block" as a display for easy embedding.

In addition to the title we also add a flag link allowing the user to unflag the term in the block.  

For testing purposes, we add the new block to the "sidebar first" region. 

At last, we create the basics for our final test. We create a page View showing only "article" nodes. We add "Content:has taxonomy term ID" as "Contextual Filter". This filter will be extended later with our context functionality.

Coding the handler

A views handler has to been included in a module to be recognized by the plugin system of Views. It is not a lot of work but despite that there are some lines where we could struggle. 

1. The .info file

A common pratice is to put views handlers and plugins in their own subdirectorys. Drupal needs to know where those files are located so we include the path into the info file.

  1. name = Views Flag Context

  2. description = Provides a flag context.

  3. core = 7.x

  4. package = Flags

  5. version = 7.x-0.1

  6. dependencies[] = flag

  7. files[] = handlers/views_argument_default_flag.inc

2. The .module file

We need to implement hook_views_api to tell Views that our module is of interest: 

  1. /**

  2.  * Implements hook_views_api().

  3.  */

  4. function views_flag_context_views_api() {

  5.   return array(

  6.     'api' => '3',

  7.   );

  8. }

Now, Views looks for a file with relevant code. Best practice is to create a modulename.views.inc file.

3. The .views.inc file

By implementing hook_views_plugin we return an array in which we define the kind of plugin we like to create and the place where to find it.

  1.  /**

  2.   * Implements hook_views_plugins().

  3.   */

  4. function views_flag_context_views_plugins() {

  5.     'argument default' => array(
  6.       'views_argument_default_flag' => array(
  7.         'title' => t('Default Flags'),

  8.         'handler' => 'views_argument_default_flag',

  9.         'path' => drupal_get_path('module', 'views_flag_context') . '/handlers',

  10.       ),

  11.     ),

  12.   );

  13.   return $plugin;

  14. }

  15. ?>

With these 3 steps we prepared our handler. Now it is time to add functionality to the handler.

4. The views_argument_default_flag.inc file

In this file we define the handler as class by inheriting and overriding methods from the views_plugin_argument_default.

The tree:

  1.  /**

  2.    * Define the options form to enable selection of flags.

  3.    */

  4.   function options_form(&$form, &$form_state) {

  5.   }

  6.   /**

  7.    * Return all possible options for the view and provide default values.

  8.    */

  9.   function option_definition() {

  10.   }

  11.   /**

  12.    * Provide the default form form for submitting options.

  13.    */

  14.   function options_submit(&$form, &$form_state, &$options = array()) {
  15.   }

  16.   /**

  17.    * This function controls what to return to the contextual filter.

  18.    */

  19.   function get_argument() {

  20.   }

  21.   /**

  22.    * Initialize this plugin with the view and the argument is is linked to.

  23.    */

  24.   function init(&$view, &$argument, $options) {

  25.     parent::init($view, $argument, $options);

  26.   }

  27. ?>

First of all, we define the option form of the handler. We want to configure which flags or vocabularys we want to use later. This makes it possible to add other flags or vocabularies very easily. Furthermore, it should be possible to add multiple terms and also to define a fallback if there is no content available. 

To achieve this we get all flags from the system and create a checkboxes form item. It is important to implement the "default value" of the options correctly. The options are available in the views object:

  1. '#default_value' => $this->options['flags'],

  2. ?>

The complete function:

[gist:5264081]

As with all forms we also can add a validate and a submit function. We only add a submit function for now.

  1.   /**

  2.    * Provide the default form form for submitting options.

  3.    */

  4.   function options_submit(&$form, &$form_state, &$options = array()) {
  5.     // We filter the options on only selected ones.

  6.     $options['vocabularies'] = array_filter($options['vocabularies']);
  7.   }

Now we have to override the get_argument() function. This function will return values to the filter in the view.

We compare the flagged content with the values from the options. We filter all term ids by their vocabularies using a helper function. It is important to return the value as a string just as we would call the argument by hand.

  1.   /**

  2.    * This function controls what to return to the contextual filter.

  3.    */

  4.   function get_argument() {

  5.     // Get available flag types from the system.

  6.     $flags = flag_get_flags('taxonomy_term');

  7.     // Get all User flags.

  8.     $user_flags = flag_get_user_flags('taxonomy_term');

  9.    

  10.     // This array will collect all Term IDs which will be filtered.

  11.    

  12.     // Get the vocab foreach flag.

  13.     foreach ($flags as $flagname => $flag) {

  14.       // We only proceed with this flag, if it has been selected and the current

  15.       // user has flagged terms with that flag.

  16.       if (!empty($this->options['flags'][$flagname]) && !empty($user_flags[$flagname])) {
  17.         // Get all tids from the user flags.

  18.         $user_tids = array_keys($user_flags[$flagname]);
  19.         $vocabs = array();
  20.         // Check  which vocabularies are valid for this handler.

  21.         foreach ($flag->types as $vocab) {

  22.           if (!empty($this->options['vocabularies'][$vocab])) {
  23.             $vocabs[] = $vocab;

  24.           }

  25.         }

  26.         // We add the valid terms of the flag set to our default argument list.

  27.         $valid_tids = _views_flag_context_filter_tids_on_vocabularies($user_tids, $vocabs);

  28.       }

  29.     }

  30.     // If no tids are valid, we can fallback to a given value.

  31.     if (empty($tids)) {
  32.       // Fall back to the exception value (by default this is 'all')

  33.       return $this->options['fallback'];

  34.     }

  35.     // If there are term ids available we return them by concating the terms

  36.     // with the multiple operator (AND or OR).

  37.     else {

  38.       return implode($this->options['multiple_operator'], $tids);
  39.     }

  40.   }

The helper function for the .module file:

  1. /**

  2.  * Helper to filter tids on given vocabularies.

  3.  *

  4.  * @param array $tids

  5.  *   array of term ids

  6.  * @param array $vocabularies

  7.  *   array of vocabulary machine names

  8.  *

  9.  * @return array

  10.  *   array of terms that live in one of the given vocabularies.

  11.  */

  12. function _views_flag_context_filter_tids_on_vocabularies($tids, $vocabularies) {

  13.   $query = db_select('taxonomy_term_data', 't')

  14.     ->fields('t', array('tid'))
  15.     ->condition('t.tid', $tids);

  16.   $query->innerJoin('taxonomy_vocabulary', 'v', 't.vid = v.vid');

  17.   $query->condition('v.machine_name', $vocabularies);

  18.   return $query->execute()->fetchCol();

  19. }

This is all for the handler.

Next, we add the handler to the contextual filter we recently created.

After applying the changes there should be the article which matches our favorite pet in the preview.

Summary

The Flag modules makes it easy to handle user preferences. Page elements can react to different flags and return relevant content to the user by using the handler we created. With a few tricks we can extend the functionality, i.e. adding automatic flagging with rules reacting on content views.

Github: https://github.com/Cyberschorsch/views_flag_context

Mar 26 2013
Mar 26

Sometimes you would like to add a map to a node or block without the need for detailled configuration options. You simply want to display a map and be done with it.
Fortunately this is an easy task using Leaflet.

Say you have a value for the location and one for the country and would like to print this "address" in a map.
So you need to first install Leaflet and Geocoder and then use this function to generate the map:

  1. /**

  2.  * Generate a simple map with a location pointer.

  3.  *

  4.  * @param string $location

  5.  *   Location to use (for example the address).

  6.  * @param string $country

  7.  *   Name of the country to use.

  8.  *

  9.  * @return string

  10.  *   The rendered map.

  11.  */

  12. function mysimplemap_map_create($location, $country) {

  13.   $map = '';

  14.   // Join the address parts to something geocoder / google maps understands.

  15.   $address = sprintf('%s, %s', $location, $country);
  16.   // Try to create a geographic point out of the given location values.

  17.   if ($geo_point = geocoder('google', $address)) {

  18.     // Create a JSON equivalent to the point.

  19.     $geo_json = $geo_point->out('json');

  20.     // Get map implementation provided by http://drupal.org/project/leaflet_googlemaps.

  21.     $map = leaflet_map_get_info('google-maps-roadmap');

  22.     // Set initial zoom level.

  23.     $map['settings']['zoom'] = 16;

  24.    

  25.     // Decode the JSON string.

  26.     // Create settings for the map.

  27.     $map_features = array(
  28.         'type' => 'point',

  29.         'lon' => $geo_data->coordinates[0],

  30.         'lat' => $geo_data->coordinates[1],

  31.       ),

  32.     );

  33.     // Render the map with a fixed height of 250 pixels.

  34.     $map = leaflet_render_map($map, $features, '250px');

  35.   }

  36.   return $map;

  37. }

  38. ?>

Easy, isn't it?

Mar 22 2013
Mar 22

As part of working on the site upgrade, our messaging and notifications system is discussed. Unfortunately we don't have a reliable number on how many people rely on email notifications from localize.drupal.org (due to automated signups) and the complexity of the upgrade process for notifications (given that we need to migrate to new modules) might set back our upgrade process even longer.

While the Drupal Association is busy with updating drupal.org itself to Drupal 7, the DA is not directly involved with the localize.drupal.org site maintenance and upgrade (similar to other subsites), so we are not tied to the main site upgrade in any way (neither do we get financing for this work).

One possible shortcut is to sidestep email notifications for the immediate upgrade and get back to the feature later. Are you heavily relying on notifications from localize.drupal.org? Please speak up at http://drupal.org/node/1392694! Thanks!

Mar 20 2013
Mar 20

We're delighted to announce that Annertech will be sponsoring the upcoming Drupal Developer Days 2013 being held in Dublin later this year. We've signed up as a silver sponsor and are really looking forward to the event.

Drupal Developer Days is an annual European event that brings together the people who develop, design and support the Drupal platform. The event will feature dozens of sessions and panels from some of Drupal's best contributors, providing developers and site builders with a chance to both learn and share their knowledge with other Drupal professionals.

This year's event is particularly special. Drupal Developer Days 2013 coincides with the final days of development prior to Drupal 8 code freeze. In light of this, this year there will be a week-long Drupal 8 core code sprint in the week leading up to the Drupal Developer Days event.

It should be a brilliant event, and there are only a limited number of sponsorship slots still available, so please help support the event.

Support Drupal Developer Days too!

Mar 03 2013
Mar 03
Introducing fluxkraft - web automation simplified with the power of Rules! fago Sun, 03/03/2013 - 16:36
Feb 28 2013
Feb 28

Ever wanted to give form elements a custom look? Theming entire forms is straightforward, if laborious. But theming individual textboxes, checkboxes and radio buttons is slightly more obscure. Keep reading to find out how to fully customize your form elements, from the input itself to the label.

You might be wondering: can't I just implement a theme_checkbox() function? Well, that might meet your requirements, but this doesn't handle the label at all. Let's set up our theme to give us maximum flexibility.

Let's start with a use case: Bootstrap likes to have checkboxes inside the label element. I happen to like this markup too. But Drupal places the checkbox next to the label. How can we achieve this without hacking core?

Start by copying over theme_form_element() into the template.php file in your theme. Let's assume the theme is called example. You can find the body here. Be sure to replace the word theme with example.

Now find the switch statement and replace it to look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

if ($element['#type'] == 'checkbox') {
  $variables['rendered_element'] = ' ' . $prefix . $element['#children'] . $suffix . "\n";
  $output .= theme('form_element_label', $variables);
}
else {
  switch ($element['#title_display']) {
    case 'before':
    case 'invisible':
      $output .= ' ' . theme('form_element_label', $variables);
      $output .= ' ' . $prefix . $element['#children'] . $suffix . "\n";
      break;

    case 'after':
      $output .= ' ' . $prefix . $element['#children'] . $suffix;
      $output .= ' ' . theme('form_element_label', $variables) . "\n";
      break;

    case 'none':
    case 'attribute':
      // Output no label and no required marker, only the children.
      $output .= ' ' . $prefix . $element['#children'] . $suffix . "\n";
      break;
  }
}

Notice we wrapped it with an outer if statement. Here, we a define custom behavior for our checkbox. This allows us to give checkboxes special handling for labels using a custom theme_form_label(). Let's implement that, by copying the existing theme_form_element_label() from here and changing theme to example. But let's make a change. Remove the return statement (and the comment above it) and replace it with:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

// Bootstrap wants us to add a class to the label as well.
if ($element['#type'] == 'checkbox') {
  $attributes['class'] = 'checkbox';
}

// The leading whitespace helps visually separate fields from inline labels.
if (!empty($variables['rendered_element'])) {
  return '  . drupal_attributes($attributes) . '>' . $variables['rendered_element'] . $t('!title !required', array('!title' => $title, '!required' => $required)) . "</label>\n";
}
else {
  return '  . drupal_attributes($attributes) . '>' . $t('!title !required', array('!title' => $title, '!required' => $required)) . "</label>\n";
}
''

Last but not least, Bootstrap also likes us to add the class checkbox to the input element itself. Though this isn't necessary for any CSS rules, the guide shows the markup this way. So it's best to be safe. We do this by overriding the theme_checkbox() function. Change the _form_set_class() function call to:


_form_set_class($element, array('form-checkbox', 'checkbox'));

And that's it. You now have a fully themed Bootstrap checkbox. More importantly, adding future elements and theming is very quick. You only need to set up all these theme functions once.

Check out the full template.php file for reference.

Feb 26 2013
Feb 26

Following up on the first blog post on Theming in Drupal 8 with Twig, this second part will cover Twig's syntax.

In order to explain these changes more clearly I want to compare the known PHPTemplate syntax with the new Twig template syntax. All examples mentioned in this blog post are based on the latest development state of the Drupal 8 Twig Sandbox.

Usually themers only have to care about the output of predefined variables or simple control structures (if conditions or for loops that output lists of content). Complex logic or further processing of variables should not be placed in the template files - I think everybody can remember a situation in which they inserted a view programmatically in a template or placed custom field on theme layer. In most cases the ability to use custom PHP in template files lead to more complexity and a lack of clarity.

Using variables in templates:

Example for the usage of variables and simple if condition in PHPTemplate:

[gist:5011792:taxonomy-term.tpl.php]

Example for the usage of variables and simple if condition in Twig:

[gist:5011792:taxonomy-term.html.twig]

In the example shown above you can clearly see the differences. Instead of using the PHP'S print function or Drupal's render() function, Twig offers the {{ }} syntax for outputting variables. Twig also improved the output of array and object elements on the theme layer. In the future we will be using the following consistent "point-syntax". Here it does not matter if we want to access array or object elements.

Example of a Views exposed filter in PHPTemplate:
[gist:5011792:syntax.php]

Example of a Views exposed filter in Twig:
[gist:5011792:syntax.twig]

Control Structures

In addition to the already mentioned if conditions there is the possibility to use for loops with the {% %} syntax.

for loop example in PHPTemplate:
[gist:5011792:book-all-books-block.tpl.php]

for loop example in Twig:
[gist:5011792:book-all-books-block.html.twig]

As in earlier versions of Drupal you should avoid putting too much logic in template files. For this purpose you should use the known preprocess function provided by Drupal. Besides the new {% %} syntax in the templates the example also shows the usage of the new nav elements of HTML5 standard, which also has been introduced in other Twig templates.

Comments in templates

For documentation in templates, Twig provides the {# #} syntax.

Example for comments in PHPTemplate (phpdoc style):
[gist:5011792:comments.php]

Example for comments in Twig:
[gist:5011792:comments.html.twig]

Reusability of templates via includes

Even though it was possible to use the PHP include or include_once function for re-using existing parts of a template, it was rarely used in the previous versions of Drupal. In Twig these includes will be used much more often. A good example for the use of includes is the image module - based on the imagestyle another template file will inlcuded by the include function of Twig.
[gist:5011792:image-formatter.html.twig]

With the help of the paramater "with" you can specify variables that are available in the scope of the included template.

Usage of filters in templates

Twig filters offer the possibility to influence the output of variables on the theme level. The following examples show a common use case for filters in Twig - they replace the check_plain function and the t-function with its Twig counterpart.

Translation in templates

Example for translations in PHPTemplate:

Example for translations in Twig:
[
gist:5011792:node-admin-overview.html.twig]

As already known from the  t-function you can use placeholders in Twig to make the translatable strings more dynamic. In the example above the variable $votes and type are replaced in the strings.

Escaping special chars in templates

In previous Drupal versions you were able to convert special chars to HTML with the help of the check_plain function.

Example in PHPTemplate:
[gist:5011792:checkplain.php]

Example in Twig:
[gist:5011792:checkplain.twig]

The escape filter can also be used via the alias e and supports parameters for different escape strategies.

  • html: escape a string for HTML body context
  • js: escapes a string for JavaScript context
  • css: escapes a string for CSS context
  • url: escapes a string for URI or parameter context
  • html_attr: escapes a string for HTML attribute context

Further information on Twig filters can be found in the Twig's documentation.

Feb 18 2013
Feb 18

I recently evaluated the Bakery Single Sign-On System aka Bakery SSO aka Bakery on behalf of clients. This article describes how I moved from finding a small weakness in version 2.x-alpha-3 to an exploit.

If you haven't updated all your sites to Bakery 2.0-alpha4 (6.x, 7.x), go and do so now.

About Bakery

Bakery provides a "single sign on" feature for Drupal based sites that are on the same second-level domain (i.e. example.com, subsite.example.com, subsite2.example.com). The Drupal.org family of sites uses Bakery so you can login on Drupal.org then visit and browse groups.drupal.org, association.drupal.org or drupalconlatest.drupal.org as an authenticated user without having to login again.

Very handy indeed.

How Bakery works

Bakery knows two types of sites, the master and slaves. The Bakery master is the only site that actually handles user authentication via username and password. Slaves forward login data to the master, but do not authenticate the user themselves via username and password.

Bakery works by passing around data in what it calls "cookies", confusingly both in actual (shared) cookies and in POST requests. These cookies contain the passed data together with a signature (a HMAC) in a base64-encoded encrypted string. The signature should ensure message authenticity (ie actually originates from master or slave). The encryption and signing key is shared between master and slaves. While Bakery makes a number of cookies, the two most important for this entry in the bug2exploit series are Oatmeal and Chocolatechip.

If you login on slave S, it will forward the username and password to the Bakery master M in Oatmeal via a browser redirect. The Bakery master does some checks, logs in the user and creates a Chocolatechip cookie only to redirect the user back to the slave. The slave checks the Chocolatechip cookie, updates relevant information and logs the user in.

While Chocolatechip serves to authenticate a user to the slaves, it also works to authenticate to the master. The cookie carries the username, mail and init together with some housekeeping fields.

Encryption & signatures

Before we continue to the weakness in Bakery, we need to take a look at the structure of cookies. Whenever Bakery bakes a cookie it starts by building an array with the data to send and a HMAC-based signature.

The next step in baking a cookie is to serialize the array structure, encrypt it via AES in ECB mode using the shared private key and finally base64-encode the result.

When the Bakery master receives such a cookie, it runs the process in reverse. First, it base64-decodes the contents, decrypts the result, then unserializes this into a $cookie array. Bakery then calculates a signature and checks if it matches the signature delivered in the cookie array.

Weakness: Unserialize on cookie value

Any security researcher will perk up when she sees an unserialize on user-supplied data. Such a call can be used to invoke object destructors with values chosen by an attacker. On Drupal 7, this can be used to delete files from the filesystem. As signature verification happens after unserializing the data, it cannot interfere with our nefarious purposes.

Alas, as Bakery decrypts the cookie before it calls unserialize on it, we need to find a way to get our payload encrypted with the private key or the decryption step will result in random garbage. This is where the next weakness comes in.

Weakness: ECB mode

AES is a block cipher that can be used in a variety of modes. In Electronic cookbook mode (ECB) used by Bakery, the plaintext is split in 128-bit blocks and each block is separately encrypted with the key. The content of one block doesn't influence the ciphertext of the next block.

Here's a short example, with a reduced blocksize of 64-bits to illustrate the consequences. A string that fits into 2 64-bit blocks has been encrypted in ECB mode. We then take the ciphertext, split it in 64-bit blocks and swap their positions. When we decrypt, the corresponding plaintext blocks have also switched position.

h e i n e j a n  d e e l s t r

4e2387db4b9713f0 5e9275df5301bf8b

5e9275df5301bf8b 4e2387db4b9713f0

d e e l s t r  h e i n e j a

This property of ECB means that we can swap, shuffle and delete blocks or even insert them into other encrypted messages to get control over the decryption result. All we need to take care of is to align injected data to block borders and to identify these blocks in the ciphertext. Note that injected data may span multiple blocks.

So, in order to exploit the unserialize call, we need to have Bakery encrypt our payload and make sure this payload gets aligned to an AES block start. The best candidate is the Oatmeal cookie, as it contains data we supply via the login form.

Here's the PHP data-structure of Oatmeal after a slave login, just before it is sent to the encryption pipeline:

$cookie = array (
     // data comes from form_state['values']
    'data' => array (        
        'name' => 'username',
        'pass' => 'mypassword',
        'op' => 'Log in',
    ),
    'name' => 'username',
    'calories' => 320,
    'timestamp' => 1358887168,
    'master' => 0,
    'slave' => 'http://slave.yawning-angel/',
    'signature' => 'complicated_looking_signature_here',
);

Under special circumstances 'data' may contain additional fields. On most sites however, this is what we have to work with. Serialized and split in 128-bit blocks this looks like:

a:7:{s:4:"data";

a:3:{s:4:"name";

s:8:"username";s

:4:"pass";s:10:"

mypassword";s:2:

"op";s:6:"Log in

";} .......... }

Remember that these blocks correspond 1:1 with the blocks after encryption. Supplying an 8 character 'username' gets the password value aligned to the start of the fifth block. One can simply delete the first four blocks to get a decryptable cookie that starts with user-supplied data. Because unserialize doesn't care about trailing characters after valid input, there's no need to worry about the extra blocks on the end, provided the "password" itself can be unserialized.

With all that encryption out of the way, lets see if we can do something fun with the payload that works on both Drupal 6 and Drupal 7 sites. Can we bypass the signature check and forge valid Chocolatechip authentication cookies?

Yes we can.

Weakness: Type juggling

Here's a typical signature comparison executed by Bakery. The received signature is compared to a hash derived from several received fields and the Bakery secret key. The age of the cookie is determined as well.

// $signature is calculated as a hash_hmac over some cookie fields.
if ($signature == $cookie['signature'] &&
    $cookie['timestamp'] + variable_get('bakery_freshness', '3600') >= $_SERVER['REQUEST_TIME']) {
  $valid = TRUE;
}

The comparison operator == has some interesting type juggling properties. If you supply a boolean TRUE on one side and a non-empty string on the other (such as 'impressive hash' == TRUE), the expression evaluates to TRUE. Because an attacker can control the type of $cookie['signature'] via the unserializable payload the signature check is no obstacle. A serialized boolean true for signature would look like s:9:"signature";b:1;.

Exploit

In order to forge a valid Chocolatechip for a targeted user account, the payload needs to meet the following requirements:

  • contain boolean signature
  • contain a valid timestamp
  • contain the username of the target
  • contain the email address of the target
  • all this serialized …
  • …but in 128 chars or less

Here's a payload to attack the user account 'deelstra' with [email protected] as mail address. The exponential notation for a future timestamp (2E9) allows additional room to spend on username and email.

a:4:{s:9:"signature";b:1;s:9:"timestamp";s:3:"2E9";s:4:"name";s:8:"deelstra";s:4:"mail";s:15:"[email protected]";}

Using the string above as the value for password gets us the following structure of Oatmeal. The actual Oatmeal cookie is of course encrypted, but will correspond to this block layout. Throw away the first 4 blocks and Oatmeal is transformed into a valid Chocolatechip cookie.

a:7:{s:4:"data";

a:3:{s:4:"name";

s:7:"usernam";s:

4:"pass";s:112:"

a:4:{s:9:"signat

ure";b:1;s:9:"ti

mestamp";s:3:"2E

9";s:4:"name";s:

8:"deelstra";s:4

:"mail";s:15:"in

fo@ustima.com";}

";s:2:"op";s:6:"

Log in";} .....}

While this specific exploit depends on knowing the email address of a targeted user, do not base your upgrade decision on this detail. Exploits that do not require any additional information beyond the user id are also possible.

Solution

Bakery 2.x-alpha4 has been modified to guard against these types of attacks. It still uses ECB (this will change in another release) but cookie handling has markedly improved:

  • Signature is sent outside of the encrypted data
  • Signature verification uses the === operator
  • Signature covers the entire encrypted cookie (block order, all fields)
  • Cookie type information has been added to the cookies and is checked on receipt
Feb 18 2013
Feb 18

Besides fundamental changes in the backend of Drupal 8 (i.e usage of several Symfony 2 components) a new theme engine called Twig is also introduced to Drupal.

In the course of further development of Drupal it is necessary to look beyond our community boundaries and to be open to new ideas and technologies. This is the only way to ensure a sustained and stable growth of the community. With the introduction of the Symfony 2 components Drupal took a major step forward to the PHP community.

In addition to PHPTemplate themers can use Twig as a new theme engine in Drupal 8. This helps lowering the entry barrier for new developers since learning PHP for theming Drupal is no longer necessary.

In our multi-part blog post on Twig we want to explain the fundamental changes of the new theme engine and show new possibilities in theming introduced by its implementation in Drupal Core.

Why Twig?

Twig has its origin in the world of Django and was developed by Armin Ronacher as an alternative theme engine for writing templates directly in PHP. Since 2009 Twig has been an integral part of the Symfony framework and has proven to be a reliable and fast theme engine. Since then the further development of Twig is promoted by Fabien Potencier, the leader of the Symfony project. Further information on his thoughts about templating engines in PHP can be found in his blog post Templating Engines in PHP.

The following list provides an overview on PHP frameworks that also support Twig:

Drupal & Twig

Apart from an easier tag-based syntax, Twig offers a lot more security on theming level. The fact can be described very briefly by a quote of John Albin at DrupalCon Denver.

We hand themers a loaded gun and tell them to hammer in a nail with it. Oh and be careful!

For instance, it would be possible to access the Drupal database via PHPTemplate and delete individual tables or access the file system of your server.

Advantages using Twig

  • more security on theme level due to limited possibilities given by tag-based syntax
  • no more PHP/HTML mix in template files
  • simplification of theming system due to the reduction of preprocess functions
  • consistent and easy-to-learn tag-based syntax, that is limited to the most common use cases in template 'development'
  • Reusability of template files (via Twig includes)
  • IDE integration (Netbeans, vim, PHPStorm, Eclipse, Textmate, Sublime)
  • good documentation - Twig documentation
  • will bring Symfony and Drupal community closer, because more components can be shared

Unfortunately there is not so much documentation on drupal.org yet but the latest coding standards can be found here: Twig Coding Standards.

Disadvantages

  • learning a new template syntax
  • performance and memory issues caused by an extra theme engine
  • differing update cycles of Drupal Core and Twig regarding to the code freeze of Drupal 8

In my opinion only the first issue should be considered as a real disadvantage - in addition to the fundamentely changed backend in Drupal 8, Drupalistas also have to get used to a new template syntax. But if you have a look at other Content Management Systems or Frameworks you'll also find custom syntax for templating, unless they are using PHP as their main language for templating.
The performance issue will be adressed in one of our following blog posts on Twig. The already mentioned update cycles are a well-known problem in the Drupal community (jQuery is a good example in Drupal 7) - but I don't think that Twig will change dramatically in future releases.

Twig is part of Drupal Core

In the latest Drupal 8 release Twig is available as an additional theme engine and is ready to use in custom themes (Integrate Twig into core: Implementation issue). Currently the built-in themes are still using PHPTemplate as their theme engine - but you can try the Drupal 8 Twig Sandbox instead. The goal should be to reduce all Drupal Core templates and their related theme functions to just one Twig template and to simplify the theming system. The Stark theme located in the sandbox is a good example for the new approach - besides the module templates all parts of administrational pages are provided as Twig templates and can be easily overriden.

We won't get rid of the big "render array" (of doom) in Drupal 8, but the usage of Twig as our new templating engine will be much easier.

In the second part of our blog post series we want to talk about the syntax of Twig and show some simple examples to explain how we can benefit from these changes.

The Twig logo is © 2010-2012 Sensio Labs

Feb 14 2013
Feb 14

Image Style Basics

Drupal offers the possibility to easily process images via image styles (formerly known as imagecache presets). In our current case of a photo heavy website we had to integrate a watermark image on all images supplied by the site. For this purpose we used the image style action Overlay (watermark).

The configuration of the style is really simple - but could be enriched with scaling and all other available actions.

A major advantage of using Drupal's built in image style is the fact that all images are only processed once and cached in Drupal's file system for later usage. In case of our image gallery the first call of the page would take a long time, because all thumbnails would be rendered for the first time. But on second call all images would be available immediately.

Domain Access Basics

With the help of the Domain Access module you are able to share configuration, content and users between different domains. Compared to the known Drupal multisite approach, which uses a single database per site, Domain Access shares all data in a single database.

In one of our latest projects we used the module for a German city website called ROW-People (row-people.de / hb-people.de). Besides sharing content and configuration between these two sites we had to ensure that all images get a unique watermark rendered via Drupal's image styles. For further information on using the Domain Access module we will write an extra blog post explaining the setup process and usage.

Our Domain Access configuration for testing:

  • Domain 1 (machine_name: domain1) Default Domain
  • Domain 2 (machine_name: domain2)

If you want to test the setup on your local machine, you will have to set up two virtual hosts pointing to the Drupal installation.

Problem and our solution

Due to the fact that image styles are only rendered once, all images on both domains would get the same watermark image. But as mentioned before, we want a unique watermark per domain.

In order to achieve this goal we had to create our own image style per domain (the original style name suffixed by the domains machine_name) that contained the desired watermark image.

  • watermark (default-watermark)
  • watermark_domain2 (suffix matches the machine_name of the domain)

For testing purposes we created a simple content type with the following fields

  • Title (Standard)
  • Body
  • Image (Style watermark)

All further processing had to be done via a preprocess function supplied by the theming layer of drupal.

While configuring our domains we needed to set up a unique machine_name per domain that could be used in our template_preprocess_node().

Provide appropriate watermark image via preprocess functions

With the help of the template_preprocess_node() function you are able to rewrite the output of fields on node level. Whenever the page is called the preprocess function will rewrite the name of the image style name based on the machine_name of the actual domain.

In case of the city-portal site (row-people.de/ hb-people.de) we also had to add some preprocess functions to overwrite the output of the image fields used in several views ( e.g. (Image gallery))

Feb 06 2013
Feb 06

For all Drupal's features, power, and flexibility, there is one area it has issues with - Content Migration. Getting content into Drupal in mass or from an older Drupal version can be brutal. So far each version of Drupal has had some sort of support for upgrading between versions included, but the chances of success can waver depending on how complex a site is and the modules used. Over the years a couple modules have risen to the challenge of addressing the migration problem, Feeds and Migrate

Migrating to Drupal 7 from Packt Publishing, and authored by Trevor James, tackles the often challenging task of getting content into Drupal from an outside source, and avoiding countless node/add forms. The books cover states "Learn how to quickly and efficiently migrate content into Drupal 7 from a variety of sources including Drupal 6 using automated migration and import processes."

Who this book is for

This book is for Drupal users, website managers, webmasters, content editors, or developers who have already installed and configured a Drupal site and understand its web-based administration; and who want to import data from other sources and websites into the Drupal framework.

This book will have little in terms of programming or code. Everything we do in the book will be configured easily by using the Drupal administration interface and module admin screens. We will look at some code briefly when we set up our Feature modules. You do not need to have any previous MySQL or PHP experience to work through these examples.

I want to call out "Who this book is for". Migrations aren't for the faint of heart and are a fairly advanced topic. I am not sure that is properly expressed above. "Drupal users" is pretty vague, and typically Content Editors are not behind the scenes folks, rarely will they be the people installing and configuring Drupal. However an intermediate site builder will do just fine with this book.

The book doesn't waste any time tediously explaining Drupal or how to install it, an advanced topic book shouldn't require it. Chapter 1, "Preparing Drupal for Content Migration", does touch on tweaking PHP memory, although focuses mostly on MAMP. The next 8 pages go on to detail how to install Admin Menu, CTools, Views, Job Scheduler, Features, Feeds, and Feeds Tamper. Considering the target audience should be familiar with Drush, this could have been shortened considerably, by omitting the install instructions and just explain why we needed the module and a brief synopsis. Note: The CSV file was not where the book said it would be in the sample files I downloaded. Instead it was in 0540OS_08/code/migration6/sites/$new_url/files/feeds/.

Chapter 2 is titled "Starting a Migration Path". It briefly talks about planning content types around the data to be imported, but focuses mostly on preparation for the supplied CSV file and less on theory. Again, some of the steps are more detailed than needed for an advanced topic book. More modules are introduced, without install instructions: Location, Location Feeds, Link, and References. A simple content type is created around the CSV, step-by-click-by-click.

Chapters 3, 4 & 5 deal with setting up, running and altering a Feeds import. This is the core of the book in my opinion. It does a good job running through things and explaining what is going on. Considering how detailed some of the earlier mundane steps where, there are a few things that feel a bit over assumed, but an experienced Drupaler should be able to figure them out. One thing that threw me was after preparing the import, "To run the import you need to go to the import URL for your site". This actually means /import. An Import link is also added to the Navigation menu, assuming you still have that block active - it is one of the first things I remove. A better wording or simply changing the sentence to "...URL for your site, 'mysite.com/import' ", would have saved me a trip the Feeds documentation and reading through the project documentation to figure it out. Reading the documentation on a module is always a good idea, but this feels like an oversight. Since the link isn't in the Admin area, it wouldn't have hurt to point out a somewhat irregularity from typical Drupal module admin structure.

Note: Feeds 2.0 has been released since the book was released, and the UI has changed slightly.  The book leads you to believe that referenced nodes will be created on import. Referenced Taxonomies do get created, but try as I might, I couldn't get the referenced nodes to be created. So my imports failed until I manually created the 24 Organization nodes for the referenced node type. Maybe I did it wrong, but there wasn't sufficient coverage on the topic to create reliable expectations on the outcome of this. 

While covering Feeds Tamper, there are two ways to access the settings mentioned. As of beta 4 there is a bug that breaks the link on the top of the Mappings page. A fix has been committed in the dev release, which also alters the UI some; Tamper now lives on a tab.

Chapter 6 covers packaging up the Feeds settings and Content Types with Features so they can be moved from a dev site to production.

Chapter 7 introduces the Migrate module as an additional method of importing content. It runs through importing the sample data included with the module. I am not really sure much is accomplished with this, as all the data is actually in the code included with the module. Migrate is a beast and requires coding to make it do anything and is considerably more complicated than Feeds. It also can be used to build on top of, such as the Wordpress Migrate module does later in the book. However the book doesn't go any deeper than that, and I suspect the only reason Migrate is covered as such is to provide a segue for the Wordpress Migration. However, for that reason this chapter should have been pushed back with the Wordpress chapter. Migrate is a beast, worthy of its own 100+ page book, so the dozen pages here really don't get you far. For a deeper look into Migrate, there is the Drupal.org community documentation and there was a technical session presented at Dupalcon Denver.

Chapter 8 really just rehashes the "official" method of upgrading Drupal 6 in place to Drupal 7. I was hoping there would be some alternative processes presented on doing this, as the upgrade can be quite troublesome for more complex sites. However unlike the official instructions included in the Drupal 7 UPGRADE.txt, the book introduces the use of the Upgrade Status module and the Drupal 7 CCK module as helpers. If you read enough about the upgrade on Drupal.org you will learn about these 2 modules, however UPGRADE.txt doesn't mention them. Upgrade Status is very helpful in getting a site ready to upgrade and identifying blockers. Fields moved into core with Drupal 7, instead of handled by CCK as in Drupal 6. So the Drupal 7 CCK module is used to convert them. I have ran through this upgrade a couple times with Upgrade Status and CCK before reading this book. The information on Drupal.org was a bit clunky and no where near as concise as it is presented in the book. I accomplished the upgrade following the steps in the book in record time!

Chapter 9 covers using the Wordpress Migrate module, which is dependent on the Migrate module. As such, swapping the Migrate chapter to follow the Core Upgrade chapter would make more sense to me. The chapter does a good job describing the process, unfortunately there are no sample files included. It would have been nice if the WXR export that was used in the book had been included in the code samples. I was able to pull an export from a clients existing Wordpress site and run through importing it into Drupal with decent success.

Conclusion

It is worth noting that both the Feeds and Migrate methods only bring over content, not menus. So there will still be work required to piece together the data imported.

The book does a very good job going through the process of importing with the Feeds module. It also covers the official D6 to D7 upgrade process in a very clear and easy to follow manner. The chapter on Wordpress to Drupal is a nice bonus. Additionally, it does that all in a manner that doesn't require coding skills. So an intermediate site builder should be able to handle them fine. 

However, the chapter on the Migrate module doesn't really do much beyond lay the ground work for the Wordpress chapter. Migrate really needs its own book, which requires considerable knowledge of coding and Drupal APIs. 

There are several screenshots that show content that isn't covered till later in the book, and a few places where it is assumed you have already created some fields and content types. With a little editing, omitting some unneeded tedious steps and shortening up the Migrate part, the book might have been reduced in size to about 100 pages.

Disclaimer

The copy of Migrating to Drupal 7 that was used for this review was provided at no charge to Scott Wilkinson of HaloFX Media LLC by Packt Publishing. However, no other compensation was received for this review and this review was published without prior review or any influence from Packt Publishing.

Jan 24 2013
Jan 24

Sometimes you have to do a whole lot of stuff during an update. Sure you could just write a standard hook_update_n() function and try processing 50,000 nodes but if you do you're almost guaranteed to run out of memory or timeout or generally make your web servers unhappy. Luckily you can use Drupal's batch API within your update hook.

Jan 14 2013
Jan 14

It's well known that websites have traffic patterns and that there are those times of the day/week/month/year were traffic predictably increases or decreases. Just like the 405 in LA, or the LIE in New York, or the 75/85 connector in Atlanta, traffic is likely to be pretty light at 2:00am on a Sunday, but travel those same roads 30 hours later and, well, bring a book...

Jan 08 2013
xjm
xjm
Jan 08
That thing you're complaining about? Someone worked really hard on it.

Working on any software project can be frustrating. You inevitably encounter things that are confusing, buggy, poorly documented, over-architected, under-architected, and so on. In an open source project, this frustration is compounded when uncertain resources and volunteer contributions lead to inconsistent quality or completeness. Working on the development version of the software adds additional challenges, as you have to learn new systems and development paradigms that may not yet be very refined.

Think twice, though, before publicly venting your frustrations about how awful that current pebble in your shoe is. In a project like Drupal, chances are, someone you're talking to worked really hard on some component of the thing that's frustrating you. They probably donated their time. They did the best they could with the resources they had.

Negativity is contagious. Even if you have good intentions, and the person you're talking to has good intentions, disparaging remarks will quickly take your discussion off track and make both of you less interested in solving the actual problem.

If you disagree with something in Drupal, first, take a deep breath, and remember that the thing frustrating you is the work of real people. Then, take the time to file an issue that neutrally states the change you suggest and the reasons for it. Be constructive. Bonus if you provide a patch to get the issue started. The same feedback that started an argument when you threw in a couple hyperbolic adjectives may be received with enthusiasm if you phrase it professionally.

Be nice. Sometimes it's hard, but it's worth it.

xjm Tue, 01/08/2013 - 05:36
Jan 04 2013
Jan 04

I'm writing this at 4am after my 2-year old woke me up because he was cold and hasn't figured out how to put a blanket on himself yet. Surely, if he knew the kinds of pains I go through every day to plan the future and make a living without losing my sanity, he'd learn how to put on his own blanket. This is also after waking up at 3am when my 4-year old climbed into bed between my wife and I. There was more room in his bed, so from 3-4 that's where I slept.

It's my fault, I suppose, for deciding to have kids - something I'd been planning for long before I had any. I started teaching myself programming and graphic design 12 years ago because I wanted a viable skill set that would allow me to support a family and spend a significant amount of time with that family. After numerous career steps, I eventually founded BuildAModule, in a large part as a way to leverage work-related time more carefully. And you'd think that by now I'd have balance, but even after over a decade of practice I struggle every day with juggling work time and family life with everything else that comes packaged with typical human existence.

So I write this as a way to connect with any of you out there who also feel like you're trying to figure out how to straddle multiple worlds without ripping your pants. There's a few things that have helped me, and I'm hoping to hear some voices chime in on this as well. Some of this stuff you've probably heard before, but all of this I've tried putting into practice, and this is a summary of my experiences. This is also partially a response to a question about post-child life in a Drupal world.

There is always something amazing happening you will never know about

I've started to realize that no matter how much time I spent learning, consuming and observing, there will always be more awesome things happening than I can possibly ever even hear about, much less dive into. Even in the microcosm of Drupal, I'll never have enough time to learn it all, or keep abreast of everything that's happening.

The cool thing about this is that it means that everyone else is in the same boat. Chances are that if I choose a small niche to develop skills in or stay current on, that I'll know something or contribute something that will help other people, and that's really what it's all about, right? We learn in order to help others in some way, sometimes as part of our living, sometimes as our life's work.

Even when free time is cut in half or quarters by the addition of children into the picture, I've found that the same rules apply. Freaking out about how much I don't know just saps the energy I need to do something worthwhile with the time I do have. And when I can find joy in what sometimes feels like the most minor contributions I make in this world, I tend to want to make even more.

There is a difference between the amount of time you want to spend with your kids, and the amount of time they need

As I was prepping for family life - building skills, saving money, trying not to annoy my wife too much - I wanted to make sure that eventually I could be available to my kids as much as I wanted to be, and as much as they needed. Now that I have kids, I realize that there are actually two separate things. How much time I want to spend with my kids is not necessarily the same as what they need.
Before I had kids I thought that I would be happy spending all day, every day with them. Now that I have them, I think the idea of staying sane in that scenario is a little wonky. The stay-at-home parents I know all feel a little nuts, and judging from time I've spent with my kids solo for long periods of time, I completely understand. They're awesome in every way except that they have a completely different agenda than I do and operate at a very different pace than adults. It's endearing at first, it even helps me see things in a fresh, youthful way, but after a while it becomes impossible to feel like a normal person around it. Even though part of me really wants to want to spend all my time with my kids, there's some other part of me that really needs outside input, and if I don't get that input, I'm not going to be very awesome with my kids.. And I've noticed my kids are the same way. Without more diverse input from other people, I drive them crazy, too.

If my goal as a parent is to make sure that my kids develop normally (even amazingly) and don't experience unnecessary trauma, then they actually need to spend a good bit time with other adults and kids, because what I can offer them is relatively limited when compared to what a community can offer. As a parent, though, I can offer consistency and stability from day to day. I can be a fixture, whereas the community is fluid, and I don't think it takes that much time to function as that fixture when you're present nearly every day.

The ideal time ratio between work and family

In our current balance of work and family life, I spend 3-4 hours with the family in the evening, and then I'm around most of the weekend, with maybe a 4-hour break during the weekend to do some introspection or play some games. Every 2-3 months I end up taking a week or two off, and during that time I spend the bulk of my time with the family. Even though I'm grateful for this time and I feel like I get the opportunity to spend more of my time with my family than a lot of people do, it often feels like it's not enough.

Something interesting happens when I'm around the family more. Things that are normally stresses - kids having tantrums, silly bickering between the adults - tend to fade away. We all still have moments, but the extra time also seems to build a buffer of understanding and compassion. I tend to be happier, and so does everyone else. It's that effect that makes me wonder what would happen if I didn't work as much on a regular basis. It makes me think about how different work ethics are in other places around the world, and how maybe those families might potentially be a lot happier on the whole.

Even within this country, I get the feeling that each person's comfort level around the work and family life ratio is enforced by their individual communities as well. If you hang out with people who work a lot, you'll probably be more comfortable working more, and more thankful for small breaks to be with family. If you're surrounded by stay-at-home parents, you'll probably feel like work is keeping you from immersing yourself in family life. I have a little bit of both, since I work at home and also hang out with some pretty hard workers. So, I go back and forth between feeling bad about not spending enough time with the kids, and not spending enough time with work. Sometimes I'm right in the middle, which is what I'm attempting to cultivate more of.

Employeeship, freelancing, and product-based businesses

I pursued a freelancing career a few years after graduating from college so that I could work from home and maximize income. Ideally, I wanted to make enough to work part-time when I had kids. I got a job in a campground that afforded me the time to learn what I needed, and the isolation to keep distractions away. I think I got pretty lucky there.

A couple years ago I shifted my focus to work on a product-based business, which - I learned from The 4-Hour Workweek - was maybe the only way to break through an income ceiling that comes from working hourly. Since making the shift, I significantly increased my income, and have been able to take large chunks of time off with family, which is exactly what I was hoping to do from the start.

For those who crave more time with their families and are currently employees, I have a hard time thinking of another way to move forward besides starting to work for yourself. Freelancing comes with less risk than attempting a product-based business right off the bat, and gives you enough wiggle room to start building a self-funded product while also giving you the same wiggle room for family. Lowering your financial requirements can also add flexibility. I've given a couple talks on this subject, and if you're thinking about making some steps forward, definitely post something here so I can add a few words of encouragement.

That said, starting a business can also be a bigger time-drain than being an employee, depending on how you approach it. I gave myself a 6-month window of time to get to a certain level of stability with my business before going back to freelancing. If it hadn't have worked out, I would have needed to give myself some time to re-connect with the family.

Talk to yourself

It's hard to find the right people to talk to at just the right times when if you're trying to figure out how to slice the work and home life pie. And usually I don't even know what questions to ask. Luckily, there are two of me. Or at least, there's two parts of my brain that can talk to one another to some degree. So, when I'm feeling a bit lost, I'll start writing (there's a nice, distraction-free program called Writeroom that I use), and usually after 30 minutes or so, I begin making progress. I start being able to articulate the problems and think about some possible re-orientations or re-organization that I could try. 

There's been plenty of research into how words affect our thoughts, and I've noticed that whenever I start using language more in my life - either talking to other people, writing blog posts or journaling - that my thoughts also become clearer. I'm not sure if this works as well for other people, but it's definitely a key technique in my arsenal, and what I use whenever I'm trying to re-balance family and work life.

Meditation on the meaning of work

It's been easy in the past for me to take the idea of work for granted. If we want to sustain a certain standard of living, we need money and thus have to work. But a few other things come into play that affect how much we decide to work. 

Through work we can potentially impact lots of people and affect world around us, and that can be a very powerful draw, one that sometimes conflicts with being present as a parent (or any other role, for that matter).

For me, it's helped to think about what my life would look like if I no longer had to worry about money. This allows me to recognize the distinction between the desire for financial stability and the desire for other benefits that work brings, like challenge and positive impact on other people.

Because financial troubles tend to cause a huge amount of stress in families, I think it makes sense to trade the time I could be spending with my kids to achieve the goal of financial stability. But when I have that foundation set, it gets a little more difficult to draw the line between work and family life when it comes to the other benefits associated with working hard. 

At what point do the more meaningful aspects of work become less or more important than spending time with my family? That answer changes all the time for me, but I know that I only have a few more years before my children start to have richer lives in the community with school and activities, which means less time at home, so I feel like there's something special in the next few years that I don't want to miss out on. Most of us only get to be parents once, right?

I try to take time regularly to think about the benefits of pursuing work versus spending more time with family. It's easy to fall into a pattern of work and family life and forget why I established the pattern in the first place, and sometimes when I'm working through the thought process around it, I remember something important that completely changes what I do or why I'm doing it.

Put the oxygen mask on yourself first

Most parents will suggest that you have to take care of yourself before fretting over your kids (beyond their basic needs, of course), because if you become compromised in some way by forgetting yourself, your kids will suffer. When I find myself going nutso over kid meals and activities and cleaning, and I re-orient myself to think this way, it can make a big positive impact. Usually when I'm on the more happy side of the emotional meter I end up being a much better parent (or at least feeling like I am). Also, kids are pretty dang resourceful and all of their mechanics are bent towards self-preservation. So even if you forget to feed them because you're reading a book or building a lego sculpture, they'll give you a heads up before they get even close to passing out.

To me, this means that if I'm getting a lot of satisfaction from work, and it's important for me to focus on it to keep a certain amount of mental and emotional momentum - or that I'll feel some resentment by having to give it up - then my kids are going to be fine if I keep that focus. But as I mentioned before, the more time I spend with them, the better we get along and understand each other, so when things start getting a little too stressful in our relationships, I'll put aside work for a while until the family is running a bit more smoothly.

Kids brains know what they need to learn, so relax

This took me a while to figure out, but it's taken a lot of stress out of parenting and also frees me up to optimize the time I do spend with the kids. Research shows that kids know when a part of their brain is ready to develop, and all we have to do is take queues from them and do what we can to support it. This is much, much easier than trying to get up to speed on everything kids are meant to be learning at different stages, and trying to force it. There are still lots of things that I've initiated, like reading aloud and singing songs, but I don't feel like I'm a complete failure if they don't pick up on it.

Any random kid is funnier than virtually any adult

I've laughed more since having kids than I did for the previous 15 years. I've noticed that when I view my child as a comedic relief, rather than a responsibility, I can derive a lot more enjoyment from them. This perspective doesn't work all the time, but sometimes it's perfect, and with me finding it funny when they spill the milk for the 5th time today, they end up having a lot better time with their own foibles, too, making us have a much better time with each other.

Kids also make excellent fodder for humor, and I actually think about certain comedians a lot when I'm with my kids, it helps me not take it too seriously. Louis CK and Bill Cosby are the two I find the most helpful.

The role of exercise

Exercise takes time. But for many of us it also will extend our lifespan. So, save the time now and potentially kick the bucket earlier? Or hit the gym every day for an hour or two and miss out on making progress on some front, whether it's family or work?

Part of the answer might be had with a treadmill desk. I've been using one for 4 years when I work. It can be cheap and doesn't negatively impact focus. I've turned on a  few people on to it over the years, and most have lost a significant amount of weight as a side effect and feel a lot better since using it. I find it helps me focus, much like a stroll outside would. This isn't really the cardiovascular workout you'd need to meet the requirements for exercise by certain measurements, but it's way better than nothing, and it feels pretty awesome.

I also go in and out of phases of exercising or going the route of using that time for work or family. But I do find that when I'm exercising, I have a noticeable increase in the amount of energy after work for playing with the kids and being more interactive with them. I'll also go back and forth on if I take this time from my work day or if I take it from family time.

Avoiding burnout by finding focus

The demands of a household are infinite. You really can occupy a team with everything that could be done in a home. And working in Drupal feels the same way. There's always something new to learn, a potential client to pursue, an awesome idea to make a reality. There are about 400 items on my list of things that I'd like to do if I had the time. Maybe half are Drupal-related, and the other half has to do with family and other skills I'd like to develop, or things I'd like to learn. I have to go back and forth between Drupal and family and focusing on other aspects of personal development because if I didn't, I'd start to resent whatever was keeping me from everything else. In order to stay active at all with Drupal, I have to find a balance, and I do that by focusing. 

I really want to release a new Evernote module, I desperately want to climb the core contrib ladder, I crave participating in IRC and I want to hit up twice the camps I do. But, I'm focusing now on building a really good learning resource for Drupal, and that might be all I'm able to get to for a little while, because I also really want to be around when my kids do amazing things.

I think it's important to want to do more than you can do, but also to somehow be okay with your own limitations. I focus on a couple things at a time, and I've found that pattern works the best for me.

Jan 02 2013
Jan 02

Over the Christmas break I came across gource, a software version control visualization tool. Gource produces really nice visual representations of software projects growing. About 2 years ago David Norman produced a gource video of development of Drupal up to the 7 release. This is pretty cool, but it only shows who committed the patch, not who contributed to it.

After some searching I found the Drupal contribution analyzer sandbox project. This module allows you to produce contributor tag clouds and code swarm videos. This was closer to what I was after, but I had to patch patch drupal_log_generator.py to support the gource custom log format.

The result is a 5:23 minute video showing the growth of Drupal.

The first few years things are pretty consistent and easy to follow. The Drupal 8 development cycle shows how much the community of contributors has grown. Towards the end of last year things look really chaotic.

To produce the video I used a clone of the Drupal 8 branch as at some time on January 1, 2013. The gource command I used was: gource --log-format custom -i 500 -s 0.0001 -a 0.01 -r 30 --title "Drupal" --highlight-users --disable-progress --hide filenames -1280x720 drupal --bloom-intensity 0.2 --bloom-multiplier 0.2 --stop-at-end /tmp/commit.log -o -| ffmpeg -y -r 60 -f image2pipe -vcodec ppm -i - -vcodec libvpx -b 10000K \~/Videos/drupal.webm

I considered writing a script to find and download user avatars from groups.drupal.org but after reviewing the video without them I decided it would be too cluttered.

Can you find your name in the video?

Dec 27 2012
Dec 27

A small problem in Drupal occurs when developing new modules, or installation profiles. Content. It goes missing, and without it developing can go wrong very quickly. Also, starting up a site’s install profile that reveals a site without content gives a feel of walking into an empty bakery shop. There’s ovens, flour, and bakers, and that great bakery smell all around, but no bread. And possible clients go to other bakeries if you run out of bread. So to solve this issue, Migrate module helps us fill up the shop with bread. Oh, and you can also migrate data from other (older) sites.

What is it good for?

Migrate fills up your Drupal installation profile, or just fresh new module, with appropriate content that helps the user or developer experiment with it, so the new module’s learning curve is easier. When developing, if you mess something up - say, drop the user table - it won’t be as bad as it could’ve been, since you kept around a backup in a CSV file. Some company want their site upgraded? No problem. Migrate can import from CSV, SQL databases, XML and much more.

_For the less technical: if you think I’m just naming random letters, relax: if you have Microsoft Excel, or even OpenOffice (it’s free!), you can start creating content before you even have a site. Just open up a new table in either software, and save it as a CSV file. Your developer should be able to guide you through in a few minutes. If he can’t, well, you might want to consider a different one.

Migrate provides a strong advantage when developing installation profiles. When you rebuild a profile, the existing database is thrown away and replaced with a new one. But most of the time, you will need content to work with, test your logic, and see the resulting displayed page. With Migrate, when you install a profile, you can automatically import content to work with.

Also, Migrate is great for it’s obvious main purpose - to migrate sites’ content to a drupal installation. This is great if you’re building a site for a client that already has a working site but is looking for an upgrade and is afraid of losing all of the site’s content.

However, the best use of Migrate is during the initial development of a new site. Think of it like that: the bakery is still being built, but the baker can’t wait to start baking. He has all those little recipes he’s anxious to try, so in the meantime he prepares dough and keeps it in his household refrigerator. The same principle occurs with Migrate: while Dave, the developer, is building on the structure of the database, content types and whatnots, Sam, the site’s manager, can start creating content, and customize it’s structure according to his needs. All Sam needs to do is create a table and save it in some Migrate-able format (like a CSV file) . This saves time for Sam till he can launch the site, since he can start preparing the content – and put the dough straight in the oven from the refrigerator, without hassling around with flour and eggs. The site will have a quicker ‘cool down’ time between the development is finished until the site is actually launched, since the content is already stored and ready to be used.

Not only time is saved. Dave receives less stress from Sam while developing, since Sam is busy creating content for the new site. Confusion is reduced when Sam decides some content’s structure needs to be altered during development (which occurs quite often) to better suit his needs. A more finely defined content is created as a result, better suited to Sam’s needs, and with less last-minute fixes. A few lines of migration code and voila! Everything is ready for the newly structured content.

So, how do I use Migrate?

Data storage (a.k.a - Sam’s side of the story) The migrate module reads information stored in .csv files. An example of one of these is shown below:

Notice that ‘company’ and ‘status’ columns are filled with references to other entities’ IDs.

That’s it for site developers. Create a table, make sure everything sits the way you want it, and send it to the developer.

The Drupal side: (a.k.a - Dave’s side of the story)

If you’re reading this, I’m assuming you are a Drupal developer (it’s quite technical from now on). I’ve written a simplified example, divided into parts. In this example, I’ve shown how to migrate content from CSV files. A complete, working example of an abstract migration class can be found here - and in the same git repository there is a complete implementation of a custom Migrate module. All the CSV files under under a csv direcrory using a naming convention csv/entity_type/bundle.csv. Example of using this abstract class to import taxonomy terms is shown below: < img src=”/sites/default/files/s033.png” />


/**
 * Migrate measurement unit terms.
 *
 * Use garmentboxMigration class to import taxonomy terms.
 */
class garmentboxMeasurementUnitTerms extends garmentboxMigration {

  // Define entity type and bundle, used in garmentboxMigration.
  public $entityType = 'taxonomy_term';
  public $bundle = 'measurement_units';

  // Map columns to import. garmentboxMigration already mapped "id" and
  // "name".
  public $csvColumns = array(
    'field_unit_type',
    'field_conversion_ratio',
  );

  public function __construct() {
    parent::__construct();
    // Map the mapped columns to fields.
    $this->addFieldMapping('field_unit_type', 'field_unit_type');
    $this->addFieldMapping('field_conversion_ratio', 'field_conversion_ratio');
  }
}
?>

And that’s it. garmentboxMigration takes care of stitching everything together, and putting the data in the content’s fields. You can obviously extend this abstract class to fit your own business logic. Importing files and images is also easy:


$this->addFieldMapping('field_image', 'field_image');
$this
  ->addFieldMapping('field_image:file_replace')
  ->defaultValue(FILE_EXISTS_REPLACE);
$this
  ->addFieldMapping('field_image:source_dir')
  ->defaultValue(drupal_get_path('module', 'example_migrate') . '/images');
?>

For multiple values in one field, use the separator(‘|’) function when defining field mapping:


$this
  ->addFieldMapping('field_inventory_line_inline', 'field_inventory_line_inline')
  ->sourceMigration('exampleInventoryLines')
  ->separator('|');
?>

For additional data tweaking, we can use the prepare(), prepareRow(), and complete() functions.


public function complete($entity, $row) {
  // Do something, e.g. flag a flag on the node.
}
?>

After everything is built, we stroll over to the UI (admin/content/migrate) and select what we want to import, and import away. Alternatively, Migrate can run through drush, which you’ll probably want to use when using an installation profile.

Dec 14 2012
Dec 14
Semantic content enhancements with Drupal, Apache Stanbol and VIE.js fago Fri, 12/14/2012 - 13:02
Nov 12 2012
Nov 12

At the 2012 Bay Area Drupal Camp, I presented a session laying out a possible dev-test-live development workflow branching model using Git. This continues on from the session I presented last year, Beginning Git.

DrupalDrupal PlanetGit
Oct 31 2012
Oct 31

Drupal 6 had Messaging & Notification Drupal 7 has Message-subscribe & Message-notify

Message-subscribe is a new module in the Message stack, that finally ties everything together to give us the ability to subscribe to content, and send notifications.

If you know Message you already know that some development skills, or knowing Rules really really good are needed. This post will go over the main concepts and flow of the notification process.

Message-subscribe, just like Message module has zero knowledge about your business logic - it is up to you to implement the Message creation. What Message subscribe gives you, though, is one simple function your implementing module will call – and that’s it. It will take care of it from there.

For example imagine an existing node, that people in the site want to subscribe to, and get notified about new comments.

Subscribing to content

Subscribing is naturally done using Flag module. There’s no assumption related to the amount of Flags you have - as long as you prefix your flag name with subscribe_ Message-subscribe treats it as a valid flag.

Message creation



/**
 * Implements hook_comment_insert().
 *
 * Send a message when a new comment is created to subscribed users.
 */
function foo_example_comment_insert($comment) {
  // Create a new message, assigned to the node author, and add a
  // reference to the comment, so we can later use tokens related to that
  // comment.
  $message = message_create('comment_insert', array('uid' => $comment->uid));
  $wrapper = entity_metadata_wrapper('message', $message);
  // There will be probably a reference field to the comment on the message itself.
  $wrapper->field_comment_ref->set($comment);

  // Let Message-subscribe save and send notifications.
  message_subscribe_send_message('comment', $comment, $message);
}

?>

As seen in the above code, our module created the Message it wanted and passed it along to message_subscribe_send_message(). Let’s see what happens from there.

Getting “context”

Drupal abuses the word context with so many meanings. Admittedly Message-subscribe is now also guilty. message_subscribe_get_basic_context() gets the entity type, in our case the comment, and extracts the related context – the comment author, the node its related to, the taxonomy terms related to the node, the OG groups related to those nodes (you can add our own context as-well). This context will help us in finding all the users that are subscribed directly or _indirectly to our node.

Getting the subscribers

Next, Message-subscribe queries the valid flags, iterating over the results to get all the users that are subscribed. An alter hook allows you to change the list, or the values in that list. The values are which flags are responsible for the subscription and the “Message notifiers” (about that in the next section).

Sending notifications

Before we talk about sending notifications, note that I never wrote the word “email”. Message-notify is a pluggable module, that allows you to add “Message-notifiers”. There’s a default implementation for email, but you are also able to add Text-messages, IRC, Fax or Postal-pigeons.

So, Message-subscribe iterates over all the users, and all over the notifiers, as in fact a user can get the same Message as email _and as a Text-message. I say same _Message as in same Message entity, not same message as message text – In Message-notify you can have different text for different notifiers thanks to the module’s integration with view-modes.

Bonus

  • There’s a UI! We are showing tabs for every valid flag, and the content of that tab is a View. You can easily change the used View via admin settings
  • It’s a Message entity, right? So it means that the email you’ve just sent, can be saved and be used for an activity stream. 2 for the price of 1
  • For a scalable solution, we are currently working - with a great help from fubhy - on implementing some sort of DrupalQueue to be able to process lots of Messages
Oct 24 2012
Oct 24

Background

SA-CORE-2012-003 fixes an issue in the Drupal installer that enables an attacker to cause the site to use a different attacker-controlled database. This database can be an external server or an SQLite file. The vulnerability also causes the installer to leak database information such as the database type, name, host and the username used to connect to the database. The only item not leaked is the database password.

The installer vulnerability was found while preparing my DrupalJam presentation (NL) on security audits and reported via the SecuriTeam Secure Disclosure program (also via [email protected]). As promised on IRC & Reddit, here's some additional information on the root cause(s).

Installer system

The vulnerability is caused by an assumption of the install system combined with a bug. The assumption is that errors generated while contacting the database indicate a system that still needs installation. To aid in understanding the additional bug, some global information on the installer follows first.

The problem revolves around the function install_begin_request. It verifies the validity of the current settings.php via install_verify_settings, then determines what the install state is in order to decide if installation should continue or if the site has already been installed.

Here’s the function (elided for brevity):

function install_begin_request(&$install_state) {
  // ...
  $install_state['settings_verified'] = install_verify_settings();

  if ($install_state['settings_verified']) {
    // Initialize the database system. Note that the connection
    // won't be initialized until it is actually requested.
    require_once DRUPAL_ROOT . '/includes/database/database.inc';

    // Verify the last completed task in the database, if there is one.
    $task = install_verify_completed_task();
  }
  else {
    $task = NULL;

    // Since previous versions of Drupal stored database connection information
    // in the 'db_url' variable, we should never let an installation proceed if
    // this variable is defined and the settings file was not verified above
    // (otherwise we risk installing over an existing site whose settings file
    // has not yet been updated).
    if (!empty($GLOBALS['db_url'])) {
      throw new Exception(install_already_done_error());
    }
  }

  // Modify the installation state as appropriate.
  $install_state['completed_task'] = $task;
  $install_state['database_tables_exist'] = !empty($task);
}

When install_verify_settings does not succeed in verifying settings.php the else path is taken, and installation will proceed, provided there's no global db_url set (this global indicates a D6 settings.php and marks a system that is to be updated).

install_verify_settings will ultimately run db_run_tasks (a wrapper for the DatabaseTasks->runTasks method). The tasks executed in db_run_tasks are listed in install.inc in the abstract class DatabaseTasks and consist of table creation, insertion, deletions and dropping.

Any exceptions generated during db_run_tasks will be reported as errors to install_verify_settings. This means that when errors of any kind (eg database is down) are generated by the db, the settings.php file fails verification and installation will proceed by displaying a partially filled in Database configuration page.

If required information is send in a POST request, the installer will mark settings.php as writable (if possible) and then update settings.php with this information.

Bug

Being able to do a new install when the database is down, or settings.php is temporarily broken is bad enough, but not what makes SA-CORE-2012-003 so grave. The big problem is the possibility to force the verifier to error out on a working database.

To see how, let's look at the database tasks install_verify_settings runs:

  • CREATE TABLE {drupal_install_test} (id int NULL)
  • DROP TABLE {drupal_install_test}

You probably see the problem now. If not, consider two POST requests fired nearly simultaneously at install.php:

  • Request 1 executes the db-task CREATE TABLE {drupal_install_test} (id int NULL); The table will be created.
  • Request 2 executes the db-task CREATE TABLE {drupal_install_test} (id int NULL); Oops…
  • Request 1 executes the rest of the db-tasks and then deletes drupal_install_test

Trying to create an already existing table in Request 2 will generate an exception that will be reported as a db/settings error to install_begin_request and thus a signal for a continued installation. By sending the right information in the POST request, a settings.php is written that points to the attacker controlled database.

Notes

I'm comfortable releasing this information after a week, because the mitigation step is pretty easy and without consequence: delete install.php or block access to it. If you didn't upgrade or follow the mitigation steps, do so now, then review your update procedures. I conciously did not provide a weaponized example of the exploit; While it's easy to create such an exploit based on this article, it is a step many attackers cannot or will not take.

Should you learn of a serious vulnerability in Drupal or another piece of software, consider the SecuriTeam Secure Disclosure program (also via [email protected]). I had a great experience.

Oct 15 2012
Oct 15
Drupal 7: How to add a custom node view and force a node template to use it admin Mon, 10/15/2012 - 15:11

This is a small Drupal 7 tip if you wanted to create a custom node view and force a node template to use this custom node view.

Use case

Let's say you are creating a custom node template for a node of type "grades". You may need to show/hide specific fields in your content type for your Grades template. (This is just an example but you can use this tip in various ways).

The Steps

  1. You can create a new view mode by putting this code into your custom module:

  2. /** * Implements hook_entity_info_alter(). */
    function MYMODULE_entity_info_alter(&$entity_info) { $entity_info['node']['view modes']['custom_teaser'] = array( 'label' => t('Your Custom Teaser'), 'custom settings' => TRUE, ); }
  3. Then go to Manage Displays in your content type and set which fields you want to show/hide in the "Your Custom Teaser" view mode.
  4. Now you have to force the viewing of your node to use this custom node view "Your Cusom Teaser". Put this code into your custom module: function MYMODULE_preprocess_node(&$variables) { if ($variables['elements']['#view_mode'] == 'custom_teaser') { $variables['theme_hook_suggestions'][] = "node__grades"; } }
    This will force the viewing of your custom content type "grades" to use grade.tpl.php

Further Reading Here is how you would make a View use your custom node view for Drupal 7.

Tags Drupal Drupal 7 tip custom node view modes template Drupal Planet

Anonymous (not verified)

Mon, 10/15/2012 - 18:24

Found this module to be useful : http://drupal.org/project/view_mode_templates

Add new comment

Language Not specified
Oct 08 2012
Oct 08

One of the many hats I wear these days is Development and Coding Track Chair for DrupalCon Sydney 2013. As outlined in the track description we are planning on showcasing what is awesome today in Drupal 7 and the cool stuff that is coming in Drupal 8. Given that there are no core conversations in Sydney we are trying to put together a more intermediate-to-advanced level track. I want people to come to these sessions and go away with their heads full of ideas about what they can do better in their next project.

If you have a session that you think fits that brief then please submit something. If you want to ask me anything before submitting your session, feel free to contact me. The decision on which sessions are accepted will be made in late October / early November by the track team, the global track chairs, the content chair and myself in a collaborative decision making process. The accepted sessions will be announced on 13 November.

Although the event won’t be as big as a northern hemisphere DrupalCon, it is going to be full of great people. The initial 100 early bird tickets sold out in less than 8 hours!

Please be aware that there is no financial support available for speakers and you will be required to buy a speakers ticket at a cost of 165USD.

Submissions close at 23:59 AEST (UTC+10) on 26 October so submit a session today!

Oct 01 2012
Oct 01

In August 2012 the Federal Ministry of the Interior published a press release with the title "Federal Ministry of the Interior publishes survey about "Open Government Data Germany"". Of course this is very exciting and interesting, because Germany also wants to join the idea of making all public documents freely accessible for general public. This can have, among others, several advantages as follows:

  • the support of organizations in achieving ther goals,
  • access to all relevant information as basis of participating in political decisions and processes and
  • complement and improve scentific results.

On the other side concerns were expressed that citizens could be overwhelmed by their limited motivation and time through informations overloads. The quality of information also has to be controlled permanently, what is associated with additional time and financial outlays. But why should the idea of "Open Government Data" be realized in Germany? First of all, because the participation through transparancy is possible, as well as better traceablility of information and facilitations of tasks can take place in each target group. Many countries have already shown how the idea of "Open Data" could be realized in the US, United Kingdom or Australia.

What Drupal has to do with it

And now, what does Drupal have to do with this topic? In the full paper (German only) of the Federal Ministry of the Interior Drupal was named as one of the recommended platform technologies among Java-based frameworks, with which a user-friendly realization of "Open Government Data" is possible. It is assumed that Java-based content management systems are better suited for major projects than PHP based CMS. This is the reason why Liferay was recommended. Liferay, like Drupal, is Open Source Software. The Java Specification is also a reason for recommending Liferay. It is able to support the JSR-168/286-Specification, while PHP based CMS like Drupal are not (However, we have found a module which apparently supports this specification. So a junction seems to be possible.) In addition to that Liferay guarantees the compability of the Java Portal System.

The consequences for Drupal

Even though in the summary of this study Liferay is preferred over Drupal, it is still a really good sign that they mention and suggest Drupal for building Open Government systems. Also, Liferay Portal Community Edition is Open Source but Liferay Portal Enterprise Edition comes under a non-free commercial licence, which guarantees long-term support as well as a support package. Also the importance of the study should not be underestimated, because it is a big and far-reaching concept. Drupal is therefore on its best way to be one of the most siginificant CMS for German Government websites.

Sep 30 2012
Sep 30

I recently worked on a project which required a updated version of the jQuery library. While there is the jQuery Update module, it only allows you to upgrade Drupal 6 to jQuery 1.3. If you really know what you're doing and want to upgrade beyond that version, you can either hack core or create your own simple module to do it. While hacking core is certainly the easier approach (simply overwriting misc/jquery.js with a newer version), it is very bad practice. You do not want to get yourself in the habit of altering Drupal core unless you want to kill your upgrade path and deal with a new slew of bugs and unpredictable behavior.

Let's start by creating an admin area for configuring the version of jQuery we want to use.

/**
* Module configuration admin form.
*
*/
function mymodule_admin_form() {
 
$form['mymodule_custom_jquery'] = array(
   
'#type' => 'checkbox',
   
'#title' => t('Override Jquery'),
   
'#description' => t('Replace the version of jQuery that ships with Drupal
      (jQuery 1.2.6) with the jQuery library specified below. You will need to
      flush your cache when turning this feature on or off.'
),
   
'#default_value' => variable_get('mymodule_custom_jquery', 0),
  );
 
$form['mymodule_custom_jquery_file'] = array(
   
'#type' => 'textfield',
   
'#title' => t('Jquery file location'),
   
'#description' => t('Specify the location of the updated jQuery library
      you would like to use. The location is relative to the docroot and
      so should probably begin with "sites/".'
),
   
'#size' => 128,
   
'#default_value' => variable_get('mymodule_custom_jquery_file', NULL),
  );

  </span>$form = system_settings_form($form);

  return </span>$form;
}
?>

Next, we're going to write a validation hook that simply makes sure the file exists.

/**
* Admin form validation callback.
*
*/
function mymodule_admin_form_validate($form, &$form_state) {
 
$file = trim($form_state['values']['mymodule_custom_jquery_file']);
 
$form_state['values']['mymodule_custom_jquery_file'] = $file;

  </span>// Only validate this value if js override is turned on
 
if (variable_get('mymodule_custom_jquery', 0)) {
    if (!
file_exists($file)) {
     
form_set_error('mymodule_custom_jquery_file', t('The file you specified does not exist: !file.', array('!file' => $file)));
    }
  }
}
?>

And lastly, we use hook_preprocess_page() to safely replace the jQuery that ships with Drupal core with our own version.

/**
* Implementation of hook_preprocess_page().
*/
function mymodule_preprocess_page(&$variables) {
  if (
variable_get('mymodule_custom_jquery', 0)) {
   
$file = variable_get('mymodule_custom_jquery_file', 0);
   
$scripts = drupal_add_js();

    </span>// remove core jquery and add our own
   
unset($scripts['core']['misc/jquery.js']);
   
$add_scripts['core'][$file] = array(
     
'cache' => TRUE,
     
'defer' => FALSE,
     
'preprocess' => TRUE,
    );
   
$scripts['core'] = array_merge($add_scripts['core'], $scripts['core']);
   
$variables['scripts'] = drupal_get_js('header', $scripts);
  }
}
?>

Make note of the line:

    $scripts['core'] = array_merge($add_scripts['core'], $scripts['core']);
?>

We are careful to add our new jQuery include at the beginning of the array (where the original jquery.js include was) in order to meet any jquery dependencies in the scripts that follow.

Sep 26 2012
Sep 26

Company director of Drupal Centric, a web design and Drupal development company, with over 20 years experience in the trade. On this blog I share my knowledge of web design, Drupal, CRM and App development, in an easy to follow and hopefully fun way.

Sep 25 2012
Sep 25

I've known that TomRandall and the rest of the crew at Level 10 have been working on Open Enterprise for a while, and I'm excited to have a reason to give it a go. I'm in the process of numerous upgrades to BuildAModule, one of which is making the current bare-bones blog more integrated with the technical advances of the last decade.

Since it's been a while since I've spun up a Drupal-based blog, and since I know there's numerous aspects of blogging and content creation I'd like to finally wrap my mind around and implement (like RDF, HTML5, pingbacksSEO Tools), I'm thinking that the right distribution could save me time. I'm not entirely sure Open Enterprise has everything I'd like to integrate, but I know it has some good minds behind it. Or, it could turn out that I'm not the target use case for Open Enterprise and I could spin my wheels a bit, but I'm going to take the gamble so I can get familiar with a product that I know is on the edge of some awesomeness. 

So, here's an 'unboxing' of Open Enterprise. In particular, I'm using the Enterprise Blog version, which I'm thinking might be the same as Open Enterprise at it's core?

As per a typical Drupal installation, I downloaded the source, set up an empty database, added a virtual host via my MAMP Pro, and went to the new site.

The installation process gave me an Open Enterprise installation profile, and prompted me to install a set of 'apps'. I was tempted to check everything:

After all that installed, it had me fill in the default admin user and site information screen. After I submitted that, I hit a screen asking me for my FTP information. That threw me off a bit:

I tried to submit the form with nothing in it, then filled it with bogus information, thinking that maybe I could deal with it later. But neither of them let me pass. I did a quick search in the issue queue, and pulled this up which said that I could just make the sites directory writable, which I did by cd-ing into the Drupal directory via Terminal and doing a

chmod -R 777 sites

I remember running into something similar on Pantheon, I imagine for similar security reasons. But I'd sure like to have had a few more details on the screen about why this was needed.

After I did the chmod, I went ahead and refreshed the page, which seemed to push the installation process to the next step. However, not without a few errors (a lot of which just have to do with my level of reporting):

Some of these errors had to do with images, and as I started clicking on bits to look at content, I saw that there were no images and I was getting errors like:

Messing about with SEO tools

So, the first thing that I want to play with is the SEO tools to see if there was anything in there that would clearly be a benefit to my current workflow, so I go to create a piece of content and just copy over a blog entry from my personal blog.

Next, I scrolled through the vertical tabs at the bottom of the content and clicked on the Content analysis one. Everything was enabled except for the Alchemy item, which got me curious, so I followed the steps to set it up, which required the following:

  1. Downloading the PHP SDK files from here.
  2. Putting the files in the AlchemyAPI folder in the alchemy module folder (but you have to take the files out of the expanded folder)
  3. Going back to the content and trying again. This time I need an API key, so I follow the instructions
  4. I sign up for an Alchemy account, verify my email, request the API key, copy it and paste it in

Another disabled tool was the Readability section, and I didn't want to dig into that after going through the 10-minute Alchemy setup. But I did watch some of Tom's video on it  which was good (Tom has a nice screencasting voice).

Once Alchemy was set up, the check seemed to go through and I got some additional info:

After checking out the 3 tabs there, nothing appeared to be particularly useful for this article at least.

The Quick SEO tab had some good tips, mostly I think for folks just getting a feel for the amount of content to plug into a typical article:

At this point I think that maybe I should save the article, since it's showing as just having 12 words when there's definitely more, and realize that I pasted the content into a WYSIWYG. My content contains html tags, so I want to paste source in, and I don't see any source code icon (that would have been nice to have).

So I went ahead and copied my content straight from the public facing page into the WYSIWYG, which got everything including some object tags I had in there, and after that there were some better results from Alchemy:

I played around with the SEO tools for a bit, and couldn't get the keyword search to work. I noticed that you could hover over keywords like you see above to show menus for adding keywords to some kind of list, but after adding a few items I wasn't able to find where the list was. I imagine there's a bit of a learning curve here, but it seems like there's got to be some potentially good stuff in these tools.

Next, I wanted to see what the output of a typical blog post looked like, so I checked out the source code. Woah! I haven't seen that many CSS and JS files on a page before. But, I know you can turn on aggregation, so I ceased the freak out and scrolled down to where the body started:

I've been diving deeper into HTML5 over the last several weeks, and I was hoping to see some

and stuff with oddly named attributes, and I wasn't disappointed (though I'm curious why the footer was at the head of the article). I'm still wrapping my mind around best practices around HTML5, and I imagine that looking at this might contain some good lessons. 

In the Open Enterprise description it mentioned that it's using a flavor of Omega for a responsive theme, so I played with the browser window size a bit to see what changed, and got a re-sized logo and stacking in the navigation on the smaller window size. At an even smaller window, the sidebar drops down below the content. Cool.

When I go to check out the blog page as an anonymous user (after publishing the page), I get Access Denied. Bummer! ;)

Okay, so permissions are set up to keep anonymous users at bay. NO BLOG FOR YOU! Well, I kind of want people to read this stuff, so I went to the permissions page and made a couple minor adjustments:

And

Hey, that blog page looks pretty good! Except for a little floating issue with the comment buttons:

So at this point, I want to see how hard it would be to integrate Disqus with this blog. I first go to the modules listing page on the off-chance that it's already included. It's not, and I noticed while I was there that Drupal core and the theme is already a bit outdated:

It's tough to keep up with Drupal, man. 

I knew from talking with the Level 10 folks that they were really getting into integrating Apps, so I wanted to check out the App listing to see if I might have missed anything. In the admin bar I clicked Apps and then took a look. I didn't see anything there, so now it's time to download a module and see how well it integrates with the distro.

Okay, that took a few minutes but everything went smoothly. I now have Disqus enabled on blog posts, and normal commenting disabled.

Now I want to click around a bit. I click on the Images tab, and WHAT'S THIS? Whirlyball? And who's the psycho int he middle who clearly runs the operation?

This default placeholder content is way better than what my real content is going to be. ;)

When I click on HOME, I get a Page not found:



Bummer again! The Add URL redirect link is enticing, but when I click that it looks like it would set the redirection for any 404s, rather than just the home page:

Okay, so I'll leave that as is for now.

As my next task, I want to set up those social links at the bottom of the page to point to something functional. Icons would be nice, too, but I'm suspecting this is a standard Drupal menu and getting those icons might take a little pulling of teeth:

Indeed, there's a little cog wheel that's a bit hard to see against the blue, that points me to edit the Social menu, where I update my links.

Getting my share on

Now, I want people to be able to share posts, ala ShareThis. So, I take a gander through The Configuration menu and see a Social media item, which I click on. As I scroll down, it looks promising:

Okay, but how do I get these to show on my blog posts? As I'm looking around, I run into this, which looks a little off:

I'm also getting some overlay screens where there's no way exit because (I'm guessing) the close button is under the admin toolbar:

Then I browse to the modules page and see that maybe the social module I'm seeing isn't fully functional yet:

I check out the Help page and it mentions something about one's profile, so I think that maybe I have to associate the social media accounts with my user, rather than the site, but when I go to my user account page, I just get a submit button:

So I scroll down the modules listings page, just to see what's in there. And what's this?

This looks promising. So here's what I do:

  1. Enable the Widgets module
  2. Check my blog post to see if social icons magically appeared. They didn't.
  3. Went to the Widgets module page to see if there was any insight there. Indeed there was!
  4. Based on what I read, I went to the blocks configuration page and enabled the Widgets: socialmedia_share-default block in the Content region, above the Main page content block.
  5. I went back to the blog past, and whammo!

Okay, so I kind of like the ShareThis versions that include details about the different networks' activity (though I just found out you can up the count just by clicking on the icons), and based on the info on the Widgets module page, I probably just need to include the Service Links module for that.

Things to investigate and update

So I'm liking this so far. I have a responsive theme that doesn't look like crap, I have a lot of my needs anticipated in terms of blogging setup. I'm on Drupal 7, which feels like the future after working on Drupal 6 so much these last 6 months, and it seems like the feature modules / apps are thoughtful and goal-oriented, meaning I can probably get some nice functionality pretty cheap if I have the need for additional features later on.

At this point, there's a few things I want to check out / fix / accomplish / learn:

  1. Get the site branded.
  2. Figure out how to interlink pages with automatic Related to this article functionality for better SEO.
  3. Set up archives and a tag listing blocks or pages for better SEO (correct me if that's old school thinking).
  4. Check out how forms do on the responsive front.
  5. See if enabling the view source on the Wysiwyg is going to break a feature module
  6. Research current best practices on working with Features in a distro (I'm curious if anything has changed since I put together a video series on the subject)
  7. See if I can use Alchemy to automatically tag and enrich posts for search engines, ala the AlchemySEO service / product. And also, is that kosher?
  8. See why I can't get keyword searches to work. Is there some API key I need?
  9. Check out RSS feeds. And is there any integration with PubSubHubUb, and do I really need to care about that?
  10. What about trackbacks and pingbacks? Do people use those? Any reason they weren't included in the distro?
  11. Play with OpenGraph MetaTags to see how it impacts mentions in Facebook
  12. I'm curious what the UUID module means for nodes. Can I use them in feature modules now (the last time I tested that it was buggy)? Is that what it's intended use is in Open Enterprise?
  13. Integrate a menu-search tool like Coffee, or maybe finally check out the Drupal 7 port that Amit Goyal made of Navigate.
  14. Do some Adobe Shadow (or Edge Inspect as it was newly released today) to check out how this Omega theme looks on a few different devices. And make sure images are 100% max-width (they get cut off as the screen gets smaller).

Thanks!

Thanks to the Level 10 folks for putting this distribution together. I know it's super hard to build features that work for lots of use cases, but it seems like they're doing a really good job so far. I'm looking forward to seeing how fast I can get this to production. :)

Apps Images enterprise I ma e 5. Is ani app for m ? h'1 Images content and displaying image ApPS are the next generation In usability for Drupal They contain bund les of galleries tuncncnatnv for your bS'lte. Select any apps you want to install Fig hr new You a add rmln? jailer rim line I'll pm page i Link Mil.11.l1gr!'; rmf ns- &quot;,uluru' link. It can be used to keep short I ? gf valuable resources In odder lay Inman apps. you Muni be able Il I FTP er SSH in! C tour server This uses v. r, nr the basis for fiJi blqw;!1 link directory like dmo.T.org. same [process as the update module. locations rr-r APPS TO ? ALL Enables r&quot;r'hl:rl.l4.e me n L od locations as content address are automatically go eneere map displays you Cari 'i?iY groups or locations on Of as listing april rotating balder j blog rotating banner of sudes with links Vern ()ll·illmC'1 on the homE page multiuser full featured avg app. Has many at the features .M' SEa essentials Development essential &quot;1';trrh engine cptlmI7.,tifl a features for your sea is il, more this app ands the most commonly used devel omern medulas to your site if yogi are limited set of than thaw In CFO ToXIcs. If you have Into ? SRN doing development In D r pal I this app its essential Ned robed CIO events a seo Tools provides t(lil1? for managing time bated events includes r h ? Liam Suite of advanced Seo tools including keyword reseercn and content analysis Seo Tools of includes all of the features of co essentials plus many ether tools for going Seo In fan drupal lode to bug In apps you must install 90olille_anaMlcs I'll a n a to. fore this enables creation of answers oLD frequently asked questions using a standard Iraq form, app will install o Forum DNAa! May, community discussion boards with topics and threeued comments help GNU socialize tour Swe. images I'd videos enterprise images is il 11 App for noIr aging images cnntr-n and displaying image provides video content management [naMes playing 0.11 uploaded Dr timbered videos galleries l' them Y,iliU T,u Un ? di WebfOrm Manages online resource links it ean be used tO keep short list or Ell valuable resource eneerea you to add web terms to your Site that can be lemalled and/or saved to a uenaase includes contact us example form but allows you to build any number of cUStom forms DiE:FAULT content ? In Stal default content B, ?r?jr.tli ng this box default confirm will me ,imt;tilDll;ld fgr each app without default content the site may look empty before yOU start addl fl to It. YOU can remove the .iI'efw! l content later by going to line apps caring page install apps SkIp this step Home Update downloaded successfully. WARNING You are not uSing an encrypted connection wyour password Will rnccse profile be sent in plain text, team mere cnccse language TO continue provide your server connection details Verify requirements Connection method se up database 'FTP install profile FTP connection settings Configure site username install apps Finished Password your password not sated In the database and only used to establish cc-reecucn ADVANCED SETTINGS Continue Alchemy has been installed Content Analysis has been installed Content Optimizer has been installed Keyword research has been installed Kilvword research google has been installed wcrestream has been Installed AppS enabled successfully Imponed node 43. Contact uS of i nodes oJt imported Some values may have been reset depending on Node experts configuration Choose profile Choose language Verify requirements Set up database rnstan profile Notes. Undefined vanabte replacements in quid token (line ? 9Sof of jChrlsjWebsilesjloc.b1og bullddmodule.comfslteS/dlljmodules/uuld/uulo taken NotiCe Undefined variable replacements in uuid_tokensO (line 9501 of jChf;sjWebsit?sjlo' blog. bui!d.Jmodul?.com/ sites J/f/rmxlule l/uuid lIuid.1 ket Notice Undefined variable replacements in lJuid_tok.?n'JO (line ? 95 of of IChf;'JIWf'bJit?J/loc blog. build.modulO com/silf'S .t!f/modulu/ uuid/uuld take i NotIce. undefined variable replacements in uujd_tok?nfO (line 95 of /ChrISjWt'!bsilesjloc.b1og bul/d.dmodule.comfsltesfd//fmodult'!sfuu/d, Eula takei Notice Undefined vartabte replacements in uuid rokens(J {line 95 of /Chr;s/Wt'bsites/lrx.bJog,bui/ddmtJdu?.comlsitt'sldll/mtJdu!t's/uu;d/UUid,lClk Notice undefined variable replacements in uujd token (line 95 of IChf;sIWcobsit?'J/loc blog hutld..Jnmdulr com/silf's I .tll/modulu/ uuid/uuld take i notice undefined vanable o. replacements In quid token [line as of ()f /Oms/Web5ltt's/foc.blog,bu,!ddmcdu!e.com/Srles/.l/1lmcdlr!es/uwd/uwd,t(lAw Notice Undefined variable replacements in uuid_tokens(Hline 9501 of IrIdic Ill tgif hills Jri&quot;, in, ,J[ configure sue rnstan Apps Finished Notice: getimagesizeO [function.getimagesize): Read error! in image_gd_get_infoO (line 349 of IChrisIWebsitesl/oc.blog.buildttmodule.comlmoduleslsystemlimage.gd.inc). least 2 to times is recommended Consider mcreasm Content Analysis Results Analyzers Alchemy Alchemy Keywords Concepts Entities Quick SED Term Relevance Keyword research voice testimonial service 95.7% Analyzer checklist Content Analysis Results Close Window Quick SEa Analyzers ep ick Quick SEe rd: testimonial long analysis Analyzer checklist yzer page tnte Char count 95, Word count ? 14, Keyword count ? I. Keyword density 7.1, Keyword prominence ? 28.6 Your page title contains 95 characters No more than so characters to 75 characters is recommended Consider reducing the length of your page title sou Char (ounl.76, Word count. 12. Keyword ccunt-, I. Keyword density_S.l, Keyword prominence. 16.7 You body contains 12 words At least zoo to 800 words is recommended Consider Increasing the number of words. The targed keyword 'tes1imoniar occurs in the body I time At least lto times is recommended Consider Increasing the number of keyword ccecrenees in your body copy Meta keywords Char count o. Word count o. Keyword count o. Keyword density.O.O, Keyv,.(Hd promlnence-O.O You meta keywords contains o words. At least to SOwards IS recommended Consider Increasing the number of words. Content Analysis Results Analyzers Alchemy Alchemy Keywords ? Con Entities Concepts Quick SEO Term aerevaeee Keyword research can widget 99.3% Analyzer checklist Google Voice 98.90 wtb page 83.00 voice testimonial service 63.4% Google Voice page 57.1% Click Call Widgets 55.5% new call wjdge? 53. lg new greeting 44. embed code 35.1&quot; OPEN ENTERPRISE OPEN ENTERPRISE HOME Notice: Undefined variable replacements In BWIA rokensQ fine 95 of /C'm.sIWebsltesJ? tllOg.ooA'damodlJ.le.comlsimslsNimodul'esluuidIUUJd. rokens. Nance: Undefined variable: replacements in ulna tokens (line 95 of of Johns ? ? byog OlJi'd.lrn odlJ ? comlsitrJsfaY/ft'KXk.sPos/uuidfuukJ. tokens l. ACCESS DENIED You are not aumcruee to access this pa ge USER LOGIN E-mail p as s word Create now account Request now password LOGIN CA ? main content ? ay &lt;hI ClASS M title idea pa9c-title&quot; show to quickly set up an Qwcso:r &lt;dlv class: class-&quot;tabS elearfix-&gt;&lt;h2 ? class ? element-invisible&quot; Primary classeN active?&gt;&lt;a ? href ? M /blog/chris-shattuck/how-quickly-set-awesome-customized-f class-&quot; element-invisible-&gt;{active tll:b hrefa·/node/6S/edit? ? href·-/node/6S/kwresearch&quot; ? iliaci idea hret-&quot;/n In class-&quot; action-links&quot; ? hret-M/node/add/enterprise-b1og&quot;&gt;Add blog post! all &lt;11&gt;&lt;a href-&quot; /admin/contont/node/enterprise blog Administer blog ? blog &lt;luI&gt; &lt;div id-&quot;b1ock-system-main&quot; c j as a=vb block block-system&quot; c Laa s=vb block bloc title. id-&quot;block-system-main&quot;&gt; &lt;div class=&quot;content&quot; class: content clear fix &lt;article about=&quot;/blog/chris-shattuck/how-quickly-set-awesome-customized-free­ class=&quot;node node-enterprise-blog node-promoted node-published node-not-sticky sel enterprise-blo9-65 -s- &lt;footer class submitted &gt;&lt;span property: dc: date dc:cre4ted&quot; content ? 2012-0 21:36&lt;/span&gt; &lt;span ? rel=&quot;sioc:has creator pga href=&quot;/users/chris-shattuck&quot; tit! In ? t r. h r; a a hnt&quot;. eueje&quot; T. vn&quot;,n ? RI II a p. r ? n t. nrnnArt-.v= fnltf!nllmlR&quot;&gt;r..hri tsi MW modulI! module D Modules There are updates available for your version of DrupaL To ensure the proper functioning of. updates page for more information and to install your missing updates, There are security updates available for one or more of your modules or themes. To ensure the available updates page for. more Information and to install yOur missing updates. Download additional contributed mOdU'o extend Orupal's functionality. mod Regularly review and install available updates to maintain secure and current site. Always REGISTER for to post comments LOG IN Access the content overview page VIew published content View own 0i;own unpublished content ?content revisions View content IJ.'JCD Administer comments and comment settings View comments Post comments by i oj NRA PAGE NOT FOUND Add URL redirect from this page to another location URL redirects FROM http:!{loc.blog.bulldamodule.com/ navigatlon4Q4 TO. TO Advanced options Add URL redirect fro The requested page could not be Ic Unk8dln Twitter YouTube Submission form settings Title Publishing options Published Promoted to front page Display settings display author and date information Comment settings closed Threading SO comments per page Content Analysis Settings Menu settings Ml sneman Indusio excluded Scheduler settings PubU,hlng enabled Iran DEFAULT ICON style i 32)(32 ill rcee set Styles teverren 16x16 Glossy 32x32 by teverren 48x48 ? in t ? L, J S I'd a media Social media Enabled ? Name Social media Version Description I ? J Example module to demonstrate module media. beta 11 Operations Help Permissions Configure Widgets Enabled Name Widgets Version Description Operations xl. a. Enables easy management of code snipes like Twitter, Facebook and Google buttons. betaS Required by: Widgets Service links (disabled) Widgets 7. x-I. 0- Enables links from Service links module to be used as widget elements Service links betaS Requires: Widgets (disabled), Service) inks (miSSing) hello on, matter ode export Key ords kill ? ion o. CHRIS SHATTUCK View Ed. 1- Shortcuts Social profiles SUBMIT File browser Devel HOW TO QUICKLY SET UP AN AWESOME, CUSTOMIZED FREE VOICE TESTIMONIAL SERVICE a ? f, '&lt;1 Mon. 2012-09-2421 :36 Chris Shattuck Ever since heard about the VoiP Drupal I've been itching to set up some kind of service that allows people to call in and leave testimonials for BuildAMocMe. But. haven't had chance and decided today that it was time to do something about it. ran into this awesome article outlining the process of using Google Voice to set up the service, and 1; I. it I. pH

Sep 12 2012
Sep 12

In my last blog post I outlined how to use per project installation profiles. If you read that post and want to use installation profiles to take advantage of site wide content changes and centralised dependency management, this post will show you how to do it quickly and easily.

The easiest way to switch installation profiles is using the command line with drush. The following command will do it for you:

drush vset --exact -y install_profile my_profile

An alternative way of doing this is by directly manipulating the database. You can run the following SQL on your Drupal database to switch installation profiles:

UPDATE variable SET value = 'my_profile' WHERE name = 'install_profile';
-- Clear the cache using MySQL only syntax, when DB caching is used.
TRUNCATE cache;

Before you switch installation profiles, you should check that you have all the required modules enabled in your site. If you don’t have all of the modules required by the new installation profile enabled in your site, your are likely to have issues. The best way to ensure you have all the dependencies enabled is to run the following one liner:

drush en $(grep dependencies /path/to/my-site/profiles/my_profile/my_profile.info | sed -n 's/dependencies\[\]=\(.*\)/\1/p')

Even though it is pretty easy to switch installation profiles I would recommend starting your project with a project specific installation profile.

Edit: Jaime Schmidt picked up a missing step in the instructions above. You need to enable the installation profile in the system table. The easiest way to do that is with this drush one liner:

echo UPDATE system SET schema_version = 0 WHERE name = 'my_profile' | drush sqlc && drush cc all
Sep 11 2012
Sep 11

At Drupalcon Munich, one of the awesome things was seeing so many people show an interest in helping out with Drupal 8’s Mobile Initiative. On the Friday after Drupalcon’s session, at that Code Sprint, there were four tables full of people helping out with JavaScript issues, Drupal’s administrative screens, responsive images, and HTML5. And, as Dries’ recent blog post shows, now is the perfect time for you to help out with the Drupal 8 Mobile Initiative. Feature freeze is coming on December 1st and we still have lots of things to accomplish.

If you’ve never worked on Drupal core before, no worries! The community is currently building a “Drupal Ladder” learning curriculum to help people level up their abilities as they climb its skill-based ladder. You can quickly learn how to contribute to Drupal development and then use your new skills to make Drupal 8 mobilicious! Jump on the Drupal 8 Core ladder at http://learndrupal.org/ladder/ee503327-50be-1904-8d04-9499098cad64

If you’ve already climbed the ladder, you can start helping out the Mobile Initiative immediately. With feature freeze so close, we’re having weekly initiative meetings to create as many opportunities as possible for newcomers. In fact, there’s a meeting today! http://groups.drupal.org/node/252318

What’s our focus?

The scope of the Mobile Initiative includes a few varied subjects, so there’s lots of different things you can work on:

  • Responsive Web Design
  • Front-end performance
  • Mobile administration
Slide from the progress report given at Drupalcon Munich.

Because we will still be able improve features and increase performance after December 1’s feature freeze, we will be focusing on those items that are “new features” for Drupal 8. Here’s three items I’d like to tackle first:

Admin navigation

When a user first installs a Drupal site and loads it up in their mobile device, the administration section of the site is literally the first impression they will have about Drupal’s mobile capabilities. Having mobile-friendly admin screens is a great way to quickly say “Hell, yes, Drupal loves mobile!”

In addition, being able to edit and draft Drupal content while waiting in line or or on the train (all the typical mobile locations) would be a big boon to productivity when administering a site.

Lewis Nyman’s early passion for this subject convinced me of how important mobile administration is to Drupal 8’s success. It’s why I made sure it was in the top goals of the Mobile Initiative. And now Acquia has taken up the cause with their Spark project; originally proposed as a Drupal 7 project, Spark received such wide praise during Drupalcon Munich, that the Acquia team has decided to first work on a Drupal 8 version before backporting to D7. But only if the Drupal community helps with the work!

Jesse Beach put up a demo site showing the current state of the work. (There's also a video showing possible admin navigation interactions a few comments earlier.) She’s currently looking for people to work on the code; just create an issue in the Git sandbox and Jesse can give you commit access.

Issue #1137920: Redesign Administration Tool bar for Small Screens & Touch Screens

Assetic

Assetic is a Symfony component providing pluggable processing of JS and CSS, making coffescript, Uglifyjs and custom aggregation groups easy and painless. Drupal 7’s current aggregation is a decent, general-purpose tool, but to achieve optimal front-end performance, engineers often have to work around the system and create custom solutions. A pluggable Assetic solution would make it really easy to extend and tweak the existing system instead of re-writing it.

Much of the front-end performance work we have planned can be continued through feature freeze, but this clearly falls in the "new feature" category, so we need to work on it now.

Théodore Biadala (nod_) has been heading up the JavaScript changes for Drupal 8. Since Symfony is a new part of Drupal, we need either need people with some Symfony or Assetic knowledge or people with a strong desire to learn these exciting new technologies.

Issue #1751602: Use Assetic to handle JS and CSS files

Responsive images

Responsive images is a serious front-end performance problem that the entire web industry is struggling with. Fortunately, the Drupal community has been very active in the proposed HTML element and we’ve made a commitment to provide a reasonable forward-looking solution to this thorny problem.

The current work is being actively worked on by Peter Droogmans (attiks). We’ve recently decided to break this task into several sub-tasks, listed below by importance. Drupal 8 will need a way to configure the different sizes used for a responsive image. And, while a UI for setting breakpoints for the site would be mighty useful, and API for setting breakpoints is needed first.

Issue #1775530: Move picture into core
Issue #1775774: Allow themes to identify their breakpoints to Drupal
Issue #1734642: Move breakpoints into core
Issue #1170478: Responsive images [meta issue]

Even more stuff

After feature freeze, the Mobile Initiative will be pivoting to focus solely on front-end performance and improving the features that make it into Drupal 8 before December 1. This includes improving our already-responsive themes and continuing to add to our Drupal 7 Mobile Guide.

Let’s get to work!

Sep 09 2012
Sep 09

Unbeknown to many users, installation profiles are what is used to install a Drupal site. The two profiles that ship with core are standard and minimal. Standard gives new users a basic, functional Drupal site. Minimal provides a very minimal configuration so developers and site builders can start building a new site. A key piece of a Drupal distro is an installation profile.

I believe that developers and more experienced site builders should be using installation profiles as part of their client sites builds. In Drupal 7 an installation profile is treated like a special module, so it can implement hooks - including hook_update\_N(). This means that the installation profile is the best place for controlling turning modules on and off, switching themes or any other site wide configuration changes that can’t be handled by features or a module specific update hook.

In an ideal world you could have 1 installation profile that is used for all of your projects and you just include it in your base build. Unfortunately installation profiles tend to evolve into being very project specific. At the same time you are likely to want a common starting point. I like to give my installation profiles unique names, rather than something generic like my_profile, I prefer to use [client_prefix\]_profile. I’ll cover project prefixes in another blog post.

After some trial and error, I’ve settled on a solution which I think works for having a common starting point for an installation profile that will diverge overtime using a unique namespace. My solution relies on some basic templates, a bash script with a bit of sed. I could have written all of this in PHP and even made a drush plugin for it, but I prefer to do this kind of thing on the command line with bash. I’m happy to work with someone to port it to a drush plugin if you’re interested.

Here is a simple example of the templates you could use for creating your installation profile. The version on GitHub is closer to what I actually use for clients, along with the build script.

base.info

name = PROFILE_NAME
description = PROFILE_DESCRIPTION
core = 7.x
dependencies[] = block
dependencies[] = dblog

base.install

php
/**
 * @file
 * Install, update and uninstall functions for the the PROFILE_NAME install profile.
 */

/**
 * Implements hook_install().
 *
 * Performs actions to set up the site for this profile.
 *
 * @see system_install()
 */
function PROFILE_NAMESPACE_install() {
  // Enable some standard blocks.
  $default_theme = variable_get('theme_default', 'bartik');
  $values = array(
    array(
      'module' => 'system',
      'delta' => 'main',
      'theme' => $default_theme,
      'status' => 1,
      'weight' => 0,
      'region' => 'content',
      'pages' => '',
      'cache' => -1,
    ),
    array(
      'module' => 'user',
      'delta' => 'login',
      'theme' => $default_theme,
      'status' => 1,
      'weight' => 0,
      'region' => 'sidebar_first',
      'pages' => '',
      'cache' => -1,
    ),
    array(
      'module' => 'system',
      'delta' => 'navigation',
      'theme' => $default_theme,
      'status' => 1,
      'weight' => 0,
      'region' => 'sidebar_first',
      'pages' => '',
      'cache' => -1,
    ),
    array(
      'module' => 'system',
      'delta' => 'management',
      'theme' => $default_theme,
      'status' => 1,
      'weight' => 1,
      'region' => 'sidebar_first',
      'pages' => '',
      'cache' => -1,
    ),
    array(
      'module' => 'system',
      'delta' => 'help',
      'theme' => $default_theme,
      'status' => 1,
      'weight' => 0,
      'region' => 'help',
      'pages' => '',
      'cache' => -1,
    ),
  );
  $query = db_insert('block')->fields(array('module', 'delta', 'theme', 'status', 'weight', 'region', 'pages', 'cache'));
  foreach ($values as $record) {
    $query->values($record);
  }
  $query->execute();

  // Allow visitor account creation, but with administrative approval.
  variable_set('user_register', USER_REGISTER_VISITORS_ADMINISTRATIVE_APPROVAL);

  // Enable default permissions for system roles.
  user_role_grant_permissions(DRUPAL_ANONYMOUS_RID, array('access content'));
  user_role_grant_permissions(DRUPAL_AUTHENTICATED_RID, array('access content'));
}

// Add hook_update_N() implementations below here as needed.

base.profile

php
/**
 * @file
 * Enables modules and site configuration for a PROFILE_NAME site installation.
 */

/**
 * Implements hook_form_FORM_ID_alter() for install_configure_form().
 *
 * Allows the profile to alter the site configuration form.
 */
function PROFILE_NAMESPACE_form_install_configure_form_alter(&$form, $form_state) {
  // Pre-populate the site name with the server name.
  $form['site_information']['site_name']['#default_value'] = $_SERVER['SERVER_NAME'];
}

Some developers might recognise the code above, it is from the minimal installation profile.

The installation profile builder script is a simple bash script that relies on sed.

build-profile.sh

#!/bin/bash
#
# Installation profile builder
# Created by Dave Hall http://davehall.com.au
#

FILES="base.info base.install base.profile"
OK_NS_CHARS="a-z0-9_"
SCRIPT_NAME=$(basename $0)

namespace="my_profile"
name=""
description="My automatically generated installation profile."
target=""

usage() {
  echo "usage: $SCRIPT_NAME -t target_path -s profile_namespace [-d 'project_descrption'] [-n 'human_readable_profile_name']"
}

while getopts  "d:n:s:t:h" arg; do
  case $arg in
    d)
      description="$OPTARG"
      ;;
    n)
      name="$OPTARG"
      ;;
    s)
      namespace="$OPTARG"
      ;;
    t)
      target="$OPTARG"
      ;;
    h)
      usage
      exit
      ;;
  esac
done

if [ -z "$target" ]; then
  echo ERROR: You must specify a target path. >&2
  exit 1;
fi

if [ ! -d "$target" -o ! -w "$target" ]; then
  echo ERROR: The target path must be a writable directory that already exists. >&2
  exit 1;
fi

ns_test=${namespace/[^$OK_NS_CHARS]//}
if [ "$ns_test" != "$namespace" ]; then
  echo "ERROR: The namespace can only contain lowercase alphanumeric characters and underscores ($OK_NS_CHARS)" >&2
  exit 1
fi

if [ -z "$name" ]; then
  name="$namespace";
fi

for file in $FILES; do
  echo Processing $file
  sed -e "s/PROFILE_NAMESPACE/$namespace/g" -e "s/PROFILE_NAME/$name/g" -e "s/PROFILE_DESCRIPTION/$description/g" $file > $target/$file
done

echo Completed generating files for $name installation profile in $target.

Place all of the above files into a directory. Before you can generate your first profile you must run chmod +x build-profile.sh to make the script executable. You need to create the output directory, for testing we will use ~/test-profile, so run mkdir ~/test-profile to create the path. To build your profile run ./build-profile.sh -s test -t ~/test-profile. Once the script has run you should have a test installation profile in ~/test-profile. I will continue to maintain this as a project on GitHub.

Sep 07 2012
Sep 07

Back when I was at Optaros and CMIS was first showing up in Alfresco in draft form, we developed the Drupal CMIS module. We had a few customers interested in combining the two technologies but I think we were a few years ahead of our time. Now it seems I come across people wanting to combine the two nearly every week in IRC, the forums, or in internal discussions. Alfresco has contributed additional modules for Drupal integration. And multiple partners have full-fledged solutions or service offerings built on top of both.

Optaros has changed their web site a bit since those days and a couple of screencasts I recorded back then have been removed. Because so many people are still interested in this topic, I’ve posted them on YouTube, with Optaros’ permission (thanks!).

So, if you want to see some examples of Drupal and Alfresco working together, here are a couple of screencasts from the archive…

Drupal Plus Alfresco (Original post, 4/7/2009)

[embedded content]

Open Atrium Plus Alfresco (Original post, 10/13/2009)

[embedded content]

Drupal, Open Atrium, the CMIS modules, and Alfresco have all progressed since then, but the general gist is the same.

Thanks to Optaros for letting me make these available!

Sep 07 2012
Sep 07

Occasionally a node reference or entity reference autocomplete widget will not operate as expected, specifically when it is based off a view reference display. Other widgets, the select box, or list of checkboxes, will still function correctly.

This will happen if the view is depending on a contextual filter (an argument), but is not being provided one. Normally a view can try to automatically fill in the argument if one is not provided based on the current page url. If the view fails to receive an argument and is unable to infer its value from the url path then it will fail to provide any results.

Outlined below is a possible scenario that would cause an autocomplete node reference field to fail.

  1. You are editing an autocomplete node reference field on a taxonomy term edit page.
  2. The view reference display you have configured is setup to only show content associated with the 'current' taxonomy term.
  3. In the view, the taxonomy term argument is provided by the current context if no value is available.

Here is how we have configured the view:


View configuration screenshot

Beneath 'Contextual Filters' clicking on 'Context: Has Taxonomy Term ID' will provide more information on this filter (argument received by views):


Note: In this screenshot we are running the openpublish beta2 profile, which currently has not updated to the latest patches.


Contextual filter configuration screenshot 1

Notice that the view will try to fill a value into it's contextual filter if none is provided. It will try to do this based on the current url:


Contextual filter configuration screenshot 2

If your widget is setup to be list or select box then the view will be able to determine the current context (a taxonomy term) and provide a default value. Views can do this because the context is determined while the form is being loaded. But if you are using an autocomplete field, the json callback to drupal provides no context and the view has no idea what page it is being accessed from.

A solution can be achieved by providing a custom function that handles the autocomplete callback from json. The function will then explicitly set the views argument to the correct taxonomy term id.

  1. Alter the existing form field to use a different callback path with hook_form_FORM_ID_alter()
  2. Create the path router in hook_menu()
  3. Design the callback function itself to invoke views

Alter The Existing Form Field

In this particular case, after viewing the source of our taxonomy form, we find the form tag id is 'taxonomy-form-term'. This translates into taxonomy_form_term as the FORM_ID when declaring the hook_form_FORM_ID_alter() function. The node reference field itself has been named 'field_myfield_nref' and contains up to 3 descrete values.

/**
 * Implements hook_form_FORM_ID_alter().
 */
function mymodule_form_taxonomy_form_term_alter(&$form, &$form_state$form_id) {
  
// We will get our term id argument from the from build itself.
  
$term_id  $form['#term']['tid'];
  
// This is the path we will create in hook_menu().
  
$new_path "mymodule/autocomplete/{$term_id}";
  
// Maximum number of descrete values (deltas) that are present.
  
$max_delta $form['field_myfield_nref']['und']['#max_delta'];
  
// Hijack the autocomplete callback for each of our values.
  
for($x=0$x&lt;=$max_delta$x++) {
    
$form['field_myfield_nref']['und'][$x]['nid']['#autocomplete_path'] = $new_path;
  }
}
?>

The above hook is enough by itself to get the autocomplete widget to begin polling a different path as you type in characters. Make sure that you have flushed all caches, and that your browser is not caching the old javascript. Now Drupal needs to be configured to do something useful when the above path is being requested.

Create the Path Router

/**
 * Implements hook_menu().
 */
function mymodule_menu() {
  
// The index will specify which path is responded to.
  
$items['mymodule/autocomplete/%'] = array(
    
// This is the function to be envoked.
    
'page callback' => 'mymodule_autocomplete_callback',
    
// Which url path segments, delineated by the forward slash (/), should be
    // sent to our function as arguments. Zero based.
    
'page arguments' => array(2),
    
'access callback' => TRUE,
    
'type' => MENU_CALLBACK,
  );
  return 
$items;
}
?>

Now, when the autocomplete field accesses the path /mymodule/autocomplete/{integer_value} Drupal will execute the function mymodule_autocomplete_callback. Next, the function must be configured to Invoke the correct view and return something useful to the requesting javascript.

Design the Callback Function

/**
 * Autocomplete callback.
 *
 * Recieve a field autocomplete json request from a taxonomy term edit page.
 * Returns a list of article nodes whos titles matches what has already
 * been typed into the field so far.
 *
 * @param int $term_id
 *   Unique taxonomy term identifier. This is the variable that is represented
 *   by the % sign of the path in hook_menu().
 * @param string $string
 *   Contents of the json submission. This will be what the user has typed into
 *   the node reference field so far.
 *
 * @return drupal_son_output()
 *   A json formated string containing possible matches constructed by a view.
 */
function mymodule_autocomplete_callback($term_id$string '') {
  
// We know the name of this field specifically because this is an edge case
  // solution. More flexible code could be put in place so as to not hard code
  // this. The field settings will store which view to use.
  
$field field_info_field('field_myfield_nref');
  
// These options will be received by views. Within the result set that views
  // will provide, we want to further limit by comparing the field 'title'
  // against what was submitted by the javascript ($string). We will compare
  // by 'contains', meaning the title must contain $string. The total results
  // returned should be no more than 10.
  
$options = array(
    
'string'      => $string,
    
'match'       => 'contains',
    
'ids'         => array(),
    
'limit'       => 10,
    
'title_field' => 'title',
  );
  
$settings $field['settings']['view'];
  
// This is the important part below. This view requires an argument for the
  // contextual filter to operate when the context can not be determined
  // automatically
  
$settings['args'] = array($term_id);
  
$matches = array();
  
// This is where the view is run that is reponsible for creating the possible
  // selections for autocomplete. Now we can pass in the argument that would have
  // otherwise been empty.
  
$references references_potential_references_view('node'$settings['view_name'], $settings['display_name'], $settings['args'], $options);
  foreach (
$references as $id => $row) {
    
// Markup is fine in autocompletion results (might happen when rendered
    // through Views) but we want to remove hyperlinks.
    
$suggestion preg_replace('/([^<]*)<\/a>/' href="https://activelamp.com/blog/drupal/customize-autocomplete-fields-with-results-you-want/([^'$2'$row['rendered']);
    
// Add a class wrapper for a few required CSS overrides.
    
$matches[$row['title'] . " [nid:$id]"] = ' class="reference-autocomplete">$suggestion '
';
  }
  return 
drupal_json_output($matches);
}
?>

By crafting our own module and using the above hooks and callback functions we now have modified the autocomplete field to work as desired. While editing our taxonomy term, the node reference field that allows us to select any node that is already associated with this taxonomy term works correctly. The first two values have already been filled out, while the third is in the process of displaying possible options.

Sep 03 2012
Sep 03

In a recent post, I explained how to dynamically alter the AJAX behavior of an autocomplete field in Drupal. But what if you have the similar node reference field, and want to modify the result set? This post will show you how.

Let's go over some assumptions:

  1. For simplicity, we will alter the field to show only nodes in English.
  2. This will only alter the autocomplete behavior, not the validation. If your field is set to accept a limited set of nodes, the suggestions have to be a subset. Altering the validation of the field is a post for another - longer - day.
  3. Our field's machine name is field_reference.
  4. Our field is set to accept only 1 value.

First, let's add the menu hook and the autocomplete behavior. Fairly standard stuff:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

/**                                                                           
 * Implements hook_form_alter().                                              
 */                                                                           
function example_module_menu() {                                              
  $items = array();                                                           
  $items['example_autocomplete'] = array(                                     
    'type' => MENU_CALLBACK,                                                  
    'page callback' => 'example_module_autocomplete',                         
    'access callback' => TRUE,                                                      );                                                                          
  return $items;                                                              
}

function example_module_autocomplete($input) {                                
  $results = array();                                                         
  $resource = db_query_range("SELECT nid, title FROM {node} WHERE language = '%s' AND title LIKE '%s%%'", 'en', $input, 0, 10);

  while ($node = db_fetch_object($resource)) {                                
    $key = sprintf('%s [nid: %d]', $node->title, $node->nid);                 
    $results[$key] = $node->title;                                            
  }                                                                           
  print json_encode($results);                                                
  exit();                                                                     
}

As you can see, all we're doing is providing a new AJAX callback. In that callback, we select nodes from the database and limit them by language. Now, let's define our form alter hook. But remember, like all CCK fields, hook_form_alter is too early in the process to modify a field. So instead, we define an #after_build function that will get called further down the road. Like so:

1
2
3
4
5
6
7
8
9

/**                                                                           
 * Implements hook_form_alter().                                                   
 */                                                                               
function example_module_form_alter(&$form, &$form_state, $form_id) {          
  if ($form_id == 'post_node_form') {                                         
    $form['#after_build'][] = 'example_module_after_build';                   
  }                                                                           
}

Finally, we'll actually alter the form field in our #after_build function. If you're alter a field set to multiple values, be sure to loop over the field and alter all rows.


function example_module_after_build($form, &$form_state) {                    
  $form['field_reference'][0]['nid']['nid']['#autocomplete_path'] = 'example_autocomplete';
  return $form;                                                               
}
Sep 01 2012
Sep 01

I'm not sure how it happened, but today I noticed that Drupal's menus were behaving very oddly. After upgrading to Drupal 6 and installing several additional modules, I noticed duplicate menu entries as well as other disturbing oddities. Items I was placing into the menu were not showing up. Menu items that I moved around were apparently saved but they did not appear properly in a dropdown context. 

Looking further into it via the absolutely awesome SQLYog tool, I verified that there were dozens of duplicate entries. Some items were duplicated up to six times. It was a real mess.

The database tables involved here are menu_links and menu_router. These are two of the more mysterious tables in Drupal. I haven't had the need to spend much time with them in the past, and I know now that this is a good thing. Fortunately, you do not have to know anything about them to fix this problem. While I spent a couple hours carefully deleting items from this table, ultimately I gave up. I was able to remove all the duplicates, but the menus were still misbehaving. At this point, I just wanted to do a factory reset on the menus, but it's not so simple as flushing cache. However, that is not far from the solution.

This solution will do a 'factory reset' on your menus. You will lose any customizations you have made. However, all core and contrib module entries will be restored very nicely.

Please backup your entire database before doing any destructive database manipulation. 

Step one is to empty the corrupted tables:

In your favorite SQL client, run the following commands:

truncate menu_links;
truncate menu_router;

At this point, your site will be completely unusable. But not for long. To complete the final step, you will need to be comfortable with the Drupal admin's best friend, drush.

Simply run the following commands from your terminal (Viva tcsh!):

drush php-eval 'menu_router_build();'
drush cc menu

Now my menus are as fresh as the day they were installed.

Though I could not clearly identify the cause of this problem, I would suggest backing up your database before installing the Taxonomy Menu module

Aug 30 2012
Aug 30

Typically I find that learning by doing works best for me, and there are some subjects (Drupal) that lend themselves to this approach as well. Drupal 7 Development by Example sounded like just the ticket. Many of the big keywords are hit on while flipping through the preface. Initially this sounds great, but perhaps there are too many? This book appears to have all the right intentions, but maybe it bites off a bit too much?

Who this book is for

This book is for people who have some experience building websites and who want to learn to do so with Drupal 7. You should have experience with HTML markup, CSS, and jQuery. Experience with previous versions of Drupal would be helpful, but is not necessary.

The Book

Authored by Kurt Madel, the book was published in May 2012, about 2 months before I got it. It has roughly 340 pages of content, divided among 11 chapters. Glancing through the preface of what the book covers, the buz words flow: Module Development, HTML5, Theme Development, UX, Media, Views, SimpleTest and Features jump out. Additionally the preface covers some of the tools you will use: MAMP/XAMPP, GIT, Drush, Aptana Studio, Drupal core and a list of modules. The "Who this book is for" statement is also in there, I quote it for a reason. Also I would like to draw attention to the module list, which also lists the versions of the modules used, this is very important.

First Rant

Just 2 months after the book is published many of the instructions no longer work. This is because the modules have changed. While the preface list shows the version, it would have been very helpful to list the version when instructed to install a module. Then it would be fresh in the readers head that the version was different and a good clue why things don't look as described. Drupal is constantly changing and it's modules some times change at break neck speed, documenting them in print is going to have problems. Kind of the nature of the beast.

Moving On

Chapter 1 runs quickly through installing MAMP & XAMPP. I hate XAMMP, couldn't bring myself to do it (a subject for a future blog series). It does cover making a few tweaks for performance, so a few points there. Then onto installing Drush and Git, extra points for covering installing these on Windows. However when it came to installing Drupal I had to scratch my head. Drush is an awesome way to install Drupal, drush dl drupal. The reader is instructed to use Git to clone the Drupal project, while this works, it comes with a lot of baggage. As in EVERY version of Drupal from version 3 to version 8 and every point, beta and dev in between. Downloading with Drush brings down just the current stable release, which is 11.2mb and 1050 files. Cloning with Git grabs 61.8mb and 3023 files. There is a comment about how the Git method can make upgrades easy, but it is never explored. The process of installing Drupal is stepped through, like I would expect from a book that says you don't need experience with a previous version of Drupal, except the steps for copying settings.php and creating sites/$new_url/files are skipped. The chapter finishes out with Installing Aptana Studio and a quick introduction to Drupal.org. I had hoped some of the developer related features of using a full IDE would have been explored, however it turns out everything can easily be accomplished with Sublime Text 2, or even notepad.

Chapter 2 dives into Drupal with a new content type, and a lot of emphasis on schema.org. While interesting, it felt like we were losing focus and the schemas can get a little tedious, specially when they don't really relate specifically to Drupal. There is a discussion between, Core, Contrib and Custom modules. The t() function gets introduced, but kind of passed right by. This function is kind of a big deal in Drupal and probably should have got more attention. Some code gets thrown out there with instructions to modify some existing code, which is hard to follow. When instructed on creating our first module, the opening  tag is omitted from the instructions. The chapter wraps up by installing Views, building a simple block, and using Devel Generate.

Chapter 3 starts with a little HTML5 and microdata. I ran into some trouble with microdata, the descriptions didn't match with what I was seeing and then I started getting errors when the Microdata module was enabled. A new module is created for using a compound field. I had a lot of trouble with the code, and despite grabbing the example code from the website, the images stopped displaying.

Chapter 4 moves into themes, highlighting the Omega theme. Drush is used to create a base theme, but most all of the goodies in Omega are skipped over. Render Arrays get some attention and a decent first look. However the code used with hook_preprocess_node is kind of glossed over. For example twice the pages are used setting up Omega, as are used on the Render Array and the preprocess function. While Omega is a great subtheme, it didn't really add anything to the section. 

Chapter 5 focuses on UX, sort of. It programmatically creates a block with 70 lines of code, that could have been done with one line of HTML in a custom block. WYSIWYG is introduced. Then 18 pages and about 200 lines of code are used to create a custom Content Editable module. While the functionality is very cool, and a glimpse at what we might see in Drupal 8, it seemed way over the top for a beginner book, specially considering it was more javascript oriented and not so much Drupal.

Chapter 6 brings in the Media module and some major moving target issues. Additionally around this time my site melted down, Ajax errors every time I tried to add an image. Disabling all the custom modules didn't help. So to keep going I turned to Aegir and fired up a new site ( I love Aegir ). I pulled in all the example code from the website to get things back in sync, which is when I realized the sample code had some minor errors relating to CSS classes that needed fixing. There were a few minor errors in instructions and then I ran into a wall. We are instructed to use the 2.x branch of the Media module, which has some nice improvements over 1.x, but is under heavy development. The version is actually labeled 'unstable'. A lot had changed from when this chapter was wrote and now. Luckily Dave Reid, a Media module maintainer, was able to straighten me out in IRC. He pointed me to a Media 2.0 tutorial that explained how to set things up, drupal.org/node/1699054. Dave also expressed some concern about using the unstable version of Media in a book, it is just changing to fast right now. The Colorbox module is also introduced and has changed since the book was published.

Chapter 7 deals with various ways to get feedback. It starts with the core contact form and Webform. There is a nice exercise on adding some text to the core contact form. This is a great example of a beginner exercise, and functionality that should be in core! The Devel module is used to dig some info out of Webform so the HTML5 email type can be added to the form. This functionality was added to Webform in February to version 3.16. Once I realized why my code didn't match, I decided to move on. The middle of the chapter has a rather lengthy enhancement to the Colorbox module, which really has nothing to do with feedback. The chapter ends with the Fivestar module and creating a custom widget for it.

Chapter 8 is more about Views. Mostly basic Views stuff, but there is an example of creating a Views plugin. Although mine didn't look anything like the screen shot, even when using the sample code. The module list says Views 3.3, and the current version is 3.5, so not all the buttons and items are labeled the same. I was still able to follow along, but it was annoying at times. The chapter ends with promoting the Colorbox custom module to a full project and using the Coder module. Good stuff, but has nothing to do with the Views chapter it lives in.

Chapter 9 lets you know right off the bat that the content isn't all related, "Rotating Banners and Project Promotion". The banner part uses Views Slideshow and some custom CSS to tweak the display, good beginner stuff. Then it is back to our custom Colorbox module to prepare it for submitting as a full project. Not so much for beginners, although something to aspire to I suppose.

Chapter 10 is SimpleTest, which I have to say flew right over my head and think most beginners will be totally baffled by as well. While the SimpleTest module is in Drupal core, it really is a PHP framework.

Chapter 11 hits on Features, which probably should have come much sooner in my opinion. This is actually a pretty good run down on features. Bonus points. Features is just the kind of thing that new developers should be learning, and can be tricky to understand at first.

Final Rant

There are multiple times when steps are skipped in the instructions. There are no photos in the downloads, which may sound petty, except the instructions sound like we should have them. This book suffers from editing issues. I fully understand that things change from the time a book is started till released, but many errors would have been found by just running through the examples like a clueless newbie.

It may sound like I hated the book, I don't. However going back to that Who this book is for statement, it misses the beginner mark. In fact if PHP were in that statement, half my issues would go away.

Not all is lost

It is quite obvious that Kurt Madel knows Drupal development. I would say this is probably a decent intermediate book, that has some editing issues.

Disclaimer

The copy of Drupal 7 Development by Example used for this review was provided at no charge to Scott Wilkinson of HaloFX Media LLC by Packt Publishing. However, no other compensation was received for this review and this review was published without prior review or any influence from Packt Publishing.

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web