Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Dec 23 2011
Dec 23

As of Drush 4.5, migrating a Drupal site between servers became much easier. The new, little-known drush archive-dump and drush archive-restore commands make it an essentially three step process.

Overview

A basic Drupal site is made of two fundamental elements: the codebase and the database. When you migrate a Drupal site, you need to migrate both of these elements, often with a bit of re-configuration to boot.

The Old Way

Before using Drush to migrate a site, my standard procedure for site migration looked something like this:

Database Migration

  1. create a new, empty database
  2. create a new user
  3. associate the new user with the new database (with correct permissions)
  4. create a db dump from the source environment
  5. import the db dump to the target environment

Codebase Migration

  1. compress the codebase into a tar.gz file
  2. transfer files
  3. unarchive tarball
  4. edit the settings.php file to reflect the new database settings
  5. check file ownership, group membership, and permissions

The New Way (for me)

I'll run through the first iteration using psuedo [tokens] as placeholder for file and sitenames.

Open up the command line on the source machine (or SSH in) and cd to your Drupal installation.

  1. Create a Drush archive dump (code and db in one)
    drush archive-dump [site]
  2. Transfer the dump to your target machine (clearly, you could just FTP it)
    scp [new-backup-archive] [user]@[target-machine]:[target-folder]
  3. Now switch to the command line on your target machine, cd to the folder above your to-be-created Drupal installation and type:
    drush archive-restore [new-backup-archive] [site] --destination=./[new-folder-name] --db-url=mysql://[msql-user]:[mysql-password]@[target-server]/[db-name]

Drush will then create the new directory, unarchive your tarball into it, create a new database, populate it, and edit your settings.php file accordingly. Wow. Awesome.

Here's how the process looks for me when moving to my local MAMP stack (without tokens):

  1. drush archive-dump default
  2. scp /home/grasmash/drush-backups/archive-dump/20111256023713/grasmash.20116223_023613.tar.gz [email protected][my-ip]:/Applications/MAMP/htdocs
  3. drush archive-restore grasmash.20116223_023613.tar.gz default --destination=./grasmash --db-url=mysql://root:[email protected]/grasmash

Now it's important to note that you can use an existing directory or an existing database. If you'd like the contents of the existing directly overwritten, you can use the --overwrite flag. Likewise, you can grant Drush sudo MYSQL privileges to modify your dbs by adding a --db-su flag, followed by the correct credentials.

Furthermore, you don't need to use --destination or --db-url flags at all. If you leave these out, Drush will attempt to unarchive to the current working directory, and will attempt to use the database credentials defined in the settings.php file.

For more information on these commands, use drush's --help flag, e.g., drush archive-restore --help

Enjoy!

Quick disclaimer:  There are many ways to migrate a Drupal site between servers. You should certainly be using SCM (like git), which can be used to move the codebase. You can use drush or backup_migrate to transfer the database. Or, you could simply move around Virtual Machines.

EDIT
At present, it seems that the --db-url flag only works with Drupal 6. Please leave a comment and let me know if this is not the case!

Dec 15 2011
Dec 15

Have you ever needed or wanted to pull a remote Git Repository on Drupal.org down as a zip or tarball? You know, the way Github does? Most project releases have fairly recently built tarballs which is awesome - but Sandboxes do not (so it seems - please correct me if I'm wrong!).

The following snippet lets you "archive" a remote repository, pull it down as a tarball, and extract it in-place:

git archive --format=tar --prefix=PROJECT/ [email protected]:sandbox/USERNAME/123456789.git BRANCH | tar -xf -

Some important notes:

  • The prefix is very important - without it, tar extracts to the current folder.
  • The trailing slash on the prefix is equally important - without it all files have PROJECT at the beginning!
  • The number (next to .git) references the sandbox Node ID.

I have tried using the "zip" format, however the unzip bash command doesn't accept stdin as a source. It looks like funzip might hold some promise thought…

EDIT: - looks like you can only do this if your user has access to the repository.

Dec 12 2011
Dec 12

I guess a lot of people that have to deal with integrating videos on a website for iPhone and iPad, will  - at some point - run into the problem that some videos simply won't play. In most cases embedding the video is the simple part: either we use the fantastic HTML5 specifcation or use some weird snippet for loading a Flash Video Player. But I don't want to look at that in this post, I want to dig a little deeper into the encoding problematics. But this is not about dealing with whether to use OGG TheoraVP8 or h264, or how to use them all at once. In this case I stumbled upon an iOS specific problem. So h264 would be the codec to look at, as this is the favourite on Apple devices.

Differences for iOS devices

One might think it would be enough to simply convert any video to h264 format for it to run on iPhone and iPad. But it isn't that easy: when a video plays on an iPad, this does not mean it will play on an iPhone (in my case it was an iPhone 3GS). Instead of playing the video Safari would instead show a crossed out play button, with the Safari Debug Console displaying a "very helpful" message:

Quicktime: Movie could not be played.

The iPad played the same video just fine.

After some research on the web I came to the conclusion that it's not enough to choose the right codec. To a greater degree, you have to keep an eye on different parameters (as bitrate for audio and video). Sure, on other devices (not just Apple's) you have to deal with that too. A blog entry at Zencoder tries to summarize the settings for different devices.

Sure, you now could try to find the correct settings by trial and error. The iOS Simulator of XCode might help in that situation, as there the same problem occurs on the simulated iPhone, but not on the iPad. So you can debug on your local machine.

Use the Handbrake to quickly stop

With the hints from the zencode post I tried may way on the road, but of course the first attempt failed. The video conversion tool of my choice "Handbrake" (alternatives) luckily had a preset that would just fit the needs for Apple devices. So I tried it again with that preset - named "Apple - Universal", and, lo and behold, it worked. The video just played on both devices. I finally found the right conversion tool and settings.

Impacts your site

For site adminstrators this would mean to convert all videos they ever uploaded to the new format. This really only might be a viable solution for smaller sites, but in the current case it was fair enough to make the most important videos compliant for the iPhone.

Other solutions

Sure, converting the videos on your local machine is not feasible for every website. But it saves the need for server side processing (like ffmpeg) or implementing a third part tool (like Zencode). If you want to provide different sizes for different devices, the local conversion is no pragmatic solution. You should definitively grab an ffmpeg module or use a SaaS for video conversion for that use case.

Icons by: Dirceu Veiga

Dec 07 2011
Dec 07

Following my previous post, and after getting some great reaction from the community via IRC and in Drupalcamp Toulouse, I’d like to share both my thoughts about the sponsoring and about the technical stuff.

The sponsoring, which already has FunnyMonkey committing to pay almost half the price, and providing on top of that developer hours, was something I’ve decided to do only after I was convinced there are some big players using OG7. The thing is that _anyway I intend to do this change. The sponsoring is just giving it a boost, and making sure it will be _sooner than later!

I’m happy I was able to post that request and to get good reaction from the community. I’m equally happy that Planet Drupal isn’t swamped with such requests. It’s not about Drupal App. It’s about harnessing the community towards a goal. Companies offering to assist by providing developer hours is exactly the proof of that.

Now here’s a quick overview of the planned changes. First the conceptual changes:

  • Instead of OG having it's own field type, we easily hook into Entity reference field, as it allows registering our own handler using a CTools plugin
  • Instead of saving the field values _and the OG membership as we currently do, we save the OG membership, and use hook_field_load() to populate the Entity reference with the correct value. Yes, I know, the docs clearly say that we should not load fieldable entities in hook_field_load() as it might cause loading recursion, so we'll just make sure you can't attach group-audience fields to OG membership (which anyway doesn't make much sense). The benefit of doing that, after a long talk with the people form the community, notably fago (thanks again fago!), came out as the best solution for a problematic thing
  • OG membership is now tightly coupled with the field that it belongs to. This means we can care about the field cardinality (i.e. how many OG membership can be created per field), and since we know the OG membership type we can provide smarter metadata
  • We can have multiple fields, to allow subscribing users/ content to group with different membership type. No more hard-coding of the field name as-well

Sneak preview time… As usual we can make a content type (Clubs in this example) a group, through the content type edit.

In the new version, when you want to make a content type (Post in this example) a group content, you immediately select to which group entity type, and optionally bundle the field should point. Making the reference to only a single entity type, allows us to have really simple integration with other modules (e.g. Views, Tokens, Rules) that expect the field value to be of the same entity type.

No more OG’s own field – this is just another Entity reference field.

In the field settings, we select OG’s handler for Entity reference, where we can define different OG related settings. The “My groups” checkbox indicates the reference should show the user only groups they are member of.

As we can attach multiple fields, it means the My group/ Other groups administrators used to see in the group-audience field, is no longer needed.

Best practice will probably be, show one field set to “My group” as select list, and another field for groups a user doesn’t belong to as autocomplete (finally solving performance issues when there are lots of groups). The primary field option, is better shown by example.

In this Post node, we can see the two fields. Remember, the groups shown in the My groups/ Other groups are user specific.

After selecting one group from each field and saving, we see that the values have “jumped” from the secondary field to the primary, making sure all our OG memberships are associated with a single field.

Looking in Devel’s output we see the field values. Those values are not stored in the field’s storage, they were populated based on the OG membership associated with the field. For those using Entity metadata wrapper, this means you can do (assuming your field is called og_ref) cool things like:

Getting the groups (i.e. the node that is a group) from a field $wrapper->og_ref->value();

Getting the OG membership from a field $wrapper->og_ref__og_membership->value();

The last image isn’t related to Entity reference, but since we are about the overhaul stuff, well… We now have global roles and permissions per bundle. This means that if you have a content type called Clubs which is a group, it can have different default OG roles and OG permissions from a group called Restaurants. And we can Copy from the “super” global roles and permissions to the per bundle. Handy!

Nov 23 2011
Nov 23

The Zen theme had its fifth birthday on October 11, 2011. While that milestone just slipped past without my notice, I’ve recently been thinking a lot about things that I’m grateful for. Zen, like Drupal core, improves because of the influx of new ideas and solutions to shared problems. And I’m extremely thankful to all those that have contributed their work.

More than simply saying “Thank you” to all those who’ve contributed patches to both the code and the documentation, I’ve decided to convert each contributor’s name into an actual Git commit. That sounds pretty geeky, but the real purpose of those commits is so each person’s name shows prominently where it belongs… on Zen’s Maintainers page.

I have a really useful Git tip for project maintainers below. But I’d also ask that you please join me (in the comments of this post) in thanking all of the people who have contributed to make Zen great.

How I thanked all the contributors

Zen has had a CHANGELOG.txt since I started maintaining the project. I’ve used it for two reasons: to list all the significant changes to the software and to thank everyone who helped with a patch.

While Git (and that troll of version control systems befor it, CVS) does have a ”log” of all the commits, its not a very useful list of important changes to a project. It’s littered with trivial messages and project maintenance notes like “Fixed typo in README”, “Added docblock” and “Fixing code formatting“. So I’ll continue to use a CHANGELOG to note the most important changes to the software. This also makes it really easy for people to see what’s happening in the latest -dev release.

After today, I will stop using the CHANGELOG as the primary way to thank everyone who helped with a patch. Not enough people look in that file.

How I now thank multi-contributor patches

However, while converting every contributor’s name into a real Git commit, I realized how I can leverage the CHANGELOG so there's one commit per patch author.

The following line in the CHANGELOG is nearly identical to the commit message for the patch. But, of course, the “author” of the commit will be me even though multiple people helped had contributed. So only I would show up on Zen’s maintainers list.

#200495 by JohnAlbin, caroltron, and Toe: Split up monolithic zen.css into smaller, logical stylesheets

Since the Drupal community has moved to Git, dealing with a patch with a single author is easy. I simply follow the advice in Drupal’s Git Handbook and use this command to commit the patch:

git commit --author="rfay "@30906.no-reply.drupal.org>

For a multi-author patch, I now leverage my CHANGELOG.txt to give everyone a proper Git commit.

First, I’ll add this to CHANGELOG.txt:

#200495 by JohnAlbin, caroltron, and : Split up monolithic zen.css into smaller, logical stylesheets

And commit that new line using this command:
git commit --author="caroltron "@171342.no-reply.drupal.org>

Then I'll modify that line to read:

#200495 by JohnAlbin, caroltron, and Toe: Split up monolithic zen.css into smaller, logical stylesheets

And commit that change using this command:
git commit --author="Toe "@10587.no-reply.drupal.org>

Now both caroltron and Toe have real Git commits and show up on the Maintainers list for Zen. And, while the commit doesn’t show the actual code or documentation changes they made, it shows the issue number where you can explicitly see how much they contributed. So, while the changeset of the commit is a bit weak, I feel strongly that the commit itself is profound.

I hope other contrib project maintainers start using Git commits to thank their contributors.

And, again, thank you all for helping out! I’ll see you soon in the issue queue.

Nov 14 2011
Nov 14

According to the Drupal Coding Standards for Documenting Hook Implementations, its considered a good practice to quickly chuck a comment before any function which implements a Drupal hook (eg, hook_menu). This helps someone reading your code quickly see that the function is actually linked with a hook in Drupal and isn't just a function in your module to be called directly.

But… Well… The thing is… Does anyone else get bored of writing the following over and over again? I know do…


/**
 * Implements hook_menu().
 */

Wouldn't it be nice if you could just type in "menu" and Vim could just fill it our for you? Here follows a little Vim script for inserting a "hook implements" comment at the current cursor position.


function! DrupalImplementsComment(hook)
  set paste

  exe "normal! i/**\"
  \          . " * Implements hook_" . a:hook . "()\"
  \          . " */\"

  set nopaste
endfunction

map  :call DrupalImplementsComment(input("Enter Hook name:"))

Wherever your cursor is, press Ctrl+C 3 times, you then get prompted to enter the hook name. When you press enter, a comment gets inserted. Hopefully this will save someone some time - its already saving me time!

To install the script, I just have it in a file called DrupalCommenting.vim inside my ~/.vim/ folder. Then, inside my ~/.vimrc file, I have a line which imports the source file, eg: so ~/.vim/DrupalCommenting.vim.

Any improvements very welcome!

Nov 12 2011
Nov 12

A good night out at the Eircom spiders for two of the Annertech team this week. After Stella's little arrival came a bit early, a flu-ridden Edward was drafted in to fill the void.

Two of our clients were up for Awards. Trócaire had been nominated for two awards - best charity and best campaign, while runireland.com were in the running for the community award. Trócaire unfortunately left empty-handed this time around, but RunIreland were successful in their category.

It's a first Eircom spider award for Annertech - one of many to come hopefully!

Nov 11 2011
Nov 11

For the last few months I’ve been working for Technocrat on a new Drupal based site for the Insurance Australia Group’s Direct Insurance brands. The current sites are using Autonomy Teamsite.

The basics of the build are relatively straight forward, around 1000 nodes, a bunch of views and a bit of glue to hold it all together. Where things get complicated is the workflow. The Financial services sector in Australia is subject to strict control of representations being made about products. The workflow system needs to ensure IAG complies with these requirements.

During the evaluation we found that generally Drupal workflows are based around publishing a single piece of content on the production site. In the IAG case a collection of nodes need to be published as a piece of work, along with a new block. These changes need to be reviewed by stakeholders and then deployed. This led us to build a job based workflow system.

We are using the Features module to handle all configuration, deploy for entities and some additional tools, including Symfony, Jenkins and drush to hold it all together.

I’ve proposed the session for Drupal Downunder in January and will refine the session based on feedback from there in preparation for Denver. If you want to learn more about Drupal Workflows in the Enterprise, please vote for my session.

Oct 27 2011
Oct 27

Last Friday, October 21, the Bay Area Drupal Camp (BADCamp) played host to the first ever (or so I've been told) Higher Education Drupal Summit. This all-day event was envisioned by the BADCamp organizers as part of a suite of events to take place the day before the camp sessions.

Drupal PlanetDrupaleducation
Oct 18 2011
Oct 18
I will be presenting "Beginning Git" at the Bay Area Drupal Camp this weekend. Ever since I caught wind that Drupal was (finally!) moving off of the archaic CVS version control system, and started planning the Great Git Migration, I took it upon myself to be sure that I learn everything I could about Git. DrupalDrupal PlanetGitcoding
Oct 10 2011
Oct 10
It's done: Rules 2 is out! fago Mon, 10/10/2011 - 19:13
Oct 09 2011
Oct 09
Publishing Linked Open Data of Austria at the Vienna Create Camp 11 fago Sun, 10/09/2011 - 18:03
Sep 20 2011
Sep 20

A few weeks ago the entire Annertech team left from their respective corners of Ireland to attend DrupalCon London 2011.

I flew Cork to Heathrow and after some initial London transportation difficulties (protip: buses are never direct) I found myself safe in Croydon, wandering around under the always reassuring helium filled Druplicon.

Code

During the conference itself, I found myself running around between the sessions in the main conference building, the core conversation room and the offices hosting the Birds of a Feather.

I particularly enjoyed the core conversations. I found them to be very accessible and it was interesting to hear people's visions for the future direction of Drupal. The build up to the conference was marked with a lot of interesting discussions on some of the challenges facing Drupal. During the conference however, many of these issues were worked out through conversation, collaboration and compromise. The core conversations reflected this. People with different interests and priorities all working together to create useful, robust and beautiful software.

I also really enjoyed the BOFs (Birds of a Feather = a small group having an informal discussion). I attended a half-dozen of these discussions on different aspects of front-end development. BOFs are a great way to have a focused chat about areas of shared interest. If you have not participated in a BOF previously you should definitely add it to your to-do list for the next DrupalCon.

Stella, Alan and I all participated in the code sprint on Friday. I worked with Alan on some of the front-end focused issues for Drupal 8 (in particular some HTML 5 issues). Stella worked on upgrading some of her modules to Drupal 7 (FAQ module now has an RC release!).

Community

In the same way that Drupal is as much a community as it is code, DrupalCon is as much about socializing as it is about the sessions.

It is not everyday that you get to hang out with people who are passionately interested in all the various aspects of web design and development. As such, I always try to take maximum advantage of the opportunity to talk script loading strategy during lunch, debate the render API at dinner and chat browser reflows over drinks.

I might be accused of a (tiny) bit of bias, but my social highlight was the DrupalCon pub/table/trivia (delete as appropriate) quiz organised by Drupal Ireland. All at Annertech were involved in putting it together. The table quiz was a refreshingly analogue event with the use of all new-fangled, will-never-catch-on digital devices banned. Unfortunately, the analogue style meant that I had to score the various teams with nothing but pen and paper. Stella, Dermot and myself were the markers for the evening. We may have been a bit rusty for the first round but we progressively modified and improved our hand-crafted algorithm. By the end we were all pretty exhausted, but our table quiz marking skills were much improved!

Farewell

So, DrupalCon London: one or two buses may have broken down, the occasional lunch time may have been a bit hectic, the odd wifi connection my have timed-out, but we would not have had it any other way. Big thanks to all involved in putting on a great conference.

See you in Denver 2012!

Sep 14 2011
Sep 14

Ever needed to build a list which "sub-selects", say, 5 items from a given list of categories? This snippet should help.

Assume the following schema…


CREATE TABLE content (
  id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
  title VARCHAR(255) NOT NULL DEFAULT '',
  body LONGTEXT NOT NULL,
  status INT(11) NOT NULL DEFAULT '1',
  created INT(11) NOT NULL DEFAULT '0',
  PRIMARY KEY (id),
  KEY content_created (created)
);

CREATE TABLE tags(
  id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
  title VARCHAR(255) NOT NULL DEFAULT '',
  PRIMARY KEY (id)
);

CREATE TABLE content_tags (
  tid INT(10) UNSIGNED NOT NULL,
  cid INT(10) UNSIGNED NOT NULL,
  PRIMARY KEY (tid, cid),
  KEY content_id (cid)
);

Now we can insert some dummy data…


TRUNCATE content;
INSERT INTO content (title, STATUS, created) VALUES
  ('Ut Secundum Modo',             1, UNIX_TIMESTAMP('2011-09-12 12:00:00')),
  ('Quidem Accumsan Facilisis',    1, UNIX_TIMESTAMP('2011-09-14 17:00:00')),
  ('Vel Ut Oppeto Interdico ',     1, UNIX_TIMESTAMP('2011-09-10 09:00:00')),
  ('Iustum Nimis Venio',           1, UNIX_TIMESTAMP('2011-09-11 12:30:00')),
  ('Consequat Defui Verto Macto',  1, UNIX_TIMESTAMP('2011-09-13 19:00:00')),
  ('Quae Natu Facilisis Ille Jus', 1, UNIX_TIMESTAMP('2011-09-09 21:15:00')),
  ('Abico Meus Ullamcorper',       0, UNIX_TIMESTAMP('2011-09-01 00:00:00')),
  ('Ulciscor Antehabeo Gravis',    1, UNIX_TIMESTAMP('2011-09-05 11:00:00'));

TRUNCATE tags;  
INSERT INTO tags (title) VALUES ('alpha'), ('beta'), ('gamma'), ('delta');

TRUNCATE content_tags;
INSERT INTO  content_tags(tid, cid) VALUES
  (1,1), (4,1),
  (2,2), (3,2), (4,2),
  (1,3), (3,3), (4,3),
  (3,4), (4,4),
  (2,5),
  (3,6), (4,6),
  (1,7), (4,7),
  (1,8);

The TRUNCATE's are only there to ensure these test tables are empty and that the auto incrementing ID's starts from 1.

Now if you run the following query, you will get a list of up to 3 of the most recent posts from each category.


SELECT tag_id, tag_name, content_id, content_title FROM (
  SELECT
    CASE
      WHEN @id != t.id THEN @row_num := 1
      ELSE @row_num := @row_num + 1
    END AS rownum,
    t.id tag_id,
    t.title tag_name,
    c.id content_id,
    c.title content_title,
    @id := t.id
  FROM tags t
  INNER JOIN content_tags ct ON ct.tid = t.id
  INNER JOIN content c ON c.id = ct.cid
  JOIN (SELECT @id := NULL, @row_num := 0) a
  WHERE c.status = 1
  ORDER BY t.id ASC, c.created DESC
) r
WHERE rownum < 4

This produces…

Note how the beta tag only has 2 items; this is due to the INSERT's above. tag_id tag_name content_id content_title 1 alpha 1 Ut Secundum Modo 1 alpha 3 Vel Ut Oppeto Interdico 1 alpha 8 Ulciscor Antehabeo Gravis 2 beta 2 Quidem Accumsan Facilisis 2 beta 5 Consequat Defui Verto Macto 3 gamma 2 Quidem Accumsan Facilisis 3 gamma 4 Iustum Nimis Venio 3 gamma 3 Vel Ut Oppeto Interdico 4 delta 2 Quidem Accumsan Facilisis 4 delta 1 Ut Secundum Modo 4 delta 3 Vel Ut Oppeto Interdico

How to use in Drupal

The above example could easily be adapted for a Drupal site; to list the 4 most recent items in all terms in a given vocabulary…


SELECT term_id, term_name, node_id, node_title FROM (
  SELECT
    CASE
      WHEN @id != td.tid THEN @row_num := 1
      ELSE @row_num := @row_num + 1
    END AS rownum,
    td.tid term_id,
    td.name term_name,
    n.nid node_id,
    n.title node_title,
    @id := td.tid
  FROM term_data td
  INNER JOIN term_node tn ON tn.tid = td.tid
  INNER JOIN node n ON n.vid = tn.vid
  JOIN (SELECT @id := NULL, @row_num := 0) a
  WHERE n.status = 1 AND td.vid = 1
  ORDER BY td.tid ASC, n.created DESC
) r
WHERE rownum < 5
Aug 24 2011
Aug 24

Day two of Drupalcon London and it was definitely 'the morning after the night before' for a lot of people at Fairfield Halls in Croydon. Things were much calmer and the initial volume levels from the assembled masses more muted than on the opening day as many people nursed some sore heads. Nevertheless the main hall was packed by just after 9am in time for the days keynote from Tom Standage, Digital Editor from The Economist magazine. I wasn't too sure prior to the keynote what form his talk was going to take, but it proved to be extremely entertaining and completely different from anything else that is likely to be heard all week (or probably at any other Drupalcon for years to come for that matter).

Tom explained how the origins of social media went as far back as the ancient of Ancient Greece and illustrated many examples from throughout the ages of how the original media was social media and had been like that in one form or another for hundreds and thousands of years until the year 1883 when the New York Sun newspaper was founded. This was the first newspaper to use an advertisers funded model rather than subscriber/reader fee based model for a publication and changed the face of how media was produced and consumed. It went from being a two way exchange to a one way street, with a limited number of people (the editors) deciding what information was important for the readers to receive. The current form of social media on the Internet has reverted media and communication back to being a two way exchange, but Tom expertly led us from start to end of the history lesson in a very entertaining fashion. If the speech has been recorded and is viewable on the Drupalcon London website, I thoroughly recommend watching the whole thing, it is informative and entertaining in equal measures.

For the first more serious seminar of the day I decided to attend the one on Deployments and the features module. Features is something I first came across back at the Drupal London meet at Microsoft earlier in 2011 and wasn't really convinced about it. I've since seen the power of features bur still never had the chance to play with it firsthand. The seminar was informative covering topics like what should be in features as opposed to install profiles, use of drush make and how features are both upgradable and revertable. It went into a little too much detail from a novices perspective at times but it was a useful session overall.

After lunch (which I again skipped, 0 for 2 at eating at Drupalcon, aside from a can of Pepsi) I headed back to the main hall along with my colleagues from The Collinson Group (shameless plug) for a seminar on one of the most important topics for large Drupal websites, making Drupal perform 'like a rockstar' (I must point out at this juncture that I really don't like the term rockstar for any kind of web development stuff, leave it for musicians, anyway I digress).

Four main reasons were outlined for the slow performance of Drupal websites during this seminar, and they are listed below:

  1. Full page renders - 200 to 300 SQL queries per page?
  2. Dynamic content to anonymous users
  3. Excessive/slow/non-optimised DB queries
  4. Naughty modules

All of these were valid basic points, coupled with the fact that many drupal sites don't use any forms of caching, not even the caching built into Drupal by default! Different methods of caching such as Drupal's default caching, memcache and APC were discussed throughout the talk along with the benefits that each type of caching can provide. The use of performance measuring tools for page loading such as the devel module and something called New Relic (which I am not currently familar with but have seen it being advertised at this Drupalcon and I am very keen to try it out) was advocated as they will allow developers to work out where their load time bottlenecks are occuring (e.g. whether its the SQL (which is a common cause), the strain on the server from the traffic, or other reasons).

Another key aspect of Drupal performance that was highlighted is your sites MySQL configuration (if that is your DB of choice). Several settings were picked out as being key settings, though obviously not the only ones:

  • key_buffer_size
  • query_cache_size
  • query_cache_limit
  • table_cache
  • sort_buffer_size
  • myisam_sort_buffer_size
  • tmp_table_size

The values of these need to be adjusted depending on the server hardware at ones disposal. It was also recommended that if you are using caching to check the cache expiry to see whether its renewing too often, and other good tips such as is path alias caching and whether your home page should always be cached, even if it has dynamic content on it (with a period refresh, how frequent depending on likelyhood of updates). Again this is a presentation where I would definitely recommend getting hold of the slides.

This was shortly followed by Robert Douglass (@robertDouglass) leading a very interesting session on the current state of Drupal in regards to apps and marketplaces for those Apps. This week Acquia (for whom Robert works) have announced their intention to have an app marketplace, initally for their subscribers and are hoping to make that public by the end of 2011. What I was not aware of is that two other companies - Subhub and Phase2 (who were both represented well during the session) have already launched App markets in the last year. Of the services to be initially offered on the Acquia marketplace, the one that really interested me was Visual Website Optimiser (or VWO for short) as we made very good use of this at WikiJob during my time there and its a fantastic SAAS offering, so that should be watched very closely for anyone with an interest in conversion rates and analytics.

The main piece of news to come out of this informative session apart from the discovery of the existing Drupal app marketplaces was that the three companies (Acquia, Phase2 and Subhub) are working together on an Open App Standard (OAS) so that developers only have to work to one set of specifications when building their apps but the apps would be able to be on many app stores (if they met the app store's individual business criteria), at least from the coding perspective. I think this is great for many reasons, not least of all they have seen the issues that have arisen with some of the mobile app development (android in particular) and are doing their best to make sure there are not the same issues with any Apps for Drupal.

Anyone wanting more information about the Open App Standard initiative, you can read about it at http://groups.drupal.org/open-app-standard/oas.

Finally for the middle day of Drupalcon the last session I attended was entitled 'A bulletproof plan for themeing' from the very good Rasmus Kalms (@kalms) from Copenhagen. He spoke a lot of common sense including how style guides can be very good tools to make use of and how they can prevent the need for designers to design every element of every page allowing the themers to use their brains, so long as they know what the basic page elements such as form elements, lists, content boxes, typography and images are supposed to look like. He also advocated close communication between designers, developers and project managers along with code review between front and back end developers. All things that may sound like common sense to some people but I'm sure its also something that a lot of people out there are not doing enough of (or at all in some cases).

Another sensible recommendation was for themers to organise their themes. Inside your main theme folder, to have subfolders for each type of template e.g. views, panels and nodes to name but three possibilities. We were also encouraged to take the plunge with HTML5 and CSS3, and to use things like CSS pre-processors which allow the use of such CSS3 functionality as variables, mixins (which look amazing), nested css and even basic functions. If anyone reading doesn't know what a mixin is, below is a basic example (I may not have the syntax 100% right but you get the idea):

.myclass {
   color: red;
}

.mydivclass {
   .myclass; 
}

Two drupal projects to look at in terms of preprocessors are Sass and Less (drupal.org/project/sass (or less)). I haven't investigated these myself yet but will be doing so shortly.

Overall, another great day at Drupalcon. I hope those who went to Batman enjoyed themselves, stay tuned for the final instalment tomorrow night.

Aug 08 2011
Aug 08

At DrupalCon Chicago earlier this year, Stella spoke with Kent Bye from Lullabot. They discussed recent improvements to the Coder module which can help you automate code reviews and upgrade your modules. They also discussed future plans for the module that will increase the number of security checks on Drupal modules.

The interview has now been released as a Drupal Voices podcast.

Jul 15 2011
Jul 15
Nginx clean-URLs for Drupal installed in various sub-directories fago Fri, 07/15/2011 - 11:29
Jul 08 2011
Jul 08
tag: furnituremobiledrupalseoteakdrupal planet

In 2008 I built my first Drupal site, for a furniture business. The site proved to be a big success and we started some other furniture sites as well later on. I also worked on many other Drupal sites and the web in general hasn't been standing still. So last year we decided it was time for an upgrade. Not as easy as thought with a site that is ranking extremely well for a wide range of keywords. It also appeared more difficult to get to a new design.

I moved the entire teakdump site to teakmeubelen.nl, which is super easy with Aegir, added the new theme, and many of the custom modules I had developed for the new Meubels website. With that most of the site was ready except for some settings that weren't easy to grab in code.

As early adopters of technology (especially within the furniture world) we'd also been entertaining the thought of having a site that works well on mobile phones. I had already seen and heard of Mobile Tools, so I gave it a quick try.

The Drupal modules

I was pleasantly surprised by the integration with Display Suite and Context. That plus the Fusion Mobile theme gave me fairly decent mobile site in a very short time.

The mobile build mode make it a snap to get a slightly more usable rendering of nodes on mobile devices and the mobile context makes it easy to quickly rearrange the entire site for mobile devices.

I noticed an issue though: the Display Suite build mode was showing on the desktop site! I had to disable Mobile Tools and look back into this the next day.

Then I checked WURFL but that seemed to just complicate matters a bit more.

As it seems now there was a little bug in the code that sets the build mode, at least I wrote a patch for the Mobile Tools that works very well for me!

The main other thing I ran into was the caching of the theme. I'm using two different themes, one for the desktop version and one for the mobile version of the site. This doesn't work at all if you have the same URLs. So finally I had to take the redirect out of Aegir and into my nginx configuration so that m.teakmeubelen.nl also points to the same install (without a redirect).

SEO

From blog posts I gathered that Google doesn't seem to consider the same content on both desktop and mobile URLs as punishable duplicate content. I just wonder how Google is supposed to detect the mobile URLs (which is one more reason for a nice blog post about my first mobile site).

Now things still need some work here and there but overall I think this site looks great, both on the desktop and on mobiles. And I wonder where to put the iPad and other tablets, as I think the desktop site is just fine on an iPad but Mobile Tools probably considers it mobile hardware anyway.

Jul 06 2011
Jul 06

Steve Fisher and I have talked about “designing in the open” on our Using Blue video podcast. The current design on that website is a work in progress; as we figure out how users are interacting with the content, we’ll tweak and refine the design. The concept is particularly useful in combating “the perfect is the enemy of the good.”

With my personal website, I’d like to extend the designing in the open concept right down to the roots. This site needs to be upgraded to Drupal 7, so I’ve decided to document and illustrate the entire process of rebooting my website.

This will not be a simple upgrade. I’ll be re-thinking the purpose and goals of my site and rebuilding it from the ground up. And I’ll be attempting to incorporate a myriad of design principles and best practices as I go.

  • Drupal 7 upgrade process
  • Content strategy
  • HTML5
  • Sass
  • Zen 7.x-5.x
  • Design techniques in this order: typography, color, composition
  • Mobile first
  • Responsive Design
  • Flexible grids
  • Flexible images + Drupal magic
  • Front-end performance

I’ve actually been meaning to re-design my site since reading Mark Boulton’s Designing for the Web almost two years ago. But, if you’ve listened to Episode 3 of Using Blue, you’ll already know that I think the web is about to undergo a profound shift in the way we build sites. So now’s the time to stop listening and discussing and to start building and discussing.

Sounds good, but what’s with the crappy design?

Some of you are asking “Why should I bother following this discussion when your site looks like a time traveler from Netscape 1.0?” A fair question, actually.

Most designers would be horrified to present something so unpolished and lacking in any good design principles. (And to add insult to injury for many designers, I’ve added Photoshop-nauseu-inducing blue guide lines to the background.) But I am unafraid. Or rather, I’m shameless.

The current “design” is a mixture of:

  1. Browser defaults
  2. Drupal 7 defaults
  3. Zen 7.x-5.x-dev defaults
  4. A very minimally tweaked Zen sub-theme (Adding support for my blog posts’ title/tagline combo.)

Since I’ve had a hand in creating many of the defaults in Drupal 7 and in Zen, I should own them.

The markup and sytling you currently see is the Pǔ (樸), or the “uncarved block”. It has no right or wrong, no beauty or ugliness. It is markup in the receptive state, ready to become.

Or to put it in site builder terminology: One of the purposes of Drupal’s and Zen’s default markup and CSS is to be easy to alter and extend. And to be a reasonable starting point. But it’s just a starting point, you will need to alter it to meet the needs of your site.

What’s next?

As you can see from the site, I’ve already done some of the work. In fact, you’re looking at a brand spankin’ new Drupal 7 site. The next post in this series will describe the upgrade process I used to get my site off of Drupal 6.

When are you rebooting?

Jun 27 2011
Jun 27

The user modal is an interesting module we’ve been working lately in gizra for Medico.com, and I’d like to share our experience.

From the README:User modal module allows opening the Register/ Login/ Reset password menu items as tabs. Since the tabs are shown via JS and not AJAX, this may lead to a better user experience, as there is no time waiting for the selected tab to load. Furthermore, the user modal form can be used by other implementing modules, and for example allow a user to submit a node and register in the same time.

The idea is that we want to lower the barrier for a new user to participate in the site – and the barrier is assumed to be too many clicks to just create a node or write a comment.

The final implementation is different from where we started:

  1. We thought about creating our own custom forms and call the right validate and submit handlers, and suppress errors of non-submitted forms (e.g. the user has decided to login, instead of register so we don’t need to validate the register fields) using D7’s #limit_vaidation_errors property. The idea was to avoid form API complexity. It’s a no go. There are many advantages to _re-using the current form. So we went on to the next step
  2. Using subform module. Easier said then done, as by the time of writing user-modal subform was pretty broken. With a few patches from our side and a lot of help from the knowledgable casey, Subform maintainer, that provided lots if fix and support, we were able to re-use the forms.
  3. So now it was just a matter of sticking it in the modal. Which modal? At first we used CTools modal (if you’ve been following my blog posts you know I’m a big fan of CTools), however in this case, we’ve replaced CTools modal with the overlay module that comes with core. Why? For several reasons (non of them is CTools modal fault).

CTools modal opens a modal dialog via ajax in the same page as the parent. The overlay opens the dialog in an IFrame, so you have “parent” and “child” pages. Silly things we love to hate, like IE, don’t allow (without hacks) to add new CSS to the page via AJAX. Another case was that Janrain (a module to connect to 3rd party providers like facebook) currently use JS to bind the click event to a link, but only to links it “sees” on page load. Links that are added via ajax are not binded. One final example, and again IE – in our case we saw (and provided a patch) that when we post the form in the modal, $_POST returns empty. Thanks IE! The advantage of using Overlay, is that the new content is in an IFrame, which means it’s actually loading the same way it would have in a separate window. Also, it should be mentioned that Overlay is a great piece of code, and I learned a lot from its JS (and a lot has went over my head).

We found that Overlay was easier to theme. CTools modal seems to have fixed width & height and requires a class to be added to the link that opens the modal with the name of the modal type that is required. With overlay it’s easier, as it’s a full page refresh , only inside the IFrame.

The disadvantage, or maybe better – challenge is that unlike CTools modal, Overlay’s modal being in an IFrame means one has to do some “tricks” to pass information from the child to the parent window.

Another minor advantage is that overlay is a core module, so it’s considered a better practice to use it. It should be noted, that the Overlay gives an impression it can/ should be used only for admins - we think it’s just a matter of showing it could be used for other stuff as well.

Instead of going in details in the blog post, you should grab the module and enable the example module that comes with it. There’s some info in the README, and a lot of comments in the example module. The example shows a simple case of creating an article node and logging-in with a singe click!

Jun 19 2011
Jun 19

I've been asked what I meant by "The Drupal Way" in my Drupallets blog post Hiring Drupal professionals, part 1: Know what you need.

In the Drupal community, "Drupal Way" is used in two related, erm, ways. The primary meaning is a general ethos or guiding philosophy, but it also gets used to refer to specific applications of that ethos. In my blog post, I was referencing the general principle, The Drupal Way, as a whole. But google "the drupal way" and you will find lots of hits talking about specific applications of the Drupal way, that is, the Drupal way to do X or the Drupal way to do Y.

The Drupal Way, the ethos, permeates every aspect of Drupal, from coding to content. It is the very essence of the Drupal community. It can be explained many different ways, but I like to boil it down to this:

Don't re-invent the wheel.

Or, put another way:

Share and share alike.

Drupal is not just open source; it is also modular by design. Thus it is not just licensed to share, it is designed to share —easily and flexibly— because by sharing (by not re-inventing the wheel) we all benefit. We can build better, more robust websites more quickly if we take advantage of one another's work than if we do everything ourselves from scratch.

This design and ethos goes beyond the mere existence of contributed modules to add features and functionality. If modules are like Lego bricks, the API(s) are the pegs that let you snap them together firmly to build your creation. Yes, you could lash some bricks together using duct tape, but the result just isn't going to be as good; further, it'll make it harder down the line to make improvements and updates.

Even more fundamental is Drupal's separation of visual appearance (themes) from structure/functionality (core & modules) from site content. Keeping these distinct makes sharing and reusing that much easier. For example, no need to build a whole new site structure and migrate all your content every time you want to totally change the site's visual appearance —just upload a new theme and click a few buttons.

And then there is content. I don't think it is a coincidence that CCK (Content Construction Kit, now in Drupal 7 core as fields) and Views were developed in Drupal. These contributed modules are simply —brilliantly— extensions of The Drupal Way to content. They, and various other Drupal modules as well, make it easy to enter content once but use it many times in many places in many different ways in your site. (For you relational database fans out there: think of a Drupal site as one gorgeous web-enabled relational database for your content. Or, as I heard someone put it: a Drupal site is essentially "things and lists of things". But more on that in another post…)

Finally, it should be noted that The Drupal Way is a guiding ethos, not rigid step-by-step instructions. Not re-inventing the wheel doesn't mean using the exact same wheel for every project. As often as you will hear "There's a module for that…" you will hear "There's different ways you can do that…"

So there you have it: Don't re-invent the wheel. This is the whole Drupal Way. The rest is commentary — and now go learn. (with apologies to Hillel…)

Update: Others' commentary on The Drupal Way

I'll be keeping an ongoing list of other good discussions of The Drupal Way (and/or not re-inventing the wheel). Please post or contact me if you know of any others!

Jun 09 2011
Jun 09

One of my development goals is to learn how to set up continuous integration so that I’ll always remember to run my automated tests. I picked up the inspiration to use Hudson from Stuart Robertson, with whom I had the pleasure of working on a Drupal project before he moved to BMO. He had set up continuous integration testing with Hudson and Selenium on another project he’d worked on, and they completed user acceptance testing without any defects. That’s pretty cool. =)

I’m a big fan of automated testing because I hate doing repetitive work. Automated tests also let me turn software development into a game, with clearly defined goalposts and a way to keep score. Automated tests can be a handy way of creating lots of data so that I can manually test a site set up the way I want it to be. I like doing test-driven development: write the test first, then write the code that passes it.

Testing was even better with Rails. I love the Cucumber testing framework because I could define high-level tests in English. The Drupal equivalent (Drucumber?) isn’t quite there yet. I could actually use Cucumber to test my Drupal site, but it would only be able to test the web interface, not the code, and I like to write unit tests in addition to integration tests. Still, some automated testing is better than no testing, and I’m comfortable creating Simpletest classes.

Jenkins (previously known as Hudson) is a continuous integration server that can build and test your application whenever you change the code. I set it up on my local development image by following Jenkins’ installation instructions. I enabled the Git plugin (Manage Jenkins – Manage Plugins – Available).

Then I set up a project with my local git repository. I started with a placeholder build step of Execute shell and pwd, just to see where I was. When I built the project, Hudson checked out my source code and ran the command. I then went into the Hudson workspace directory, configured my Drupal settings.php to use the database and URL I created for the integration site, and configured permissions and Apache with a name-based virtual host so that I could run web tests.

For build steps, I used Execute shell with the following settings:

mysql -u integration integration < sites/default/files/backup_migrate/scheduled/site-backup.mysql
/var/drush/drush test PopulateTestUsersTest
/var/drush/drush test PopulateTestSessionsTest
/var/drush/drush testre MyProjectName --error-on-fail

This loads the backup file created by Backup and Migrate, sets up my test content, and then uses my custom testre command.

Code below (c) 2011 Sacha Chua ([email protected]), available under GNU General Public License v2.0 (yes, I should submit this as a patch, but there’s a bit of paperwork for direct contributions, and it’s easier to just get my manager’s OK to blog about something…)

// A Drush command callback.
function drush_simpletest_test_regular_expression($test_re='') {
  global $verbose, $color;
  $verbose = is_null(drush_get_option('detail')) ? FALSE : TRUE;
  $color = is_null(drush_get_option('color')) ? FALSE : TRUE;
  $error_on_fail = is_null(drush_get_option('error-on-fail')) ? FALSE : TRUE;
  if (!preg_match("/^\/.*\//", $test_re)) {
    $test_re = "/$test_re/";
  }
  // call this method rather than simpletest_test_get_all() in order to bypass internal cache
  $all_test_classes = simpletest_test_get_all_classes();

  // Check that the test class parameter has been set.
  if (empty($test_re)) {
    drush_print("\nAvailable test groups & classes");
    drush_print("-------------------------------");
    $current_group = '';
    foreach ($all_test_classes as $class => $details) {
      if (class_exists($class) && method_exists($class, 'getInfo')) {
        $info = call_user_func(array($class, 'getInfo'));
        if ($info['group'] != $current_group) {
          $current_group = $info['group'];
          drush_print('[' . $current_group . ']');
        }
        drush_print("\t" . $class . ' - ' . $info['name']);
      }
    }
    return;
  }

  // Find test classes that match
  foreach ($all_test_classes as $class => $details) {
    if (class_exists($class) && method_exists($class, 'getInfo')) {
      if (preg_match($test_re, $class)) {
        $info = call_user_func(array($class, 'getInfo'));
        $matching_classes[$class] = $info;
      }
    }
  }

  // Sort matching classes by weight
  uasort($matching_classes, '_simpletest_drush_compare_weight');

  foreach ($matching_classes as $class => $info) {
    $main_verbose = $verbose;
    $results[$class] = drush_simpletest_run_single_test($class, $error_on_fail);
    $verbose = $main_verbose;
  }

  $failures = $successes = 0;
  foreach ($results as $class => $status) {
    print $status . "\t" . $class . "\n";
    if ($status == 'fail') {
      $failures++;
    } else {
      $successes++;
    }
  }
  print "Failed: " . $failures . "/" . ($failures + $successes) . "\n";
  print "Succeeded: " . $successes . "/" . ($failures + $successes) . "\n";
  if ($failures > 0) {
    return 1;
  }
}

I didn’t bother hacking Simpletest output to match the Ant/JUnit output so that Jenkins could understand it better. I just wanted a pass/fail status, as I could always look at the results to find out which test failed.

What does it gain me over running the tests from the command-line? I like having the build history and being able to remember the last successful build.

I’m going to keep this as a local build server instead of setting up a remote continuous integration server on our public machine, because it involves installing quite a number of additional packages. Maybe the other developers might be inspired to set up something similar, though!

2011-06-09 Thu 09:51

Jun 06 2011
Jun 06

(This is the first installment of a multipart series.)

As a Drupal trainer and consultant, I've been getting a lot of phone calls lately either asking if I have trainees to recommend or else hoping that the "consultant" in my job title is a synonym for developer. (It's not: I'm the kind of consultant who helps you figure out what to do rather than the kind that does it for you.) People are having a really hard time finding experienced Drupallers to hire.

At the same time, I've become aware of more and more Drupal projects that went horribly awry because the freelancer or shop hired, though perfectly good PHP coders, didn't really know or understand Drupal. (In just in the last few months, I've personally had not one but two clients who were site-rescue refugees from the same freelancer!)

Unfortunately, the increasing popularity of Drupal can add up to a double whammy for those trying to hire Drupal help. Not only is there a shortage of experienced Drupallers, but there is an increasing number of inexperienced Drupallers offering their services. And these difficulties are compounded by the fact that many of those seeking to hire, quite naturally, don't know very much about Drupal.

So how do you find good Drupallers so your project actually gets the power, flexibility, and ease of use for content creators/managers that led you chose Drupal in the first place?

The first step is to ask for the right thing. There are different kinds of Drupal professionals and most clients and companies seeking to hire Drupallers don't understand or ask for the kind they need.

Besides end users, there are three broad categories of Drupallers: themers, site builders, and module developers. Of these, only module developers actually need to be PHP ninjas

Themers are responsible for the site's graphic design, which in Drupal is independent from the site structure and content. A Drupal site's entire visual design can be completely changed with literally just a couple mouse clicks. (For a demonstration of the independence of content and theme, visit Drupal Gardens —note how the content remains constant as you change from theme to theme.)

Themers come in two flavors: those who customize existing themes (subthemers) & those who create entirely new themes. Only the latter need significant PHP skills, but even then, graphic design and CSS mastery are much more important. For subthemers, PHP doesn't hurt, but isn't vital; often only custom CSS is needed.

Site builders put together the site's structure —the data types, displays, menus, navigation, entry forms, access control, administration, and other functionality for the site's content.

Site builders have even less need for mad PHP skills. Much more important is information architecture, usability, accessibility, and the like. Very complex, feature-rich Drupal sites can be built using only existing core and contributed modules. Indeed, while having familiarity with PHP can help, you actively do not want someone whose first instinct is to code to solve problems. For Drupal, custom PHP should be the last resort, not the first. Re-coding the wheel defeats the purpose and advantage of using Drupal!

If custom PHP is needed for a site —that is, if there is some functionality needed that can't be achieved using existing contributed modules— then it should go in a custom module, which is where module developers come in.

For module developers, of course, PHP is a must. But to be a good module developer, you also need to have good site building skills; you need to understand and appreciate The Drupal Way and also have mastery not just of PHP, but of the Drupal API (and certain contributed module APIs).

I should clarify that there aren't impenetrable divides between these three broad groups. Many themers are also site builders, many site builders are also at least subthemers, etc. and naturally there are various specializations that overlap or even fall somewhat outside these broad three, such as performance optimizers who get into server configuration, etc.

Unfortunately, clients and companies new to Drupal rarely know much about Drupal beyond that it is a CMS based on PHP and databases. So they understandably, but detrimentally, focus on the PHP bit, advertise for "Drupal developers", and emphasize PHP skills in their criteria.

But in a labor market where any kind of experienced Drupaller is in short supply and where there are more good site builders than good module developers, advertising for module developers (which is effectively what advertising for a "Drupal developer" with PHP skills is) when what you need is a good site builder is not the best recipe for success. Good site builders know when to bring in a themer or module developer, but trying to hire a module developer before you even know if you actually need one just frustrates everyone.

Worse, remember those decent PHP coders who don't really know Drupal? They tend to be attracted to "Drupal developer" positions that emphasize PHP skills, too. And that's how sites end up with custom PHP code that is not only totally unnecessary but located in the wrong place (e.g., hacked core/contributed modules or everything in a single theme page.tpl.php file instead of in a custom module where it belongs) and an unsustainable site that doesn't work properly.

You're much more likely to be successful finding and hiring a good Drupal professional if you know what kind you need and ask for it by name. If you're new to Drupal, start by looking for a "Drupal site builder" and emphasize Drupal knowledge and web best practices, not PHP skills. If you already have a site builder but need a graphics professional for your visual design, look for a "Drupal themer" and emphasize graphic design and CSS skills as well as Drupal theming knowledge. If you already have a site builder but determined that existing modules can't do what you need, then it's time to look for a "Drupal module developer" (not just a "Drupal developer") and emphasize Drupal module development knowledge, especially Drupal core and contributed module APIs and best practices.

So, you're advertising/searching for the kind of Drupaller you need, now what? How can you know if you've found them? See the next installment in this series: Hiring Drupal professionals, part 2: Know who they are

[Added 5 Jun 2012] Especially for larger projects, a more expansive discussion of the different roles involved in creating (Drupal) websites can be found in Randall Knutson's great blog post Why Web Development is Like Building a House over at LevelTen. (Just make sure to translate their use of "Developer" to "Site builder"!)

Jun 06 2011
Jun 06

We did it. Drupal shops and other website companies are sometimes very fast in creating a new site for a client. But not so with their own website. It took us about half a year from initial idea to launch - guess this was pretty good.

We wanted a new design and we wanted a better structure, and of course we wanted to present ourselves better to potential customers and everyone else that visits our site at undpaul.de. Apart from some blog posts, the site is all German at the moment, but we are working on it.

The spark of conception

We thought about what makes undpaul special, and quickly realized: it is the people. All members of our company are very active in the community - and of course nice people you would like to make the acquaintance of :). So how about putting the people out in the open? The idea was to create a slideshow of big photos of us on the top of the front page. Darn, we did not have the photos. So hire a photographer and off we went to the studio. We were lucky to know someone who was good at people shots. The surprising thing was - we were done in under two hours and had a lot of great shots.

It was a lot of fun rotating onto the photo stage in several rounds and being shot in all our - ehm - glory. I can recommend the trip to the photographer to everyone, you might find out you can look better than you thought. Now we are able to present a shot of every team member along with a statement about their personal area of work or thoughts about Drupal to directly address the viewer.

Presenting your work showcases right

What is the most interesting part of a webdesign shop for the client? Probably the sites the webshop has built to prove its competence. We wanted to highlight the best and create an easy navigation through the rest. So we tagged every sample in different categories and created multiple Views and slideshows to easily navigate through the showcases. This is a work in progress and feedback on how it works are very welcome. Jcarousel and Views Slideshow are great Views plugins for slideshows, especially Views Slideshow has great options that make almost everything possible.

Upgrading to Drupal 7

Our old site was running on Drupal 6 and this was a great chance to collect some experience in upgrading a site with a lot of CCK fields, comments, images and stuff. The upgrade went smoothly in general, but we had to touch a lot of things manually: Formatters get lost, so you have to rework views, image styles still get lost so you have to recreate them, text areas lose their input formats (might be fixed by now). Loads of stuff, some of which we still did not file issues of (shame on us). But in the end - having all our content transferred to Drupal 7 was worth the ride. Working in Drupal 7 - ah, like a breath of fresh air.

Fun with CSS3

This is the modern web. We do not have to limit ourselves to Helvetica, Times and Verdana. So we wanted a cool header font. Having found out that @font-face works perfectly well, I looked out for a nice font. After some experimenting and finding out that a cool font is not automatically one that looks good on the web we settled on Yanone Kaffesatz. But as this is a bit over-used on the web, we chose the light variant, which you do not see as often. Comminication about how that looks revealed something interesting (no secret really): Anti-aliasing makes thin fonts look a lot bolder on Mac that it does on PC. So find a size that looks good on both...

Box-Shadow is easy in modern browsers. Not so in IE - including IE8. In the end (and since our dropshadow Jquery plugin strangely did not work anymore) we settled on using IE's native shadow filter . The filter works perfectly well after some getting used to it. Alas, it has one drawback... It creates extra padding for all the shadows you add to elements. So conditional stylesheets for IE and all elements that have shadows on them. I doubt I will do that again :(. Rather search longer for a better script...

Putting it all together

The feedback we get to the site is quite encouraging. The personal touch we wanted to add appears to get across to the visitor. Having used some modern techniques may have been a learning experience partly, but hopefully has added a spark here and there. The site is very much a work in progress and more areas need work and the entire thing needs more content.
All in all it was and is a very rewarding experience to work on a site and being able to closely follow your own vision and ideas. A bit like decorating a great new flat and being able to use the best materials and furniture. What could be nicer?

May 28 2011
May 28

Updated! This is the version that I presented at Drupal Camp Sacramento Area at 10 am on 28 May 2011, updated & expanded from the version I presented at DrupalCamp @ Stanford at 10 am on 2 April 2011.

Session description

More than three score and ten useful contributed modules for building Drupal sites.

There are many really useful contributed modules to take your site beyond the basics of Drupal core. There are modules to improve, allow, and/or help with everything from accessibility to workflow, from images to input formats, and beyond.

This session will be of interest to beginner and intermediate Drupallers, as well as those who manage or hire Drupallers or who are just trying to decide whether to use Drupal.

(This functionality is part of core in Drupal 7.) indicates some or all of the module's functionality is part of core in Drupal 7.

May 17 2011
May 17

Drush make is a wonderful tool for constructing Drupal platforms. A lot of Drupal developers are used to adding a list of modules, a few libraries and theme or 2 then running drush make to build their platform. It all seems pretty easy. What if I told you module developers could make things even easier for site builders?

Some contrib modules depend on third party libraries, and due to various reasons they can’t always be stored in git repositories on drupal.org and included in the module release. To solve this problem module developers can include a .make file for their module. Drush recursively processes make files, so the module make file would be processed once found by drush make.

A good example of where this could be useful is the SMTP module, which depends on the LGPL licensed PHPMailer library. The module also requires a patch to be applied to the library, which drush make can apply for us. The following .make file could be included in the SMTP module as smtp.make:

core = 6.x
api = 2

libraries[phpmailer][download][type] = "get"
libraries[phpmailer][download][url] = "http://downloads.sourceforge.net/project/phpmailer/phpmailer%20for%20php5_6/Previous%20Versions/2.2.1/phpMailer_v2.2.1_.tar.gz"
libraries[phpmailer][download][md5] = "0bf75c1bcef8bde6adbebcdc69f1a02d"
libraries[phpmailer][directory_name] = "phpmailer"
libraries[phpmailer][destination] = "modules/contrib/smtp"

libraries[phpmailer][patch][drupal-compatibility][url] = "http://drupalcode.org/project/smtp.git/blob_plain/2acaba97adcad7304c22624ceeb009d358b596e3:/class.phpmailer.php.2.2.1.patch"
libraries[phpmailer][patch][drupal-compatibility][md5] = "2d82de03b1a4b60f3b69cc20fae61b76"

Now when the SMTP module is included a normal drush make file it will be downloaded, the PHPMailer library will be downloaded and patched ready for use.

Unfortunately there are some limitations to this approach. Firstly it assumes that the SMTP module will be installed under the modules/contrib directory, which is accepted best practice, but may not suit everyone’s needs. When I tested this with the current stable version of drush make (6.x-2.2) it failed, and drush make 6.x-3.x from git needed to be patched. Hopefully a fix for this can be backported to the 6.x-2.x branch and included in a future release.

Update: I have posted the make file for the SMTP module as patch in issue #1159080.

May 10 2011
May 10

I've had several people ask me recently how I managed to get the nice date effect on my blog headers. It's quite simple really. All you need is:

  • An Image - A background image (a CSS Sprite) which contains the days, months and years.
  • Some HTML - A VERY basic HTML template.
  • Some CSS - To align the image sections.
  • A PHP Snippet - A Drupal preprocess function.

How to make a Nice Date block

The Image

The image I use here was knocked up in Photoshop. It's just a grid of "output". Everything must be lined up pixel perfect to make the CSS easier to generate.

Dates grid

As you can see, on the left we have the Months (Jan -> Dec), then the Days (01 -> 31) and finally the years. You can also make out the grid layout; months are "2 rows per day" and day is "2 rows per year".

Using CSS, we can specify which "section" of the image appears in the HTML template. This is known as "spriting". It's a technique for clipping bits of a single image. Without the spriting technique, we'd need 12 (months) + 31 (days) + 7 (years) = 50 images!

The HTML

Using the following simple HTML, we can apply CSS to style appropriately.



  {MONTH_LONG}
  {DAY_LONG}
  

{YEAR_LONG}

In the above, the values in curly braces are variable placeholders which should be replaced by appropriate data. SHORT implies a shortened date (eg, the year 2011 shortens to 11, the month January shortens to 01). LONG is the opposite of SHORT and is only there so that Search Engines and screen readers have some content to use (ie accessibility).

For example, 1st January 2011 would result in:


January

1st

2011

With CSS disabled (or to a Search Engine), this still reads January 1st 2011.

The CSS

The following CSS is used to align the sprite images into the HTML above.


.nice_date {
  float:right;
  position:relative;
  width:41px;
  height:40px;
}

.nice_date .month,
.nice_date .day,
.nice_date .year {
  position:absolute;
  text-indent:-1000em;
  background:transparent url(i/dates.png) no-repeat;
}
.nice_date .month { top:5px;  left:0;  width:25px; height:10px; }
.nice_date .day   { top:20px; left:0;  width:25px; height:20px; }
.nice_date .year  { bottom:0; right:0; width:15px; height:40px; }

.nice_date .m-01 { background-position:0     0; }
.nice_date .m-02 { background-position:0  -10px; }
.nice_date .m-03 { background-position:0  -20px; }
.nice_date .m-04 { background-position:0  -30px; }
.nice_date .m-05 { background-position:0  -40px; }
.nice_date .m-06 { background-position:0  -50px; }
.nice_date .m-07 { background-position:0  -60px; }
.nice_date .m-08 { background-position:0  -70px; }
.nice_date .m-09 { background-position:0  -80px; }
.nice_date .m-10 { background-position:0  -90px; }
.nice_date .m-11 { background-position:0 -100px; }
.nice_date .m-12 { background-position:0 -110px; }

.nice_date .d-01 { background-position:-25px      0; }
.nice_date .d-02 { background-position:-25px  -20px; }
.nice_date .d-03 { background-position:-25px  -40px; }
.nice_date .d-04 { background-position:-25px  -60px; }
.nice_date .d-05 { background-position:-25px  -80px; }
.nice_date .d-06 { background-position:-25px -100px; }
.nice_date .d-07 { background-position:-25px -120px; }
.nice_date .d-08 { background-position:-25px -140px; }
.nice_date .d-09 { background-position:-25px -160px; }
.nice_date .d-10 { background-position:-25px -180px; }
.nice_date .d-11 { background-position:-25px -200px; }
.nice_date .d-12 { background-position:-25px -220px; }
.nice_date .d-13 { background-position:-25px -240px; }
.nice_date .d-14 { background-position:-25px -260px; }
.nice_date .d-15 { background-position:-25px -280px; }
.nice_date .d-16 { background-position:-50px      0; }
.nice_date .d-17 { background-position:-50px  -20px; }
.nice_date .d-18 { background-position:-50px  -40px; }
.nice_date .d-19 { background-position:-50px  -60px; }
.nice_date .d-20 { background-position:-50px  -80px; }
.nice_date .d-21 { background-position:-50px -100px; }
.nice_date .d-22 { background-position:-50px -120px; }
.nice_date .d-23 { background-position:-50px -140px; }
.nice_date .d-24 { background-position:-50px -160px; }
.nice_date .d-25 { background-position:-50px -180px; }
.nice_date .d-26 { background-position:-50px -200px; }
.nice_date .d-27 { background-position:-50px -220px; }
.nice_date .d-28 { background-position:-50px -240px; }
.nice_date .d-29 { background-position:-50px -260px; }
.nice_date .d-30 { background-position:-50px -280px; }
.nice_date .d-31 { background-position:-50px -300px; }

.nice_date .y-06 { background-position:-75px      0; }
.nice_date .y-07 { background-position:-75px -040px; }
.nice_date .y-08 { background-position:-75px -080px; }
.nice_date .y-09 { background-position:-75px -120px; }
.nice_date .y-10 { background-position:-75px -160px; }
.nice_date .y-11 { background-position:-75px -200px; }
.nice_date .y-12 { background-position:-75px -240px; }
.nice_date .y-13 { background-position:-75px -280px; }

As you can see, the first part just sets up the element sizes and positions. The last chuck of code defines the background offsets for the sprite.

Note: You may need to adjust bits of this based on your own settings. For example, if you remake the Dates PNG Sprite, you will need to adjust ALL the background positions (unless you keep to the same grid).

The PHP

The following PHP is used to embed the HTML Template into a Node.


function THEMENAME_preprocess_node(&$vars) {
  $vars['nide_date'] = _THEMENAME_nice_date($vars['created']);
}
function _THEMENAME_nice_date($timestamp) {
  // Nice Date
  $ys = date('y', $timestamp);
  $yl = date('Y', $timestamp);
  $ms = date('m', $timestamp);
  $ml = date('M', $timestamp);
  $d  = date('d', $timestamp);

  return "

{$ml}

{$d}

{$yl}

"; }

You've probably guessed, but you should replace THEMENAME with the name of the theme (eg, this theme is currently called "tmj2"). You now have a variable, $nide_date to print into your node.tpl.php.

You can also use the same function in hook_preprocess_comment too, if your site has comments enabled.

Other Tips

  • You could bundle this up into a module to re-use across several sites. The preprocess hooks in Drupal 6+ are accessible from Modules.
  • You could alter the Sprite PNG to use a different font; I used Helvetica to fit with the site's clean/simple font design. Someone like Morten (the King of Denmark) might prefer to remake it using Bello to match his blogs header title.
  • Altering the layout is possible too; maybe you're prefer to the date along the top and the month on the side?

If you implement this on your site, please share your link below! (Note to spammers, my site uses No Follow and I check for link spam, so save us both some time ;-) hehe).

May 03 2011
May 03

A quick Google search shows the “Death to Lorem Ipsum” meme is a reoccurring one that is once again hitting the twittersphere this week while An Event Apart is in Boston. Their points about understanding the content during the design phase are completely essential when creating websites, but their rallying cry is completely off base.

Crying “death to lorem ipsum” because real content keeps breaking our design is like crying “death to hammers” because we keep hitting our thumb.

Imagine if Vera Wang was asked to design outfits for a team of people.

Let’s say her client doesn’t initially tell her anything about the people she needs to design clothing for. So, Vera uses Elle McPherson as the model. And the client approves of the design because, of course, it looks fantastic on Elle.

But when Lebron James and the Miami Heat show up for their outfits and look completely ridiculous in misshapen clothing, let me be clear…

Do not blame Elle McPherson!

Lorem ipsum is just a model of real content. If the designer uses the wrong model, its not the model’s fault.

How did we get here?

“Death to
Hammers!”

Lorem ipsum has been a tool we’ve been using for decades. Because real clients never hand their homework in on time. And designers almost never get the content before they are required to start the design.

If we use realistic-looking, but fake content, the client often freaks out saying “That text doesn’t reflect our brand message. You can’t use that!” And even though we assure them its just dummy text, the unease they feel sets the tone for the rest of the design review. Its a losing formula for designers. And Karen McGrane elegantly describes why using real draft content in designs is a never-ending battle.

So we invented Lorem Ipsum. Completely fake content that is so far off message that the client can’t even attempt to improve the content or disapprove of it. At worst, it creates initial confusion about what’s going on. But after some reassurances and eduction, the client can settle down and get down to the business we want them to focus on: our designs.

So where did we go wrong? We repeated the phrase “just ignore the lorem ipsum” one too many times. It’s a mantra that we used to get clients to ignore the fake content. But unfortunately, we now ignore the fake content too.

We’ve forgotten the fake content is supposed to be a model of the real content. By ignoring Lorem Ipsum we’ve ignored the living, breathing content that it is supposed represent.

Getting out of the trap we built for ourselves

We’ve been doing web design for nearly 20 years and we still fall into the trap of wanting perfectly sized content. There is no such thing on real websites. If your designs have features that require the content to be limited to a certain number of characters, I highly recommend you re-think that approach. Why?
  1. The people signing off on your design are often not the people writing the content. This is a reality of large organizations. The people with the authority to approve representations of the company’s brand are not the same ones in charge of writing the copy. It’s likely the copy writers will never hear your recommendations about content length.
  2. The content that fits in your design today will not be the same content that is put into the website tomorrow. Organizations change. Whether its their writing style, the people, or their goals, you shouldn’t rely on what you “know” about the content today. Because it will change over time. And your designs have to be able to cope with those new parameters.

I know. I know. Design loves constraints. (I do listen to Mark Boulton.) And, yes, your design may not be as pretty when the client puts a “too long” piece of content into your designs. But it shouldn’t break! Create a design that looks great for the 80%, but also looks fine for the other 20%. Your design must be flexible.

The perfect solution to the problem of not knowing what the real content looks like is to use Lorem Ipsum. But you have to learn how to use it properly. Stop editing it to fit perfectly in your design.

Learning to use “Real Lorem Ipsum”

I’m not a designer (as you can see from my blog’s design); I’m a professional site implementer. And I know where the pain points are when putting real content into carefully crafted designs. Here’s a short checklist for you to double check your designs against Real Lorem Ipsum™:

  • Headlines. If you have a list of content and they all have titles with roughly the same length, then you’ve only designed for the 80%. You’ve forgotten about the odd-ball 20% that every site has. Make some really long titles. Make some really short ones.
  • Tables. Those grids of text next to images where the bottom of the text perfectly aligns with the bottom of the image? That never happens with real content. Make some of your Lorem Ipsum text be longer than the image for some examples.
  • Names. If you’ve designed any name tags, you’ll already know what I’m talking about. Al Grey and Dikembe Wamutombo both have to fit.
  • Small boxes/labels. Those tiny boxes of text better be able to handle a concept longer than 4 words if you know what I mean.

There’s lots more examples, but that should get your left brain thinking in the right direction. ;-)

Finally, a compromise

I hate ending on this note, but I’m radical centrist by nature. Sometimes, not often, but sometimes, you really do need to constrain the content. In those rare cases, I cannot stress enough that you need to:

  1. Get the client’s buy-in now. If there is a limitation to the length of content, make sure the client agrees to every single limitation.
  2. Educate the client. Provide documentation so that everyone can understand why the constraints are necessary.
  3. Re-educate the client. You’re building the site in a CMS, right? Make the CMS notify the user about the constraints as they enter the content. I’m a big fan of (and contributor to) Drupal. In Drupal, you would add a description for the field with the constraint. Make the inline help short and to the point so the user will actually read it.
  4. Optionally, force the client to listen. Again, using a CMS, you should be able to validate the content as they submit it. I’d recommend against cutting off the extra characters. Instead prevent the form from submitting, display all the content and display a helpful error message so the user can edit the content to match the required limitations. In Drupal, you’d just write a custom validation function for that form field.

A summary slightly too long for twitter

Instead of just dumping fake Lorem Ipsum into your design and editing it to fit your design, create “Real Lorem Ipsum” where the content challenges the design to think about real world problems.

Update: Oooh. Rachel Lehman has a great idea in the comments below. “You can even ask for the draft content, then place the Lorem Ipsum in the length of the draft content! Then you get the best of both worlds - no distractions from unfinished content, but realistic length representation.” Fantastic!

Apr 27 2011
Apr 27

In the LinkedIn group Drupal Community Network, there has been a discussion started by the question: "Drupal for a novice or newcomer? Good Bad , Ugly and why?"

This was my response to that question:

Drupal = good, whether novice or expert. The thing to remember is that novices don't stay novices. Why start with something "easy" just to have to start all over again with something else when you outgrow it? But you can't outgrow Drupal. Drupal will more than keep up with you as your site grows and as your knowledge grows.

A good part of why Drupal is (in)famous for having "a steep learning curve" is because when people hear of all the things Drupal sites can do, they tend to want their Drupal site to do those things, too. And they want to be able to do everything right away — but there is no system in the world that lets you do everything and anything —easily— right away.

Right out of the box, Drupal —even Drupal 6.x— can easily be used for a personal blog or basic news site. Add one of many free themes (just a matter of uploading & expanding a file and then clicking a few buttons), and it becomes an even better looking site. Start with Drupal 7.x (released in January) and it is even easier and nicer right out ofthe box.

But it is unreasonable to expect to be able to create complex sites like Whitehouse.gov or Grammy.com easily as a novice. Of course, if you are building such complex sites, it is easier to build them using Drupal than in anything else I'm aware of (which is more than a good part of why those sites were built with Drupal).

So, yes, Drupal is good —great, even— for a novice or newcomer, whether you're just new to Drupal or new to website building entirely. It is incredibly powerful, but you can start simply and learn as you go along.

As the discussion progressed, and as people shared their own experiences with Drupal and other CMSes, coding was mentioned. I've noticed a lot of discussions of the merits of various CMSes, especially Drupal, tend to get bogged down in discussions of coding, as if being able to code was essential to building websites using Drupal. This is not the case at all! As I noted in the discussion:

I build flexible, complex sites using Drupal 6.x and 7.x without touching any code at all. The few times I have done any coding, it was only in the subtheme. (Most of the time all that is needed for the subtheme is to change the CSS.)

Coding is a last resort in Drupal. "The Drupal Way" mainly comes down to: Don't re-invent the wheel.

So, with regard to coding, that means use existing contributed modules to add functionality. If you do have to create a custom module, do your best to contribute it back to the community -- if you needed that functionality, chances are others want that ability, too. Share and help others, as others are sharing and helping you, so everybody can avoid re-coding the wheel.

(The Drupal Way extends to structuring your site for content, too: sites can be and should be set up so users enter content once, and then it shows up in as many different places as desired, automagically.)

After some more talk of coding (which I contributed to —my bad!), Steve Ringwood made an excellent point:

There is a fair amount of talk of coding here. I like to encourage new folks to look at the "legos". Between content types, views, panels and others (I like the display suite) you can achieve a lot without any coding. I think it valuable to understand what you can do first, before writing code.

The Legos analogy really suits Drupal. In Drupal, just as with Legos, you can build very simple things (CayucosLioness.org) or you can build very complex things (MTV.co.uk). You can use just the original rectangular Lego bricks (Drupal core) or you can add specially shaped Lego parts for added functionality (contributed modules). You can even, if you want, mold your own pieces (custom modules —though I'd say the analogy breaks down a bit here, because it is much easier to write your own modules than mold your own Lego pieces!)

Yet despite the Lego professionals who build incredibly detailed custom models or even the Lego fans who build the large and complex Lego kits of the Taj Mahal, Tower Bridge, and the like, nobody talks about Legos having a "steep learning curve". To the contrary, we tend to think of Legos as a child's toy, suitable for 5 year olds!

Just like Legos, Drupal grows with you. And, like Legos, your imagination is the only limit to what you can build with Drupal —including building simple websites easily.

So let's stop perpetuating this false notion that Drupal has a "steep learning curve". Drupal is Lego. Come play!

Apr 07 2011
Apr 07

I recently came across LiveReload and was impressed. Actually impressed is an under statement. I was amazed. LiveReload has really improved the way I work with css. As it says on its github page LiveReload is browser extension & a command-line tool that:

  1. Applies CSS and JavaScript file changes without reloading a page.
  2. Automatically reloads a page when any other file changes (html, image, server-side script, etc).

Getting it to work is straightforward. You simply need to download the LiveReload gem and install the extension/plugin in your browser of choice (as long as your browser of choice is Chrome, Safari, or Firefox!). When you have all that set up you can make edits to your files and have the changes display instantly in the browser. Even better if you are doing css and js changes the page will only load the css or js file. As such it is like using firebug to tweak properties except that you can do it in your IDE of choice.

I got this to work fine on Drupal 6 but I ran across a bit of a problem using Drupal 7. I was not sure what the problem was but eventually tracked it down to the fact that LiveReload will not work with css files imported using @import. As anyone who has been using Drupal 7 or following its development knows Drupal 7 uses @import extensively to get round the IE 31 link/style tag limit.

LiveReload has such a positive impact on my workflow that I was not going to let something small like that stop me using it! After a little bit of digging around I came across a solution to the problem.

Thanks to the new hook_css_alter it is easy to change the properties of a css file. So something simple like this

function MYTHEME_css_alter(&$css) { // Alter css to display as link tags. foreach ($css as $key => $value) { $css[$key]['preprocess'] = FALSE; } }

will render css links as a link tag rather than a style tag with @import. Perfect! It works because the default style if you do not preprocess is a
tag. You can dig into to drupal_pre_render_styles to learn about how css files in Drupal 7 are rendered.

Mar 30 2011
Mar 30

I would like to claim an utter hatred of race conditions. This is where code is written in such a way that it doesn’t fully consider the possibility of another thread (e.g. another website hit) or threads occurring concurrently. Consider the following which has been increasingly frustrating me recently:

Drupal stores variables in the ‘variables’ table. It also caches these in the ‘cache’ table so that rather than doing multiple SELECT queries for multiple variables, it simple gets all the variables straight out of the cache table in one SELECT then unserializes them.

cron_semaphore is one of these variables which is created when cron starts, then it deletes it when finishing. If it isn’t deleted it should mean that cron hasn’t finished running yet, so the next time cron tries to run it will quit straight away. But due to a certain race condition it doesn’t always get properly deleted as follows (p2 is an abbreviation for an unrelated process running concurrently, e.g. a visitor to your website):

1, cron starts, cron_semaphore variable inserted (and variables cache is deleted)
2. p2 starts, variables cache is empty so “SELECT * FROM {variables}” then…
3. cron finishes, cron_semaphore variable deleted and the variables cache is cleared
4. … p2 inserts result of “SELECT * FROM {variables}” into cache, but that SELECT was called before cron deleted the variable
5. you now have no mention of cron_semaphore in the variables table, but there it is back in the variables cache!

Consider many visits to your website concurrently and you soon realise this can become a very common occurrence. In fact, this exact problem inflicts us at least a handful of times every day. As a result cron keeps trying to run but immediately quits when it sees the semaphore variable still there. After an hour it finally deletes the semaphore but in the meantime crucial stuff doesn’t get done.

Web applications can quickly become riddled with race conditions such as these. I’ve spotted at least two more in Drupal’s core code in the past. When the ‘bugs’ occur as a result they can be tricky to pin down, appearing to be freakish random occurrences. Worse yet, even when found they can be a royal pain to untangle.

Share this:

Like this:

Like Loading...
Mar 11 2011
Mar 11
Drupal 8: Entities and data integration fago Fri, 03/11/2011 - 19:41
Mar 09 2011
Mar 09
Drupal 8: Approaching Content and Configuration Management fago Wed, 03/09/2011 - 19:02
Mar 05 2011
Mar 05

DrupalCon Chicago is only a few days away and here at Annertech we're really looking forward to it. Alan and I are both flying out tomorrow and will be there for the week, allowing some extra time for sight-seeing and the code sprint on Friday. Both Alan and I are presenting at the conference. Alan and Randy have a session on Git on Drupal.org: It's easier than you think! and of course, my own one (with my co-presenter Jim Berry) on Coder: Upgrade your modules to Drupal 7.

Other highlights of the trip for me are going to the Welcome Party in the Field Museum, meeting an old colleague of mine from my Doolin Technologies days and, of course, the Drupal Trivia night on Thursday. This is actually an event the Drupal Ireland gang are organising. Many thanks to Conor, Síle and Deirdre for helping Alan and myself put together the questions. Word of warning, there are some tricky ones in there, so don't forget to study! We've even got a few prizes, including some Irish whiskey and Drupal Ireland t-shirts. Also major kudos to Tiffany and the rest of the DrupalCon Chicago team for all their assistance.

So here's to a great DrupalCon - we hope to see you there!

Feb 16 2011
Feb 16

Isn't it a pain when you have dozens of Views setup and they are all marked as "overridden" because you just pulled in an updated feature file from somewhere. Features doesn't always notice when the Views on your site aren't up to date.

The following snippet (which you should use with caution) will batch "delete" (or Revert, once the view is in code) all Views which are marked as Overridden. This took a few seconds to run on our development machine.


$views = views_get_all_views();

foreach ($views as $view) {
  if ($view->disabled) continue;

  if ($view->type == t('Overridden')) {
    $view->delete();
    views_object_cache_clear('view', $view->name);
  }
}

I ran this from the Devel PHP page (http://www.example.com/devel/php). It essentially does the same as the view module does when you individually revert views, but this does it without confirming on ALL VIEWS MARKED AS Overridden. I cannot stress enough - use at your own risk, and backup your database first!

Feb 09 2011
Feb 09

Last night, myself and a colleague on my current contract assignment went to the February Drupal Drop-in from the Drupal London community, held at Microsoft's very swanky offices near Victoria Tube station. The theme of the evening was useful modules that maybe not everyone would be aware of, so there were not going to be any talks on CCK and Views and "standard" modules such as those.

What followed were a wide range of presentations and copious amounts of free Domino's pizza (again, thanks to Robert Castelo (sorry for the mix up!) and the nice people at Code Positive for shelling out to feed a large group of us and there was some really good information to be had, as well as great contacts to be made, though having done a long day at the office I was at my unsocialable best. I'll try to be more friendly and actually talk to people next time!

After an initial mini sales pitch from Microsoft about their new platform Microsoft Web Matrix, designed to be their answer to Xampp for local development, combined with a means of trying to sell hosting to people, the Drupal module sessions got underway. I summarised a lot of the points for the team on my contract assignment, so thought I would also share these notes with the wider community. There were some very good modules mentioned, some I had prior experience of, some I knew of but never had used, and some I'd never heard of! Please keep reading for details:

Flag Module + Flag Weight Module: Flag module can be used to "flag" things (nodes, users, comments) in order to create lists of whatever is flagged. You can have different levels of granularity (e.g. one global flag list, or a list per user). Flag Weight module allows for the ordering of these lists. This could be used to create anything from an Amazon wish list to an internal favourite content / bookmark list.

This also combined with the "Draggable Views" module, which will allow you to re-order your flagged lists by drag and drop, which is rather nice. I can see a lot of potential uses for this module and I hadn't heard of it before so I am definitely going to make use of this in the future.

If anyone wants to check out the slides then please see http://bit.ly/flag-slides for the full presentation.

Panels Module: For those who are not familiar with Panels, Panels is a dynamic form of laying out the content section of pages in Drupal. You can use different panel layouts for different contexts, e.g. when a user is logged in or logged out, or one layout per content type (or combinations, you can have endless panel variants, though having too many would be difficult to manage).

I used Panels extensively on my last job at www.wikijob.co.uk and found it to be incredibly useful. There are various modules such as Panels everywhere and Panelator for extending Panels but I have not used these yet. If you require a website with a lot of different content layouts then panels is a great flexible alternative to using a lot of template files.

Masquerade Module: Simply, impersonate another user when logged on as admin or someone who has "masquerade" permissions. Good for checking out users with different roles or different sets of permissions. I don't have first hand experience of this yet but will be definitely putting it to good use.

Content Management Filter (CMF): Aptly described in the presentation as the "Admin content screen on steroids", the standard admin content screen is, let's face, pretty crap for lack of a better technical term. This module adds far more robust functionality such as searching for content by date, author, title rather than just by content type or published/unpublished.

I've been so annoyed with the admin content screens many times and I've considered writing something like this myself, it just appears someone kindly beat me to it. Another one I will definitely be using!

Backup / Migrate: Automatically create Database back-ups. It stores the backup in the files directory so we would have to be careful about file and folder permissions or risk exposing the DB to the wide world (I'm sure this is all configurable to stop this from happening). I want to investigate this module a bit more before using it but it handles a very necessary task without the need to delve into your hosting or get your system admin's help.

Apache SOLR & SOLR Suite of Modules: Again, for those not familiar, SOLR is another form of Search. It requires a Java based server (Tomcat?) to be set up with the apache solr module installed. SOLR offers all sorts of search functionality not provided by the core Drupal search such as content weighting, "did you mean" on mis-spelt search terms and can do related content for your site (see Dries's blog at buytaert.net for an example).

SOLR and it's associated modules are very awesome, we were running this on the WikiJob site. You can also get the Acquia SOLR module and pay for them to handle your SOLR search if you don't have the capacity to have a Java Server yourself. According to the presentation last night costs start at something like $350 per year - please see Acquia.com for further details.

Module Filter: Provides an Ajax filtering form for the modules screen, useful when you have a billion modules. I would imagine there are quite a lot of production websites out there with an awful lot of modules on their modules screen whether they are installed or not and this just allows for nice, quick filtering. A useful niche tool module.

Context: Allows you to display blocks or breadcrumbs depending on whether certain conditions are met (e.g. you are on a certain page, user has a certain role, combinations of these conditions). It also allows you to do things like display different themes based on conditions. I'm sure it has many other powers too. This is a module I've been aware of for a long time but I've previously been in the panels camp on the long side of the panels vs context argument. Having seen this, I'm probably now part of the argument that each one is better in different circumstances, and I'd like to get more familar with panels.

There were several other modules discussed, including a really nice demonstration of the power of the Amazon.com API and the 960 gridder module which looks like a great tool if you design your websites to the 960 framework or use 960 layouts, but the ones I covered in details were the ones where I took the most notes.

Unfortunately, most of the sites on my current contract are Drupal 5 sites, so at least in the day job I can't take advantage of most of this information and these great modules. I am hopeful that all this module info will add weight to the case of getting everything upgraded to Drupal 6 and possibly beyond!

Hopefully this information proves useful to some people and I look forward to seeing people at more Drupal drop-in's in the future!

Just as a footnote, thanks to @manarth and @ncameron amongst others for some great presentations at the drop-in, keep it up everyone!

Feb 09 2011
Feb 09

As we move into 2011, I have been doing a lot of work with comments in Drupal lately (Drupal 6 to be specific). One thing I have always wanted to know how to do, but never got round to investigating until recently, was the ability to customise the "Post new comment" tagline at the top of the comment form.

Anyone with a passing familarity of the inner working of Drupal would assume that you need to use a "_form_alter" (or Form Alter in English) hook to modify this piece of text on the form. That was certainly my first assumption, and funnily enough, it proved to be incorrect. I spent a fair bit of time with my head stuck in XDebug in Eclipse (the FiveRDesign IDE of choice, for lack of a better option) following the different steps through and changing different variables but nothing would affect the title.

After much digging around Drupal.org unsuccessfully for help on this matter I stumbled across a post which thankfully gave the simple answer (I unfortunately can't find the original post now to give them credit). What is required to override the comment form's title is actually a preprocess function, specifically, preprocess_box. Boxes are one layer of the Drupal theme API I haven't touched before so I was a little bit surprised about this.

All that is required is the following code:

function module_name_theme() {
	return array(
    	    'preprocess_box' => array(
      		'arguments' => array('vars' => array(), 'hook' => NULL),
    	    )
	);
}

function module_name_preprocess_box(&$vars, $hook) {
	switch ($vars['title']) {
		case 'Post new comment':
			$vars['title'] = 'Your new title';
		break;
	}
}

And hey presto, a customised comment form title. This could probably also be achieved with a custom template or overriding the theme function for the comment form if you wanted to take the forms customisation further, but if you just want to alter the title, knock up a quick custom module with this hook in it (and with a _theme hook to register your theme_preprocess_box function) and your comment form will have a custom title in no time.

Jan 31 2011
Jan 31
Restful web services in Drupal 7 fago Mon, 01/31/2011 - 14:07
Jan 26 2011
Jan 26

I spent the weekend at Drupal Downunder in Brisbane. The venue was excellent. I’m a fan of not using “traditional” venues for conferences, to help make them even more memorable for attendees.

I managed to catch up with a bunch of people. The relaxed feel about the event was great. Most conferences I’ve attended recently have either been large or I’ve helped organise them, this time I could relax and enjoy.

On the Saturday I presented Building Distributions with Drupal 7, which had a small turn out as I was up against Josh Koenig and his Pantheon presentation. My presentation was hampered by lack of internet connectivity, but I think it went well. I used Lego, Duplo and Quatro blocks to demonstrate the evolution of Drupal distros.

Saturday night involved a pub crawl with various DDU and LCA folks. The highlight of the crawl was the Mana Bar, which is a gamers bar, that has a good collection of retro consoles and games on display.

I spent a fair bit of Sunday in the hallway track. I discussed the D7 port of UUID with Dries, which helped confirm the direction I was heading with it. Several people wanted to discuss my $100 Drupal site blog series. I also gave my Horizontally Scaling Drupal presentation, which was very well attended. Unfortunately due to people torrenting there was no usable internet access for my presentation. I had to skip the post event BBQ so I could fly back to Melbourne.

The lack of mobile signal and wifi made it frustrating to prepare and present. I would have liked to have seen an inclusive social event organised on the Saturday night. Overall I really enjoyed DDU and the organisers are to be congratulated. The vegetarian food options were excellent.

Thanks for Four Kitchens for funding me to get to DDU. I have just started contracting with them, so I really appreciated them covering my trip.

Jan 24 2011
Jan 24

This is the 2nd part of the tutorial. You can see the 1st part here.

Now that we have imported the users using the basic tools of migration module. We will learn to import references (nid or uid) and files.

Install the migrate extras module . Migrate extra module enables import of file field.

The following example demonstrate how to import the old cars table.

Here is the table structures of old cars , ‘cars’:

Here is Drupal’s car content type fields structure (The machine name is: car).

Create a new extended class to import the cars :


class OldCarsMigration extends Migration {
  public function __construct() {
    parent::__construct();
    $this->description = t('Migrate cars from the source database');
?>

Now we ll set the cars table primary key, see Migrate Part 1 for more information :


$this->map = new MigrateSQLMap($this->machineName,
  array('car_id'  => array(
    'type' => 'int',
    'not null' => TRUE,
    ),
  ),
  MigrateDestinationTerm::getKeySchema(),
);
?>

Here we set the query to get the source object:


$query = db_select('old_users', 'users')
  ->fields('users' ,array(
    'car_id' ,
    'user_id' ,
    'car_model',
    'picture',
    'failures',
  ));
$this->source = new MigrateSourceSQL($query);
?>

It’s important to define migration dependencies to avoid the possibility that cars will be imported before users, In such a case the system won’t know how to handle the user reference field since those user were not imported yet.


  $this->dependencies = array('OldUsersMigration');
?>

Now we will use the primary key mapping that was defined in Migrate Part 1 to keep the same references defined in the old database. After that Migrate module will take care of figuring out what is the new UID corresponding to the old UID.


  // We want to reference the fields to the old user ID.
  $this->addFieldMapping('uid', 'user_id') //uid(destination), user_id(source)
    ->sourceMigration('OldUsersMigration')// the name of the class in part 1
    ->defaultValue(1);
?>

Define your destination as car content type.


  $this->destination = new MigrateDestinationNode('car');
?>

Map all the other fields.


  $this->addFieldMapping('field_car_model', 'car_model');
?>

Now, lets see how to import serialized string values :

In the old database we have a field that saves a multiple value list of failures, the values are saved using a single (serialized) string field and semicolon (;) as seperator between each value, for example if a car has the following failures: p1 and p2, the value of the string will be p1;p2. Therefore this field cannot be imported as is, we need to explode the values before importing them, this is done using the ->separator property as shown bellow:


  $this->addFieldMapping('field_failures', 'failures')
    ->separator(';')
?>

In Drupal the structure is (standard CCK multilple field):

field_failures :
	[0]['value'] = p1
	[1]['value'] = p2
	[2]['value'] = p3

Now we will import the image files, thanks to Migrate module all we need to do is the following :


  $arguments = MigrateFileFieldHandler::arguments('', 'file_copy', FILE_EXISTS_RENAME);
  // Mapping of old field (picture) to new CCK field (field_image)
  $this->addFieldMapping('field_image', 'picture')
    ->arguments($arguments);
?>

This will take care of copying the files to Drupal /files directory as well as the creation of file objects.

Map the DNM (Do not map) fields.


  $this->addFieldMapping('nid')
    ->issueGroup(t('DNM'));
?>

An other very useful function in the prepare function, the function allows us to manipulate the mapping result just before the node save function, a bit like a hook. In this example I use this function to convert a string value to lower case.

The function signature is:

 public function prepare(stdClass $account, stdClass $row)

$node argument represent the destination node, just before it’s saved into the database. $row argument represent the source object returned by your query, it can be used as reference to make changes to your node.



public function prepare(stdClass $account, stdClass $row) {
  $account->field_car_model[0]['value'] =  strtolower($account->field_car_model[0]['value']) ;
}
?>

Here is an other example where I used the $row argument to modify the node.



$account->field_car_model[0]['value'] = strtolower($row->field_car_model) ;
?>

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web