Sep 03 2011
Sep 03

Unfuddle is a very convenient and fairly cheap SaaS hosted Redmine (with extensions) and SVN/Git, which I use regularly on customer projects. Their service includes reassuring "Backup" options, going as far as to include the ability to backup to your own Amazon S3 bins in addition to their local backups.

Recently, however, I had to go back to an archived project for which I had dutifully taken a dump before closing it on Unfuddle, and looked for a way to restore it to my Unfuddle account. To no avail. Unfuddle support then kindly confirmed that the service did include a backup feature, but no restore:

Backups of projects within Unfuddle are for personal use only and cannot be restored into accounts via the interface. This however may be possible with some custom scripting on your part and with the use of our API (http://unfuddle.com/docs/api).

What then ? I will not comment on the business sense of such a "backup / no restore" proposition, but in this specific case, it prompted me to find a way to take my business elsewhere, and what better way than to Drupal ? Project and friends being a bit too complicated for these needs, I figured this would be a good occasion to flex my Migrate chops and built a Migration enabling the import of Unfuddle dumps to CaseTracker, the project management component in Open Atrium. So here it is:

Unfuddle backup restored to CaseTracker, main screen

As you can see, this code, even in its first incarnation, can restore a bunch of information from your Unfuddle backups to Casetracker:

  • Unfuddle Users, as Drupal users
  • Unfuddle Projects, as CaseTracker projects
  • Unfuddle Tickets, as CaseTracker basic cases
  • Unfuddle ticket Follow-ups, as CaseTracker comments

The image below shows the mapping of Unfuddle Ticket fields to CaseTracker Cases.

Unfuddle Ticket mapping to CaseTracker case

Of course, this being a one-off effort, the features are limited, but for anyone interested, the code is available from the OSInet Git repository for the migrateunfuddle module. Should anyone be willing to work on it, it could obviously be moved to Drupal.org, like my previous Migrate Remote Migrate extension for over-the-wire migrations using XML-RPC.

To use it, just install the module normally, uncompress your Unfuddle backup in the sites/(your site)/files/unfuddle directory, and head over to the Migrate v2 administration screen.

Messages ?

In case you are very observative and noticed the non-empty list of messages on the Migrate UI screen copy, these are useless "Comment was created" from comment module, which happen to be the reason for core issue #1268636 to allow watchdog() to ignore low-priority messages.

Aug 04 2011
Aug 04

So the scenario is this: you notice that you are no longer able to login on your Drupal site:

  • the {users} table entry for your account looks just fine
  • the login and access timestamps on your account are just a few seconds old, matching your latest attempt to login
  • you reset your password in the DB, just in case, and it still does not work
  • the telltale is that Drupal will not even tell you your login failed: it actually works, as the {users} table shows, but yet you are not logged in

Can you guess what can have been going wrong and fix it ?

Actually, in such cases, and at least on most sites without high-end technologies like Memcached or MongoDB sessions, there is one other table you should be looking at, and that is the {sessions} table. If, like most Drupal sites, you are running on MySQL, chances are that your {sessions} table has crashed. This is most common in Drupal versions up to 6, which use the MyISAM engine by default, typically unchanged on smaller sites, whereas Drupal 7 defaults to InnoDB, and larger sites will also have installed Drupal 6 on InnoDB too.

What is happening in such a case is that Drupal actually updates your login entry into the {users} table, but then when the page ends, PHP tries to invoke the Drupal DB-based session handler to write back information about your session into the {sessions} table, and fails because the table is crashed. If you look at your web server logs, you will then notice tons of messages like this (typical Apache: /var/log/apache2/error.log):

[Thu Aug 04 09:34:27 2011] [error] [client (IP masked)] PHP Warning:  Table './(site masked)/sessions' is marked as crashed and last (automatic?) repair failed\nquery: UPDATE sessions SET uid = 0, cache = 0, hostname = '(IP masked)', session = 'language|s:2:\\"en\\";', timestamp = 1312443267 WHERE sid = '1c3a573a10ff949d31c193c42365c7d7' in /(path masked)/includes/database.mysql.inc on line 174, referer: http://(site masked)/user

These happen when Drupal tries to update a session entry. And then tons of similar messages on SELECT queries instead of UPDATE for the attemps by Drupal at locating an existing session for the user.

The solution is then obvious: ssh to your server and run something like:

mysqlcheck -rev (your database) sessions

Do not forget to add the DB prefix if your are using one, like this:

mysqlcheck -rev mysingledb site1_sessions

This should restore your table to sanity. If you do not have access to a command line, as can often happen with smaller sites on shared hostings, phpmyadmin includes an option to "repair" a table, which does essentially the same thing.

One point to notice about this is that it will lock the table during the entire repair process, so your site will be frozen in the meantime.

Afterwards, you might want to consider taking a backup of your site, then converting it to InnoDB. Do not forget that this means a different configuration in /etc/mysql/my.cnf to attain good performance: you will likely want to reduce the memory space allocated to MyISAM and increase the InnoDB buffer pool, for a start ; then follow one of the InnoDB optimization tutorials available everywhere.

Oh, and if you are still running a Drupal 5 site, it is high time to upgrade: Drupal 5 has been out of support for over six months now ! Time to rebuild on Drupal 7 !

Mar 27 2010
fgm
Mar 27

I noticed today that one of my sites returned 403 Access denied on various pages with URLs like format/<foo>, although it was an alias for a taxonomy/term/<tid> taxonomy path which was actually available when not aliased. What could be going on ?

It turned out that this issue is caused by the http://drupal.org/node/28776 patch introduced in DRUPAL-6-7 to protect various VCS paths.

That patch introduced by this issue modifies the FilesMatch clause to match on ^format$ (for SVN), which causes any path containing the format string to be denied, causing that problem. Which gives the solution if this is a problem for you: either modify the relavant FilesMatch clause or rename every path containing format on your site.

Note that this specific subpattern has been rolled back and reworked after http://drupal.org/node/581706 for DRUPAL-7-0-ALPHA2, so Drupal 7 does not have this problem.

Feb 23 2010
fgm
Feb 23

Received this Harris Interactive poll today, on behalf of Tarsus, the french company organizing the Solutions Linux expo.

One of the choices was a list of the top 30 Internet Technologies. And guess what ? For once, Drupal was on the list: first time I see it mentioned in this type of business context. Interesting.

Drupal among the top 30 internet technologies

Dec 11 2009
fgm
Dec 11

Most of the time, when working on some piece of code, I'll resort to the configured debugger in my current Zend Studio configuration. And you probably do too :-)

However, I often have to access debug-type information on live sites where installing a debugger is out of the question, and I find myself often resorting to parameter dumps like the following:

<?php
// lazy version for simple cases
function foo_bar($x, $y, $z) {
 
dsm(func_get_args());
 
// [...]// less lazy version for more hairy cases
function foo_baz($x, $y, $z) {
 
dsm(array('in foo_baz, x' => $x, 'y' => $y, 'z' => $z));
 
// ...
?>

You've probably being using it too and, of course, after the first few dozen times, it becomes a bit used. So here's a tiny snippet that makes such dumps simpler to type and use :

<?php
function foo_quux($x, $y, $z) {
 
dsm(func_get_args_hash());
 
// [...]/**
* Provide a better func_get_args()
*
* @return array
*   Hash of parameters, keyed by parameter name
*/
function func_get_args_hash() {
 
$stack = debug_backtrace();
 
$stack = $stack[1];
 
$rf = new ReflectionFunction($stack['function']);
 
$params = $rf->getParameters();
 
$ret = array();
  foreach (
$params as $index => $param) {
   
$ret[$param->name] = $stack['args'][$index];
  }
  return
$ret;
}
?>
Nov 16 2009
fgm
Nov 16

Sometimes, you want to delete a whole bunch of users, terms, or nodes, say to cleanup a site while developing and still keep its configuration, so reinstalling is not really an option, and the normal content- or user-administration pages get in your way because they only show a limited number of entries, meaning you have to delete page after page of entries.

Of course, you could just use node_delete or user_delete in D6, user_cancel in D7, and loop on all values. But there is a simpler way: just install devel module, and enable its devel_generate submodule.

The main purpose of devel_generate, as its name indicates, is to create content, be it nodes, terms or users. However, much like you use the Start button on Windows to Stop the OS, you can use devel generate to delete content instead of creating it, by enabling its optional Delete all content in these node types before generating new content checkbox. If you then choose to generate 0 entries, you'll get just a nice cleanup, without content generation, and nothing to write.

And for you Drush users, there's even an additional bonus: devel_generate has a Drush plugin implementing three equivalent commands: drush generate users|content|taxonomy. These three commands take as parameter the number of entries to generate, plus a "kill" option, which is here to remove existing content in the same way the checkbox specifies in the UI.

Sep 14 2009
fgm
Sep 14

In case you've found Jeff Miccolis' post about the Extendr module for Flickr great, but have been bothered by the fact that the link on the blog was broken and that comments on that article were closed, the good news is that he made his code available on drupal.org too, at:

As explained on the original post and as you should guess given the source code of Views, this needs a patch to Views 2 to allow overriding views_plugin_query_default, or could likely be used on Views 3. The beauty of it is that it's so short when you look at the implementation. Hopefully someone at Devseed will eventually fix the link...

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web