Jun 29 2014
Jun 29

The Drupal community web site has a profile field to list "My mentors"

For example, on my profile I say I was mentored by:
  • robbiethegeek - how to appreciate Drupal awesomeness and its limitations
  • Alex UA - how to run a business providing Drupal services
  • forestmars - how to be involved in the Drupal community
  • smerrill - how to be an engineer with platform tools like Jenkins, Vagrant, Redis
  • snugug - how to make web sites responsive
  • ericduran - how to experiment with new doodads like HTML5, Android
  • zroger - how to use Drupal hooks and APIs in code

I started thinking about my dumb luck picking Drupal as a tool about 9 years ago. I was looking for a Content Management System that made sense.

I was awfully interested in a project called PAWS (PHP Automatic Web Site) -- and it's a good thing I didn't ride that horse, which was long ago put out to pasture.

A client asked me to convert his static PHP site so that he could manage the content in the include files without editing code. I built my first Drupal 4.x site, with the crazy hack of creating a node for every include, and then printing the includes/nodes inside a main node (Panels, sort of, which did not exist in Drupal then). I also customized the front end of the TinyMCE wysiwyg editor to add buttons to apply his brand's pink and blue colors. The client smoked a lot of pot, drifted away, came back a year or two later for more work -- without a database. Oh well, not the first - or last - time the db was lost by a client.

That experience convinced me that a lot could be done with Drupal that I had not been able to do without a lot of custom coding just to build the base web application. Other projects with early versions of WordPress and Mambo (predecessor to Joomla) left me unimpressed with their extensibility. I have often said since then that "WordPress is like the smaller sibling of Drupal, but Joomla is the evil cousin."

Then Earl Miles conjured up his merlinofchaos wizardry for Sony Music, creating Views and Panels and Ctools, and that was around the time that a lot of developers took notice of Drupal. I was profoundly convinced that Drupal had outgrown being a CMS enabling writers to (more or less) easily edit content without (much) coding, and had become a Content Management Framework that could perform elegant and dynamic manipulations of the content in its database.

So I had to add dumbluck to my mentors - not just for my early experiment hacking the node system, but for each solution that I was able to implement afterwards, because my choice of Drupal provided me with an extensible framework allowing complex algorithms for presentation of content, and the Drupal project improves with every contributor's enhancements.

I think I'm dumb, maybe just happy

[embedded content]

I noticed in preparing this post that some Drupal user profiles are accessible by username, eg. https://www.drupal.org/u/decibel.places and https://www.drupal.org/u/robbiethegeek, while others, like merlinofchaos and smerrill, are only accessible by their UIDs https://www.drupal.org/user/26979 and https://www.drupal.org/user/77539 respectively.

May 05 2012
May 05

Over the years, I've accumulated a large collection of e-books and digital music albums, not to mention family pictures. Information overload is not a philosophical point of view, it's a real problem that forces me to devote time, effort and money to maintain that collection.

That's probably why so many media organizers exist. Because I believe that all applications should be delivered from the Web, and because no ready-made Web media organizer struck me as fulfilling my needs, I started to write my own using Drupal 6, dubbed Mediatheque. Here are the most important design goals I had in mind:

  • The main job of the organizer is to "ingest" media files by processing them to extract metadata, which is made available for searching and browsing.
  • I should be able to point the system to different "volumes" containing my media. These volumes are essentially folders that are present somewhere on the network.
  • No extra file space (beyond database growth) should be required to ingest media files.
  • The system should be able to store arbitrary metadata about the media.
  • I should be able to add new information handlers for media at any time - the system would silently re-process the files.
  • New version of media handlers should also be supported by re-processing the files.
  • The system should be smart about recognizing files across name changes and metadata updates.
  • The system should be robust in the face of plugin errors.
  • Media display should be dependent on its type.

Based on these goals, here are the significant implementation details of the current system:

  • Drupal Queue is used to process the files in the background. In fact, I am using 3 queues as a processing pipeline:

Starting with a volume root, the folders queue finds and enqueues the files, the files queue creates Drupal documents (nodes) and enqueues them for plugin processing, and the plugins queue applies the registered plugins to extract metadata.

  • Each processing step produces a log entry. This allows to track the errors that are produced during media processing. The log is a Drupal table that is integrated with Views via hook_views_data. Each log entry contains information about the processed document, the file hash, the plugin name and version, and the status code of the processing, allowing to detect the cases where re-processing should occur. Mediatheque log

  • Plugins are metadata extractors that are associated with MIME type patterns. For example, Mediatheque currently comes with an ID3 extractor that uses the getID3 library and an e-book metadata extractor that uses the Google Books API via Zend Gdata. Mediatheque plugins

  • Metadata extracted by the plugins is stored in the document node using my CCK Metadata module. This is a simple name/value CCK field and the document node includes this field with unlimited cardinality. The metadata pairs returned by each plugin are prefixed with a unique plugin prefix to be able to handle re-processing. CCK Metadata also allows custom formatting of specific metadata entries via hook_cck_metadata_fields:

/**
 * Implementation of hook_cck_metadata_fields().
 */
function mediatheque_cck_metadata_fields() {
  return array(
    'isbn:thumbnail' => array(
      'formatter' => 'mediatheque_formatter_thumbnail',
    ),
  );
}

Eventually, this metadata system should reuse RDF instead of using a custom design. Mediatheque plugins

  • Finally, the main mediatheque view is created as a regular view with filterable metadata. The metadata name exposed filter is converted to a drop-down via my Views Hacks' Views Selective Exposed Filters module whose job is to restrict the available filter values to those found in the view results. Mediatheque plugins

Mediatheque is still very much a work in progress. However, many conceptual challenges have already been solved, and I would love to hear your feedback!

AttachmentSize 132.92 KB 53.99 KB 50.21 KB 87.44 KB 163.26 KB
Jan 18 2012
Jan 18
Lullabot logo

Lullabot has trained thousands of Drupal developers & guided the development of some of the largest Drupal websites.

Dec 06 2011
Dec 06
Lullabot logo

Lullabot has trained thousands of Drupal developers & guided the development of some of the largest Drupal websites.

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web