Jun 14 2006
Jun 14

The Drupal way in programming is 60% about hooks 10% about nodeapi and the rest is ingenuity. That's allright, but one fact has always troubled me. And that is - how much effort is spent on discovering which module implements the hook I need. Far enough, the old caching/memoizing trick does the job to ensure that the cpu time is not expensive. Yeah, but still it's outside simplicity might be, just might be implemented in a more elegant fashion.

Drupal hooks implement a nearly Aspect Oriented API. The cross-cut is the point where the hook is invoked. aspect(cross-cut) === hook. The different aspects are declared as moduleName_hook. The disatvantage is that these are indiscriminate. Hooks are always fired. Pros - simplicty, cons - wasted cpu cycles.

I wonder. Is it possible to achieve a similarly simple syntax, but with the added advantage of not firing unnessesary hook calls.

Problem (1st iteration)

We need to call all functions interested in this particular execution point. The current context and the run history should determine which functions are actually interested in this particular point/label/crosscut.

So what if:

.....
hook('view','a callback);
.....
//in a code piece far away
invoke('view', .... );

In invoke() we can have something similar to the current code

function invoke($hook,$args) {
...
while($call = next($hooks[$hook])) {
$call($args);
}
...
}

How to implement hook(...)?

function hook($hook,$callback) {
invoke( $hook, $callback, 'add');
}

We would need an unhook() though, so that the interest is fully managed.
function unhook($hook,$callback) {
invoke( $hook, $callback, 'remove');
}

And finally the modified invoke:
function invoke($hook, $args, $op) {
static $hooks;

if($op == 'add') {
//memory waste, but can do for now
$hooks[$hook][$args] = $args;
return;
}
elseif($op == 'remove') {
unset($hooks[$hook][$args]);
return;
}

while($call = next($hooks[$hook])) {
$call($args);
}
}

Order please

Partial order that is. It would be handy to be able to have before($hook) and after($hook) calls. This will allow manipulating the sequence of calls, without modifying the grand plan request workflow.

before($hook) adds the callback before the hook point. The order of the other 'before' callbacks for this hook is unknown.

after($hook) does the same, but after the execution of the hook

The arguments of the callbacks added with before and after are the same as for hook for a given cross-cut/hook point

The implementation is similar to hook

function before($hook,$callback) {
invoke( $hook, $callback, 'add-before');
}

function not_before($hook,$callback) {
invoke( $hook, $callback, 'remove-before');
}

function not_before($hook,$callback) {
invoke( $hook, $callback, 'add-after');
}

function not_after($hook,$callback) {
invoke( $hook, $callback, 'remove-after');
}
....
//in invoke, the rest is similar
if($op == 'add-before') {
unset($hooks[$hook]['before'][$args]);
return;
}
....

An interesting 'theoretical' consequence is that this is a kind of a higher-order functions based implementation of aspect style programming in an imperative language. That is if you are into such words. It is more interesting thing to notice is that the $var() syntax allow you to code wanted patterns and execute them later on by simply passing new $var to the pattern function. This technique parallells lisp and scheme style macros, you just can't easily manipulate them.

Second iteration or cleaning up the nest

There are a few problems with the above code. It doesn't reflect faithfully the problem, that is the treatment of arguments and results. It is too verbose and can do with some optimisation.

Let's start with the model. The type of an aspect function is ($args→$result). We are not interested in actual shape of either, but their correct treatment. It suffice to say that the only requirement to both is that they are the same for each separate hook. The shape of the aspect computation is (∥($args→$result))→(∥($args→$result))→(∥($args→$result)), where ∥ denotes parallell or unordered behaviour of a bunch of functions. The last remark is important, since we are introducing order or synchronisation with the before and after operations, but that order is only partial, relevant to the bigger picture. In each phase the order of execution should remain unknown. This doesn't break the abstraction boundaries, which is good. This as well means that the functions in each phase must not change $args and must not overwrite the same part of $result. This restriction won't be enforced for both simplicty and performance, it should be tested with unit tests in real life code. Failing to obey this discipline will cause an unexpected behaviour.

Let's change the code then. What is the shape of a typical before function?
function a_before($args, &$result) {
....
}
As it happens the shapes of the at and after functions are the same. We are enforcing a partial order of evaluation and enhancing a function, not changing it's signature.
The evaluator can be written out as looping over each of the different phases
....
while(current( $hooks[$hook]['before'] )) {
$cb = key( $hooks[$hook]['before'] );
$cb($args);
next( $hooks[$hook]['before'] );
}

while( current( $hooks[$hook]['during'] ) ) {
$cb = key( $hooks[$hook]['during'] );
$cb( $args, $result );
next( $hooks[$hook]['during'] );
}

while( current( $hooks[$hook]['after']) ) {
$cb = key( $hooks[$hook]['after'] );
$cb( $result );
next( $hooks[$hook]['after'] );
}
return $result;
....

This code can be optimised by parametrising the hook selection, add and remove operations, so that we end up with the following short hook invoke function
function invoke($hook, &$args, $op = 'run', $phase = '' ) {
static $hooks = array();

switch($op) {
case 'run':
foreach( array('before', 'at' ,'after') as $phase ) {
while( current( $hooks[$hook][$phase] )) {
$cb = key( $hooks[$hook][$phase] );
$cb( $args, $result );
next( $hooks[$hook][$phase] );
}
}
return $result;

case 'add':
$hooks[$hook][$phase][$args] = true;
break;
case 'remove':
unset( $hooks[$hook][$phase][$args] );
break;
}
}

We can sweeten the syntax by adding explicit hook_before, after, at and the respective unhook functions, to hide the message passing notation of the above function. For example
function hook_after( $hook, $cb ) {
invoke( $hook, $cb, 'add', 'after');
}
function unhook_before( $hook, $cb ) {
invoke( $hook, $cb, 'remove', 'before');
}

Adding honey to the pud

The invoke function does too many things, it is unfortunate. It is a natural 'object', purists of any kind don't go for me, that is not a flame bate. We can use php's object oriented features to make this a bit more abstract and enforce separation of concerns, rather than a static and a switch.

class hook_namespace {
var $hooks;

function run( $hook,$args ) {
foreach( array('before', 'at' ,'after') as $phase ) {
while( current( $this->$hooks[$hook][$phase] )) {
$cb = key( $this->$hooks[$hook][$phase] );
$cb( $args, $result );
next( $this->$hooks[$hook][$phase] );
}
}
return $result;
}

function hook($hook, $phase, $cb) {
$this->$hooks[$hook][$phase][$cb] = true;
}
function unhook($hook, $phase, $cb) {
unset($this->$hooks[$hook][$phase][$cb]);
}
}

While I could have gone oveboard abstracting further and further, the above class is good enough for my personal taste. It is concise. It does what it says on the box. Ok, fair enough, there are bits to keep in your head, but that is (hopefully) allright. The last two versions practically implement the drupal hook wiring without the broadcast effect for trying out all possible hook definitions. On an abstract level it is a tad more powerful. It has the disadvantage that you need to hook up at runtime, not definition time.

If php had more abstract features we could have done more. With a proper macro facility, we could have shifted the work further towards compile time, now there are extra operations to be performed.

To killes - you are more right than you ever thought. The above code is monadic for all practicall purposes. Of course in haskell it will look very differently, but hey it is a step.

Update I realised that it is better to name the class hook_namespace, since the instance of that class is providing a encapsulation of hook in a single scope, it's a bit verobose, but at least a better name. This has the dubious benefit of renaming add and remove to hook and unhook.

Attachment Size aspect.php.txt 1.71 KB
Jun 06 2006
Jun 06

At the Yearly Kos convention on June 8th the CivicSpace community will be teaching a workshop on building netroots campaign websites. Experts will be avaliable all day to teach participants at every skill level. The day will culminate in a barn raising of a real world netroots campaign website thought up by the DailyKos community and built by workshop participants and facilitators.

This thread:

http://www.dailykos.com/story/2006/6/6/13713/38510

is being used used as a campaign idea incubator with the best concept being built three days from now at the YearlyKos convention. If any of you are DailyKos members we could use your help recommending the thread so that it gets exposure within the DailyKos community, also please submit your ideas if you have them. If any of you want to help out please send me an email and let me know:

  1. If you will be attending YearlyKos
  2. If you want to help facilitate the workshop
  3. What specifically (if anything) you would like to pitch in on.

Hope to I'll see some of you in Las Vegas shortly.

May 24 2006
May 24

What a good occasion. After so much time spent on Google SOC (the bit about Drupal), after dealing with students, mentors, and other creatures. After the results are out, you finish it off with a birthday.

Well done, Happy Birthday Rob. (true or not it is your birthday)

May 23 2006
May 23

I've updated the code. Now the scripts can convert a module into a new module and tpl file.

The script can produce a _widget hook from the old module file.
usage
bash$ ./mparse.php filename option
for example try:
bash$ ./mparse.php path/to/drupal/modules/node.module --all

update: fixed several bugs, so a new file is attached

Attachment Size mparse.tgz 3.46 KB
May 22 2006
May 22

I kind of finished the parser bit of the transformer routines for splitting off the theme functions from the drupal modules. It ended up a bigger thing than anticipated. The parser doesn't handle yet functions returning references.

At the moment only a parser and a pretty printer exists. It can be modified to enforce appropriate style and is not drupal specific.

I'll be adding the module_templates() functions generator this evening and tomorrow and maybe a specific split driver, so you can do the generation of the name.module, name.tpl.php in one pass, rather than running separate programs.

I attach a working as of this morning for your amusement. Warning: ugly code.

Attachment Size mparse.tgz 2.36 KB
May 12 2006
May 12


Alaa - a blogger, an activist, a drupal contributor and more importantly just a man was jailed for being on a demo. Disgusting.

While that doesn't surprise me, things like that happen everywhere, not just in countries with regimes similar to Mubarak's, it still pisses me off.

I can't stand the response of the 'powers of the day' to bully you into submission, if you dare not to support them.

If you happen to be in London on 13/5 and fancy making noise in front of the Egyptian embassy just join the protest.

Oh, yes have you heard the carpet bombers for peace, here's the Egypt version?

May 09 2006
May 09

Spencer Critchley recently interviewed me about CivicSpace, Drupal, and DeanSpace for the O'Reilly OnLamp blog.

May 09 2006
May 09

As I go through a test site for One Inch Frame’s update to Drupal 4.7, I’m doing something different with my wiki nodes and web link nodes. Right now, I’m using to build a custom wikinode type, and am using for the large amount of links that I accumulate in my links directory, the sort of thing that I used to post to del.icio.us. The weblinks module has been discontinued in favor of the , but that promising project is still not ready, and my needs are such that that project might just be overkill. I’ve still got my eye on it, though.

With the 4.7 version, I’ve decide to take a good hard look at the , as it is basically the future of flexinode and Drupal. And it’s also really, really cool.

So I created a simple node type for wikinodes, and a type for web links. One of the things I want to do differently this time around is build in attribution to my web links, so I can give credit to the source where I found the link. So, another CCK content type for web link source is needed, and I can use the built-in ‘nodereference’ field type to link a web link to its source.

And, out of the box, it works great. I can add attribution sources and they show up in a drop-down in the web link content creation form. Super cool. The default look of this field when viewing a web link makes the attribution hotlinked to the attribution’s source node. It boggles my mind to think of some of the ways smart people are going to take advantage of the CCK, and, particularly, its nodereference field type. There’s also a userreference which does the same thing for users, for completeness.

The way CCK-created content looks is a fine default, but, theming them will be necessary in almost every case. Fortunately, the CCK download comes with a theme/ directory with documentation and samples.

So, the first thing I want to do for all my CCKreated (heh, like it? made it up myself) nodes is to remove the label. The CCK allows theming at the field level or up at the node level. To remove the labels globally for all CCKreated content, there’s some code to add to the template.php file, then your PHPTemplate-based theme will call field.tpl.php files to render all CCK fields. Just removing the printing of the label is fine here:

So, my field.tpl.php looks like this:

<div class="field field-type-<?php print strtr($field_type, '_', '-') ?> field-<?php print strtr($field_name, '_', '-') ?>">
<div class="field-items">
  <?php foreach ($items as $item) { ?>
    <div class="field-item"><?php print $item['view'] ?></div>
  <?php } ?>
</div>
</div>

Simple.

While that takes care of theming the individual fields, for my web links I want to theme the whole node. PHPTemplate allows for different themes to be applied to different content types, and CCK extends that to its own types. So, to theme a web link CCKreated node, I write a node-content-web_link.tpl.php. This will override the theming of the individual fields, and give me control over the content as a whole. As CCK exposes its fields via a $field_<fieldname> array, this, too, is pretty simple:

<div class="node<?php if ($sticky) { print " sticky"; } ?><?php if (!$status) { print " node-unpublished"; } ?>">
  <?php if ($picture) { print $picture; }?>
  <h2 class="headerstyle"><a href="http://oif.eafarris.com/cck_theming_in_drupal_4_7/<?php print $node_url?>"><?php print $title?></a></h2>
  <span class="submitted"><?php print $submitted?></span>
  <span class="content">
    <p class="weblink_description"><?php print $field_description[0][view] ?></p>
    <p class="weblink_url"><?php print $field_url[0][view]?></p>
    <p class="weblink_attribution">via <?php print $field_attribution[0][view]) ?></p>
  </span>
  <span class="taxonomy">Categories: <?php print $terms?></span>
  <?php if ($links) { ?><p class="internallinks">&raquo; <?php print theme('links', $node->links, '&nbsp;&middot;&nbsp;')?></p><?php }; ?>
</div>

That’s nice, and generic. I need a couple more things to make this just what I’m after:

  • The url isn’t hot. It should be linked to itself.
  • The attribution goes to the attribution node, but I’d much rather it go directly to the attribution’s site, which is contained in a URL field as part of the attribution node.

The first one is very simple, just wrap the url in an href of itself:

<p class="weblink_url"><a href="http://oif.eafarris.com/cck_theming_in_drupal_4_7/<?php print $field_url[0][view]?>"><?php print $field_url[0][view]?></a></p>

Getting to the last part will require a tiny bit of programming. What we want, instead of a link to the attribution node (which is a cck-derived content type of content-web_link_source) we want the link to go directly to the url within that node. To start, we’ll need to call a custom theme function instead of printing the $field_attribution stuff:

<p class="weblink_attribution">via <?php print theme('oif_attribution', $field_attribution[0][nid]) ?></p>

Now, we write that theme function. It goes in template.php, and, for completeness’ sake, we call it andreas03_oif_attribution(), since andreas03 is the name of the theme. It gets the nid of the attribution node passed to it::

<?php
function andreas03_oif_attribution($nid) {
  $attribution = node_load($nid);
  $output = '<a href="http://oif.eafarris.com/cck_theming_in_drupal_4_7/' . $attribution->field_url[0][value] . '">' .  $attribution->title . '</a>';
  return $output;
}
?>

And, there it is. A fully themed CCK node. Drupal is so freakin’ cool. You’ll see these new nodes when I roll out the new version of One Inch Frame, based on Drupal 4.7, which is still several weeks away. There are just so many cool things that I want to do, that are now possible.

May 09 2006
May 09

I've just uploaded into the bryght svn a basic filter, actually a couple of them (I was lazy). They parse a php file, and output (leave out in the case of the no_filter), all functions starting with a given prefix.

I've done it in order to be able to play with separate theme_xxx files.

usage:
vlado:~$ no_filter function_prefix php_file
read the php_file, print it to stdout, leaving out all functions starting with function prefix.

vlado:~$ yes_filter function_prefix php_file
read the php_file, print it to stdout, leaving out all functions not starting with function prefix.

both filters know about associated comments, etc...

May 04 2006
May 04

Funny things these summer of code applications. Some of them come a very well thought out projects, others are more like cloudy ideas.

If you want to submit a proposal, and want to be successful, please put a little effort in spec-ing properly. Tangible goals are a very good thing. Good language, not really importnat, just put them in bullets if you fancy it that way.

If unsure, get to #drupal on irc.freenet.org, and ask. Mention SOC and project proposal, and you'll get attention and tlc. That is if you are applying for a drupal project, for others, well don't expect much help in that channel.

Good luck

May 04 2006
May 04

Watch this screencast to learn how to use Drupal to create Google Maps mashups of virtually any arbitrary data or content with no coding in minutes. For the example shown in this screencast I took a csv file of crime data provided by the San Francisco government and turned it into a usable google maps mashup in about 10 minutes.


14 minutes / 60megs

To play along at home you will need to install the following:

Please feel free to leave any comments, questions, or feedback.

May 03 2006
May 03

One of the great features of Drupal is its ability to run any number of sites from one base installation, a feature generally referred to as multisites . Creating a new site is just a matter of creating a settings.php file and (optionally) a database to go with your new site. That's it. More importantly, there's no need to set up complicated Apache Virtual hosts, which are a wonderful feature of Apache, but can be very tricky and tedious, especially if you're setting up a large number of subsites.

No worries, there is a solution.

Create a new LogFormat

Copy the LogFormat of your choice, prepend the HTTP host field, and give it a name:

LogFormat "%{Host}i %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vcombined 

Get the script

Next, download the attached script (split-logfile) and store it somewhere like /usr/bin (don't for get to chmod 755 that baby!)

Now, tell apache to use pipe logfiles to your script rather than writing them directly to disk:

CustomLog "| /usr/bin/split-logfile" vcombined 

Restart Apache

/etc/rc.d/init.d/httpd restart

That's it.

Naturally, you may have to modify split-logfile if you don't store your logfiles in the default location.

 

 

May 02 2006
May 02

Google accepts studet applications for the summer of code 2006. Drupal has interesting projects for you to choose from. If you are a student anywhere, interested in php and/or drupal development, want to spend your summer coding and earning some money, apply. You might get accepted. If none of the proposed projects interests you, you can suggest your own. Come on, be brave. You can get a rock-star status in a part of the net.

May 02 2006
May 02

Drupal 4.7 is out. It features a lot of genetic improvements. Features support for a lot of hot features, but more importantly, it started the slimming trend.

I'm delighted that although there are loads of additional functionality, the modules are getting slimmer. Probably due to forms api. I hope this trend will continue.

May 02 2006
May 02

On Saturday night Drummy, Tony, and I went to Super Happy Dev House IX. I was a bit drunk. I bet Neil and Tony that I the "business guy" would beat them the "hackers" at hacking. My goal for the night was to integrate the gmaps module with views to allow users to create maps of any view.

I won handily:
http://zacker.org/mapdev/?q=crimemap

The how-to is posted on this handbook page.

May 01 2006
May 01

Wheee! Drupal 4.7.0, the new stable release of the CMS, has been released. After more than one year of development, we are now finally able to enjoy all the goodies which this new release brings.

For German speaking people we can now proudly claim to have a completed translation of the user interface, thanks to all the contributors! If you don't speak German, you can choose from one of the other 40+ translations for your site.

Read the release announcement for more information, I'm off upgrading a couple of sites now ;)

Apr 28 2006
Apr 28

I finally came about to changing the look of this blog. It is a new css-based design. It might get heavyish on some machines+browsers, but to be honest I can't care less about it.

This design is kind of a proof for the themability of drupal. It took me about 20 minutes to convert this from a plain html with embedded stylesheet to page.tpl.php+css.

If I can do it, any monkey should be able to.

Update: thanks to rkerr & |gatsby| I've fugured out that I have a slight problem in IE - unreadable. The bloody boxes and png transparency problems. Now it should be much better.

Thanks guys

Apr 21 2006
Apr 21

This article explains a practical implementation of a technique outlined in the article "Sharing Drupal tables between databases using MySQL5 Views".

Problem

You have multiple (multisite) Drupal sites and you would like to manage the content for all of these sites through a single interface. Depending on the nature of a given piece of content, you may want the content published on one, several or all of your subsites, but you do not want to have to create copies of the same content for each site.

Solution

Taxonomy plus MySQL5 views. (NOTE: this solution will not work with versions of MySQL prior to 5.)

Assumming you have your subsites properly set up and running, the first step is to create a special vocabulary which you will use to target content.

Go to [your site's baseurl]/admin/taxonomy/add/vocabulary and create a vocabulary. We'll call it simply "sites".

Next, go back to your taxonomy page (/admin/taxonomy) and select "edit vocabulary" for the "sites" vocabulary.

Add a name for each of the subsites you would like to manage. For our example, we'll have two subsites, "foo" and "bar", and one master site, "master".

Now add at least three pieces of test content. Target one piece of content for each of foo, bar and both.

Next, we're going to create a node view for each of our subsites that we'll use to replace the actual node table.

The SQL is as follows:

CREATE VIEW [subsite, eg. "foo"]_node AS SELECT n.* FROM node n, term_data td, term_node tn, vocabulary v WHERE v.name = '[vocabulary name, eg. "sites"]' AND td.vid = v.vid AND td.name = '[subsite vocab term, eg. "foo"]' AND td.tid = tn.tid AND n.nid = tn.nid ;

Because the terms that serve as our subsite labels may very well exist within other vocabularies, we also need to join on the vocabulary table to ensure our solution works reliabley.

Finally, we need to have our subsites use the views we have created instead of our master nodes table, which only the "master" site will have access to directly.

In your drupal's sites directory, you should have directories that correspond to each of your drupal sites (both master and subsites). Edit the settings.php file for each of your subsites, and use the db_prefix variable to point the site to your view. So sites/foo.example.com/settings.php would contain the following:

$db_prefix = array( 'node' => 'foo_', );

At this point, you'll want to disable creation of content from within each of your subsites. You can do this in the from the admin/access page. If you attempt to create content from within the subsites, you'll likely get a 'duplicate key' error.

I hope that explanation is clear. These articles are written rather hastily. If you questions or suggestions regarding this solution, please leave a comment.

Apr 20 2006
Apr 20

Jeff Robbins of Lullabot posted a podcast interview of me here. I babble on for an hour or so about the Dean Campaign, DeanSpace, and CivicSpace.

Apr 11 2006
Apr 11

Drupal is an open-source Content Management System (CMS), which is particularly suited to customization through use of blocks, themes and modules. Drupal's innovative use of taxonomy and its unique menu system allow for effective management of content as a site grows or evolves.

Driving the ongoing development of Drupal is the desire for a simple and powerful framework which developers can work within to create custom web applications without having to build the pieces that nearly all websites have as a foundation (user authentication, content creation interface, layout managment).

And unlike other similar CMSs, Drupal takes its role as a framework very seriously. Take a look at the database structure of any other CMS that has been around for a while, and you'll almost certainly find a labyrinth of tables with mysterious names and seemingly redundant purposes. Drupal's database design is clean and constantly being re-factored. Tables that have outlived their purpose are removed from the schema and the code. Table design is very generic, allowing different types of content and modules to share the same table space.

Drupal also has an actual API. It too is refactored on a regular basis as the web and user needs change. Drupal 4.7, for example, has a completely redesigned Forms API which now allows developers a much broader reach into the entire Drupal framework, allowing her to tweak parts of Drupal that were previously pretty much set in stone.

From a webmaster's point of view, Drupal may not have the most eye-candy or largest feature set when compared to other CMSs. However, there's virtually nothing that can't be built into Drupal, making it the most flexible CMS out there.

Go with Drupal, and you will have no regrets.

Apr 11 2006
Apr 11

A few weeks ago we held our first ever "Drupal Camp" in San Francisco at Compumentor's office. I blogged about this previously when we were hashing out the idea. Today I posted a pretty lengthy report back about it on Drupal.org:

http://drupal.org/node/58182

Apr 05 2006
Apr 05

Yes, conceivably within a few years Ruby on Rails could emerge as a dominant web application development environment. But I am betting strongly against it. Why all the hype then? True innovation, great presentation, lots of screencasts & brilliant marketing. But in the end, RoR is fighting an uphill battle. E.G. LISP has a much greater market share than Ruby (.721% vs .2%).

Programming language market share numbers are taken from this study. It covers the languages in general and is not specific to web application development. If anyone has any better analysis please let me know.

*Update*: There is a great blogpost from a Ruby on Rails devotee here that comes away with much the same conclusion but instead of just numbers he provides a well reasoned argument.

Apr 01 2006
Apr 01

Frustrated with the never-ending betas of the imminent Drupal 4.7, I have decided to switch to a better project, Drup.us.

Mar 28 2006
Mar 28

Today, I woke up and decided that I will give my drupal modules away, in the hope that they will be better maintained. If you’re a Drupal developer who would like to take over the Captcha, TextImage and Similar modules, please contact me. Note, I’m not unmaintaining them, I’m only looking for a person who has more time and energy than me to work out the kinks and make the modules work for every Drupal version and not bloat the features and prefers to make things more efficient. If I do not find such a person, I’ll just keep sporadically maintaining them like I was. Interested? Just send me your drupal.org username, and what module you’d like to take over.

Mar 27 2006
Mar 27

Recently, I've had the pleasure to try out something new - reviewing a book before it is published.

I have been acting as technical editor/reviewer (or whatever that's called in English) for the first German book about Drupal, written by Hagen Graf: "Drupal: Community-Websites entwickeln und verwalten mit dem Open Source-CMS". The book covers the Drupal 4.7-beta series and is a good introduction to Drupal and it's concepts. It's a nice book for people who want to learn more about creating websites with Drupal. More details in this post over at drupal.org.

Reviewing books is a lot of fun - I might do that more often in future ;)

Mar 25 2006
Mar 25

Ok, ok, maybe there actually is something to this Web2.0 business. A lot of the Web2.0 hype stems from the maturation of blogging, the software that helps people blog and RSS. All of these are good and bad things. Good in that there's more information available to more people today than ever before. Bad because there's more information available to more people today than ever before. Even before the term "blog" was coined, people were publishing personal journals, ramblings and general rants on the web. It was as true ten years ago as it is now, most of this content is... well... not very interesting to most people. Maybe it's poorly written blather about why someone hasn't posted poorly written blather recently. Or maybe it's well written content on a very specialized topic that is only relevant to a small number of people.

Good, bad, relevant, meaningless, it's all out there somewhere. The key to finding it is RSS. I used to subscribe to the Los Angeles Times. I'd poke through it every day and read a sentence or two and then tossed it in the recycling bin. The fact is, my interest in polotics, world affairs, sports, gardening, real estate and used cars is pretty limited, and that's all you're going to read about in most newspapers. My interests are elsewhere. Software programming, guns, literature, skiing, and so on. The best way to feed my desire for information is to turn to the web. But turning to the web for information is like visiting Niagra Falls for a sip of water. It's just too much. You need something like Google, Yahoo or now RSS to filter the information for you.

Will Web2.0 ever mean anything? Absolutely. I believe Web2.0 will eventually become something even grandma can understand. Presentation will be fast and flexible (AJAX, DHTML). Content will be tightly filtered with RSS. Folks can easily take advantage of all this stuff now with sites such as Bloglines, Feedster (update: now dead) and Feedburner. And all of this falls under the annoying label of Web2.0. This site is dedicated to Drupal. One of the things I wanted to provide was an aggregate feed of the most relevant Drupal news available. So using Drupal's aggregator module, I began creating the Drupal news page. You'd think that would be a simple task, but it isn't. With so many feeds, and so many feeds of feeds, and so much information available, pulling down just the right content is still very hard, and I still don't have the desired page I want. But the fact that it is even possible is something I'm very excited about. Here's a link to the story I was going to originally write about before my ADHD kicked in: The bottom line on blogging.

Mar 24 2006
Mar 24

Problem:

You have a "master" database that contains data from multiple Drupal sites and you want to share it among them. Normally, you could use table prefixing to allow each of your sites to point to a single table. But what if you do not want content from one site to "bleed" across to the other sites? Let's say you have a network of Drupal sites sharing a user database. You want to share that user's information across your entire network of sites, but only make the information visible from those sites to which the user has subscribed. Or maybe you want to populate baz.com with users who meet some arbitrary criteria. There are lots of possibilities here, but nobody's paying me to write this, so let's get on with it.

Solution:

Use MySQL5 Views. For our example, we'll use the users table. Our "master" users table (the table that contains all users for all of our sites) resides in a database called "master". The database of the our example site that will have restricted access to our masters users table is called "banana". Assuming you're starting with a fresh instance of the Drupal schema in your database "banana", do this:

use banana;
drop table users;
CREATE VIEW users AS
SELECT *
FROM master.users
WHERE uid IN (
SELECT uid
FROM somedatabase.sometable
WHERE uid = 0
OR label = 'foo'
)

Drupal will use banana.users just as it would a normal users table. No other modifications are necessary. Now only "foo" users will be included in the users table for your banana website. Note: Drupal has a dependency that is not really documented. Every users table must have an entry that contains uid=0. It's a "stub" entry that Drupal needs to function properly when a user is anonymous. A workaround for this dependency is to include "user 0" in the results set that defines your view. See, wasn't that easy? Please post questions here, and I'll update these instructions as needed.

Mar 22 2006
Mar 22

Consider this:

  • My job is to focused around developing web applications that help communities collaborate yet the majority of the day to day collaborative work I am personally involved in is faciliated by standard mailman mailing lists, not community focused web applications. Mailing lists are functionaly no different than they were more than twenty years ago when they were invented.
  • Without a doubt the most pervasive and powerful organizing tool the Dean campaign grassroots groups used beyond Meetup to self organize was Yahoo groups. Yahoo's business is centered around "user produced content" and community. YahooGroups (formerly eGroups) with 50M registered users is their #1 community tool. Yet the toolset has barely changed in the 6+ years since eGroups was bought and made a part of Yahoo.
  • In my experience as a community organizer (DeanSpace, PeopleFinder) I have found that there are only two indespensable tools: wiki's and mailinglists. With both in place 85%+ of your web app needs are covered and groups are more than capable of self-organizing effectively.

So given all this, why does CivicSpace still not ship with working YahooGroups-like mailinglists and wiki support? Good freaking questions. Thankfully, I believe we are finally getting close to an adequate answer....

My first screecast (37 megs 10 min):

This sandbox runs on Drupal 4.7 beta 5. Modules I am using on this site:

  • Og - to create and manage the groups. Thank you Moshe!
  • og2list - to send out mail to group members
  • og_forum - to sync a forum w/ each group
  • og_mandatory_group (4.7 port included below) - to auto-join site registrants with the main group
  • og_intro (in zip below) - to send notices to the main group when users join and new groups are formed
  • Tabs - to draw the fancy ajax tabbed pages on the og nodes. Thanks nedjo!
  • Node Relativity - to handle 'sub groups'
  • Freelinking - to handle wiki-link -> node edit forms and [[wiki link]] fiters
  • Masquerade - to let me test the site as a non-admin user
  • Mailhandler - to post mail to my site off of og email lists (reads a catchall for the domain)
  • Mail Stuffer (in zip below) - my hacky helper module that associates mail sent in to mailinglists
  • Wikipage (in zip below)- to create wiki node types and manage permissions
  • Bookmark - to let users save pages in their bookmarks block

I would highly recommend waiting until og2list is fully baked and until I have a chance to clean up my code before you use this. But if you must I have included all my new modules, slightly hacked modules (ported og2list to 4.7 and added tags support to mailhandler) and my theme .tpl files. This stuff will all make it into cvs some time next week if all goes well.

Attachment Size theme.tar.gz 14.91 KB modules.tar.gz 90.38 KB
Mar 17 2006
Mar 17

I've poured almost three of my life in to CivicSpace waving my hands and willing it into existance. We have a lot to show for it: 30 major software releases, two thousands CivicSpace powered websites, a vibrant and quickly growing user community, and a network of 25+ vendors occupying a solid slice of the marketplace of advocacy / non-profit web technology services. But what we haven't had so far is a solid user facing product - something I can show my mom...

Ten minutes ago I sent a note to our mailinglists announcing that we will shortly begin alpha testing a hosted CivicSpace service and are looking for testers. Three minutes later four people signed up.

The CivicSpace hosted service is almost here and I couldn't be more excited.

Mar 16 2006
Mar 16

Good thing I had a good backup of my www directory and database!

So I tried upgrading Advanced Web Design to 4.7 yesterday. I had all of the 4.7 verions of all the modules and I was all set. One of those moduels was our friend, tax_access. I unpacked the new files into the modules directory and expected everything to be fine.

Tax_access is supposed to use the Drupal update system for upgrades. Right? Wrong. Didn't work here. No big deal. I'll drop the old tables and create the new ones manually. I only have one term restricted, I just re-do the category permissions. But my problem had nothing to do with the tables.

I don't know what tax_access does when you enable it in the settings, but there was some legacy 4.6 version of the module hanging around that I couldn't get past. For whatever reason, it doesn't start using the new module file, when it replaces the old one. Of course this legacy version was not forms API compliant. I could do nothing to get rid of it. So I reverted to my backups.

Anyone using tax_access: Disable the module BEFORE upgrading! Or else the legacy version will come back to bite you in the butt. Apparently this has happened to others, but I didn't find a lot in the forums or issue tracker about it. There's nothing in the documentation.

You have been forewarned.

Mar 15 2006
Mar 15

Today, the folks over at Performancing.com announced the realease of Performancing Metrics, a free new service for tracking statistics for Blogs. Best of all, it's all in Drupal. OK, I'll come clean, I was involved in this project. My primary role was integrating Performancing's existing website with the Metrics tool. The entire toolset runs behind a single Drupal module that acts as a wrapper for what are essentially standalone PHP scripts.

Disclosure out of the way, I've also been using this tool for some of my own websites for the last month or so. When Google Analytics launched, I was able to get an account before they blocked new subscribers due to load issues. I've been monitoring my sites with Google Analytics for about four months. It seems to be a very powerful tool, but to be honest, I don't understand how to use it. There are just too many views and parameters and summaries. Using it, I feel like I did when I first used Vim or MS Word or Paint Shop Pro, overwhelmed and confused. Hey, I just want to write a letter to Mom, and I'm staring what looks like the cockpit of a 747. Similarly, when I look at website stats, I just want to see how many folks are visiting, where they come from, how long they stay, what they like, whether they bothered to leave comments, etc... I don't need a pivot table detailing gross ROI per visitor vectored over a decreasing granular matrix margin. Eeeek! Just show me the big pic and let me get on with my day.

If you're like me, you'll appreciate this new tool. It's light on eye-candy and heavy on simplicity and quick access to what you want to know. It's also somewhat addictive.

To use it, you'll need to sign up over at Performancing.com. Then all you have to do is paste a bit of javascript into your Blogs pages (via templates or footer configuration).

What distinguishes this tool from others is that it's focused on Blogs. The javascript you use will vary depending on which blogging software you're using. This way the javascript is able to glean information from your site that other trackers simply can't. Author names, Comments, Posts, etc...

We're working on making this embedded javascript tracking more effective and specific to the various blog systems, but even this beta version of Performancing Metrics is worth the effort of signing up.

You'll also find lots of cool articles regarding professional blogging. Even if this isn't how you make your living, there is some excellent information that non-professionals can use. Stuff like how to bring more people to your site and how to "monetize" your website more effectively.

Check it out.

Mar 14 2006
Mar 14

Most website's I run will be easily updated once Drupal 4.7 rolls around. In fact, the average website won't need to be updated anytime soon. But for one website in particular, I am already having nightmares. I need to upgrade for a few reasons:

  • I need the extra functionality. There are new features in Drupal 4.7 that will maker certain tasks remarkably easier. For example, the new block regions. More than 1,500 pages all have the same footer in the content area. The owner wonders why she can't change how it reads on all of the pages every now and again (once a month or so). Isn't that the advantage of a CMS? Now it is.
  • My buddy Jeremy is working on a new Ads module. I have been requesting development on the current banner module, which prompted Jeremy to re-write it as an Ads module. The fundamental difference being that ads will be nodes, thereby increasing functionality and enabling the sort of options I am looking for. I am finding myself in GREAT NEED of said increased functionality - the ability for ads to be searchable, and/or matching option, list all option, etc.
  • I am splitting the content (some 1,900 nodes) into two websites and need to do a lot of cleanup. Not a technical issue, but a real one nonetheless. We are facing some real branding and marketing issues which require us (meaning me) to completely revamp the website as a whole. This includes structure, navigation, taxonomy, etc.
  • I want to use some of the new goodies. There are some needed modules and/or module improvements that I see in the repository. I think they will be very helpful to reach my goals for this website.

The headaches come in when I figure that I have some 27 contributed modules (Amazon Tools, Flexinode, Simplenews, etc.) and would like to use another 12 (E-commerce, Blog Theme, User Points, etc.) at least. That's not including some modules I'd like to discard (FCKeditor) in favor of others (TinyMCE). At least I don't have to worry about Menu on the Fly anymore.

There are two possible approaches.

  1. Take the site down for a day and upgrade the usual way. Fool around with the terms and menus, un-publish nodes, whatever - as I go along.
  2. Redevelop the site on my local machine using UniServer and swap a clean, brand-new 4.7 website for the other in the dark of night.

I will have to consider this some more.

Mar 14 2006
Mar 14

Web2.0 is now, and it's here. If you're not Web2.0, you're out like 2005. If you have to ask, you wouldn't understand.

Well, I have to admit, I need to ask. In fact, I have several questions about all this Web2.0 fanfare.

Why should I upgrade? How much will it cost me? How come nobody has it in stock when I try to buy it?

I've been making my living doing web stuff since 1996, and I never got my copy of Web1.0. I didn't even realize it had been released. Now I just sit and nod in meetings where people talk about "synergistic solutions", or throw in a "user comprehensible value prop" in such a way that you just scratch your head and think it must mean something. "This needs to be Web2.0" is a favorite of mine. Whatever Web2.0 is, obviously it's critical to the success of new projects. (I need to remember to remember that for my next project.)

Who owns Web2.0? For now, I guess the safe assumption is that Google does. That would explain both all the ballyhoo as well as the mystery surrounding this next generation breakthrough. Ask someone at work what Web2.0 is and you'll likely get the curly brows and a loud "Pffft!", in which case you just give them a, "... I guess some of the folks in this meeting haven't gotten the Web2.1 pre-release yet ..." and give them the reverse curly brows and a subtle sigh.

Look around for Web2.0 and you will no doubt find everyone talking about it, but you're not likely to actually find an example of it. I found no screenshots on Google Images. No info on where to order it or how to install it. However, I do think I'll be able to recognize it once I find it. I know it will have an "API". I know it will also be very clean because it requires Ajax. It will probably involve pastels contrasted with ridiculously bright primary colors.

It will also have a very fancy look to it. I imagine the email icon above probably isn't very Web2.0. (though I'm not claiming to be a Web2.0 authority, of course.

And I'm sure all the experts would agree that this RSS icon is definitely Web2.0.. However, my failure to convert the white background to transparent is probably more web 1.1 or so.

If you know as little about Web2.0 as I do, I wouldn't worry about it too much. If there's anything to it, I'm sure we'll all be getting 3000 free hours of it in the mail from AOL real soon.

Mar 14 2006
Mar 14

New versions of Drupal are out for the 4.5.x, the 4.6.x and the 4.7.0-beta branches which fix 4 (in words: four) security issues from four different categories, namely: access control bypassing, cross-site scripting, session fixation, and mail header injection.

Upgrade now!

Warning: If you're using 4.5.x, the patches for DRUPAL-SA-2006-003 will not fix the security issue immediately. You have two options: a) upgrade to 4.6.6 instead of 4.5.8, or b) upgrade to PHP >= 4.3.2.

Mar 10 2006
Mar 10

Rumor is spreading of some sort of new analytics tool from the blogging experts over at Performancing. This tool utilizes Drupal and is designed specifically for bloggers. It's a tool similar to others like Google Analytics and Sitemeter. What makes it different is that it is designed to track blogs specifically. Read more about it at Performancing, Problogger and Code Professor.

Mar 07 2006
Mar 07

Had to delete the xmlrpc.php file that come with Drupal because my host was getting slammed by hack attempts - some successful, some not. The naughty installation was 4.3.3 and I have since upgraded to the 4.6.5 version. (Gee, how did that one slip by?) Should be safe now...

In any event, I have been asked very nicely to not upload these files in the future. Will comply... As a result, Drupal ID logins are now disabled at my website.

However, it took me a while to figure out that this file was also related to the Drupal ID login feature, not just the website directory ping. That's not really clear in the documentation. Here I am making a fool of myself trying to login in to the Drupal Sites directory.

Mar 01 2006
Mar 01

John Stahl recently gave me some heat for my assertion that Elgg should be built on top of Drupal.

I think that the next few years are going to bring tremendous challenges for applications that do not easily communicate with other applications that are “outside their platform” i.e are written using a different language/framework, run on a different server, etc....The days of monolithic application stacks that try to do everything are fading fast. A new “network-centric” software ecosystem is starting to bloom.

This is wishful thinking. I've spent much of the past few years puzzling over this exact question. While I am personally very much a proponent of web standards and web services I am pessimistic as to how much immediate impact they will have in the evolving marketplace of non-profit/ngo & advocay web technology services.

Backstory

I didn't always think this way. If you told me a year and a half ago that I would be hawking CivicSpace/Drupal as the über-platform that could meet virtually every need of any size organization I would have told you you were nuts....

After the Dean campaign ended and we started work on CivicSpace our assumptions were:

  • Drupal wouldn't be able to "scale" to meet the needs of large organizations
  • We would not be able to develop the platform quick enough to meet all the core needs of organizations any time soon
  • We would integrate third party service providers to fill the functionality gap organizations required

Then some interesting and unexpected things happened:

  • It turned out Drupal could scale
  • CiviCRM showed up and filled the functionality gap
  • Lots of vendors established quickly growing businesses servicing the technology. The top half dozen firms now employ ~ 50 people between them and each are looking to hire.

This reshaped my thinking on the future of the web-technology marketplace quite a bit. We have an immediate opportunity to commodify the core web-technology organizations need to a single integrated and scalable open-source application stack and this is a very good thing for the marketplace. Over the next few years I believe the advantages afforded by this stack of technology (CS/Drupal/CiviCRM) will far outweigh the benifits realized by the integration of applications accross web-services in terms of costs saved and passed on to technology owners and innovations in technology and services.

Integration accross web-services and web-standards is relatively costly

  • Web applications that are not built from the ground up to be integrated will never play nicely with one another. Let me repeat: if an open-source application is not built to be integrated with your web-application it probably isn't worth the effort to try to integrate it. This renders 95%+ of open-source web applications useless for those looking to leverage the work of other communities.
  • If an application has a strong API it is still tricky to integrate. Even integrating with CiviCRM, an application that was built from the ground up to interface with CMS's and shares the same web-app environment as Drupal, it costs us 2-10X more to integrate with it than it does to integrate with a standard Drupal module.
  • The most cutting edge private sector web-service and web-standards are not that advanced. We still don't have workable single sign-on solution, the cutting edge of semantic information interchange across the web is to embed it in XHTML, the hottest API's from SalesForce and Flickr haven't made much of a dent in the marketplace, and please don't get me started on the semantic web.

The market currently prefers a single integrated stack

  • The majority of the market for web-technology services is owned by companies like Kintera, GetActive, and Convio that serve as one-stop shops for virtualy all an organizations web-technology needs. They have trained technology owners to make purchasing decisions under the assumption that they can cut a check to a single entity that will provide and support a complete "solution". I.E. there is not much point to shopping around for "best of breed" services to integrate, "just give us your business and all your technology problems will go away". This creates an uphill battle in the marketplace for vendors selling what will be seen as "piecemail" solutions created by amassing various web-technology providers products across web-services.
  • There are huge unanswered usability concerns created by integrating together two different applications. This is espcially felt in functionality that is presented to an organization's contituents. Technology owners demand a seamless user-experience across their user facing application space (CMS, event tools, community tools, etc). In the future as more and more functionality moves out of the back-office and onto the web these concerns will only increase.
  • Open-Source vendors eat costs when integrating third party services, mantaining integration, and licensing software, so they have economic incentives to service a complete pre-integrated stack of technology instead of servicing a suite of other providers products integrated over web-services.

The future of the application stack and the role I think webservices will play

I hope and expect that in the next year the CS/CiviCRM/Drupal stack will evolve to a point where it can compete head on with the likes of Kintera, GetActive, and Convio. When this happens open-source vendors will grow into full blown ASP's that will be able to sell services that undercut the current market of proprietary service providers and will be able to grow downmarket to smaller organizations and horizontally to for-profits with overlapping technology needs. With so many organizations and vendors based on the same codebase it will create a very efficient marketplace that supports application development and services. It will also open up the marketplace to anyone wishing to specialize their services towards a vertical, sell data services, or offer 'best of breed' applications. Since the majority of the vendors business will be based entirely around customization and hosting / support and not licensing fees to support product development and sales, they will be much more likely to partner with 3rd party providers or create specialized services themselves. Over time as web-standards evolve the third party services and specialization will grow increasingly important in the marketplace. And then we can all live happily ever after....

Feb 21 2006
Feb 21

Russell Beattie figures out MySpace, in component terms. Funny how every sentence of his reminded me of a specific Drupal module that provides that feature.

Then of course, we’re not counting the sentences that talk about hotties and risque photographs. I don’t think there’s a module (yet) to provide that.

Feb 14 2006
Feb 14

The Yahoo! UI people just opened up their widget library, complete with graded browser support. Hmm. Maybe we would use the calendar widget for the Drupal archive.

Jan 31 2006
Jan 31

Recent versions of Drupal have the oh-so-cool feature that allows you to host many websites off a single Drupal codebase. The coolest part about this is that you can share some tables accross multiple websites; which means you can do things like have a single username/password table accross all the websites. This can easily be done, as specified in the settings.php comments as:

* $db_prefix = array( * 'default' => 'main_', * 'users' => 'shared_', * 'sessions' => 'shared_', * 'role' => 'shared_', * 'authmap' => 'shared_', * 'sequences' => 'shared_', * );

Now here’s an important thing to note: The first table you have to share is the sequences table. This is the table that handles all the id counters, so if you don’t share this one, something like this can happen:

[you shared only the users table]

1. User 1 signs up on Site A, gets user id#1
2. User 2 signs up on Site A, gets user id#2
3. User 3 signs up on Site B, gets user id#….? The correct answer is not 3!

This happens because you didn’t share sequences… Site B uses it’s own sequence generator to render a duplicate userid… which the user table would not accept, and this would go on till the Site B sequence catches up with the Site A sequence, and then things would be normal. The code quality in user.module helps protect the user table from data corruption, but you will have many signups disappear into thin air with a set up like this. Hence, all you need to do is share the sequences table along with the users… and you’re all set!

Btw, hello Planet people!

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web