Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Sep 07 2012
Sep 07

Occasionally a node reference or entity reference autocomplete widget will not operate as expected, specifically when it is based off a view reference display. Other widgets, the select box, or list of checkboxes, will still function correctly.

This will happen if the view is depending on a contextual filter (an argument), but is not being provided one. Normally a view can try to automatically fill in the argument if one is not provided based on the current page url. If the view fails to receive an argument and is unable to infer its value from the url path then it will fail to provide any results.

Outlined below is a possible scenario that would cause an autocomplete node reference field to fail.

  1. You are editing an autocomplete node reference field on a taxonomy term edit page.
  2. The view reference display you have configured is setup to only show content associated with the 'current' taxonomy term.
  3. In the view, the taxonomy term argument is provided by the current context if no value is available.

Here is how we have configured the view:


View configuration screenshot

Beneath 'Contextual Filters' clicking on 'Context: Has Taxonomy Term ID' will provide more information on this filter (argument received by views):


Note: In this screenshot we are running the openpublish beta2 profile, which currently has not updated to the latest patches.


Contextual filter configuration screenshot 1

Notice that the view will try to fill a value into it's contextual filter if none is provided. It will try to do this based on the current url:


Contextual filter configuration screenshot 2

If your widget is setup to be list or select box then the view will be able to determine the current context (a taxonomy term) and provide a default value. Views can do this because the context is determined while the form is being loaded. But if you are using an autocomplete field, the json callback to drupal provides no context and the view has no idea what page it is being accessed from.

A solution can be achieved by providing a custom function that handles the autocomplete callback from json. The function will then explicitly set the views argument to the correct taxonomy term id.

  1. Alter the existing form field to use a different callback path with hook_form_FORM_ID_alter()
  2. Create the path router in hook_menu()
  3. Design the callback function itself to invoke views

Alter The Existing Form Field

In this particular case, after viewing the source of our taxonomy form, we find the form tag id is 'taxonomy-form-term'. This translates into taxonomy_form_term as the FORM_ID when declaring the hook_form_FORM_ID_alter() function. The node reference field itself has been named 'field_myfield_nref' and contains up to 3 descrete values.

/**
 * Implements hook_form_FORM_ID_alter().
 */
function mymodule_form_taxonomy_form_term_alter(&$form, &$form_state$form_id) {
  
// We will get our term id argument from the from build itself.
  
$term_id  $form['#term']['tid'];
  
// This is the path we will create in hook_menu().
  
$new_path "mymodule/autocomplete/{$term_id}";
  
// Maximum number of descrete values (deltas) that are present.
  
$max_delta $form['field_myfield_nref']['und']['#max_delta'];
  
// Hijack the autocomplete callback for each of our values.
  
for($x=0$x<=$max_delta$x++) {
    
$form['field_myfield_nref']['und'][$x]['nid']['#autocomplete_path'] = $new_path;
  }
}
?>

The above hook is enough by itself to get the autocomplete widget to begin polling a different path as you type in characters. Make sure that you have flushed all caches, and that your browser is not caching the old javascript. Now Drupal needs to be configured to do something useful when the above path is being requested.

Create the Path Router

/**
 * Implements hook_menu().
 */
function mymodule_menu() {
  
// The index will specify which path is responded to.
  
$items['mymodule/autocomplete/%'] = array(
    
// This is the function to be envoked.
    
'page callback' => 'mymodule_autocomplete_callback',
    
// Which url path segments, delineated by the forward slash (/), should be
    // sent to our function as arguments. Zero based.
    
'page arguments' => array(2),
    
'access callback' => TRUE,
    
'type' => MENU_CALLBACK,
  );
  return 
$items;
}
?>

Now, when the autocomplete field accesses the path /mymodule/autocomplete/{integer_value} Drupal will execute the function mymodule_autocomplete_callback. Next, the function must be configured to Invoke the correct view and return something useful to the requesting javascript.

Design the Callback Function

/**
 * Autocomplete callback.
 *
 * Recieve a field autocomplete json request from a taxonomy term edit page.
 * Returns a list of article nodes whos titles matches what has already
 * been typed into the field so far.
 *
 * @param int $term_id
 *   Unique taxonomy term identifier. This is the variable that is represented
 *   by the % sign of the path in hook_menu().
 * @param string $string
 *   Contents of the json submission. This will be what the user has typed into
 *   the node reference field so far.
 *
 * @return drupal_son_output()
 *   A json formated string containing possible matches constructed by a view.
 */
function mymodule_autocomplete_callback($term_id$string '') {
  
// We know the name of this field specifically because this is an edge case
  // solution. More flexible code could be put in place so as to not hard code
  // this. The field settings will store which view to use.
  
$field field_info_field('field_myfield_nref');
  
// These options will be received by views. Within the result set that views
  // will provide, we want to further limit by comparing the field 'title'
  // against what was submitted by the javascript ($string). We will compare
  // by 'contains', meaning the title must contain $string. The total results
  // returned should be no more than 10.
  
$options = array(
    
'string'      => $string,
    
'match'       => 'contains',
    
'ids'         => array(),
    
'limit'       => 10,
    
'title_field' => 'title',
  );
  
$settings $field['settings']['view'];
  
// This is the important part below. This view requires an argument for the
  // contextual filter to operate when the context can not be determined
  // automatically
  
$settings['args'] = array($term_id);
  
$matches = array();
  
// This is where the view is run that is reponsible for creating the possible
  // selections for autocomplete. Now we can pass in the argument that would have
  // otherwise been empty.
  
$references references_potential_references_view('node'$settings['view_name'], $settings['display_name'], $settings['args'], $options);
  foreach (
$references as $id => $row) {
    
// Markup is fine in autocompletion results (might happen when rendered
    // through Views) but we want to remove hyperlinks.
    
$suggestion preg_replace('/([^<]*)<\/a>/' href="https://activelamp.com/blog/drupal/customize-autocomplete-fields-with-results-you-want/([^'$2'$row['rendered']);
    
// Add a class wrapper for a few required CSS overrides.
    
$matches[$row['title'] . " [nid:$id]"] = ' class="reference-autocomplete">$suggestion '
';
  }
  return 
drupal_json_output($matches);
}
?>

By crafting our own module and using the above hooks and callback functions we now have modified the autocomplete field to work as desired. While editing our taxonomy term, the node reference field that allows us to select any node that is already associated with this taxonomy term works correctly. The first two values have already been filled out, while the third is in the process of displaying possible options.

Feb 09 2012
Feb 09

We spent a few hours trying to figure out how to use Color to make our custom Drupal 6 theme configurable. Color rewrites your CSS to include the user-configured colours, and adds the resulting stylesheet link to your header.

The first trick was to get the colour picker to show up on the theme settings page. The documentation wasn’t clear, but the easiest way to get started seems to be to copy the color/ directory from the Garland theme into a subdirectory of your theme, and then customize it from there. You will also need to follow the Drupal 6 or Drupal 7-specific instructions for calling the Color module when preprocessing pages.

Color searches your style.css (and imported stylesheets or other stylesheets defined by the ‘css’ part of your $info array) for colour definitions. Any colour that exactly matches one of the colours defined in the default scheme is replaced by the colour in the selected scheme, with the caveat that the base colour should not appear in the stylesheet. If the base colour is found in the stylesheet, it will be replaced by an empty string. In your stylesheet, make sure your base colour uses the shortened version (ex: replace #cccccc with #ccc) or use a very similar colour instead (ex: #cbcbcb).

So, the easy way to colourize your theme:

  1. Enable Color, if you haven’t yet.
  2. Copy the color directory from the Garland theme into your theme

Color will attempt to figure out unspecified colours based on those colours’ relationship with the base colour. This can lead to interesting combinations. If there are colours you do not want Color to change, put them in a section after a comment like this:

/*******************************************************************
 * Color Module: Don't touch                                       *
 *******************************************************************/

All colours specified after that comment will not be rewritten.

Some gotchas to watch out for:

  • It’s probably a good idea to add a comment to your style.css reminding developers to resubmit the colour settings after making changes to the stylesheet. Color rewrites the stylesheet, so changes aren’t picked up until the stylesheet is regenerated.
  • The Color preview appears to use hardcoded HTML. The gradient is created by color.js, and there doesn’t seem to be a way around it. Our workaround is to use CSS to hide both the preview and the header above it. Unfortunately, there is no div that encloses both the header and the preview.
Jan 31 2012
Jan 31

“Why is your window transparent?” a coworker asked me when she noticed my screen. I told her about how I do my CSS theming, and she pulled another coworker over and made me repeat the explanation. Since that seems like something other people might find handy, here it is.

Sass: Syntactically Awesome Sytlesheets

I rarely do CSS/front-end theming work, but when I do, I try to make it as fun and easy as back-end development. I use Sass (Syntactically Awesome Stylesheets) so that I can use nested selectors, variables, and mixins. This makes my code cleaner and easier to write. You’ll need Ruby in order to install Sass, but the tool will give you CSS that you can use on any web platform.

Browser-based tools

I prefer doing the initial tweaking in Google Chrome, because I like the way that the developer tools make it easy to modify the stylesheet. The Chrome CSS Reloader extension is handy, too. Most of the time, I make my CSS changes in the text editor, then use the CSS Reloader to reload the stylesheet without refreshing the page. This makes it easy to manually toggle the display of some elements while allowing me to refresh style rules. If I want to figure out the values for a few simple changes, I’ll sometimes make the changes directly in Chrome (you can use arrow keys to adjust values), then copy the values to my Sass source file.

Colors, sizes, and spaces

A second monitor is totally awesome and well worth it.

Designs rarely specify all the colours, sizes, and spacing needed. To quickly get the color of a pixel, I use WhatColor. This shows the hex code for colors, and allows me to quickly copy the code with the F12 shortcut key. If you want to change the shortcut key, the source is available as an AutoHotkey script.

To make it easier to match sizes and spaces, I use WinWarden to make my browser window 20% translucent. Then I carefully position it over my design reference until the important features match. Magnifixer makes it easier to line things up because it can magnify a fixed portion of the screen. By focusing Magnifixer on the part I’m working on, I can tweak CSS without straining my eyes.

When I know I’m going to be making a lot of changes, I use AutoHotkey to map a shortcut so that I can refresh the CSS with one keystroke instead of several. When I happen to have my USB foot pedal handy, I rig it up to refresh my stylesheet.

Regression testing

Sometimes my CSS changes modify other rules. Instead of laboriously checking each page after changes, I’ve figured out how to use Selenium WebDriver to write a Java program that loads the pages in Mozilla Firefox and Internet Explorer, capturing screenshots and numbering them according to the pages in my design reference. This means that I can run the program in the background or start it before taking a break, and then flip through all the screenshots when I get back.

Cross-browser testing

What’s CSS theming without the requirement of browser compatibility? Someday, when I need to deal with more browsers, I might look into Selenium RC. In the meantime, I develop in Chrome, my Selenium-based program makes it easier to test in Firefox and IE, and it’s easy enough to try the URLs in Safari as well. Virtual machines handle the rest of the requirements. 

So that’s how I’ve been doing CSS theming on this project. What are your favourite tips?

Sep 06 2011
Sep 06

The clients wanted a quick way to jump to the latest revision of a node. I was delighted to discover that the Finder module made it easy to create an autocomplete shortcut to nodes and users. It offered way more features than I would’ve coded myself. Finder lets you match nodes on title, author, CCK fields, and so on.

There’s a simpler module called Node Quick Find, but I’m going to go with Finder for now because of the options that Finder offers.

There was one small thing I needed to tweak. Finder Node goes to node/nid, but we’ve got Revisioning set up to view the current revision of a node and not the latest. Fortunately, Finder took that into account and provided enough hooks to let me change the behavior. Here’s what I needed to do:

function mymodule_finder_goto_alter(&$result, &$finder) {
  $finder->base_handler['#module'] = 'mymodule';
}

function mymodule_finder_goto($finder, $result) {
  $vid = revisioning_get_latest_revision_id($result->nid);
  drupal_goto('node/' . $result->nid . '/revisions/' . $vid . '/view');
}

You’ll want to use more if logic if you’re working with different kinds of Finders, of course, but this was enough to handle what I needed. Hooray for hooks!

Finder doesn’t seem to support Features, so I’ll need to configure things again once I move to production. No problem! I’ve added some notes to myself in the issue-tracking system we’re using, and I’ve asked the clients to try this new shortcut out.

Drupal: There’s a module for that.

2011-09-02 Fri 14:17

Aug 29 2011
Aug 29

Fatal error: Unsupported operand types in …../patched/rules/rules/rules.module on line 347

That was the error message Daniel sent me when he asked for my help in debugging. After some digging, I found out that the rules had been defined in two features, so Drupal got thoroughly confused. After I commented out one of the implementations of hook_rules_defaults and deleted the relevant rows from rules_rules in the database, the site worked again.

Daniel wanted to know how I figured out that problem, so here’s the story.

The line number told me that rules_sort_children was having problems. I added a var_dump to it so that I could find out what kind of unexpected data was causing the errors.

if (!is_numeric($element[$key]['#weight'])) { 
  var_dump($key, $element[$key]['#weight']); 
}

The output showed that the regular rules were fine, but our custom rules weren’t – weight was an array instead of an integer.

I looked at the difference between the features in code and the features in the database. The rules were being duplicated. I tried updating the features to match the information in the database, but the code looked wrong, so I used git to stash the changes. I also tried reverting the features, but that didn’t solve the problem either. Time to dig deeper.

I backed up the database, then deleted our custom rules from rules_rules. Still problematic. I commented out the rule definitions in our site_structure.features.inc. The rules administration page now successfully displayed – but mysteriously, the rule definitions were still available.

I looked at the rule tags to see where else they might be defined, and found that another feature had included the rules. Aha! I commented those out. As expected, the rules disappeared from the administration page. I’d identified the problem: the rules had been defined in more than one module, which had thoroughly confused Drupal.

Because it made more sense to define the rules in our site_structure feature than in the other feature, I uncommented the site_sitestructure_rules_defaults definitions and left the other feature’s rules commented. That worked.

I tried restoring the rule customizations from the database, but that gave the same error. The database copy had multiple definitions, and I didn’t feel up to picking my way through the data or writing a Drush script to manipulate the rows. I re-deleted the customizations to get a clean copy. Just in case the other feature had more recent definitions of the rules, I compared the two. Aside from the extra tag, they were identical, so I didn’t need to copy any information over. It meant that Daniel would have to make his changes again, though.

Features: When it’s good, it’s very very good. When it’s bad, it results in quirky bugs. Make sure you don’t define your rulesets in multiple features. Drupal Features still has some rough spots when it comes to Rules. I remember when I created this feature with the rules in it – it created duplicate rules, so I needed to delete the old ones. Still, it’s a great way to export rules and other configuration changes to code, even though it takes some getting used to (and the occasional bit of database-diving).

Anyway, that’s the story of how I identified that issue and worked around it.

When you’re faced with a fatal error involving unsupported operand types, figure out what kind of operands it expects and print out anything that doesn’t match. Then you can start figuring out the real problem, which is how that data got in there in the first place. I’ve used this to find form elements that were mistakenly defined, array elements that were unexpectedly null, and so on. Don’t be afraid to add debugging code to core or contributed modules, particularly if you can use source code control to restore a clean copy. If you use a runtime debugger like XDebug, you can easily explore the data and the call stack. If not, there’s always var_dump.

Hope that helps!

Aug 10 2011
Aug 10

I’m wrapping up a Drupal 6 project which was funded by one of IBM’s corporate citizenship grants. The Snake Hill folks we’ve been working with will continue working with the client until they’re ready to launch. For my part, I’ve been in user acceptance testing and development mode for almost a month, rolling out new features, fixing bugs, and getting feedback.

The project manager has shuffled some hours around and made sure that I’ve got some “support” hours for follow-up questions after we turn the project over.

What worked well

Hey, I can do this stuff after all! I gathered requirements, estimated the effort, negotiated the scope, communicated with the clients and other team members, and generally did other technical-lead-ish stuff. I’ve done that on other projects, but usually that was just me working by myself and talking to clients. This one was more complex. It was fun figuring out what would fit, how things were prioritized, whether or not we were on track, and how to make things happen. I’d love to do it again. (And with the way the world works, I will probably get an opportunity to do so shortly!)

Understanding a project deeply: I was on the first phase of this project as well, and the experience really helped. We didn’t have any disruptions in technical leadership on our part, unlike in the first phase. See, last year, the IBM technical lead who had been talking to the client ended up leaving the country, so we had to repeat a few discussions about requirements. This time, I could draw on my experience from the first phase and our ongoing discussions about the client’s goals for the project. That was fun.

I’ll be turning the project over to the other development company, and the client’s concerned about whether they’ll be able to pick things up and run with it. I’ve tried to write down as many notes as I can, and I also invested time in briefing the other developers on the overall goals as well as the specific work items. Hope that works out!

Externally-accessible issue tracking: In the previous phase of this project, issue tracking consisted of e-mailing spreadsheets around. It was painful. One of the first things I did when we started this phase of development was to set up a Redmine issue tracker on the client’s server. After we gathered and prioritized requirements, I logged them as features in Redmine and split them up over the different phases. I reviewed our list of outstanding work and filed them as bugs, too. As feedback came in, I tracked bugs. I took advantage of Redmine-Git integration and referred to issue numbers in my commit messages. When people e-mailed me their feedback or posted messages on Basecamp, I created issues and added hyperlinks.

Having an externally-accessible issue tracker helped me worry less about missing critical bugs. I also shared some reporting links with the clients and the project manager so that they could track progress and review the priorities.

On future projects, I would love to get to the point of having clients and testers create issues themselves. Wouldn’t that be nifty?

Git for version control: I’m so glad I used Git to manage and share source code between multiple developers. The other developers were fairly new to Git, but they did okay, and I figured out how to clean up after one of the developers wiped out a bit of code after some commit confusion. Git stash and git branch were really helpful when I was trying lots of experimental code.

Developing with a non-default theme: We had a lot of development items to work on while the No.Inc creative team got their Drupal theme together. Once No.Inc sent the theme, I layered it on top of the site, fixed the usual problems, and had even more fun working on a site that looked halfway done. Definitely recommend getting a reliable theme in place sooner rather than later.

Mentoring people: I helped a new developer start to get the hang of Drupal. It was a slow process (must raise estimates even more when dealing with newbies), but I hope the investment pays off. I wrote (and updated!) documentation. I identified small tasks that he could work on first. I checked on him every so often. I successfully resisted the urge to just do things myself. Slowly getting there…

Decision log: I used a wiki to keep track of the design decisions I needed to make, the alternatives I considered, and what I eventually chose. That was helpful for me. I hope it will help future developers, too.

Linux VM on a Microsoft Windows host, XMing, and Plink: I’ve tried lots of different configurations in the course of this project. Doing my development inside a virtual machine has saved me so much time in terms of restoring from backup or being able to tweak my operating environment. I started with a Linux VM on a Windows host, using Samba to access my files and either Eclipse or Emacs to edit them. That was a bit slow. Then I shifted to a Linux VM on a Linux host, SSHing to the VM and using Emacs from the VM itself. That was great for being able to do Linux-y stuff transparently. But then I found myself wanting to be back on Microsoft Windows so that I could use Autodesk Sketchbook Pro (Inkscape and MyPaint aren’t quite as awesome). I ran XMing to create an X server in my Windows environment, used plink to connect, and then started a graphical Emacs running on my virtual machine. Tada! I could probably make this even better by upgrading to 64-bit Microsoft Windows, adding more RAM, and upgrading to a bigger hard disk. (Alternatively, I could host the VM somewhere else instead of on my laptop…)

What I’m going to work on improving next time

Better browser testing, including cross-browser: I’m getting slightly better at testing the actual site, motivating myself with (a) interest in seeing my new code actually work, (b) the remembered embarrassment of gaping bugs, and (c) the idea of slowing down and getting things right. Juggling multiple browsers still doesn’t make me happy, but maybe I can turn it into a game with myself. Selenium might be useful here as well.

Continuous integration: I set up Jenkins for continuous integration testing, but it fell by the wayside as I wasn’t keeping my tests up to date and I wanted more CPU/memory for development. I ran into a number of embarrassing bugs along the way, though, so it might be worth developing stricter discipline around this. I’m still envious of one of the Drupal projects I heard about in IBM, which got through UAT without identified defects thanks to lots of manual and automated testing. If I add more power to my development machine or offload testing to another machine, that might be a good way to stick to this process.

Closer communication with clients and external developers: We set up short daily meetings for the developers, but sometimes people still felt a little lost or out of touch. On future projects, I’ll make sure the clients have it on their calendar as an optional meeting, and maybe see about getting e-mail from people who can’t join on the phone. If I’m the tech lead on a future project, I’ll sit in on all client status update meetings, too. We found out about some miscommunications only when I handled one of the status calls. Fortunately, it was early enough that we could squeeze in the critical functionality while reprioritizing the others. Tense moment, though!

Better vacation planning: I realized we had a 4-day weekend the week before we had it, and we forgot about some people’s vacations too. Heh. I should get better at looking at the entire project span and listing the gaps up front.

Earlier pipeline-building: I nudged some project opportunities about a month before our projected end date, but that wasn’t long enough to deal with the paperwork lag. Oh well! Next time, I’ll set aside some time each week to do that kind of future pipeline-building, and I’ll set myself reminders for two months and a month before the project ends. Not a big problem.

My manager’s been lining up other Drupal and Rails projects for me to work on. Looking forward to learning all sorts of lessons on those as well!

Other Drupal lessons learned:

2011-08-10 Wed 17:08

Aug 08 2011
Aug 08
Update 2014-02-19: See LittleDynamo’s comment with a link to this StackOverflow answer.
Update 2014-02-11: See Claus’ comment below for a better way to do this.

Drupal autocompletion is easy – just add #autocomplete_path to a Form API element, set it to something that returns a JSON hash, and off you go.

What if you want to pass form values into your autocompletion function so that you can filter results?

Searching, I found some pages that suggested changing the value in the hidden autocomplete field so that it would go to a different URL. However, that probably doesn’t handle the autocomplete cache. Here’s another way to do it:

Drupal.ACDB.prototype.customSearch = function (searchString) {
    searchString = searchString + "/" + $("#otherfield").val();
    return this.search(searchString);
};

Drupal.jsAC.prototype.populatePopup = function () {
  // Show popup
  if (this.popup) {
    $(this.popup).remove();
  }
  this.selected = false;
  this.popup = document.createElement('div');
  this.popup.id = 'autocomplete';
  this.popup.owner = this;
  $(this.popup).css({
    marginTop: this.input.offsetHeight +'px',
    width: (this.input.offsetWidth - 4) +'px',
    display: 'none'
  });
  $(this.input).before(this.popup);

  // Do search
  this.db.owner = this;
  if (this.input.id == 'edit-your-search-field') {
    this.db.customSearch(this.input.value);
  } else {
    this.db.search(this.input.value);
  }
}

Drupal.behaviors.rebindAutocomplete = function(context) {
    // Unbind the behaviors to prevent multiple search handlers
    $("#edit-your-search-field").unbind('keydown').unbind('keyup').unbind('blur').removeClass('autocomplete-processed');
    // Rebind autocompletion with the new code
    Drupal.behaviors.autocomplete(context);
}

You’ll need to use drupal_add_js to add misc/autocomplete.js before you add the Javascript file for your form.

Hope this helps!

2011-08-08 Mon 19:16

Aug 05 2011
Aug 05

I know, I know. I shouldn’t allow IFRAMEs at all. But the client’s prospective users were really excited about images and video, and Drupal’s Media module wasn’t going to be quite enough. So I’ve been fighting with CKEditor, IMCE, and HTML Purifier to figure out how to make it easier. I’m hoping that this will be like practically all my other Drupal posts and someone will comment with a much better way to do things right after I describe what I’ve done. =)

First: images. There doesn’t seem to be a cleaner way than the “Browse server” – “Upload” combination using CKEditor and IMCE. I tried using WYSIWYG, TinyMCE and IMCE. I tried ImageBrowser, but I couldn’t get it to work. I tried FCKEditor, which looked promising, but I got tangled in figuring out how to control other parts of it. I’m just going to leave it as CKEditor and IMCE at the moment, and we can come back to that if it turns out to be higher priority than all the other things I’m working on. This is almost certainly my limitation rather than the packages’ limitations, but I don’t have the time to exhaustively tweak this until it’s right. Someday I may finally learn how to make a CKEditor plugin, but it will not be in the final week of this Drupal project.

Next: HTMLPurifier and Youtube. You see, Youtube switched to using IFRAMEs instead of Flash embeds. Allowing IFRAMEs is like allowing people to put arbitrary content on your webpage, because it is. The HTML Purifier folks seem firmly against it because it’s a bad idea, which it also is. But you’ve got to work around what you’ve got to workaround. Based on the Allow iframes thread in the HTMLPurifier forum, this is what I came up with:

Step 1. Create a custom filter in htmlpurifier/library/myiframe.php.

#i', '', $html);
    return $html;
  }
  public function postFilter($html, $config, $context) {
    $post_regex = '#]+?)>#';
    return preg_replace_callback($post_regex, array($this, 'postFilterCallback'), $html);
  }
  protected function postFilterCallback($matches) {
    // Whitelist the domains we like
    $ok = (preg_match('#src="http://www.youtube.com/#i', $matches[1]));
    if ($ok) {
      return '';
    } else {
      return '';
    }
  }
}

Step 2. Include the filter in HTMLPurifier_DefinitionCache_Drupal.php. I don’t know if this is the right place, but I saw it briefly mentioned somewhere.

// ... rest of file
require_once 'myiframe.php';

Step 3. Create the HTML Purifier config file. In this case, I was changing the config for “Filtered HTML”, which had the input format ID of 1. I copied config/sample.php to config/1.php and set the following:

function htmlpurifier_config_1($config) {
  $config->set('HTML.SafeObject', true);
  $config->set('Output.FlashCompat', true);
  $config->set('URI.DisableExternalResources', false);
  $config->set('Filter.Custom', array(new HTMLPurifier_Filter_MyIframe()));
}

Now I can switch to the source view in CKEditor, paste in my IFRAME code from Youtube, and view the results. Mostly. I still need to track down why I sometimes need to refresh the page in order to see it, but this is promising.

2011-08-05 Fri 16:34

Jun 09 2011
Jun 09

One of my development goals is to learn how to set up continuous integration so that I’ll always remember to run my automated tests. I picked up the inspiration to use Hudson from Stuart Robertson, with whom I had the pleasure of working on a Drupal project before he moved to BMO. He had set up continuous integration testing with Hudson and Selenium on another project he’d worked on, and they completed user acceptance testing without any defects. That’s pretty cool. =)

I’m a big fan of automated testing because I hate doing repetitive work. Automated tests also let me turn software development into a game, with clearly defined goalposts and a way to keep score. Automated tests can be a handy way of creating lots of data so that I can manually test a site set up the way I want it to be. I like doing test-driven development: write the test first, then write the code that passes it.

Testing was even better with Rails. I love the Cucumber testing framework because I could define high-level tests in English. The Drupal equivalent (Drucumber?) isn’t quite there yet. I could actually use Cucumber to test my Drupal site, but it would only be able to test the web interface, not the code, and I like to write unit tests in addition to integration tests. Still, some automated testing is better than no testing, and I’m comfortable creating Simpletest classes.

Jenkins (previously known as Hudson) is a continuous integration server that can build and test your application whenever you change the code. I set it up on my local development image by following Jenkins’ installation instructions. I enabled the Git plugin (Manage Jenkins – Manage Plugins – Available).

Then I set up a project with my local git repository. I started with a placeholder build step of Execute shell and pwd, just to see where I was. When I built the project, Hudson checked out my source code and ran the command. I then went into the Hudson workspace directory, configured my Drupal settings.php to use the database and URL I created for the integration site, and configured permissions and Apache with a name-based virtual host so that I could run web tests.

For build steps, I used Execute shell with the following settings:

mysql -u integration integration < sites/default/files/backup_migrate/scheduled/site-backup.mysql
/var/drush/drush test PopulateTestUsersTest
/var/drush/drush test PopulateTestSessionsTest
/var/drush/drush testre MyProjectName --error-on-fail

This loads the backup file created by Backup and Migrate, sets up my test content, and then uses my custom testre command.

Code below (c) 2011 Sacha Chua ([email protected]), available under GNU General Public License v2.0 (yes, I should submit this as a patch, but there’s a bit of paperwork for direct contributions, and it’s easier to just get my manager’s OK to blog about something…)

// A Drush command callback.
function drush_simpletest_test_regular_expression($test_re='') {
  global $verbose, $color;
  $verbose = is_null(drush_get_option('detail')) ? FALSE : TRUE;
  $color = is_null(drush_get_option('color')) ? FALSE : TRUE;
  $error_on_fail = is_null(drush_get_option('error-on-fail')) ? FALSE : TRUE;
  if (!preg_match("/^\/.*\//", $test_re)) {
    $test_re = "/$test_re/";
  }
  // call this method rather than simpletest_test_get_all() in order to bypass internal cache
  $all_test_classes = simpletest_test_get_all_classes();

  // Check that the test class parameter has been set.
  if (empty($test_re)) {
    drush_print("\nAvailable test groups & classes");
    drush_print("-------------------------------");
    $current_group = '';
    foreach ($all_test_classes as $class => $details) {
      if (class_exists($class) && method_exists($class, 'getInfo')) {
        $info = call_user_func(array($class, 'getInfo'));
        if ($info['group'] != $current_group) {
          $current_group = $info['group'];
          drush_print('[' . $current_group . ']');
        }
        drush_print("\t" . $class . ' - ' . $info['name']);
      }
    }
    return;
  }

  // Find test classes that match
  foreach ($all_test_classes as $class => $details) {
    if (class_exists($class) && method_exists($class, 'getInfo')) {
      if (preg_match($test_re, $class)) {
        $info = call_user_func(array($class, 'getInfo'));
        $matching_classes[$class] = $info;
      }
    }
  }

  // Sort matching classes by weight
  uasort($matching_classes, '_simpletest_drush_compare_weight');

  foreach ($matching_classes as $class => $info) {
    $main_verbose = $verbose;
    $results[$class] = drush_simpletest_run_single_test($class, $error_on_fail);
    $verbose = $main_verbose;
  }

  $failures = $successes = 0;
  foreach ($results as $class => $status) {
    print $status . "\t" . $class . "\n";
    if ($status == 'fail') {
      $failures++;
    } else {
      $successes++;
    }
  }
  print "Failed: " . $failures . "/" . ($failures + $successes) . "\n";
  print "Succeeded: " . $successes . "/" . ($failures + $successes) . "\n";
  if ($failures > 0) {
    return 1;
  }
}

I didn’t bother hacking Simpletest output to match the Ant/JUnit output so that Jenkins could understand it better. I just wanted a pass/fail status, as I could always look at the results to find out which test failed.

What does it gain me over running the tests from the command-line? I like having the build history and being able to remember the last successful build.

I’m going to keep this as a local build server instead of setting up a remote continuous integration server on our public machine, because it involves installing quite a number of additional packages. Maybe the other developers might be inspired to set up something similar, though!

2011-06-09 Thu 09:51

Mar 30 2011
Mar 30

I would like to claim an utter hatred of race conditions. This is where code is written in such a way that it doesn’t fully consider the possibility of another thread (e.g. another website hit) or threads occurring concurrently. Consider the following which has been increasingly frustrating me recently:

Drupal stores variables in the ‘variables’ table. It also caches these in the ‘cache’ table so that rather than doing multiple SELECT queries for multiple variables, it simple gets all the variables straight out of the cache table in one SELECT then unserializes them.

cron_semaphore is one of these variables which is created when cron starts, then it deletes it when finishing. If it isn’t deleted it should mean that cron hasn’t finished running yet, so the next time cron tries to run it will quit straight away. But due to a certain race condition it doesn’t always get properly deleted as follows (p2 is an abbreviation for an unrelated process running concurrently, e.g. a visitor to your website):

1, cron starts, cron_semaphore variable inserted (and variables cache is deleted)
2. p2 starts, variables cache is empty so “SELECT * FROM {variables}” then…
3. cron finishes, cron_semaphore variable deleted and the variables cache is cleared
4. … p2 inserts result of “SELECT * FROM {variables}” into cache, but that SELECT was called before cron deleted the variable
5. you now have no mention of cron_semaphore in the variables table, but there it is back in the variables cache!

Consider many visits to your website concurrently and you soon realise this can become a very common occurrence. In fact, this exact problem inflicts us at least a handful of times every day. As a result cron keeps trying to run but immediately quits when it sees the semaphore variable still there. After an hour it finally deletes the semaphore but in the meantime crucial stuff doesn’t get done.

Web applications can quickly become riddled with race conditions such as these. I’ve spotted at least two more in Drupal’s core code in the past. When the ‘bugs’ occur as a result they can be tricky to pin down, appearing to be freakish random occurrences. Worse yet, even when found they can be a royal pain to untangle.

Share this:

Like this:

Like Loading...
Dec 01 2010
Dec 01

I've been working with custom PHP MySQL apps for quite a few years. Early on I was building grant systems for nonprofit and State Arts Agencies in the United States. About 4 years ago, I fell into Drupal quite by accident at a little think tank that my then employer, The Western States Arts Federation, had organized in Vancouver. Several of the participants were employees of Bryght. I began to drink the cool aid. While at WESTAF I was responsible for the build of a couple of Drupal sites and found myself more and more drawn to the modular LEGO-like way you could snap pieces together to create something new. It was just about at that point that I starting going to Drupal meetups and transitioned to a Web Producer position with a Drupal shop. This was the beginning of my seeing Drupal as my bread and butter.

Fast Forward to November 2010. I've been to Drupalcons in Barcelona, Boston, Washington, Paris, Copenhagen, and San Francisco. I was involved in helping set up the original Drupalcamps in Colorado when you could count the participants on two hands and that has grown to several hundred. Austin was the first camp I attended outside of Colorado and I was impressed. Six out of the Seven Vintage Digital Co-op members attended.

I approach each of the meetups, cons, and camps as a place to potentially grow my understanding of Drupal as a profitable platform. They give me a chance to meet with leaders in our community, with other Drupal Professionals, and interact with different shops of different sizes. The sessions often validate and sometimes surprise me. In Austin, I attended nine sessions including Jeff Robbins' keynote. I wrote about each of the sessions on my personal blog. They are really just notes with video from the event - but if you have interest you can see them on dogstar.org tagged with drupalcamp austin. I was happily met with a few new community modules, mostly surrounding SEO, that I hadn't heard of before. I enjoyed the panel on Managing A Drupal Business. Panels like this allow people to validate what they think they know by hearing similar stories from other businesses.

I am seeing more and more of the office managers, accountants, content creators, marketers, and account managers at these events. This should be encouraged and more business track sessions ought to be solicited. As Drupal continues to mature and ease itself into the Enterprise we need to grow with it widening the pool of talents beyond project managers, developers, and themers. We need to thinking in strategic ways and leverage those who have traditionally not been within our community. All of what I'm seeing is encouraging.

It is easy to discount the social time at the camps and conferences. However, meeting people in a looser way at places like Austin's Dog and Duck Pub is also very important. These are the times that professional contacts are made, deals are often brokered, and jobs are offered. Conversations can be had that help you avoid technical and project oriented pitfalls that other developers have found themselves subject to. You might find out about an RFP that a friendly competitor isn't going to bid on because they are out of bandwidth.

Austin was an inexpensive way to feed the need for professional development and to continue to forge and support relationships. I, personally, am looking forward to more camps.

Jul 28 2010
Jul 28

(This is content originally posted at http://jcfiala.net/blog/2010/07/28/magic-command-invocation-drush.)

I've recently been digging into the Aegir Hosting System, both because we're starting to use it at my current NREL gig, and because I've proposed to do a session on it at DrupalCamp LA. In short, Aegir allows you to easily administer a number of Drupal sites from a common site, itself built on Drupal. It's really slick, and a lot of the functionality is built on Drush. (Also see http://drupal.org/project/drush .)

So I'm looking into how Aegir handles tasks, which are just what they sound like - nodes representing commands that Aegir fires off every minute or so from a queue - if the queue's empty then that's that, but otherwise it calls drush hosting-task #, where the # is a task node id. Wanting to know what that actually does, I looked around, found hosting.drush.inc, and discovered that it didn't have a callback.

Huh. I'd thought that Drush commands had to have a callback, but here I discover that they don't. So, I wondered, what's actually going on there? And, following the advice handed to me by Greg Knaddison when I started in Drupal, I started following the code. And here's what I found:

Generally, you want to use the callbacks on drush commands when they're fairly simple - the task is doing something basic that can be handled easily. If it's something more complex, you can skip the callback, and gain a fair amount of control over how your command is invoked, and even cleanup help if something goes south.

If a drush command doesn't have a callback, then the command info is handed off to a function called drush_invoke - you can find it in the command.inc file. (For the purposes of this, I'm going to call the command hosting-task, because that's what I was looking for when I found this out, and it's a good example.) The first thing drush_invoke does is look for an include file named after the reverse of the command name - so our hosting-task gets inverted and the dashes are converted to dots, leaving us with task.hosting.inc. It then looks around to find this file in any directory that contains a *.drush.inc file - the folks working on Aegir in this case put task.hosting.inc in the same directory as hosting.drush.inc, which makes sense. Now, there doesn't have to be an include file like this, and if Drush doesn't find one, it'll still continue to the next part. But if you've got a complex command that you're handing off to drush_invoke to handle for you, why not put it in it's own file for neatness' sake?

Continuing, after the possible file load, the drush command is turned into a base $hook by the simple expedience of changing any spaces or dashes into underscores - so for the example, we'd have a base hook of 'hosting_task' With that constructed, drush looks for functions named off of this command as follows:

  1. $hook_validate (ie, hosting_task_validate)
  2. pre_$hook (ie, pre_hosting_task)
  3. $hook (ie, hosting_task)
  4. post_$hook (ie, post_hosting_task)

And then, for each module that defines a module.drush.inc file, drush looks for a drush_$modulename_$function... which means that for the hosting module, it looks for:

  1. drush_hosting_hosting_task_validate
  2. drush_hosting_pre_hosting_task
  3. drush_hosting_hosting_task
  4. drush_hosting_post_hosting_task

If you look at task.hosting.inc, you'll see that most of these are defined - they're not actually using drush_hosting_pre_hosting_task, but that's because they're doing their preparation work in the drush_hosting_hosting_task_validate function. So, the drush_invoke happily continues along this series of commands, until it either reaches the end (oh, happiness), or one of these functions invokes drush_set_error() - in which case the whole stack of commands reverses and runs backwards, and drush_invoke sees if there's a function named exactly the same, only with '_rollback' appended to the name. (For our example, there is a drush_hosting_hosting_task_rollback() defined in task.hosting.inc.)

A note on naming - although above we've referenced drush_hosting_hosting_task_validate and drush_hosting_hosting_task, the folks who maintain Drush recognize that these are pretty long names to have to construct, and as such they check for when one of these function names they construct starts with drush_module_module and change it over to drush_module. So, in task.hosting.inc, they really should be naming the callback functions drush_hosting_task_validate and drush_hosting_task... but for now Drush recognizes the old form and still will call them.

So to summarize what I've said so far, if you run into a Drush command without a callback, what you really should be looking for is a file named the reverse of the name in the same directory as the hook_drush_command implementation, and that file should contain the drush_module_command functions. And alternately, if you want to create a Drush command that can rollback/clean up after itself, then you want to create your commands in a reverse-order name file.

But... there's something else.

drush_invoke goes through all of the modules which define drush hooks when looking for functions to call when invoking a drush command. So, if you wanted to do extra work on hosting-task drush commands, you could create an example.drush.inc in your example.module, and then define your own drush_example_hosting_task function. Heck, you could define any of:

  1. drush_example_hosting_task_validate
  2. drush_example_pre_hosting_task
  3. drush_example_hosting_task
  4. drush_example_post_hosting_task

This is pretty exciting, as it opens up a lot of customization not only in Drush, but also in Aegir. I think as Aegir continues to mature, we're going to start seeing a bunch of modules which extend it in various ways.

Jul 07 2010
Jul 07

Alfresco wants to be a best-in-class repository for you to build your content-centric applications on top of. Interest in NOSQL repositories seems to be growing, with many large well-known sites choosing non-relational back-ends. Are Alfresco (and, more generally, nearly all ECM and WCM vendors) on a collision course with NOSQL?

First, let’s look at what Alfresco’s been up to lately. Over the last year or so, Alfresco has been shifting to a “we’re for developers” strategy in several ways:

  • Repositioning their Web Content Management offering not as a non-technical end-user tool, but as a tool for web application developers
  • Backing off of their mission to squash Microsoft SharePoint, positioning Alfresco Share instead as “good enough” collaboration. (Remember John Newton’s slide showing Microsoft as the Death Star and Alfresco as the Millenium Falcon? I think Han Solo has decided to take the fight elsewhere.)
  • Making Web Scripts, Surf, and Web Studio part of the Spring Framework.
  • Investing heavily in the Content Management Interoperability Services (CMIS) standard. The investment is far-reaching–Alfresco is an active participant in the OASIS specification itself, has historically been first-to-market with their CMIS implementation, and has multiple participants in CMIS-related open source projects such as Apache Chemistry.

They’ve also been making changes to the core product to make it more scalable (“Internet-scalable” is the stated goal). At a high level, they are disaggregating major Alfresco sub-systems so they can be scaled independently and in some cases removing bottlenecks present in the core infrastructure. Here are a few examples. Some of these are in progress and others are still on the roadmap:

  • Migrating away from Hibernate, which Alfresco Engineers say is currently a limiting factor
  • Switching from “Lucene for everything” to “Lucene for full-text and SQL for metadata search”
  • Making Lucene a separate search server process (presumably clusterable)
  • Making OpenOffice, which is used for document transformations, clusterable
  • Hiring Tom Baeyens (JBoss jBPM founder) and starting the Activiti BPMN project (one of their goals is “cloud scalability from the ground, up”)

So for Alfresco it is all about being an internet-scalable repository that is standards-compliant and has a rich toolset that makes it easy for you to use Alfresco as the back-end of your content-centric applications. Hold that thought for a few minutes while we turn our attention to NOSQL for a moment. Then, like a great rug, I’ll tie the whole room together.

NOSQL Stores

A NOSQL (“Not Only SQL”) store is a repository that does not use a relational database for persistence. There are many different flavors (document-oriented, key-value, tabular), and a number of different implementations. I’ll refer mostly to MongoDB and CouchDB in this post, which are two examples of document-oriented stores. In general, NOSQL stores are:

  • Schema-less. Need to add an “author” field to your “article”? Just add it–it’s as easy as setting a property value. The repository doesn’t care that the other articles in your repository don’t have an author field. The repository doesn’t know what an “article” is, for that matter.
  • Eventually consistent instead of guaranteed consistent. At some point, all replicas in a given cluster will be fully up-to-date. If a replica can’t get up-to-date, it will remove itself from the cluster.
  • Easily replicate-able. It’s very easy to instantiate new server nodes and replicate data between them and, in some cases, to horizontally partition the same database across multiple physical nodes (“sharding”).
  • Extremely scalable. These repositories are built for horizontal scaling so you can add as many nodes as you need. See the previous two points.

NOSQL repositories are used in some extremely large implementations (Digg, Facebook, Twitter, Reddit, Shutterfly, Etsy, Foursquare, etc.) for a variety of purposes. But it’s important to note that you don’t have to be a Facebook or a Twitter to realize benefits from this type of back-end. And, although the examples I’ve listed are all consumer-facing, huge-volume web sites, traditional companies are already using these technologies in-house. I should also note that for some of these projects, scaling down is just as important as scaling up–the CouchDB founders talk about running Couch repositories in browsers, cell phones, or other devices.

If you don’t believe this has application inside the firewall, go back in time to the explosive growth of Lotus Notes and Lotus Domino. The Lotus Notes NSF store has similar characteristics to document-centric NOSQL repositories. In fact, Damien Katz, the founder of CouchDB, used to work for Iris Associates, the creators of Lotus Notes. One of the reasons Notes took off was that business users could create form-based applications without involving IT or DBAs. Notes servers could also replicate with each other which made data highly-available, even on networks with high latency and/or low bandwidth between server nodes.

Alfresco & NOSQL

Unlike a full ECM platform like Alfresco, NOSQL repositories are just that–repositories. Like a relational database, there are client tools, API’s, and drivers to manage the data in a NOSQL repository and perform administrative tasks, but it’s up to you to build the business application around it. Setting up a standalone NOSQL repository for a business user and telling them to start managing their content would be like sticking them in front of MySQL and doing the same. But business apps with NOSQL back-ends are being built. For ECM, projects are already underway that integrate existing platforms with these repositories (See the DrupalCon presentation, “MongoDB – Humongous Drupal“, for one example) and entirely new CMS apps have been built specifically to take advantage of NOSQL repositories.

What about Alfresco? People are using Alfresco and NOSQL repositories together already. Peter Monks, together with others, has created a couple of open source projects that extend Alfresco WCM’s deployment mechanism to use CouchDB and MongoDB as endpoints (here and here).

I recently finished up a project for a Metaversant client in which we used Alfresco DM to create, tag, secure, and route content for approval. Once approved, some custom Java actions deploy metadata to MongoDB and files to buckets on Amazon S3. The front-end presentation tier then queries MongoDB for content chunks and metadata and serves up files directly from Amazon S3 or Amazon’s CloudFront CDN as necessary.

In these examples, Alfresco is essentially being used as a front-end to the NOSQL repository. This gives you the scalability and replication features on the Content Delivery tier with workflow, check-in/check-out, an explicit content model, tagging, versioning, and other typical content management features on the Content Management tier.

But why shouldn’t the Content Management tier benefit from the scalability and replication capabilities of a NOSQL repository? And why can’t a NOSQL repository have an end-user focused user interface with integrated workflow, a form service, and other traditional DM/CMS/WCM functionality? It should, it can and they will. NOSQL-native CMS apps will be developed (some already exist). And existing CMS’s will evolve to take advantage of NOSQL back-ends in some form or fashion, similar to the Drupal-on-Mongo example cited earlier.

What does this mean for Alfresco and ECM architecture in general?

Where does that leave Alfresco? It seems their positioning as a developer-focused, “Internet-scale” repository ultimately leads to them competing directly against NOSQL repositories for certain types of applications. The challenge for Alfresco and other ECM players is whether or not they can achieve the kind of scale and replication capabilities NOSQL repositories offer today before NOSQL can catch up with a new breed of Content Management solutions built expressly for a world in which content is everywhere, user and data volumes are huge and unpredictable, and servers come and go automatically as needed to keep up with demand.

If Alfresco and the overwhelming majority of the rest of today’s CMS vendors are able to meet that challenge with their current relational-backed stores, NOSQL simply becomes an implementation choice for CMS vendors. If, however, it turns out that being backed by a NOSQL repository is a requirement for a modern, Internet-scale CMS, we may see a whole new line-up of players in the CMS space before long.

What do you think? Does the fundamental architecture prevalent in today’s CMS offerings have what it takes to manage the web content in an increasingly cloud-based world? Will we see an explosion of NOSQL-native CMS applications and, if so, will those displace today’s relational vendors or will the two live side-by-side, potentially with buyers not even knowing or caring what choice the vendor has made with regard to how the underlying data is persisted?

Mar 23 2010
Mar 23

The Idea

A number of months back, a group of us had the idea to create a software co-operative. There were several tenets that we decided to follow:

  • The company wouldn't have any employees -- everybody involved would be have 1099 status and would be an independent contractor
  • The company would be formed as an Limited Liability Company - we chose the state of Delaware
  • The company would seek to have a $0 cash and asset value at the end of each year
  • The company would be virtual to keep costs low
  • We would focus on working with open sourced projects like Drupal

The Setup

We set the company up using Instacorp. The benefit was the speed at which we could set up the company with an automatic legal presence in Delaware. The people at Instacorp made the process incredibly simple, asking a few questions. Within days the legal documents were delivered. That, in itself, really didn't make the company real.

After receiving the legal documents, it was necessary to obtain an FEIN for tax purposes. This is a simple process on the IRS site - it just takes a few minutes and you get the documentation electronically.

We needed to decide how to be taxed.
LLC's report taxes in one of 3 ways:

  • Disregarded entity (limited to one member LLC's)
  • Partnership (default if other elections are not made)
  • Corporation, electing to be taxes as pass-through entity, called S corporations.

In the case of Vintage Digital, workers are paid for what they do which is reported to them on FORM 1099 as commissions.
Great variation will occur in compensation since it is entirely based on hours worked and percentages for those who find clients and shepherd clients through the contracting process. The LLC is a virtual corporation, with exceeding low or non-existent overhead. There is no intent to use the LLC other than as a distribution method for sharing work; profits/losses will be kept to a minimum. There is no intent to hold fixed assets or incur debt.

In the final analysis both partnership and S-corporation reporting would be the same.

Our accountant indicated that the following things were recommended:

  • Use "S" corporation for tax reporting because the laws are better understood and simpler.
  • Majority of LLC's elect to report taxes as a pass-through corporations, hence even the IRS is more familiar with these tax laws.

We had an S-corp election to indicate how we were going to be taxed - after the election that document needs to be sent to the IRS.

We needed a bank account and opted for a bank that had free business checking and had online bill pay. The bank required our Articles and two forms of identification. We also needed a copy of our FEIN letter from the IRS.

The Tools
All companies need tools to help run things on a day to day basis. A virtual venture is no different. We needed management tools, communication tools, invoicing/book keeping software, and ways to manage contracts. To that end we sought out different solutions that would provide us with ways to sensibly manage ourselves and our projects.

  • Skype for Communication (both voice and chat)
  • Open Atrium for a client intranet and as an internal planning tool.
  • Bamboo Invoice for invoicing clients (although that might change as we transition to QuickBooks)
  • Drupal for our Web presence
  • dotProject for time record keeping and ticketing
  • Office for estimates, calculating commission shares, and contracts
  • Google Voice for incoming phone calls - the rest of us use our own mobiles

The Team

The team is comprised of:

All of us have (and continue to) contribute to the Drupal Project and are heavily involved in our local communities.

The team keeps in pretty much constant contact through Skype. We try to meet about once a month together to have Member/Board meetings. They have occurred in restaurants for brunch, member's homes, and also at a bowling alley - pretty much anywhere that is quiet and you can get through company issues. Fortunately for our crew, we all live with 30 miles or so of one another which makes getting together fairly easy.

Each project gets its own Skype room, project in Open Atrium, project in dotProject, and commission spreadsheet.

As clients come in, we assess who has bandwidth for a given project - the goal ensuring that each co-op member has enough work (in and outside of the co-op) to make a reasonable living. Co-op members are free to work as much or as little as they want (given the work is available). This arrangement was designed to give our team as much flexibility as possible.

Mar 22 2010
Mar 22

A lot of people have been asking for the files we used to integrate Alfresco CMIS with Drupal Open Atrium (See ecmarchitect.com blog post). I’ve happily mailed those to whomever asked. I’ve had the intention of testing them with the latest version, cleaning them up, and putting somewhere more appropriate like the Open Atrium feature server, or at the very least, Google Code or GitHub. But it hasn’t happened yet so I figured I’d make them available here and appeal to the Community to give them a good home.

The zip includes a readme file with (very) rough install/config directions.

Good luck!

Feb 16 2010
Feb 16

Fellow Optaros colleague, Chris Fuller, and I want to present on the Alfresco-Drupal integration at Drupalcon in San Francisco (April 19-21). If you’re interested in Alfresco, Drupal, and CMIS (any or all of the above), please vote for our session.

Jan 07 2010
Jan 07

Recently I had the privilege of working on the conversion of Novus Biological's website to Drupal 6, selling scientific supplies using Ubercart. I did this as a freelancer for the talented folks at SpireMedia, working with them as well as with fellow Vintage Digital member Ben Jeavons. It was a fantastically intense experience.

Ubercart's a great system out of the box, but there were a number of features on the site which made this implementation difficult.

  1. There were about 70,000 products, which is probably two or three orders of magnitude greater than the usual Ubercart site.
  2. The site needed to work with three currencies, and four prices, depending on the user's location or selection. Ubercart doesn't handle this well.
  3. The site needed to have two kinds of discounts - ones available to most any user, and ones only available to users who had bought the discount with userpoints.

I'll probably write about all of these in time, but at the moment I want to comment on that second item.

A close-up of the novusbio website, showing the region-change text
If you take a look at www.novusbio.com, or the image above, you'll see a bit of text in the upper right hand corner which says something like 'US site', or 'Europe Site', or 'Great Britain site', or 'World Site', with a little arrow next to it letting you change your region. We would start by pre-selecting a region for the user based on which IP they were browsing from, and then users could change it. If you go to a product page on Novus' site, you'll see different prices based on which of these are currently selected, as well as different currency signs. Since doing something like this is what some people would like to be able to do, I wanted to take a little time to go over how we did this.

Note: If you're not a coder, and you need this functionality, then you're going to want to go find a coder to handle this for you. It's not currently possible to do by flipping a switch or installing a module. I'm going over the changes you need to make here, but the full set of changes, along with the needed patches to Ubercart, are contained in uc_example.zip, attached to this post.

To start with, you're going to need to define the regions or currencies you're doing prices for. I use the term regions, because you could well have two different areas of the world using the same symbol but different price amounts. So, I use a define to mark which regions I'm keeping track of.

define('UC_EXAMPLE_DOLLAR', 1);
define('UC_EXAMPLE_EURO', 2);
?>

There are basically two things you want to handle when you allow for more than one currency in Ubercart:

  1. You want to change the currency symbol.
  2. You want to change the numeric amount of the price. You probably won't want to have a factor that you apply to the base price - I would rather suggest setting up alternate prices as CCK fields on your product nodes, much the same way that books in the US have set US and Canadian prices printed on them.

(As an aside, although the changes I'm discussing will allow you to have, say, both US and Euro prices on your site, you'll also at some point need to go through and write up some new reports that are multi-currency aware. After all, a report that says you sold 54,382 in the last two months doesn't mean anything if 24,382 was in US Dollars and 30,000 was in Euros. And, if you do up some reports like this, why not share them with the rest of us?)

There's two different places where you need to keep track of the current currency type as well - for when a user is browsing the site, or putting together an order, or reviewing her cart, or whichever, you'll want to store the current currency type in the session. On the other hand, if you're viewing an order - either in the very last step before the order is committed, (where a row has been written to uc_orders) or when viewing an order in the history, you need to have stored the currency type in the order data itself.

Changing the amount/magnitude of the item

Now, the slightly simpler part is changing the amount of the charge - if you're looking at $360, then we're talking about the 360 part of it. This amount needs to change on cart items or on products when they're viewed. We don't need to worry about changing the amount on the items in completed orders, because that amount quite sensibly is fixed when the order is committed.

I'm not going to go over every change you need to make - you might need to do a little work in your views to decide which price to use when displaying a view (indeed, views_customfield may be a good idea for that), but when you're changing how the product is displayed, you're basically doing a little work with hook_nodeapi:

/**
* Implements hook_nodeapi().
*/
function uc_example_nodeapi(&$node, $op, $a3 = NULL, $a4 = NULL) {
  if (
$op == 'view') {
   
$region_price = uc_example_get_node_price($node, $_SESSION['region']);
   
   
$context['class'][1] = 'sell';
   
$context['field'] = 'sell_price';
   
$node->content['sell_price']['#value'] = theme('uc_product_price', $region_price, $context, array('label' => !$a3));
   
   
$context['class'][1] = 'display';
   
$context['field'] = 'sell_price';
   
$node->content['display_price']['#value'] = theme('uc_product_price', $region_price, $context);

  }
}

/**
* Returns the price from the node, from the region passed in.
*
* @param stdClass $node Product node to get price for.
* @param integer $current_region Region to get the price for.
*/
function uc_example_get_node_price($node, $current_region) {
  switch (
$current_region) {
    case
UC_EXAMPLE_DOLLAR:
      return
$node->field_price_us[0]['value'];
    case
UC_EXAMPLE_EURO:
      return
$node->field_price_euro[0]['value'];
    default:
      return
0;
  }
}
?>

When viewing the price in a person's cart, we'll want to use hook_cart_item($op, $item) to change the price for products.

/**
* Implements hook_cart_item()
*
* This lets us change the base price based on the user's current region.
*
* See http://www.ubercart.org/docs/api/hook_cart_item
* @param String $op     Operation being done
* @param stdClass $item Cart item - usually a node of some sort.
*/
function uc_example_cart_item($op, &$item) {

  </span>//dpm('hook_cart_item '. $op);
 
if ($op == 'load') {
    global
$user;
   
   
$product_node = node_load($item->nid);

    </span>//dpm($_SESSION);
   
$region_id = 0;
   
// If we're viewing the cart item on the review page, we want to use the region
    // that's embedded in the order item, not the region that the user is using.
   
if ($_GET['q'] == 'cart/checkout/review') {
     
$order = db_fetch_object(db_query(
       
"SELECT uo.order_id, IFNULL(uom.region, 1) as mc_region,
        IFNULL(uom.currency_sign, '$') AS mc_currency_sign
        FROM {uc_orders} uo
        INNER JOIN {uc_order_products} uop ON (uo.order_id = uop.order_id)
        LEFT OUTER JOIN {uc_order_multicurrency} uom ON (uo.order_id = uom.order_id)
        WHERE uid = %d and order_status = '%s' AND uop.nid = %d ORDER BY uo.order_id DESC"
,
       
$user->uid, 'in_checkout', $item->nid
     
));
      if (
$order) {
       
$current_region = $order->mc_region;
      }
      else {
       
$current_region = $_SESSION['region'];
      }
    }
    else {
     
$current_region = $_SESSION['region'];
    }

    if (</span>$product_node) {
     
$item->price = uc_example_get_node_price($product_node, $current_region);
    }
  }
}
?>

Changing the Currency Symbol

With that done, lets consider the currency sign. I like storing the user's currently selected symbol in $_SESSION['currency_sign'], and we'll also be storing it in orders as well. The first impulse you might have would be to start mucking about with variable_set('uc_currency_sign') - but changing that changes the symbol for everything at once, and you're no doubt hoping to have more than one customer at a time.

Instead you want to register a price handler with hook_uc_price_handler. Then, whenever a price is being determined by uc_price(), your price alteration function can determine the proper currency sign to use, and provide it.

/**
* All we're doing with this price alteration function, is telling it to use for the
* currency sign the sign that we've stashed in $_SESSION.
*/
function _uc_example_price_alter(&$price_info, $context, &$options) {
 
// default currency sign.
 
$options['sign'] = $_SESSION['currency_sign'];
  if (isset(
$context['subject']['order']) && $context['subject']['order']->order_id) {
   
$order = $context['subject']['order'];
    if (isset(
$order->mc_currency_sign)) {
     
$options['sign'] = $order->mc_currency_sign; // use sign from order!
   
}
    else {
     
$currency_sign = db_result(db_query("SELECT currency_sign FROM {uc_order_multicurrency} WHERE order_id = %d", $order->order_id));
      if (
$currency_sign) {
       
$options['sign'] = $currency_sign;
      }
    }
  }
}
?>

What I'm doing there is setting $options['sign'] - which is an array passed in by reference - and changing it to the sign we want it to have. First we set it by the $_SESSION, and then we check and see if the context - a collection of information about what this price is and where it's being presented - contains the order which this price is a part of. If it is a part of an order, then we use the sign saved as part of that order.

If you're wondering how this information gets to be part of the order, unsurprisingly the answer is we implement hook_order in our code.

/**
* Implements hook_order().
*
* This is where we finalize the region_id and currency_sign.
*/
function uc_example_order($op, &$arg1, $arg2) {

  if (</span>$op == 'new') {
   
// $arg1 is a reference to the order object.

    </span>$arg1->mc_currency_sign = $_SESSION['currency_sign'];
   
$arg1->mc_region = $_SESSION['region'];
  }
  if (
$op == 'save' && $_GET['q'] == 'cart/checkout') {
   
// Now we need to save our updated region data to the order!
   
if (db_result(db_query("SELECT count(*) FROM {uc_order_multicurrency} WHERE order_id = %d", $arg1->order_id))) {
     
db_query("UPDATE {uc_order_multicurrency} SET region = %d, currency_sign = '%s' WHERE order_id = %d",
              
$_SESSION['region'], $_SESSION['currency_sign'], $arg1->order_id);
    }
    else {
     
db_query("INSERT INTO {uc_order_multicurrency} (order_id, region, currency_sign) VALUES (%d, %d, '%s')",
              
$arg1->order_id, $_SESSION['region'], $_SESSION['currency_sign']);
    }
  }
  if (
$op == 'load') {
   
$multicurrency = db_fetch_array(db_query("SELECT region, currency_sign FROM {uc_order_multicurrency} WHERE order_id = %d",
                                            
$arg1->order_id));
   
$arg1->mc_region = $multicurrency['region'];
   
$arg1->mc_currency_sign = $multicurrency['currency_sign'];
  }
}
?>

And... then this is where we get to the unfortunate part. Remember that price handler above, where we pull the order information out of the context? Unfortunately, Ubercart is really inconsistent about providing us that needed information, even when the call to uc_price is happening in a function where $order is already just sitting there, all loaded up. So, I'd like to introduce these patch files, which are provided to you inside of uc_example.zip:

  • ubercart-621494.patch
  • ubercart-display-symbols.patch
  • ubercart-order-price.patch

These three patch files need to be applied to Ubercart, and force it to provide the $order as part of the $context when uc_price is being called. If you look through these files, you'll see that most of the time this $order object is already there - I'm just adding it to the $context['subject'] array. (Unfamiliar with patches? Copy the three files to the Ubercart directory, change your directory to that Ubercart directory, and then execute patch -p0 < {filename} for each one.)

Enclosed is a module that contains the changes I've mentioned above. It assumes that you've used cck to add a 'price_us' and 'price_euro' field to the product nodes, but otherwise you can use it as is, plug it into a test Ubercart installation, run, and try it out. The patches are only necessary when viewing a finalized order, particularly in the admin area, so you don't even need to use that when just experimenting with the code.

AttachmentSize uc_example.zip7.1 KB
Oct 13 2009
Oct 13

UPDATE: Screencast now lives here:

[embedded content]

I recorded a quick screencast of a simple integration we did to show Open Atrium leveraging Alfresco as a formal document repository via CMIS. This leverages the CMIS Alfresco module we developed and released on Drupal.org.

As I point out in the screencast, there’s not much to the integration from a technical standpoint. Open Atrium is Drupal and the CMIS module already has a CMIS repository browser. So, all we had to do was expose the module as a “feature”, which is something Open Atrium uses to bundle modules together that create a given chunk of functionality.

Readers familiar with Alfresco Share will instantly recognize the Open Atrium concepts. Instead of “sites” Atrium uses “groups”. Instead of “pages” or “tools”, Atrium uses “features”. The overall purpose, self-provisioned team-based collaboration, is the same and many of the tools/features are the same (blog, calendar, member directory). I’m not advocating using one over the other–as usual, what works best for you depends on a lot of factors. I just thought Atrium provided a nice way to show yet another example of Drupal and Alfresco together (post).

Sep 15 2009
Sep 15

People want intranets that are fun and easy to use, full of compelling content relevant to their job, and enabled with social and community features to help them discover connections with other teams, projects, and colleagues. IT wants something that’s lightweight and flexible enough to respond to the needs of the business that won’t cost a fortune.

That’s why Drupal + Alfresco is a great combination for things like intranets like the one Optaros built for Activision and why we had a record-breaking turnout for the Drupal + Alfresco webinar Chris Fuller and I did today. Thanks to everyone who came and asked good questions. I’ve posted the slides. Alfresco recorded the webinar so they’ll make it available soon, I’m sure. When that happens, I’ll update the post with a link. Until then, enjoy the slides.

[UPDATE: Fixed the slideshare link (thanks, David!) and added the links to the webinar recording below]

1. Streaming recording link:
https://alfresco.webex.com/alfresco/lsr.php?AT=pb&SP=TC&rID=42774837&act=pb&rKey=b44130d69cc9ec5f

2. Download recording link:
https://alfresco.webex.com/alfresco/ldr.php?AT=dw&SP=TC&rID=42774837&act=pf&rKey=c50049ac82e1220a

Apr 28 2009
Apr 28

I’ll be in Chicago tomorrow for the Alfresco Meetup. I’ll be speaking during the Barcamp on Alfresco and Drupal integration with CMIS (module, screencast). I’ll also have the Alfresco-Django integration running on my laptop. I may not have time to show Alfresco-Django during my slot, but I’ll be happy to stick around and do informal demos and talk about either integration if you’re interested because I’d like your feedback on it.

Apr 20 2009
Apr 20

People often need to build a custom user interface on top of the Alfresco repository and I see a lot of people asking general questions about how to do it. There are lots of options to consider. Here are four options for creating a user interface on top of Alfresco, at a high level:

Option 1: Use your favorite programming language and/or framework to talk to Alfresco via REST or Web Services. PHP? Python? Java? Flex? Whatever, it’s up to you. The REST API is nice because if you can’t find a URL that does what you need it to out-of-the-box, you can always roll-your-own with the web script framework. This option offers the most flexibility and creative freedom, but of course you might end up building constructs or components that you may have gotten “for free” from a higher-level framework. Optaros‘ streamlined web client, DoCASU, built on Ext-JS, is one freely-available example of a custom UI on top of Alfresco but there are others.

Option 2: Use Alfresco’s Surf framework. Alfresco’s Surf framework is just that–it’s a framework. Don’t confuse it with Alfresco Share which is a team-centric collaboration client built on top of Surf. And, don’t assume that just because a piece of functionality is in Share it is available to you in the lower-level Surf framework. You may have to do some extra work to get some of the cool stuff in Share to work in your pure Surf app. Also realize that Surf is brand new and still maturing. You’ll be quickly disappointed if you hold it to the same standard as a more widely-used, well-established framework like Seam or Django. Surf is a good option for quick, Alfresco-centric solutions, especially if you think you might want to leverage Alfresco’s browser-based site assembly tool, Web Studio, at some point in the future. (See Do-it-yourself Alfresco Surf Code Camp).

Option 3: Customize the Alfresco “Explorer” web client. There are varying degrees to which you can customize the web client. On one end of the spectrum you’ve got Freemarker “presentation templates” followed closely by XML configuration. On the other end of the spectrum you’ve got more elaborate enhancements you can make using JavaServer Faces (JSF). Customizing the Alfresco Explorer web client should only be considered if you can keep your enhancements to an absolute minimum because:

  1. Alfresco is moving away from JSF in favor of Surf-based clients. The Explorer client will continue to be around, but I wouldn’t expect major efforts to be focused on that client going forward.
  2. JSF-based customizations of the web client can be time-consuming and potentially complex, particularly if you are new to JSF.
  3. For most solutions, you’ll get more customer satisfaction bang out of your coding buck by building a purpose-built, eye-catching, UI designed with your specific use cases in mind than you will by starting with the general-purpose web client and extending from there.

Option 4: Use a portal, community, or WCM platform. This includes PHP-based projects like Drupal (Drupal CMIS Screencast) or Joomla as well as Java-based projects like Liferay and JBoss Portal. This is a good option if you have requirements that match up well with the built-in (or easily added-on) capabilities of those platforms.

It’s worth talking about Java portal servers specifically. I think people are struggling a bit to find The Best Way to integrate Alfresco with a portal. Of course there probably is no single approach that will fit every situation but I think Alfresco (with help from the community) could do more to provide best practices.

Here are the options you have when integrating with a portal:

Portal Option 1: Configure Alfresco to be the replacement JSR-170 repository for the portal. This option seems like more trouble than it is worth. If all you need is what you can get out of JSR-170, you might as well use the already-integrated Jackrabbit repository that most open source portals ship with these days unless you have good reasons not to. I’m open to having my mind changed on this one, but it seems like if you want to use Alfresco and a portal, you’ve got bigger plans that are probably going to require custom portlets anyway.

Portal Option 2: Run Alfresco and the portal in the same JVM (post). This is NOT recommended if you need to scale beyond a small departmental solution and, really, I think with the de-coupling of the web script engine we should consider this one deprecated at this point.

Portal Option 3: Run the Alfresco web script engine and the portal in the same JVM. Like the previous option, this gives you the ability to write web scripts that are wrapped in a portlet but it cuts down on the size of the web app significantly and it frees up your portal to scale independently of the Alfresco repository tier. It’s a fast development cycle once you get it set up. But I haven’t seen great instructions for setting it up yet. Alfresco should document this on their wiki if they are going to support this pattern.

Portal Option 4: Write your own portlets that make services calls. This is the “cleanest” approach because it treats Alfresco like any other back-end you might want to integrate with from the portal. You write custom portlets and have them talk to Alfresco via REST or SOAP. You’ll have to decide how you want to handle authentication with Alfresco.

What about CMIS?

CMIS fits under the “Option 1: Use your favorite programming language” and “Portal Option 4: Write your own portlets” categories. You can make CMIS calls to Alfresco using both REST and SOAP from your own custom code, portlet or otherwise. The nice thing about CMIS is that you can use it to abstract the underlying repository so that (in theory) your front-end code will work with different CMIS-compliant back-ends. Just realize that CMIS isn’t a fully-ratified standard yet and although a CMIS implementation is in the Enterprise version of Alfresco, it isn’t clear to me whether or not you’d be supported if you had a problem. (The last response I saw on this specific question was a Peter Monks tweet saying, “I don’t think so”).

The CMIS standard should be approved by the end-of-the-year and if Alfresco’s past performance is an indicator of the future, they’ll be the first to market with a production-ready, fully-supported CMIS implementation based on the final spec.

Pick your poison

Those are the options as I see them. Each one has trade-offs. Some may become more or less attractive over time as languages, frameworks, and the state of the art evolve. Ultimately, you’re going to have to evaluate which one fits your situation the best. You may have a hard time making a decision, but you have to admit that having to choose from several options is a nice problem to have.

Dec 15 2008
Dec 15

We needed to disable all of Drupal’s CSS files from our theme. Here’s how we did it:

function THEMENAME_preprocess(&$variables) {

  // Get rid of all of Drupal's CSS files
  $css = drupal_add_css();
  foreach ($css['all']['module'] as $file => $status) {
    if (!strstr($file, 'modules/MYMODULE')) {
      unset($css['all']['module'][$file]);
    }
  }
  $variables['styles'] = drupal_get_css($css);

We also wanted (no *real* need) to use screen.css rather than style.css, so we edited THEMENAME.info to have this:

stylesheets[all][] = reset.css
stylesheets[all][] = screen.css

… and we removed the line for style.css from it.

Finally, as the superuser we went to /admin/build/modules (or /themes, can’t remember now) to refresh the theme cache. We also had to tick to enable the theme at /admin/build/themes as although we’d been using the theme for ages quite fine, it wasn’t actually ticked before.

And hey presto, it worked. Should probably add that it took waaaay too long to do though, so though we’d add this snipped for others to read.

Share this:

Like this:

Like Loading...
Sep 11 2008
Sep 11

Lately, I’ve had various frustrations with Drupal which have moved me away from using it for various things. I’d like to go through where I’ve moved away from Drupal, why I’ve made those changes, and my future Drupal decisions.

WordPress rather than Drupal blog

To begin with, this blog is now on WordPress rather than Drupal – and I have to say that I’m loving it… and so are my non-geeky colleagues. It ticks all the right boxes. Its *really* user friendly. Its much easier to add photos (and videos) to posts. And it’s hardly taken long at all to setup.

So where did Drupal go wrong with this? Well, I guess its the ‘kitchen sink’ approach back-firing. In trying to be something to everyone, Drupal runs the risk of being less than perfect to to any one specific task too. WordPress, on the other hand, has the ability to focus on being the very best blogging software out there, and nothing else gets in the way of that or deters it from its ultimate goal.

But also, we needed to seaparate the business end of our website (the creation of yearbooks, using tens of thousands of nodes) from the nodes and users to do with the blog, partly because around this time every year we flush out the old site and start a new. So we were either going to have a separate Drupal install for the blog or use WordPress. We chose WordPress.

Form theming frustrations

Have a look at the form here and let me know how you’d create it using FAPI (we’re on Drupal 6.4). I’m talking specifically about the theming of the form. Yes, its a very simple form. A search form. But let’s have a look at what’s going on with it and discuss the Drupal way versus the way we ended up doing it.

In FAPI, you’d probably have a ‘textfield’ element for the search box and a ‘submit’ element for the ‘Go’ button. Easy enough, one minute of code. But what about the title ‘Name of your school or group’? Probably a title for the textfield element, no? But then how do we get it centred above both the textfield and the submit button? And what about the text under the two fields? A description, right? Again… how do we get it to appear *exactly* where we want it? The look and feel of the form are to me absolutely crucial. I don’t just want a textfield’s title (with annoying colon after it), a textfield, a description, and then a Go button all one on top of the other.

The Drupal solution? Using a theme hook. We define the form in our implementation of hook_form() and then theme that form separately. The ‘programmer’ cares about the functionality of the form but not the visual design of the form. The ‘designer’ doesn’t care about the functionality and instead works on how it looks. But I’m both the programmer and the designer here, and I want my work to be as easy as possible! So, let’s say i go down this route (I tried, I really did). I need to register my theme function in hook_themes(). Okay, I know why, it saves extra code running on every page load. But its still annoying. Now I create my theme function… urgh. You’ve got to really know your FAPI stuff to get this to work. I try for a while but then I give up. It just feels so messy with some code somewhere, some in another place, and then when I ask my colleague to have a look so he can learn how to do it he’s disgusted… doesn’t know what’s going on… starts bad-mouthing Drupal. So we build our own massively simple FAPI instead in about half an hour that does just what we want it to do.

So now we’re using our own super-basic FAPI for this form. Not all forms, just the ones we want complete control over, visually. Rather than using hook_form() and defining a form array, we just hard-code the HTML for the form. Some of you may be in complete horror now thinking about this but its just by far and away the easiest way to get forms to do exactly what you want. Like a forename and surname field side by side rather than one on top of the other, sharing the title ‘Name’ which is a label for the forename field.

We’re sticking with the idea of a validate() and submit() hook though. I like that one. But we’re doing it slighly differently and more simply, so that any new coders we might hire can quickly and easily pick it up.

Going nodeless

I don’t always like nodes. I really, really don’t. I don’t need revisions, and if I did I’d do them in my own way, just the right way for me, rather than a way that kind of works for everyone but not quite perfectly for anyone in specific. I don’t like the way that as uid=1 I get all the extra bits like ‘Authoring information’ which I never touch. Node hooks and the nodeapi which I once loved are now a higgeldy-piggledy mess that’s a real pain for my new hire. I try to explain to him what’s going on when I save a node. “So this function deals with the submission. But not the core node stuff, Drupal deals with that. And if we want something used for all nodes, we put it in here instead. And we can also override this specific bit here.”… he looks on in amazement, totally baffled by what’s going on and why. It would be so much easier for him (and me) to understand if everything’s just in one place.

So what do we gain from nodes? Umm… not much really. We don’t use contributed modules any more because they never do exactly what I want and always do stuff which I don’t want them doing which just make them less efficient. We put all our code in our own one module instead. A massive, hefty module with a dozen or so include files.

We gain the ability to always do $node->nid and use node_save() and other handy things. But we don’t really need nodes, and it frustrates me having to do the extra INNER JOINs on node_revisions etc. So we’re trialing not using nodes at all for one of our content types – our customers. We just have a simple ‘id’ field now in one single table. We no longer need to INNER JOIN node and node_revisions. We haven’t had any problems so far, but the new hire is finding it much easier to code now, without the ‘baggage’ of Drupal.

The future

Our current plan is to gently migrate away from Drupal, perhaps altogether. We like the idea of building our own framework again, one that does exactly what it needs to do for our site. Its not something we can do overnight. Ours is a yearly cycle, following the academic year, and the current plan is to fork the codebase in around January/February and that would mark the beginning of our own framework if we still feel that way then.

In the meantime, we’ll continue using nodes for most of our content types (if simply because migrating away from them would be a long and arduos task with little reward) and we’ll continue to use FAPI for most our forms. But I see us using our own simple FAPI for more and more forms where we need complete control over them, and I see us extending this FAPI to help us reduce using the same code multiiple times.

I think I still like Drupal. I definitely appreciate the vibrant community. But sometimes I thoroughly hate Drupal and get massively frustrated by it. But I still like it in theory at least. One framework for all my websites. But whilst I just work on one massive website it just has so much less use to me.

You’re more than welcome to urge me to stay with Drupal. In fact, I highly hope someone can manage this. I’ve put a lot of time over the last few years into learning Drupal, and it would be a great waste and a shame to lose all that.

Share this:

Like this:

Like Loading...
Nov 15 2006
Nov 15

Tailing my error log, I kept coming across annoying errors like this:


[Mon Nov 13 21:14:49 2006] [error] [client xx.xxx.xxx.xxx] PHP Warning: mysql_real_escape_string() expects parameter 1 to be string, array given in /path/to/drupal/includes/database.mysql.inc on line 350, referer: http://www.example.com/node/1234/edit

No matter how hard I tried, I couldn’t reproduce the errors locally, but somehow real users could create them on the live server. I tried to track down the bug but could only go so far – yes, it was happening when a node was edited, and it was in a database query, but which query? There were too many to look at, so I needed more information.

Because I couldn’t reproduce the bug locally, no amount of dprint_r() and such had any effect. Following some advice in #drupal I explored debug_backtrace() and added its output to a custom error log like this:


function mymodule_error($errno, $errstr, $errfile, $errline, $errcontext) {

if (in_array($errno, array(ERR_LOW, E_NOTICE, E_STRICT))) {
$level = false;
}
elseif (in_array($errno, array(ERR_MED, E_WARNING))) {
$level = ‘WARNING’;
}
elseif (in_array($errno, array(ERR_HI, E_ERROR))) {
$level = ‘FATAL’;
}
else $level = ‘UNKNOWN’;

if ($level) {
// don’t log unimportant errors

$functions = array();
foreach (debug_backtrace() AS $errors) {
$functions[] = $errors[‘function’] . ‘:’ . $errors[‘line’];
}
$trace = join(‘ < = ', $functions);

error_log($level . ' ' . $errfile . ':' . $errline . '. ' . $errstr . '. (' . $trace . ')');
}
}
set_error_handler('mymodule_error');

I put the code in a module file, and it didn’t work… an hour of hair pulling later I found the culprit to be devel.module:

function devel_init() {
restore_error_handler();
}

… this was undoing my error handling setting, so I had to comment out the restore_error_handler() line and my error handler took centre stage.

With the new code live, I just waited a few hours until the bug showed up again – this time, with extra backtrace information to help me debug:


[Tue Nov 14 18:13:14 2006] [error] [client xx.xxx.xxx.xxx] WARNING /path/to/drupal/includes/database.mysql.inc:350. mysql_real_escape_string() expects parameter 1 to be string, array given. (mymodule_error: < = mysql_real_escape_string:350 <= db_escape_string:152 <= _db_query_callback: <= preg_replace_callback:196 <= db_query:1040 <= my_custom_function:659 <= my_nodetype_update:297 <= node_invoke:494 <= node_save:1876 <= node_form_submit: <= call_user_func_array:206 <= drupal_submit_form:132 <= drupal_get_form:1620 <= node_form:2093 <= node_page: <= call_user_func_array:418 <= menu_execute_active_handler:15), referer: http://www.example.com/node/1234/edit

So now I knew exactly where the bug was coming from. ‘db_query’ was called on line ‘1040’ of my file, and of the several ways to get into the function in which line 1040 resides, I knew it had come form my_custom_function on line 659.

Armed with this extra information, tracking down the bug was a breeze. Stupidly, I’d forgotten to do a regular expression I was supposed to do on a variable before doing a db_query, meaning arrays which were supposed to be left alone were wandering into the query pretending to be strings.

Share this:

Like this:

Like Loading...
Nov 13 2006
Nov 13

In our effort to give something back to Drupal (the fantastic open source content management framework at the core of AllYearbooks), we’re aiming to make a physical yearbook for members of the worldwide Drupal community.

Presuming all goes to plan and enough people join, the idea is to produce physical copies of the yearbook (as well as allowing PDF downloads). These will be provided free to Dries and the top 10 Drupal contributors. It may also be possible for other interested members to buy a copy, though this book isn’t being run as a money-making venture.

If you’re a Drupal developer or involved in Drupal in some other way, join the yearbook now. It only takes a few seconds to join, and you can come back any time before printing to update your yearbook entry and upload some gallery photos for collages and other such pages.

So far 8 Drupalites have joined the yearbook. We’re hoping for at least 100 members for it to be worth printing.

If you have any suggestions, please get in touch.

Share this:

Like this:

Like Loading...

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web