May 18 2016
May 18
teaser image for blog post

Members of the British Council Digital team were delighted to receive the RITA2016 award last Thursday for the huge change in IT cloud infrastructure that Ixis delivered in the summer of 2015.

The award for "Infrastructure as an Enabler" reflected the innovative change in the way the British Council undertook their hosting requirement for the initial 120 sites operating in over 100 countries across the globe and delivering a clear business benefit. Moving away from dedicated infrastructure to virtual containers provided the ability to tightly control and guarantee server resources to an individual site and a quick and easy way to duplicate an environment for QA, testing, staging and feature branch development.

Ixis partnered with Drupal container expert Platform.sh to provide the underlying infrastructure and API. We'll publish further detail on our integration as a case study.

Congratulations also go another of our clients: Westminster City Council, for their award and further two highly commended positions in this years awards.

Photo courtesy of Chaudhry Javed Iqbal on Twitter.

May 18 2016
May 18
teaser image for blog post

In this blog post I'll discuss some methods of ensuring that your software is kept up to date, and some recent examples of why you should consider security to be among your top priorities instead of viewing it as an inconvenience or hassle.

Critics often attack the stability and security of Open Source due to the frequent releases and updates as projects evolve through constant contributions to their code from the community. They claim that open source requires too many patches to stay secure, and too much maintenance as a result.

This is easily countered with the explanation that by having so many individuals working with the source code of these projects, and so many eyes on them, potential vulnerabilities and bugs are uncovered much faster than with programs built on proprietary code. It is difficult for maintainers to ignore or delay the release of updates and patches with so much public pressure and visibility, and this should be seen as a positive thing.

The reality is that achieving a secure open source infrastructure and application environment requires much the same approach as with commercial software. The same principles apply, with only the implementation details differing. The most prominent difference is the transparency that exists with open source software.

Making Headlines

Open Source software often makes headlines when it is blamed for security breaches or data loss. The most recent high profile example would be the Mossack Fonseca “Panama Papers” breach, which was blamed on either WordPress or Drupal. It would be more accurate to blame the firm itself for having poor security practices, including severely outdated software throughout the company and a lack of even basic encryption.

Mossack Fonseca were using an outdated version of Drupal: 7.23. This version was released on 8 Aug 2013, almost 3 years ago as of the time of writing. That version has at least 25 known vulnerabilities. Several of these are incredibly serious, and were responsible for the infamous “Drupalgeddon” event which led to many sites being remotely exploited. Drupal.org warned users that “anyone running anything below version 7.32 within seven hours of its release should have assumed they’d been hacked”.

Protection by Automation

Probably the most effective way to keep your software updated is to automate and enforce the process. Don’t leave it in the hands of users or clients to apply or approve updates. The complexity of this will vary depending on what you need to update, and how, but it can often be as simple as enabling the built-in automatic updates that your software may already provide, or scheduling a daily command to apply any outstanding updates.

Once you've got it automated (the easy part) you will want to think about testing these changes before they hit production systems. Depending on the impact of the security exploits that you're patching, it may be more important to install updates even without complete testing; a broken site is often better than a vulnerable site! You may not have an automated way of testing every payment permutation on a large e-commerce site, for example, but that should not dissuade you from applying a critical update that exposes credit card data. Just be sure you aren't using this rationale as an excuse to avoid implementing automated testing.

The simple way

As a very common example of how simple the application of high priority updates can be, most Linux distributions will have a tried and tested method of automatically deploying security updates through their package management systems. For example, Ubuntu/Debian have the unattended-upgrades package, and Redhat-based systems have yum-cron. At the very least you will be able to schedule the system’s package manager to perform nightly updates yourself. This will cover the OS itself as well as any officially supported software that you have installed through the package manager. This means that you probably already have a reliable method of updating 95% of the open source software that you're using with minimal effort, and potentially any third-party software if you're installing from a compatible software repository. Consult the documentation for your Linux distro (or Google!) to find out how to enable this, and you can ensure that you are applying updates as soon as they are made available.

The complex way

For larger or more complex infrastructure where you may be using configuration management software (such as Ansible, Chef, or Puppet) to enforce state and install packages, you have more options. Config management software will allow you to apply updates to your test systems first, and report back on any immediate issues applying these updates. If a service fails to restart, a service does not respond on the expected port after the upgrade, or anything goes wrong, this should be enough to stop these changes reaching production until the situation is resolved. This is the same process that you should already be following for all config changes or package upgrades, so no special measures should be necessary.

The decision to make security updates a separate scheduled task, or to implement them directly in your config management process will depend on the implementation, and it would be impossible to cover every possible method here.

Risk Management

Automatically upgrading software packages on production systems is not without risks. Many of these can be mitigated with a good workflow for applying changes (of any kind) to your servers, and confidence can be added with automated testing.

Risks

  • You need to have backups of your configuration files, or be enforcing them with config management software. You may lose custom configuration files if they are not flagged correctly in the package, or the package manager does not behave how you expect when updating the software.
  • Changes to base packages like openssl, the kernel, or system libraries can have an unexpected effect on many other packages.
  • There may be bugs or regressions in the new version. Performance may be degraded.
  • Automatic updates may not complete the entire process needed to make the system secure. For example, a kernel update will generally require a reboot, or multiple services may need to be restarted. If this does not happen as part of the process, you may still be running unsafe versions of the software despite installing upgrades.

Reasons to apply updates automatically

  • The server is not critical and occasional unplanned outages are acceptable.
  • You are unlikely to apply updates manually to this server.
  • You have a way to recover the machine if remote access via SSH becomes unavailable.
  • You have full backups of any data on the machine, or no important data is stored on it.

Reasons to NOT apply updates automatically

  • The server provides a critical service and has no failover in place, and you cannot risk unplanned outages.
  • You have custom software installed manually, or complex version dependencies that may be broken during upgrades. This includes custom kernels or kernel modules.
  • You need to follow a strict change control process on this environment.

Reboot Often

Most update systems will also be able to automatically reboot for you if this is required (such as a kernel update), and you should not be afraid of this or delay it unless you're running a critical system. If you are running a critical system, you should already have a method of hot-patching the affected systems, performing rolling/staggered reboots behind a load-balancer, or some other cloud wizardry that does not interrupt service.

Decide on a maintenance window and schedule your update system to use it whenever a reboot is required. Have monitoring in place to alert you in the event of failures, and schedule reboots within business hours wherever possible.

Drupal and Other Web-based Applications

Most web-based CMS software such as Drupal and Wordpress offer automated updates, or at least notifications. Drupal security updates for both core and contributed modules can be applied by Drush, which can in turn be scheduled easily using cron or a task-runner like Jenkins. This may not be a solution if you follow anything but the most basic of deployment workflows and/or rely on a version control system such as Git for your development (which is where these updates should go, not direct to the web server). Having your production site automatically update itself will mean that it no longer matches what you deployed, nor what is in your version control repository, and it will be bypassing any CI/testing that you have in place. It is still an option worth considering if you lack all of these things or just want to guarantee that your public-facing site is getting patches as a priority over all else.

You could make this approach work by serving the Git repo as the document root, updating Drupal automatically (using Drush in 'security only' upgrade mode on cron), then committing those changes (which should not conflict with your custom code/modules) back to the repo. Not ideal, but better than having exploitable security holes on your live servers.

If your Linux distribution (or the CMS maintainers themselves) provide the web-based software as a package, and security updates are applied to it regularly, you may even consider using their version of the application. You can treat Drupal as just another piece of software in the stack, and the only thing that you're committing to version control and deploying to servers is any custom modules to be layered on top of the (presumably) secure version provided as part of the OS.

Some options that may fit better into the common CI/Git workflows might be:

  • Detect, apply, and test security patches off-site on a dedicated server or container. If successful, commit them back to version control to your dev/integration branch.
  • Check for security updates as part of your CI system. Apply, test and merge any updates into your integration branch.

Third-party Drupal Modules (contrib)

Due to the nature of contrib Drupal modules (ie, those provided by the community) it can be difficult to update them without also bringing in other changes, such as new features (and bugs!) that the author may have introduced since the version you are currently running. Best practice would be to try to keep all of the contrib that the site uses up to date where possible, and to treat this with the same care and testing as you would updates to Drupal itself. Contrib modules often receive important bug fixes and performance improvements that you may be missing out on if you only ever update in the event of a security announcements.

Summary

  • Ensure that updates are coming from a trusted and secure (SSL) source, such as your Linux distribution's packaging repositories or the official Git repositories for your software.
  • If you do not trust the security updates enough to apply them automatically, you should probably not be using the software in the first place.
  • Ensure that you are alerted in the event of any failures in your automation.
  • Subscribe to relevant security mailing lists, RSS feeds, and user groups for your software.
  • Prove to yourself and your customers that your update method is reliable.
  • Do not allow your users, client, or boss to postpone or delay security updates without an incredibly good reason.

You are putting your faith in the maintainer's' ability to provide timely updates that will not break your systems when applied. This is a risk you will have to take if you automate the process, but it can be mitigated through automated or manual testing.

Leave It All To Somebody Else

If all this feels like too much responsibility and hard work then it’s something Ixis have many years of experience in. We have dedicated infrastructure and application support teams to keep your systems secure and updated. Get in touch to see how we can ensure you're secure now and in the future whilst enjoying the use and benefits of open source software.

Apr 25 2016
Apr 25
teaser image for blog post

Over the last couple of years, we have been using Codeception at Ixis for running automated acceptance tests during development work. Over this time we attempted to distil some of the ideas and abstract custom code into Codeception modules, which are all available on GitHub.

Adopting Codeception for automated acceptance testing has been quite the learning process: getting used to the Codeception system, best practices and underlying code; facing challenges when writing tests for unusual or difficult situations; moving from the cURL-based PhpBrowser to the more advanced and complex WebDriver; problems with underlying components such as Selenium and PhantomJS; lack of integration with Drupal... phew! And that's not before we've considered processes, methods and best practices for running automating testing itself!

The latter, however, is a story for perhaps another time. This post summarises the Codeception modules we've worked on, why they might be useful and some lessons learned.

Drupal Content Type Registry

Drupal Content Type Registry is a module to provide a set of classes that encapsulate Drupal content types. This makes it much easier to quickly test standard Drupal functionality relating to content types, taking into account how they exist on your site. It enables testing of things such as the content types admin page, the 'manage fields' page for each content type, and provides createNode() and deleteNode() methods that can be used to quickly create test nodes where you can provide the test data using specific values, random values, or a range of values where one is picked at random.

Once enabled, a contentTypes.yml file should be created which defines:

  • GlobalFields - fields that will be used across all of the content types on the site. This is useful for things like title and body fields, to save you having to redefine the exact same field on every content type.
  • ContentTypes - content types on the site and the fields they have (global or otherwise).

Here's the example from the README:

GlobalFields:
    body:
        machineName:    body
        label:          Body
        type:           Long text and summary
        selector:       "#edit-body-und-0-value"
        widget:         Text area with a summary
        required:       true
    title:
        machineName:    title
        label:          Title
        type:           Node module element
        selector:       "#edit-title"
ContentTypes:
    news:
        humanName:    News
        machineName:  news
        fields:
            globals:
                - title
                - body
            field_image:
                machineName:    field_image
                label:          Image
                type:           Image
                selector:       "#edit-field-image"
                widget:         Media file selector
                required:       true
                testData:       "image1.png"
            field_icon:
                machineName:    field_icon
                label:          Icon
                type:           Text
                selector:       "#edit-field-icon"
                widget:         Text field
                skipRoles:
                    - editor
                    - publisher
                testData:
                    - smiley
                    - grumpy
                    - happy
                    - wacky
                preSteps:
                    - ["click", ["#button"]]
                    - ["fillField", ["#the-field", "the-value"]]
                postSteps:
                    - ["waitForJs", ["return jQuery.active == 0;"]]
        submit: "#edit-submit-me-please"

The definition of fields comprises the machine name, label, field type, widget and CSS ID or selector used when filling in node edit forms. Similarly, the definition of content types comprises the machine name, human-readable name and a list of field definitions. The module also provides a set of classes representing most field widgets provided by Drupal 7 core.

There are other features too, such as: preSteps and postSteps, which allow optional steps to run before and after filling the field; testData, which allows specific test data (literals or random text) to be used for each field; skippedRoles, which skips certain fields for specific user roles when creating nodes; and "Extras", which simulate the user clicking things on the node edit form that are not actually fields, like set the sticky status or the publication status of a node. For more detailed information on the module's configuration, see the README file.

This module (in combination with Drupal User Registry) also provides enough to create close to generic tests that will check your Drupal 7 site for all expected content types and fields. See this example Gist.

Credit must go to Chris Cohen for his original work on the Content Type Registry.

Drupal Drush

Drupal Drush allows the running of Drush commands in acceptance tests. It also allows the use of the following statements in tests:

// Execute "drush cc all"
$I->getDrush("cc", array("all"))->mustRun();

The getDrush() method returns and instance of Symfony\Component\Process\Process so you can read stdout, get the exit code, etc. The ->mustRun() method is useful as a Symfony\Component\Process\Exception\ProcessFailedException exception will be thrown if the command fails, meaning your test will automatically fail.

Drupal Mail System

Drupal Mail System allows the testing of the Drupal mail system.

// Test to see expected number of emails sent.
$I->seeNumberOfEmailsSent(1);
 
// Clear emails from queue.
$I->clearSentEmails();
 
// Check email fields contains text
$I->seeSentEmail(array(
    "body" => "body contains this text",
    "subject" => "subject contains this text",
));

Relies on TestingMailSystem class which stores the emails in a Drupal system variable.

Drupal Pages

Drupal Pages is a dependency of some other modules listed here and contains several PageObjects for "generic" Drupal 7 pages, based on "vanilla Drupal 7' and the default Bartik front-end or Seven administration themes. These PageObject classes can be extended to override any static properties as required for the site being tested.

Drupal User Registry

Drupal User Registry is a Codeception module for managing test users when running acceptance tests with Codeception. Once configured, it can automatically create and delete test Drupal users at the beginning and end of a test suite run. These users can then be used during acceptance tests to login and test elements of the project specific to that role.

The module is configured in the suite configuration:

class_name: AcceptanceTester
modules:
    enabled:
        - PhpBrowser
        - DrupalUserRegistry
    config:
        PhpBrowser:
            url: 'http://localhost/myapp/'
        DrupalUserRegistry:
            defaultPass: "foobar"
            users:
                administrator:
                    name: administrator
                    email: admin@example.com
                    pass: "foo%^&&"
                    roles: [ administrator, editor ]
                    root: true
                editor:
                    name: editor
                    email: editor@example.com
                    roles: [ editor, sub-editor ]
                "sub editor":
                    name: "sub editor"
                    email: "[email protected]"
                    roles: [ sub-editor ]
                authenticated:
                    name: authenticated
                    email: authenticated@example.com
                    roles: [ "authenticated user" ]
            create: true
            delete: true
            drush-alias: '@mysite.local'
  • defaultPass - use this password for all created user accounts, unless they have one individually specified.
  • users - a list of test user accounts to create, complete with username. email, password and a list of roles.
  • create and delete - whether to create and delete users at the start and end of a run.
  • drush-alias - the Drush alias to use when managing users via DrushTestUserManager.

Once configured, the module also allows the use of the following statements in tests:

// Returns a DrupalTestUser object representing the test user available for
// this role.
$user = $I->getUserByRole($roleName);
 
// Returns a DrupalTestUser object representing the test user available for
// exactly these roles.
$user = $I->getUserByRole([$roleName1, $roleName2]);
 
// Returns a DrupalTestUser object representing the user, or false if no users
// were found. Note this will only return a user defined and managed by this
// module, it will not return information about arbitrary accounts on the site
// being tested.
$user = $I->getUser($userName);
 
// Returns an indexed array of configured roles, for example:
//   array(
//     0 => 'administrator',
//     1 => 'editor',
//     2 => ...
//   );
$roles = $I->getRoles();
 
// Returns a DrupalTestUser object representing the "root" user (account with
// uid 1), if credentials are configured:
$rootUser = $I->getRootUser();

This module is used in any acceptance test suite we create in order to test specific elements where being logged in as a user with specific roles is necessary. However, there are some limitations and improvements that could be made.

The user registry can be used in combination with codeception Step Objects and Drupal Pages to provide login and logout functions to enable testing as an authenticated user. For example:
    $ php codecept.phar generate:stepobject acceptance AuthenticatedSteps
    class AuthenticatedSteps extends \AcceptanceTester
    {
        /**
         * Log in.
         *
         * @param DrupalTestUser $person
         *   The Drupal Person to log in.
         */
        public function login(DrupalTestUser $person)
        {
            $I = $this;
 
            $I->amOnPage(UserAccountPage::route('login'));
            $I->expectTo('not be redirected due to already being logged in');
            $I->seeCurrentUrlEquals('/' . UserAccountPage::route('login'));
 
            $I->expectTo('see various elements of the login page');
            $I->seeElement(UserAccountPage::$loginFormUsernameSelector);
            $I->seeElement(UserAccountPage::$loginFormPasswordSelector);
            $I->seeElement(UserAccountPage::$loginFormSubmitSelector);
 
            $I->amGoingTo('fill in the login form');
            $I->fillField(UserAccountPage::$loginFormUsernameSelector, $person->name);
            $I->fillField(UserAccountPage::$loginFormPasswordSelector, $person->pass);
            $I->click(UserAccountPage::$loginFormSubmitSelector);
 
            $I->expectTo('log in successfully');
            $I->dontSee(UserAccountPage::$loginFormCredentialsErrorMessage, Page::$drupalErrorMessageSelector);
        }
 
        /**
         * Log out.
         */
        public function logout()
        {
            $I = $this;
 
            $I->amGoingTo('log out');
            $I->amOnPage(UserAccountPage::route('logout'));
            $I->expectTo('be redirected to the front page');
            $I->seeCurrentUrlEquals('/');
        }
    }
Then in a test:
    /**
     * Test pages.
     *
     * @guy AcceptanceTester\AuthenticatedSteps
     */
    class ThePagesCest
    {
        /**
         * Test the page as an authenticated user.
         *
         * @param AuthenticatedSteps $I
         */
        public function testThePage(AuthenticatedSteps $I)
        {
            $I->login($I->getRootUser());
            // ...
            $I->logout();
        }
    }

There's potential for integrating the login(), logout() and other related functionality into the Drupal User Registry module itself, but still allow the test suite author to override or specify these procedures themselves.

Drupal Variable

Drupal Variable allows us to test Drupal system variables, for example:

// Assert that the target site has variable "clean_url" set to 1
$I->seeVariable("clean_url", 1);
 
// Set a variable.
$I->haveVariable("clean_url", 0);
 
// Delete a variable.
$I->dontHaveVariable("clean_url");
 
// Retrieve a variable value.
$value = $I->getVariable("clean_url");

This module was the key to working out different "connections" to Drupal itself. Previously, we were restricted by the assumption that everything needed to be done via the browser in acceptance testing (see the lessons learned around creating test content via the test browser, in Drupal Content Type Registry above).

Drupal Variable can connect to a Drupal site using one of three methods:

  • Bootstrapped - if the site is accessible on a locally mounted file system, fully bootstrapping the site is possible. In this case we use the Codeception\Module\Drupal\Variable\VariableStorage\Bootstrapped class and ensure we set the drupal_root configuration setting.
  • Direct connection - if the site is remote but we have access via an open MySQL connection, we use the Codeception\Module\Drupal\Variable\VariableStorage\DirectConnection class and set a dsn, user and password.
  • Drush - if the site is remote but access via a Drush alias is available, we use the Codeception\Module\Drupal\Variable\VariableStorage\Drush class and ensure we set the drush_alias setting in the module's configuration.

In review

Problems with creating test content in the browser

Creating lots of test content in the browser with Drupal Content Type Registry has proved problematic in places and in hindsight not the best way to achieve our goal. Many of our development projects and supported sites have additional elements that make things that little more difficult too, such as alternative publishing workflows with Workbench or WYSIWYG editing and media management with CKEditor and the Media suite of modules.

Whilst these are problems that must be tackled when explicitly testing the UI and user flow to verify a particular feature, it's not necessary to effectively repeatedly run this test in order to set up test data and content. Not only is it in places complex, but running a full suite (in multiple environments, i.e. in different browsers) is slow and a lot of that time is (obvious facepalm incoming) setting up test data.

Once we had a more 'useful' connection to Drupal with Drush, database or a fully bootstrapped connection as implemented in the Drupal Variable module, it became clear that there could be a better (or at least faster) way of setting up test content. We're not 100% what this will involve yet, perhaps custom code, or integration with modules such as Migrate or Node export.

Refactoring and relating the modules

Most of these module sprang from solving different problems whilst testing different projects. Whilst we have one or two suites using all if not most of them, the modules function together but may benefit from refactoring any shared code. For example, Drupal User Registry will currently check for, create and delete test users using Drush and aliases and the original intention was to have other ways of doing so via other connections. As mentioned above, Drupal Variable implements a different method but already has three types of connection, including Drush, which Drupal User Registry could utilise.

Behat?

Fairly recently Codeception introduced Gherkin and natural language features which is up there at the top of a list of advantages that Behat may have over Codeception - second only to it's much more extensive integration with Drupal, of course! Behat is being considered as an alternative or supplemental method for testing our Drupal sites.

Mar 07 2016
Mar 07
teaser image for blog post

In learning about custom Drupal 8 module development, I found plenty of very simple field module examples, but none that covered how to store more than one value in a field and still have it work properly, so it's time to fix that.

To save you typing or copy and pasting things around all the code in this post is available on Github at https://github.com/ixis/dicefield

Concepts

There are three main elements to define when creating a field type:

  • The field base is the definition of the field itself and contains things like what properties it should have.
  • The field widget defines the form field that is used to put data into your field, what its rules are and how those data are manipulated and stored in the field.
  • The field formatter is how the field will be displayed to the end user and what options are configurable to customise that display.

So far, so familiar if you've ever worked with Drupal 7 fields, and this is like so much of Drupal 8: on the surface, to the end user, it's very similar, but behind the scenes, it's a whole new world.

Use case

To create a (probably quite limited-use, in all honesty) real-world example, I decided to take on the challenge of creating a field to represent dice notation. For example, if you see 1d6 you would grab a single six-sided die and roll it. If you see 3d6-2, you would roll 3 six-sided dice and subtract 2 from the result.

There are three components here:

  • The number of dice
  • The number of sides on each die
  • The modifier: the part that is added or subtracted at the end

Although in practice you could store the whole thing as one big string, and it would be a walk in the park to set up, you would lose some of the more useful functionality, such as search indexing and sorting at a database level. Suppose you wanted to create a view that filtered only field values that involved rolling 5 dice. With a multi-value field such as the one we're creating, it's simple to do. If you store everything as one big string, it involves pattern matching or loading all the results and sifting through them.

Info file

Info files are now YAML format, and this is covered in detail elsewhere, but here's what I came up with:

dicefield.info.yml

name: Dice field
type: module
description: A way of specifying a dice value such as 1d6 or 2d8+3.
package: Field types
version: 1.0
core: 8.x
 
dependencies:
  - field

This is nice and straightforward, and, obviously, our module must depend on the core field module, or it cannot work at all.

Note that Drupal 8 no longer requires anything more than an info file to enable a module; previous versions required an empty .module file at least.

Field base

Now things get interesting. We're going to create a new plugin class to define our field type. The system generally works by extending one of the existing types and making the necessary changes to it. This is the biggest piece of advice I can give beyond reading articles like these: do as little work as possible! Copy/paste from existing things in core or contributed modules and change them to suit (although obviously give credit where it's due).

It's worth noting that, unlike in Drupal 7, our dicefield.info.yml file does not contain a list of "includes" that Drupal 8 should know about. These are loaded automatically by the PSR-4 autoloader, which is both more efficient and more convenient than the previous method. It does mean, however, that you must be careful to lay out your folder structure carefully and make sure things are named properly, because these things do matter in Drupal 8.

src/Plugin/Field/FieldType/Dice.php

/**
 * @file
 * Contains \Drupal\dicefield\Plugin\Field\FieldType\Dice.
 */
 
namespace Drupal\dicefield\Plugin\Field\FieldType;
 
use Drupal\Core\Field\FieldItemBase;
use Drupal\Core\Field\FieldStorageDefinitionInterface;
use Drupal\Core\TypedData\DataDefinition;
 
/**
 * Plugin implementation of the 'dice' field type.
 *
 * @FieldType (
 *   id = "dice",
 *   label = @Translation("Dice"),
 *   description = @Translation("Stores a dice roll such as 1d6 or 2d8+3."),
 *   default_widget = "dice",
 *   default_formatter = "dice"
 * )
 */
class Dice extends FieldItemBase {
  /**
   * {@inheritdoc}
   */
  public static function schema(FieldStorageDefinitionInterface $field_definition) {
    return array(
      'columns' => array(
        'number' => array(
          'type' => 'int',
          'unsigned' => TRUE,
          'not null' => FALSE,
        ),
        'sides' => array(
          'type' => 'int',
          'unsigned' => TRUE,
          'not null' => TRUE,
        ),
        'modifier' => array(
          'type' => 'int',
          'not null' => TRUE,
          'default' => 0,
        ),
      ),
    );
  }
 
  /**
   * {@inheritdoc}
  */
  public function isEmpty() {
    $value1 = $this->get('number')->getValue();
    $value2 = $this->get('sides')->getValue();
    $value3 = $this->get('modifier')->getValue();
    return empty($value1) && empty($value2) && empty($value3);
  }
 
  /**
   * {@inheritdoc}
   */
  public static function propertyDefinitions(FieldStorageDefinitionInterface $field_definition) {
    // Add our properties.
    $properties['number'] = DataDefinition::create('integer')
      ->setLabel(t('Number'))
      ->setDescription(t('The number of dice'));
 
    $properties['sides'] = DataDefinition::create('integer')
      ->setLabel(t('Sides'))
      ->setDescription(t('The number of sides on each die'));
 
    $properties['modifier'] = DataDefinition::create('integer')
      ->setLabel(t('Modifier'))
      ->setDescription(t('The modifier to be applied after the roll'));
 
    $properties['average'] = DataDefinition::create('float')
      ->setLabel(t('Average'))
      ->setDescription(t('The average roll produced by this dice setup'))
      ->setComputed(TRUE)
      ->setClass('\Drupal\dicefield\AverageRoll');
 
    return $properties;
  }
}

This looks quite complicated and also quite alien from most things in Drupal 7 unless you're used to working with ctools plugins or the migrate system. Let's break it down a little bit.

The namespace

We have defined our namespace at very nearly the top of the file:

namespace Drupal\dicefield\Plugin\Field\FieldType;

The standard way is to use the Drupal namespace, followed by the name of your module (exactly the same as the name of the folder your module lives in), then the other bits. It's this namespace that will tell the PSR-4 autoloader where to find the classes it needs, so make sure it's correct!

Annotation-based plugin definition

There are multiple ways of defining the plugin's core data. The standard Drupal 8 way is to use annotations, which are like code comment blocks, but contain actual code rather than a comment. Other ways include YAML files, for example, but we're going to keep things simple here.

Note that one downside to using annotations to define plugin data is that since they are effectively comments, not all IDEs can interpret them in the same way as code, so you lose the syntax highlighting and code suggestions associated with writing PHP code in a modern IDE (we use PhpStorm internally). While this might look bad, it's actually not a huge deal because:

  • The plugin definition is a tiny part of your overall code base.
  • The code is right there in front of you, instead of in a separate file.
  • There are still other options if you really don't like it.

Here's the code in question:

/**
 * Plugin implementation of the 'dice' field type.
 *
 * @FieldType (
 *   id = "dice",
 *   label = @Translation("Dice"),
 *   description = @Translation("Stores a dice roll such as 1d6 or 2d8+3."),
 *   default_widget = "dice",
 *   default_formatter = "dice"
 * )
 */

Everything starting from the @FieldType is the plugin definition and everything above is just a regular comment, so you can still write a useful description if you like (and in fact, you should).

The @FieldType part tells Drupal 8 that it is a new field type. There are other annotations that can define various things in Drupal, and we'll see a few others later in the article.

There are a number of key/value pairs in the definition, and these work as follows:

  • id is used to give this plugin a machine name. This only needs to be unique for the type of thing being defined here, so you could have a FieldType called "dice" and also a FieldFormatter called "dice" without worrying about the implications of a namespace collision.
  • label uses the @Translation() notation, which is just like using Drupal's t() function, and provides a human-readable name to be used in the admin UI and other places.
  • description also uses @Translation and just lets users know what your field is for.
  • default_widget is the machine name of the widget that will be used, by default, when this field is put in place on an entity. If there are multiple widgets available, users will be able to pick, but this will be the default. Note that this refers to the machine name of the widget, not the class name. Drupal makes this distinction a lot, so you will become used to working with two different types of notation: Drupal internal machine names, and class names. The class name is not needed here. As long as we define a @FieldWidget plugin later, with an id of "dice", we will be good to go.
  • default_formatter works the same way as default_widget, but is used for the formatter (what the user sees on the front end, rather than the way data are put into your field). Note how these both have the same name. Because they're different plugin types (one is a FieldWidget and the other is a FieldFormatter), they can have the same name and Drupal 8 won't get confused.

There are also a number of other keys that you can use here, but these are best detailed by the Drupal documentation on Entity annotation, and we've covered the ones we need.

Extending classes

Nearly every class you write in Drupal 8 will extend another class, or implement an interface, or apply a trait, or perhaps any combination of those. For example:

class Dice extends FieldItemBase {

We are extending from the base field class here and this will give us all of the functionality we need to implement a new field type. All we have to do is override the methods that we want to work in a different way.

The schema

The schema is simply the definition for how the data will be stored (in the database, or whatever storage engine you're using). We need to return an array (apparently we're still stuck in "array inception" mode for some parts of Drupal 8 but thankfully this is now a lot less common) of arrays, that contain arrays that define the columns we want to store. Yeah, that.

/**
 * {@inheritdoc}
 */
 public static function schema(FieldStorageDefinitionInterface $field_definition) {
   return array(
     'columns' => array(
       'number' => array(
         'type' => 'int',
         'unsigned' => TRUE,
         'not null' => FALSE,
       ),
       'sides' => array(
         'type' => 'int',
         'unsigned' => TRUE,
         'not null' => TRUE,
       ),
       'modifier' => array(
        'type' => 'int',
        'not null' => TRUE,
        'default' => 0,
      ),
    ),
  );
}

We are basically only defining the columns key in the outer array. In practice it's probably a good idea to define indexes and possibly also foreign keys if your fields will be linked to other data, but let's keep things simple for now.

The definitions in columns work the same way as the Drupal 8 schema API which you can use for reference if you need to.

Notice we're defining three integer fields: one for the number of dice, one for the number of sides on each die, and one for the modifier.

The number of dice and the sides are mandatory, so they do not have a default value. However, you can safely assume that unless otherwise stated, the modifier is optional, and should default to zero, which is why this one has a default value.

Note also that the first two are unsigned, because you can't have a die with -6 sides. The modifier is not unsigned, because both +3 and -3 are valid for modifiers.

The final thing worth mentioning here is that we're only defining fields that are actually stored as data. Later on, we'll see how to derive a computed field, but since the field is a calculated value (which is then cached in the render cache, so stop sweating about performance already!) it is not stored in the database and shouldn't be defined here.

isEmpty

It's very important to tell Drupal how to know if your field is empty or not. Without this, certain basic field functionality will not work properly. In our case, it's quite straightforward: the field is only really "empty" if none of the three values contain anything.

/**
 * {@inheritdoc}
 */
public function isEmpty() {
  $value1 = $this->get('number')->getValue();
  $value2 = $this->get('sides')->getValue();
  $value3 = $this->get('modifier')->getValue();
  return empty($value1) && empty($value2) && empty($value3);
}

Notice how we're using the internal method $this->get() to grab the value? The properties attached to the field will be called the same thing as those in propertyDefinitions() (see below). It makes sense for them to also match the properties we have defined in schema() above, but this does not necessarily have to be the case. Just have a good reason for doing otherwise!

propertyDefinitions

Next, we define the properties that this field will have. These will be the individual pieces of data we can retrieve from the field, and will affect things like view sorting order and how we will set up our formatter later.

/**
 * {@inheritdoc}
 */
public static function propertyDefinitions(FieldStorageDefinitionInterface $field_definition) {
  // Add our properties.
  $properties['number'] = DataDefinition::create('integer')
    ->setLabel(t('Number'))
    ->setDescription(t('The number of dice'));
 
  $properties['sides'] = DataDefinition::create('integer')
    ->setLabel(t('Sides'))
    ->setDescription(t('The number of sides on each die'));
 
  $properties['modifier'] = DataDefinition::create('integer')
    ->setLabel(t('Modifier'))
    ->setDescription(t('The modifier to be applied after the roll'));
 
  $properties['average'] = DataDefinition::create('float')
    ->setLabel(t('Average'))
    ->setDescription(t('The average roll produced by this dice setup'))
    ->setComputed(TRUE)
    ->setClass('\Drupal\dicefield\AverageRoll');
 
  return $properties;
}

Note that we have the same three properties that we defined as being stored in schema() above, plus a fourth one, called average. This is a computed field, which means that instead of storing the value in the database, we derive it from the values of the other fields. It is more useful to do it this way, because the average value is just the sum of the minimum and maximum possible roll, halved, then added to the modifier. If we were to store this in the database we would be wasting database space. You might think it inefficient to compute this value, but in fact, it's cached by Drupal's render cache system, and invalidated only when the field is updated, so except for the first time it's computed, it's not generally a performance hindrance.

Each of our four properties are basic types as defined by Drupal's typed data API. We have three integers and a float, but we could also use string or other types if we wanted to. We could even come up with our own types, but that's not necessary for this field so I won't cover it here.

We just return an array of properties by using DataDefiniton::create() and chaining the methods we want in order to create the property. As a minimum, you should use setLabel() and setDescription, but there are plenty of others that you can use. The fourth property, average, has two extra methods.

setComputed() is used to indicate that this field is computed rather than stored in the database, so there won't be a matching column in schema().

Given that it's computed, Drupal needs to know what class to use to do this computation, and this is where setClass() comes in. See below for more about computing field values in their own classes.

Widget

Now that we've set up our field base, we need to set up a widget so that people editing a node (or other entity) where this field is used are able to input or edit the data.

src/Plugin/Field/FieldWidget/DiceWidget.php

/**
 * @file
 * Contains \Drupal\dicefield\Plugin\Field\FieldWidget\DiceWidget.
 */
 
namespace Drupal\dicefield\Plugin\Field\FieldWidget;
 
use Drupal\Core\Field\FieldItemListInterface;
use Drupal\Core\Field\WidgetBase;
 
/**
 * Plugin implementation of the 'dice' widget.
 *
 * @FieldWidget (
 *   id = "dice",
 *   label = @Translation("Dice widget"),
 *   field_types = {
 *     "dice"
 *   }
 * )
 */
class DiceWidget extends WidgetBase {
  /**
   * {@inheritdoc}
   */
  public function formElement(
    FieldItemListInterface $items,
    $delta,
    array $element,
    array &$form,
    array &$form_state
  ) {
    $element['number'] = array(
      '#type' => 'number',
      '#title' => t('# of dice'),
      '#default_value' => isset($items[$delta]->number) ? $items[$delta]->number : 1,
      '#size' => 3,
    );
    $element['sides'] = array(
      '#type' => 'number',
      '#title' => t('Sides'),
      '#field_prefix' => 'd',
      '#default_value' => isset($items[$delta]->sides) ? $items[$delta]->sides : 6,
      '#size' => 3,
    );
    $element['modifier'] = array(
      '#type' => 'number',
      '#title' => t('Modifier'),
      '#default_value' => isset($items[$delta]->modifier) ? $items[$delta]->modifier : 0,
      '#size' => 3,
    );
 
    // If cardinality is 1, ensure a label is output for the field by wrapping
    // it in a details element.
    if ($this->fieldDefinition->getFieldStorageDefinition()->getCardinality() == 1) {
      $element += array(
        '#type' => 'fieldset',
        '#attributes' => array('class' => array('container-inline')),
      );
    }
 
    return $element;
  }
}

Class

Once again, we are inheriting from the base WidgetBase class because almost all of the work involved in being a widget is done for us, so we only need to lift a finger to tell Drupal what's different from the base.

class DiceWidget extends WidgetBase {

Annotation

Again, we see that the annotation at the top of the class defines basic data on this widget:

/**
 * Plugin implementation of the 'dice' widget.
 *
 * @FieldWidget (
 *   id = "dice",
 *   label = @Translation("Dice widget"),
 *   field_types = {
 *     "dice"
 *   }
 * )
 */

This time, @FieldWidget tells Drupal it's dealing with a widget, and the id and label properties work the same way as for the base field above.

We have an array this time, in the form of field_types, which tells Drupal which types of field are allowed to use this widget. Note that unlike regular PHP arrays in Drupal, you must not put a comma after the last element in these arrays.

This field_types allows us to create new widgets, even for existing field types, in case we want a better or different way of inputting data. For example, a geo-location field that stores map coordinates might have a text widget for inputting the data manually, and a separate map widget that allows the user to click on a map to choose a point.

formElement

In actual fact, we only need to override one method in this class:

  /**
   * {@inheritdoc}
   */
  public function formElement(
    FieldItemListInterface $items,
    $delta,
    array $element,
    array &$form,
    FormStateInterface $form_state,
  ) {
    $element['number'] = array(
      '#type' => 'number',
      '#title' => t('# of dice'),
      '#default_value' => isset($items[$delta]->number) ? $items[$delta]->number : 1,
      '#size' => 3,
    );
    $element['sides'] = array(
      '#type' => 'number',
      '#title' => t('Sides'),
      '#field_prefix' => 'd',
      '#default_value' => isset($items[$delta]->sides) ? $items[$delta]->sides : 6,
      '#size' => 3,
    );
    $element['modifier'] = array(
      '#type' => 'number',
      '#title' => t('Modifier'),
      '#default_value' => isset($items[$delta]->modifier) ? $items[$delta]->modifier : 0,
      '#size' => 3,
    );
 
    // If cardinality is 1, ensure a label is output for the field by wrapping
    // it in a details element.
    if ($this->fieldDefinition->getFieldStorageDefinition()->getCardinality() == 1) {
      $element += array(
        '#type' => 'fieldset',
        '#attributes' => array('class' => array('container-inline')),
      );
    }
 
    return $element;
  }

This method tells Drupal how to render the form for this field. Because we need to know three things (the number of dice, the sides per die, and the modifier), we will provide three fields for this. Note how the form keys for these fields match what we defined in schema() and propertyDefinitions() above.

The #attributes on the fieldset causes the fields to be displayed inline instead of one line after another.

These fields use the number field type, which is basically a text field but with little up and down arrows that can be used to increase or decrease the value. It also provides some basic validation in that you need to put a numerical value in here, not a string, and there's no need to write this validation if it's already done for us.

The #default_value key shows how to extract the value from the current field. In the case where we're editing a field, we want the existing values to be in the form ready to be changed, and $items[$delta]->PROPERTY_NAME will do that for us.

I have also set up a default value in the case where we're creating a completely new node, as I felt it was nice to be able to show an example of the required input. Also, since the modifier is often zero, it makes sense to set this as a default value. I could have also used #placeholder to put an HTML5 placeholder value in the field instead of real input.

The last part of this method simply adds a fieldset so that if the field cardinality is 1 (only 1 "dice roll" field value can be put in, instead of allowing unlimited, or a higher number of entries), then the label for the field will still show up properly. This can be used as-is for most field widgets.

Formatter

The last required step (and it might not even be required if you can re-purpose a core formatter from Drupal itself) is to set up a new formatter. This will output the information in the field onto the screen so that users can see it.

src/Plugin/Field/FieldFormatter/DiceFormatter.php

/**
 * @file
 * Contains \Drupal\dicefield\Plugin\Field\FieldFormatter\DiceFormatter.
 */
 
namespace Drupal\dicefield\Plugin\Field\FieldFormatter;
 
use Drupal\Core\Field\FieldItemListInterface;
use Drupal\Core\Field\FormatterBase;
 
/**
 * Plugin implementation of the 'dice' formatter.
 *
 * @FieldFormatter (
 *   id = "dice",
 *   label = @Translation("Dice"),
 *   field_types = {
 *     "dice"
 *   }
 * )
 */
class DiceFormatter extends FormatterBase {
  /**
   * {@inheritdoc}
   */
  public function viewElements(FieldItemListInterface $items, $langcode = NULL) {
    $elements = array();
 
    foreach ($items as $delta => $item) {
      if ($item->sides == 1) {
        // If we are using a 1-sided die (occasionally sees use), just write "1"
        // instead of "1d1" which looks silly.
        $markup = $item->number * $item->sides;
      }
      else {
        $markup = $item->number . 'd' . $item->sides;
      }
 
      // Add the modifier if necessary.
      if (!empty($item->modifier)) {
        $sign = $item->modifier > 0 ? '+' : '-';
        $markup .= $sign . $item->modifier;
      }
 
      $elements[$delta] = array(
        '#type' => 'markup',
        '#markup' => $markup,
      );
    }
 
    return $elements;
  }
}

Annotation

The annotation defines the basic formatter data, and works just like the others above.

  /**
   * Plugin implementation of the 'dice' formatter.
   *
   * @FieldFormatter (
   *   id = "dice",
   *   label = @Translation("Dice"),
   *   field_types = {
   *     "dice"
   *   }
   * )
   */

In fact, this is almost identical to the one for DiceWidget that we defined above, except we're now using the @FieldFormatter type.

Class

As we've seen above, it's easiest to just extend the base class, FormatterBase, since this does all the heavy lifting already and we can pick and choose what to override.

class DiceFormatter extends FormatterBase {

viewElements

We are overriding the method that actually produces the markup that will be displayed on the page:

/**
 * {@inheritdoc}
 */
public function viewElements(FieldItemListInterface $items, $langcode = NULL) {
  $elements = array();
 
  foreach ($items as $delta => $item) {
    if ($item->sides == 1) {
      // If we are using a 1-sided die (occasionally sees use), just write "1"
      // instead of "1d1" which looks silly.
      $markup = $item->number * $item->sides;
    }
    else {
      $markup = $item->number . 'd' . $item->sides;
    }
 
    // Add the modifier if necessary.
    if (!empty($item->modifier)) {
      $sign = $item->modifier > 0 ? '+' : '-';
      $markup .= $sign . $item->modifier;
    }
 
    $elements[$delta] = array(
      '#type' => 'markup',
      '#markup' => $markup,
    );
  }
 
  return $elements;
}

The most important thing here is that we loop through the $items because each field could have a cardinality of greater than one, meaning multiple dice rolls can be stored in a single field.

There is a fringe case where technically it's possible (not in the physical world) to have a one-sided die, which will always roll a 1, no matter what, so instead of writing 1d1, we tell the formatter to present it as just 1 for clarity.

Next we add the modifier, but only if it's a non-zero, because 1d6 looks cleaner than 1d6+0. Note that the modifier could be positive or negative, so we need to account for that in the code. Negative numbers, when converted to strings, already have a negative symbol at the front, but positive ones don't, so we add that on.

The last part is quite important, and that is presenting the return value as a series of render arrays, rather than just plain text. Everything should be presented as a render array where possible, because this allows Drupal to delay its rendering until the last possible moment, affording other modules the opportunity to override where necessary.

AverageRoll type

Earlier, we defined a computed field when we set up propertyDefinitions(). This means that we need to tell Drupal how to compute the value of this field, and we will create a separate class for this.

src/AverageRoll.php

/**
 * @file
 * Contains \Drupal\dicefield\AverageRoll.
 */
 
namespace Drupal\dicefield;
 
use Drupal\Component\Utility\SafeMarkup;
use Drupal\Component\Utility\String;
use Drupal\Core\TypedData\DataDefinitionInterface;
use Drupal\Core\TypedData\TypedDataInterface;
use Drupal\Core\TypedData\TypedData;
 
/**
 * A computed property for an average dice roll.
 */
class AverageRoll extends TypedData {
 
  /**
   * Cached processed value.
   *
   * @var string|null
   */
  protected $processed = NULL;
 
  /**
   * Implements \Drupal\Core\TypedData\TypedDataInterface::getValue().
   */
  public function getValue($langcode = NULL) {
    if ($this->processed !== NULL) {
      return $this->processed;
    }
 
    $item = $this->getParent();
 
    // The minimum roll is the same as the number of dice, which will occur if
    // all dice come up as a 1. Then apply the modifier.
    $minimum = $item->number + $item->modifier;
 
    // The maximum roll is the number of sides on each die times the number of
    // dice. Then apply the modifier.
    $maximum = ($item->number * $item->sides) + $item->modifier;
 
    // Add together the minimum and maximum and divide by two. In cases where we
    // get a fraction, take the lower boundary.
    $this->processed = ($minimum + $maximum) / 2;
    return $this->processed;
  }
 
  /**
   * Implements \Drupal\Core\TypedData\TypedDataInterface::setValue().
   */
  public function setValue($value, $notify = TRUE) {
    $this->processed = $value;
 
    // Notify the parent of any changes.
    if ($notify && isset($this->parent)) {
      $this->parent->onChange($this->name);
    }
  }
}

There is no annotation!

Since we're not defining a plugin here, there's no annotation at the top of this class. There's no need.

Class

We are simply extending an existing type, to do the heavy lifting for us, as with previous classes above.

class AverageRoll extends TypedData {

Caching

I mentioned earlier that Drupal's render cache means that we only need to process the value when it changes, rather than every time we see the field, and we can add a mechanism to achieve this. It's via a protected property:

/**
 * Cached processed value.
 *
 * @var string|null
 */
protected $processed = NULL;

Note how even though this is just a property on a class, it is still fully documented like anything else!

getValue

This method will, as the name suggests, get the value of the computed field when Drupal asks for it. Note that method doesn't need to be called directly. Drupal takes care of this internally which means that you can just use $item->average instead of having to write $item->average->getValue() or anything complicated like that.

/**
 * Implements \Drupal\Core\TypedData\TypedDataInterface::getValue().
 */
public function getValue($langcode = NULL) {
  if ($this->processed !== NULL) {
    return $this->processed;
  }
 
  $item = $this->getParent();
 
  // The minimum roll is the same as the number of dice, which will occur if
  // all dice come up as a 1. Then apply the modifier.
  $minimum = $item->number + $item->modifier;
 
  // The maximum roll is the number of sides on each die times the number of
  // dice. Then apply the modifier.
  $maximum = ($item->number * $item->sides) + $item->modifier;
 
  // Add together the minimum and maximum and divide by two. In cases where we
  // get a fraction, take the lower boundary.
  $this->processed = ($minimum + $maximum) / 2;
  return $this->processed;
}

At the beginning of this method is the test to see if we have already assigned a value to $this->processed. If we have, we don't need to compute the value. We only do that part if the value is null, to save on processing power.

The internals of this method first work out the minimum and maximum possible rolls, applying the modifier to each, and then divide their total by two, which gives the average. For many dice rolls this will be a fraction, which is why we have defined this field as a float rather than an integer.

setValue

Lastly, to make sure cache invalidation works correctly, we need to define a method to take care of setting our value:

/**
 * Implements \Drupal\Core\TypedData\TypedDataInterface::setValue().
 */
public function setValue($value, $notify = TRUE) {
  $this->processed = $value;
 
  // Notify the parent of any changes.
  if ($notify && isset($this->parent)) {
    $this->parent->onChange($this->name);
  }
}

Here, we set $this->processed to the new value, but we also notify the class's parent via the onChange() method. Remember that the vast majority of implementation is handled by the base class instead of making us do the work here, so we ought to be happy to pass the buck to Drupal itself where we can!

AverageRollFormatter

Lastly, it's worth bearing in mind that we have a formatter than can display the dice roll itself, and we also have a computed field that can calculate the average roll, but we have no way of actually showing the average roll to the end user. We will create one more formatter class that will take care of this for us. Because this is a separate formatter, it will allow the site administrator to choose, when displaying dice fields (in views, say), whether to show the dice notation, the average value, or even both (by adding the field twice with different formatters).

src/Plugin/Field/FieldFormatter/AverageRollFormatter.php

/**
 * @file
 * Contains \Drupal\dicefield\Plugin\Field\FieldFormatter\AverageRollFormatter.
 */
 
namespace Drupal\dicefield\Plugin\Field\FieldFormatter;
 
use Drupal\Core\Field\FieldItemListInterface;
use Drupal\Core\Field\FormatterBase;
 
/**
 * Plugin implementation of the 'average_roll' formatter.
 *
 * @FieldFormatter (
 *   id = "average_roll",
 *   label = @Translation("Average roll"),
 *   field_types = {
 *     "dice"
 *   }
 * )
 */
class AverageRollFormatter extends FormatterBase {
  /**
   * {@inheritdoc}
   */
  public function viewElements(FieldItemListInterface $items, $langcode = NULL) {
    $elements = array();
 
    foreach ($items as $delta => $item) {
      $elements[$delta] = array(
        '#type' => 'markup',
        '#markup' => $item->average,
      );
    }
 
    return $elements;
  }
}

The details here are much the same as our DiceFormatter from above, but the implementation is even more simple. Since the value of the field is computed for us already, all we need to do is print it out (by putting it into a render array) in viewElements().

One thing to bear in mind here is that we could have done the average calculation directly in the formatter. There's certainly scope for it: we have all the field data and the ability to execute whatever custom code we like. However, this would have been the wrong thing to do because it would have meant that the average can only be displayed on the front end, and not accessible internally to Drupal. By doing it the way we have done it above, we ensure that, for example, views is able to sort this field by average value, or another developer can plug into this module and grab the average value for his or her own use.

Wrapping up

Now that you have all the elements in place, you can enable the module and try it out!

Once enabled, you will see the new "dice roll" option when picking a field type to add to an entity. When added, you can try creating a new entity of this type, and see the widget in action. Then, the formatter will take care of showing the end result on your entity's view page. Don't forget: you have a choice of two formatters, so you can either show the dice notation or the average value (or even both if you use a custom template or views).

If you have any comments, corrections or observations about this tutorial, please feel free to leave them in the comments below. Hopefully the information about the principles and design decisions will be useful!

The full codebase is available on Github.

Feb 11 2016
Feb 11
teaser image for blog post

This months Northwest Drupal User Group (NWDUG) in Manchester had a lovely visit from a non developer to talk about how the Internet has changed his life and the challenges of using the Internet as a blind person.

Being blind since he was a child has meant that some activities weren't possible before the Internet became accessible to everyday people - such as reading the news or magazine articles. Now Sunil works with the Internet everyday at his job in the British Red Cross.

Many years ago JAWS was the defacto screen reader that accessible sites were tested against and made the Windows operating system more popular amongst the blind community. Since Apples VoiceOver capabilities have been baked in to their OS X software its opened up better applications to navigate the web. It was also noted that nearly all blind people tend to prefer iPads and iPhones because of their mature and built in accessibility features - and the interface controlled with flicks and swipes to the touch screen.

The group were given a demo of navigating the Guardian news website being navigated in the Safari web browser. The VoiceOver facility describing all the elements on the page read out the descriptions at a high speed to transmit the vast amount of information needed.

Sunil said he used headings to jump to content he's looking for - H1 etc. HTML tags.  Interestingly the "skip to main content" link right at the top of the page (and often found in Drupal base themes) wasn't used! A comment from the audience did suggest that those links are great for sighted users navigating sites with the keyboard.

He found the ALT attribute on images useful, if present, although TITLE attributes should contain the descriptions of what is in the image.

Forms with clearly labeled fields help associate form validation error messages with the bad input. 

The CAPTCHA anti spam systems still create a mixed problem - implementations that offer disability support are hard to hear and so difficult to solve. Sunil's favorite are the logic questions such as maths or odd word in a list. Google's more recent ReCAPTCHA now removes all these problems and replaces it with a simple tick box. A Drupal module is also available to support this service on forms.

In recent years the European cookie compliance policy pop ups have invaded many of our websites -- ones which appear at the top of the page tend to be a mild irritant but once accepted they don't re-appear - so just as relevant to all of us. Prehaps putting the cookie warning at the bottom of the site is a better solution?

In a similar vein the topic of pop-up banner advertising was asked by the group -- is it a problem or inconvenience when browsing a site? Sunil mentioned he had previously used Ad blocking software years ago but these days banners don't get in the way of his reading -- although he doesn't know what the advertising is due to it usually being image based (advertising networks take note!).

One gripe that still exists even in modern website builds is the use of PDF files for text information such as restaurant menus. if that information could be duplicated as HTML page content it would make consumption of the information possible for all.

It seemed that there's not much else that can stop Sunil from navigating the web in 2015 - which is great news as Drupal site builders!

We've had general web accessibility guidelines (WCAG) for many years but it's refreshing to see how that translates in to the real world, and a great thanks to @SunilPeck for giving up his evening to visit the Drupal usergroup.

The full evenings talks are available every month on YouTube on the NWDUG channel. You can watch Sunil in the February 2016 session.

Photo at NWDUG thanks to @ChandeepKhosa.

Nov 11 2015
Nov 11
teaser image for blog post

Launched in 2008 Drupal 6 has served a large base of sites for the past 7 years even with the more recent Drupal 7 launched in 2011.

As a provider of Drupal support and hosting services Ixis still look after a number of clients who are running the latest up to date Drupal 6 codebase. However, with the announcement of Drupal 8 launching on November 19th 2015 this starts the countdown to the end of the extended support policy - which is 3 months after the launch of Drupal 8. The final 3 months will only cover security updates - not functionality or bug fixes.

Drupal 6 Security Updates end on February 24th 2016

The 3 month window won't be long enough to allow upgrading of Drupal 6 sites to Drupal 8, especially as we'd always advise allowing a new major Drupal Core release to be tested by early adopters before upgrading and migrated your site to it.

What are the implications of this?

From Drupal.org

  • The security team will no longer provide support or Security Advisories for Drupal 6.
  • All Drupal 6 releases on project pages will be flagged as not supported.
  • At some point in the future update status may stop working for Drupal 6 sites.

From Ixis

  • We will continue to provide our support services to Drupal 6 sites and where necessary apply any security updates provided by 3rd parties.
  • We cannot guarantee your Drupal 6 site would not become vulnerable over the coming years as new attack vectors are discovered in web based applications and applied to old Drupal 6 sites.

What if I have a Drupal 6 site still?

If not already in progress, begin planning a site upgrade or rebuild. Web technologies have progressed a lot since 2008 which may mean reviewing how your Drupal 6 site works and what the business needs are - which may be better addressed in a more modern techniques thanks to the functionality of Drupal 8.

As with many of our long standing clients Ixis can assist with your journey from Drupal 6 to Drupal 8 with our in-house development team services. Now is the time to be making plans!

Any short term solutions?

  1. Reducing your sites functionality to content only would allow a quicker path to migrate to Drupal 8, and then as contrib modules are upgraded to 8.x re-introduce the functionality during 2016.
  2. Archive the site as static HTML pages - although this will remove interactive elements such as logins and comments.

What about upgrading to Drupal 7?

Depending on the timeframe and functionality required for site delivery Drupal 7 may still be the better option due to the sheer number of stable additional contributed module components available, such as Ecommerce.  At Ixis we're still developing Drupal 7 solutions for the next few months at least.

Speak to us

If you're unsure where the Drupal 6 support end of life announcement leaves you and your business website please do get in touch with us to discuss how Ixis can help in the short and long term. We're happy to chat!

Mar 11 2015
Mar 11

Come and join our well established UK team as a senior Drupal support engineer. We support interactive sites and applications of all kinds, so every client can offer different challenges and solutions each month.

You'll have the opportunity to be involved with projects ranging from international brands, enterprise public sector organisations through to charities and media sites. We also run several internal projects which you'll have chance to provide input and development for if you wish - this is where we often experiment with new ideas, techniques and technologies first!

The technical skills we're looking for:

  • Excellent knowledge of Drupal & its configuration.
  • Extensive experience with Panels, Views, Features.
  • Experience of working with a source code version control system.
  • Comfortable using a terminal command line.

The type of person we’re looking for:

  • Strong analytical, problem solving, and debugging skills.
  • Ability to forge strong relationships with colleagues, clients and suppliers.
  • Calm, considered and methodical.
  • Excellent verbal and written communication.
  • Excellent time management with the ability to handle a varying workload, while maintaining a high quality of wor

Ideally you'll have 1+ years commercial experience of working as part of a team.

Your work will cover a variety of areas, including:

  • 3rd line support for complex cases from our clients.
  • On-boarding of large Drupal sites to our service desk and platform.
  • Preparation and participation in regular client Service Reviews.
  • Extending existing Drupal projects which may have been designed and developed by a 3rd party agency.
  • Internal projects to create or improve our services.

As you may notice, we love Drupal! There's a chance to play with the latest tools and contribute back to the Drupal open source community as part of your daily work.

Most importantly we’re looking for somebody who is friendly, positive, enthusiastic and hungry to learn as part of our team.

Training

We actively encourage staff to further their web skills by attending and presenting at conferences, camps and training days.

Ixis have sponsored, organised and attended Drupalcon Europe, DrupalCamps and DrupalDevDays.

Location

Ideally we would like your work to be carried out on-site at our lovely office in Warrington, Cheshire, UK, but we may consider remote working depending on experience and ability. You will be required to visit the office throughout the year so ability to travel is necessary.

There may also be some occasional travel involved for client meetings from Scotland to the South East and anywhere in-between. Therefore this position will only be suitable for UK residents.

Why Work For Ixis?

We've been in the Drupal business for over 10 years, it's all we do, whether it's providing consultancy, architecting solutions, providing support services, developing sites or hosting large scale projects. We think it keeps our staff focused and highly skilled experts in our industry. We care about the open source software we use - and ensure we contribute our time and code back to the projects in our day to day job.

Our business is always growing and this is the perfect time to get onboard, helping us to shape the future of Drupal and its implementation in the UK and Europe.

We work with a wide range of high calibre clients and are well respected by both the Public and Private sector in consultancy, development, support and hosting. We host and support some of the biggest UK public sector sites.

We run a fun, friendly, and relaxed office environment as well as regular evening and weekend activities with the team.
As a perk of working on site you’ll be treated to the very best development environment hardware in the shape of a well specced 27” display iMac.

The Details

Position: Full-time.
Apply: Submit your cover letter and CV using the form at http://www.ixis.co.uk/job-application
Hours: We work 35 hours per week with a flexible start or finish time to fit around busy traffic periods and family commitments.
Holidays: 33 paid days per year.
Salary: £25,000 - £32,000
Training: An individual budget for training/conference/event ticket fees, accommodation, travel and time off for learning is provided each year.

No recruitment consultants to contact us please.

Jan 06 2015
Jan 06
teaser image for blog post

It was an exciting and busy year here at Ixis in 2014 as we continued to build our portfolio of Drupal clients

Highlights have included celebrating our tenth birthday and building on and growing the team’s expertise which has allowed the company to win more clients across a variety of sectors. We also celebrated one of our best financial quarter since Ixis was founded in 2004. 

Having won a string of new clients last year including national consultancies, local authorities and leading charities, our focus in 2015 will be on expanding our reach within these sectors. We are also developing plans to collaborate with our clients to deliver long term value which increasingly means investing sensibly in research and development.

The G-Cloud has had a big impact on our business so this will still remain central to our growth strategy. We also truly value retaining our clients and this of course will remain one of our biggest priorities as laid out in our core strengths.

This year, we want to continue to grow our business whilst maintaining one of the strongest and most diverse Drupal specific portfolios in the UK. Here’s to an exciting and prosperous year to all.

Interested in Ixis? You can find out what it’s like working with us here.

Oct 25 2014
Oct 25
teaser image for blog post

We are delighted to be working with the British Council on a new Drupal hosting and infrastructure support project. The British Council are valued clients, and we have worked with them for more than 6 years managing both the global suite of 150 country sites, and the prestigious suite of Drupal teaching and learning sites.

We will be working to to create four individual platforms for hosting key Drupal websites on, moving away from just one main infrastructure, to improve resilience, efficiency and increase availability to the sites which generate more than 35 million page impressions per month and are used by more than 65 million people each year alone.

Will also continue to provide 24/7 Drupal website support to the sites which include Learn English, Teaching English and Premier Skills English.

Martin Heineberg, websites and media coordinator for the British Council, said: “Having worked with Ixis for more than six years, we knew we could rely on them to provide the right solution for hosting the websites. As Drupal hosting and support experts, they are always quick to respond and resolve any rare instances involving technical issues.”

We are looking forward to getting started with this hosting and support project, which will be one of the largest Drupal infrastructures in the UK.

Read more at on the Prolific North Website

Oct 10 2014
Oct 10
teaser image for blog post

I'm Andy, a developer at Ixis and having just settled back in after my first DrupalCon I thought I’d wrap up my thoughts after attending the annual European conference for the first time.

Initially - wow - DrupalCon is big! I’ve only been to some smaller PHP conferences so to see over 2000 people in one place was quite something. What struck me was how well it was organised - everything was on time with very few technical hiccups. I found the number of sessions quite overwhelming - there was so much to choose from, so having the videos of the sessions online with in an hour or so after it finished was really helpful. I’m still ploughing through the ones I’m interested in.

Just a quick note on my favourite session: "Field API is dead. Long live Entity Field API!". It looks like the Field API has really grown up and using some solid OO practices. I think it solves a lot of Drupal-isms developers have had to work with in the past - this session seemed really well received with lots of applause. I really am excited about working with Drupal 8.

It was also the first time for Peter our project manager:

Still under the thousands of impressions what I've received during Drupal con :-) The event was absolutely magnificent, meaningful and enjoyable at the same time. The organisers appeared very keen to provide a high level of technical, catering and professionalism to the event.

What I especially enjoyed was that I could meet the faces of the Drupal community: developers, project managers, company owners, freelancers, people who are working on small to large scale projects. The diversity of the community was amazing. The socials which were organised besides the main event were great as well. I think the greatest challenge on the event was to find enough time to sleep and rest.

I had a number of favourite sessions and it's hard to pick just one, so try to my best and pick two:

How to sell Agile - Vesa Palmu  has really good entertainment skills. It was a well built presentation about Agile's advantages, pitfalls and some honest advice about the limitation of the methods.   

Part 2: Train Wrecks & Ugly Baby Client Meetings - Susan Rust's presentation was more like a discussion about everyday challenges of a project manager, rather then a real presentation, but definitely useful.

So altogether if you haven't been to a DrupalCon we have to tell you it was worth it. Consider next year's European one in Barcelona.

Sep 05 2014
Sep 05
teaser image for blog post

With the 2014 European Drupal conference fast approaching, the Ixis team members attending this year have scoured the schedule for their must see see sessions this year, and why.

For the Developers

Content Staging in Drupal 8 (Wednesday 10:45) - moving content about from dev to production has always been a huge pain in Drupal, so hopefully we'll get a chance to see how this might work in Drupal 8 and finally put an end to the question of when and where the client should start adding their conent during the development phase.

Drupal's PHP Component Future (Wednesday 14:15) - we're already using and developing modules for our work with Codeception, so we would benefit from learning how this might apply to Drupal 8, or even 9.

A Decoupled Drupal with Silex (Thursday 14:15) - decoupling Drupal from the front end has been a topic for a few years now. Learn from others who have jumped on early and ironed out the development problems. Relevant to Drupal 7and 8.

Drupal 8 CMI on Managed Workflow (Wednesday 13:00) - as projects get more complex the pain of the Features module becomes well known, CMI holds much hope for the future of automated repeatable builds.

Future-Proof your Drupal 7 site (Wednesday 14:15) - Until Drupal 8 is out and under some real world use we can ensure our Drupal 7 development choices make the upgrade path less painful. We'll see what's in Drupal 8 and how that maps back to the available Drupal 7 modules that are currenly available.

Twig and the new Drupal 8 Theme System (Tuesday 10:45) as we all know Drupal 8 will be using a new theme system called Twig. This session should provide an insight into what to expect from a support point of view.

For Business Owners / Project Managers

Project Train Wrecks (Wednesday 14:15) promises to be an insight from other agencies on the problems faced and solutions to keep projects running smoothly.

The Myth of the Meerkat: Organising Self-Organising Teams (Tuesday 13:00) finding the balance between processes and control when working on a project as a team in order to get things done. Leaving everybody to get on with it rarely works.

There's plenty more to attend during the week to keep everybodies minds happy - the countdown to September 29th begins, and tickets are still available at the slightly discounted rate until September 16th.

Aug 15 2014
Aug 15
teaser image for blog post

We are 10 today (ok strictly tomorrow but it's the weekend!)  We're all off to Centre Parcs over the bank holiday weekend to celebrate, and thought it was good time to reflect on some of the highlights and changes over the previous ten years.

  • It all started from small beginnings with a £1000 grant from the now defunct Business Link.  In their wisdom they decided to issue the grant as physical £100 cheques every few weeks which we found amusing at the time.  Myself and Mike bootstrapped the rest from our own pockets and became profitable quickly having no loans or overdraft, which is still true to this day. 
  • Strategy - In the early years whilst predominantly dealing with Drupal, we'd still occassionally work with other frameworks or custom PHP.   After regretting doing this on every single project, we made the decision to work exclusively with Drupal.  We also decided we would not become a full service 'jack of all trades' agency offering the likes of creative services or SEO.  This allowed us to become truely focused as a technical Drupal agency providing primarily Drupal hosting, support and development, generally with large public and private sector organisations.    
  • We were one of the first companies to be selected as a G-cloud supplier, and one of only 3 Drupal companies who joined in the first round.  We were an early advocate and have been vocal about the merits of the framework, and have seen a huge change in public sector procurement in a short space of time.  We even get to compete with (and win work from) the big service integrators.
  • We moved office for the third time in 2012, to a new 2000 sq foot office.  The intention was this space would comfortably allow for expansion, but we've managed to fill in just over 2 years (RIP our beloved pool table).  We have always had an office and whilst we agree with some of the benefits being distributed gives, I still think in hindsight this was the right decision given the services we provide.  After phasing in remote working more frequently, we sent an open survey to the team and the majority liked working in the office and wanted to be there!
  • Ixis acquired Leafish in 2012,  Dylan and Paul have been instrumental in taking over and leading our infrastructure and development departments respectively.
  • Community - We have sponsored DrupalCon and our staff have been heavily involved in organising local meetups and DrupalCamps.  We've tried to attend most European DrupalCons over the past few years and it has been great to see the community grow over time.   We've always tried to make an effort with schwag and our stress balls have gone down well, going through three iterations over the years.  We've evolved this and produced something different this year, check them out on our twitter account @ixisit later today!
  • Contributions - Ixis have provided patch contributions to Drupal Core and numerous popular modules over the years, such as Google Analytics, Cookie Control and EdgeCast
  • Colloboration - We've always worked and partnered with other agencies outside our core area of expertise, and as such have built a good network of other companies who we refer work to and from.  We like the way the community colloboration extends to various other parts of the business, so competitors end up working together and discussing things like how they sell to a certain customer base or architect a system.  In other sectors, competing companies wouldn't dream of sharing their 'secrets'.
  • Growth - Ixis have always aimed for sustainable long term growth, and the majority of our clients have now been working with us for many years as we've ensured we retention is high.  The business is now stable with a high percent of re-occuring revenue, and in the last few consecutive years we have doubled staff numbers to nearly 20 and increased turnover substantially.  Growing in this sustainable fashion has also allowed us to ensure we maintained quality and looked after our long standing clients.
  • Continual improvement - We have made improvements to the business every year and will continue to do so.   Various parts of the business have evolved and adapted to suit the market and the needs of our clients, from our infrastructure to our staff.   We've also consciously moved away from being a lifestyle business predominently made up of techies, by expanding with financial, sales and operational roles.      

We would like to extend a huge thanks to Drupal and the community for making all of this possible.  We also couldn't have done this without our clients or the staff that work with us, so a big thank you to our clients and our team for allowing us to make it this far, and for the enjoyable rollercoaster that is building a service based company.   

We wouldn't claim any of the above has been easy with every year of growth and change bringing new challenges (and increasing grey hairs!)  We've learnt from our mistakes and with the benefit of hindsight we'd no doubt do some things differently.       

We've had the pleasure of working on a really diverse range of projects, with everything from local historic club The Hacienda through to charities like Epilepsy Action and global organisations such as the British Council.

Thank you also must go to those companies who have helped us along the journey including branding, marketing, strategy and PR, technical services and mentoring (you know who you are!)

It's a slightly scary thought that after starting Ixis in our early twenties we've been doing this for over a third of our lives, who knows what the next few will hold!  We are looking forward to the next chapters and the future of Drupal...here's to Drupal 8! 

Jul 16 2014
Jul 16
teaser image for blog post

Since we integrated the EdgeCast CDN for one of our clients, and released a related EdgeCast Drupal module we have been encouraging more and more clients to consider a CDN layer to accelerate performance to multiple geographic locations and maintain an excellent uptime even during site maintenance periods.

A recent international client who is running many domains with federated content using the Domain module needed to make use of the content delivery network to improve performance and resiliance for their sites.

The use of the Domain module created a problem with the purge process employed in the EdgeCast v1.x module. As with most of the cache purging modules available for Drupal it sent each URL individually and waited for confirmation back to say it the request had been successful.

With a large multi domain site this multiplied the page load time upon submitting a node edit form to the point of a PHP timeout and Ngnix HTTP 500 errors. Increasing the timeouts was a short term fix, but it still meant content editors were slowed down waiting for forms to submit.

There was also another effect cause by external feed aggregation which created nodes, which in turn triggered URL purge requests, and slowed down each node creation step.

We toyed with the idea of using a Drupal batch queue to process the purge requests during the regular cron call but this delayed the refreshing of content pages far too long for the content editors.

We settled on what appears to be the little used Multi Curl facility. This allowed us to build up an array of all the URLs using the Expire module and then send a single HTTP request to the EdgeCast API followed by ignoring the response back from the API. This created a much faster content editing experience for the users.

Note: The Expire module doesn't currently implement Domain module purging - the code class is empty. See issue #2293217 for details. We worked around this using the Cache Expiration Alias module.

The side effect of the Multi Curl development work was the speeding up of the single domain sites purge requests too!

The v2.0 EdgeCast module is available now for Drupal 7.

Find out more about the content delivery network from Edgecast.

Jan 27 2014
Jan 27
teaser image for blog post

What's the point in using an IDE, especially a paid one, for PHP development when there are so many simpler, free tools?

In November 2013 I was lucky enough to be offered the chance to present a session at DrupalCamp NW on why it's time for Drupal developers to be using a proper IDE. The session used JetBrains PhpStorm as an example, but presented the case including other popular IDEs like Eclipse and NetBeans as well.

Drupal is getting more and more complicated, and, as developers, we should feel obliged to create code that looks good, does what it says on the tin, and works every time. An IDE provides modern functionality like syntax highlighting, step debugging, and other niceties, such as the ability to look up function documentation then and there in the IDE without needing to browse to a website. It also provides a whole bunch of error checking, like making sure you return the correct values from functions, or using static methods correctly, with zero effort required.

Coding standards is a big thing. Nobody's required to do it at the moment, but in the past, Drupal's contrib has filled up with poorly formatted code, or just code that is difficult to read or written in an alien style. In the past, we had to write our code, then run a command to review it before making a commit, which was a pain. With modern IDEs, we can be shown coding standards errors as we type them, so not only can we make fewer mistakes when we commit, but we learn instantly what kind of formatting to avoid and what kind works well.

As we move towards a more object-oriented approach in Drupal 8, away from procedural code, hopefully we will migrate to good object-oriented patterns. This often means jumping around in code more and more, since we will be writing one object per file, and sometimes method implementations are inherited from higher up in the hierarchy. An IDE makes the task a lot simpler, and you're able to see quickly and easily where an object gets its stuff from, and whether you have overridden or implemented methods correctly.

Although a PhpStorm licence comes at a yearly cost, I feel that the time saved more than makes up for the fee. Many people in the Drupal community are Apple users (or fans) and think nothing of laying down over £800 for a new Macbook! A PC would do, but many feel that the convenience and experience of using a Mac is a better way, and the same applies to PhpStorm!

In short, as a Drupal community, we already have a wealth of contributed modules that do a whole host of useful things, but now we should be focusing more on code quality and documentation, to involve more people more easily, and make sure things like updating modules and providing extra functionality are a breeze, not a chore riddled with semantic or conceptual problems. Nobody's forcing you to use an IDE, but if it'll help you write better code and make a more positive contribution to Drupal, at no (or little) extra cost, maybe it's time to make the switch.

Here is the video of the session:
[embedded content]

And the presentation slides:
Phpstorm & Drupal presentation from DrupalCamp NW 2013

Aug 08 2013
Aug 08
teaser image for blog post

With some seasoned and new members of the Ixis team attending Drupalcon in Prague this coming September we wanted to share our suggestions on essential sessions to attend from our teams point of view.

Matt P in our support team has been working with Drupal 6 & 7 for the past year and feels brushing up on what's coming in Drupal 8 for a site builder will be helpful as a primer when existing clients are begining to plan their site upgrades.

Also in our support team Adam T is keen to begin contributing back to the project so the session on becoming a contributor to the Drupal project looks like a good kickstarter coupled with the introduction to Your First Drupal 8 Module.

For the developers Paul B thoroughly recommends Alex Pott's talk about Not Invented Here a Drupal 8 Story which he previously saw at Drupal Dev Days Dublin. For those seasoned Drupal "developers" moving to Symfony and a more OO Drupal 8 framework will hopefully benefit from the Don't be Stupid Grasp Solid introduction to OOP talk.

In our DevOps team Michael A picked out some sessions around the current hot topic of containers with the Docker and Vagrant session by Acquia's Senior Cloud Systems Engineer Ricardo. We're looking forward to seeing where it goes when it eventually becomes production ready.

Designing Distributed Systems from CommerceGuys CTO looks to provide a good understanding the hard problems at play in distributed systems.

For myself there's so much variety on offer - from undestanding and making sure Ixis will be ready for Drupal 8 next year through to learning from others on how to ensure doing Support is a Blast for the team looking after our clients.

If it's all getting a bit much for your brain then we can definetly recommended the light hearted look at the history of Drupal Blocks which got a lot of great feedback at DrupalDevDays Dublin earlier in the summer.

If you've not yet decided on attending Drupalcon Prague then there's still time to grab a ticket. We'll hopefully see some of you there in September!

May 30 2013
May 30
teaser image for blog post

Shopify open sourced Dashing late last year, their Dashboard software created for displaying dashboards on TVs around their office. We decided to see have a play and see what statistics we could show on our office TVs.

Install is simple via gem, although as a non Ruby person I spent more time than I should have installing Ruby 1.9 (from source in the end). Once up the server runs on a port (3030 as standard) and applications can simply send data to the dashboard in a certain format and it is displayed immediately.

Open Atrium

We currently use Open Atrium, a Drupal based package for our intranet & support ticketing system. We decided to expose to the dashboard emergency and high priority tickets, plus a count of those unassigned in the queue and approaching our response SLA time.

The dashboard also lists the top users who have tickets assigned to them to highlight any possible bottlenecks, in the future this could easily be expanded to show other stats like 'top ticket slayers today' etc.

Extracting the statistics from Open Atrium uses a mixture of hand crafted SQL code and programatically calling custom Views displays. As Atrium uses a somewhat complex mix of Organic Groups & Spaces to control access to nodes we cheat and switch to User ID 1 to bypass all the usual Drupal access control.

function casetracker_stats_ticket_counts() {
  $counter = new stdClass;
 
  // Switch user to UID 1 to execute View queries with no access control limits
  global $user;
  $account = $user;
  $user = user_load(1);
  $counts = views_get_view_result('atrium_emergency', 'block_1');
 
  $counter->high = $counts[0]->num_records ? $counts[0]->num_records : 0;
  $counter->normal = $counts[1]->num_records ? $counts[1]->num_records : 0;
  $counter->low = $counts[2]->num_records ? $counts[2]->num_records : 0;
  $counter->emergency = $counts[3]->num_records ? $counts[3]->num_records : 0;
 
  $counts = views_get_view_result('atrium_emergency', 'attachment_1');
  $counter->sla = $counts[0]->num_records ? $counts[0]->num_records : 0;
 
  $counts = views_get_view_result('atrium_emergency', 'attachment_2');
  $counter->unassigned = $counts[0]->num_records ? $counts[0]->num_records : 0;
 
  $counts = views_get_view_result('atrium_emergency', 'attachment_3');
  $counter->security = $counts[0]->num_records ? $counts[0]->num_records : 0;
 
  // Reset user back to default
  $user = $account;
 
  return $counter;
}

Dashing Widgets can exist on multiple dashboards, but keeping them all up to date requires a single HTTP POST call with the widget name in the URI. The data payload is a simple JSON structure which is different depending on the type of Dashing widget.

We push the data every 15 minutes which is triggered by the standard Drupal cron run and implementing the hook_cron() function in our stats module.

/*
 * Send a data set to a specified widget on a Dashing board
 */
function casetracker_stats_dashing_widget_push($widgetid, $data) {
  $endpoint = variable_get('casetracker_stats_dashing_endpoint', '');
  $url = $endpoint . '/widgets/' . $widgetid;
  drupal_http_request($url, array(), 'POST', '{ "auth_token": "' . variable_get('casetracker_stats_dashing_token', '') . '", ' . $data . '}');
}
 
/*
 * Update Dashing board widgets with latest data
 */
function casetracker_stats_dashing() {
  $endpoint = variable_get('casetracker_stats_dashing_endpoint', '');
  if ($endpoint) {
    $counts = casetracker_stats_ticket_counts();
    watchdog('stats', print_r($counts, true), array(), WATCHDOG_DEBUG);
    casetracker_stats_dashing_widget_push('emergency', '"value": "' . $counts->emergency . '"');
    casetracker_stats_dashing_widget_push('high', '"value": "' . $counts->high . '"');
    casetracker_stats_dashing_widget_push('server', '"value": "' . $counts->normal . '"');    
    casetracker_stats_dashing_widget_push('slabreach', '"current": "' . $counts->sla . '"');
 
    casetracker_stats_dashing_widget_push('unassigned', '"current": "' . $counts->unassigned . '", "last": "' . variable_get('casetracker_stats_unassigned', 0). '"');    
    // Remember previous unassigned number and update if different
    if ($counts->unassigned != variable_get('casetracker_stats_unassigned', 0)) {
      variable_set('casetracker_stats_unassigned', $counts->unassigned);
    }
 
    // Staff Users
    $users = casetracker_stats_user_staff_tickets();
    $items = array();
    foreach($users as $uid => $data) {
      $account = new stdClass;
      $account->name = $data['name'];
      $account->uid = $uid;
      $items[] = array('label' => theme('username', $account), 'value' => $data['count']);
    }
    casetracker_stats_dashing_widget_push('staff-tickets', '"items": ' . json_encode($items));
  }
}
 
/*
 * Regular job processing for case tracker stats
 */
function casetracker_stats_cron() {
  casetracker_stats_dashing();
}

Icinga

Icinga is an open source monitoring system that was originally created as a fork of Nagios. We use Icinga to perform a range of checks on remote servers, such as load, diskspace or a HTTP string on a page to ensure the webserver is serving pages correctly.

When various thresholds are exceeded Icinga creates warnings or critical alerts that we can then send and display in Dashing.

We used Nokigiri to simply scrape the count of current alerts, such as critical alerts in the example below:

count = page.css( "//*[@title='Unacknowledged Services CRITICAL']" )[0].text
 
send_event('icinga-critical', value: count)

Twitter

Dashing comes with a widget to display tweets for certain search terms, we currently just look for any @ixisit replies/mentions but this could be expanded for any hash tag, or other useful Twitter accounts like Drupal security announcements

There is also another contributed widget on the additional widgets page that can parse Twitter followers / tweet counts etc.

Hardware

We currently have Apple TV's already hooked up to use airplay but there still isn't a native browser yet. We opted for a Raspberry Pi that is powered directly from the TV via USB. We've then configured Raspbian to boot straight into LXDE and open Chrome fullscreen. The total cost of the Pi, wireless dongle, power & HDMI cable was less than £40!

Future statistics

Our use case currently is slightly different as we support lots of different Drupal sites, so we just want some high level totals of potential issues rather than detailed site specific statistics. Dashing does however support multiple dashboards so the possibilities are endless.

It is interesting to think about what other information could be displayed on prominent dashboards like TVs around the office, especially from Drupal. There is a range of Drupal statistics from outstanding security updates, last failed logins to recent article submissions that could be really useful in a dashboard.

The list of third party widgets is growing with other widgets of interest including Jenkins build progress, and in the last week alone travel time from tomtom and Pingdom.

Dashing on github

Mar 12 2013
Mar 12

If you're trying to trouble shoot problems on a Drupal site and need to get some information out of Drupal then using the command line can get you answers quick.

Traditionally calling any of the Drupal API functions to diagnose something would require peppering your modules with temporary debug code, or writing a whole new module to call API functions and display results.

The command line tool drush is a swiss army knife of useful Drupal functionality, and to make our life easier in calling Drupal API functions for a specific site we can use the 'php-ev' command, often abbreviated to 'ev'. With this command we remove the need to create new modules or adding temporary code to existing modules.

Because the PHP code being run is executed in the context of the Drupal site who's DocumentRoot you are in, you can pull information from the database about entities, users and the inner workings of Drupal.

Examples of useful things you can do:

  • Print a list of all modules which are reacting to a hook being called. Here we're finding all modules that hook in to the cron system to find one which could be executing a drupal_goto() or something equally naughty.
drush ev "print_r(module_implements('cron'))"
  • determine the alias of a system path.
drush ev "print_r(drupal_get_path_alias('node/3614'))"

If you come up with any handy clever ones share them on DropBucket.

Feb 25 2013
Feb 25

We have recently launched a revamped site for Times Education targetting newly qualified teachers with the NUT.

New Teachers is a core asset of Times Education, providing teaching tips, resources, online live advice chat and the latest job vacancies for new teachers.

Ixis have worked with Times Education for 4 years, providing development and consultancy support to a range of sites including assets for headteachers and the provision of careers guidance for schools. We also provide a hosting and support infrastructure that ensures the sites are secure and performing especially under heavy traffic loads at peak times.

Times Education have partnered with the National Union of Teachers to update and refresh the New Teachers site and Ixis were commissioned to migrate the site from Drupal 6 to Drupal 7 and apply the new design theme. Rather than use the traditional Drupal upgrade process we opted to move just the content and users utilising the Migrate module.

We took a phased approach to the migration with key client involvement at every step of the way. Content was evaluated during migration to see if there was any room for improvement in both functionality and usability. Existing users had to be migrated in order to keep the SSO functioning as expected and to provide a seamless transition from old site to new.

The new site provides a much improved user experience and a clearer navigation, showcasing the content and maximising the resources provided for new teachers. Screen shots of before the site was refreshed and after:

 

The administrators now have full control over the layout of the site through a drag and drop Panels interface. This allows them to make fairly major changes to the layout of their site without them needing to dig down deep into Drupal configuration screens.  The Panels user interface worked and looked beautiful with the block based layout of the new design.

 

In addition to a more flexible site layout they can also import resources from the main tes.co.uk straight into the New Teachers site automatically thanks to integration with their XML datasets, saving time.

Finally we re-architected the single sign on user authentication system fixing long standing issues which speeds up the user experience when interacting with the New Teachers website.

Feb 24 2013
Feb 24
teaser image for blog post

Drupal 8 needs as much prodding along as it can get to make the anticipated winter 2013 launch.

With the launch in mind the next worldwide sprint weekend is coming up just after the London DrupalCamp weekend - on the 9th & 10th of March. Yes Mothering Sunday is also on the 10th in the UK so make sure you don't forget those ladies in your life inbetween git pushing code.

The global Drupal event calendar Drupical is doing a nice job of keeping a record of the sprint events coming up over the next few months.

In The UK you've currently got a choice of two venues to meet up and sprint together on Drupal 8 Core issues. Brighton and Somerset user groups are both hosting events across the two days.

If you're new to the idea of collaboratively code sprinting then you'd be well looked after over at the Drupal Ladder project site which aims to get 1% of the huge Drupal community contributing to Drupal core by 2014.

If you're involved with a user group and have some time on March the 9th and 10th to organise your own contribution to the global sprint then you should jump over to the Issue Sprint Guide for a quick read and then get it promoted on Drupal Groups UK as soon as possible.

If you don't live near a publisised sprint venue then fear not - you can still contribute to Drupal 8 issues through the Drupal Office Hours initiative which does a superb job of categorising problems for different types and levels of user wanting to get involved - you don't need to be a developer!

Jan 30 2013
Jan 30

At Drupalcamp North West it hit me that we don't actually see other Drupal companies as much we should outside of a Drupalcon/camp, so I decided to do something about it.

The first Drupal Social event was Drupal Karting, which saw a few Northern Drupal companies get together on a weekend away from work.

The guys from Livelink were a few seconds a lap quicker than the rest of us, with Jonny Glancy and Richard Lyon leading joint first into the final with Chris Cohen from Tigerfish 3rd and myself and Ashley Wright ending up 4th and 5th.

The final saw some changes with Jonny finishing 1st, Chris Cohen taking 2nd and Ashley Wright (standing in for Ixis ;) taking 3rd.

Thanks to Livelink, Hydrant, Code Enigma, TigerFish, Bashton and everyone who attended despite the weather conditions!

The feedback was all really positive, so I see no reason why we can't do more events like this once per quarter etc, anything from a walk up Snowdon to some #drupalpaintballing ;)

View more of the photos from the day in our flickr gallery.

Nov 22 2012
Nov 22

We have agreed and completed an acquisition of fellow UK North West based Drupal agency Leafish.

This latest move marks the culmination of a significant 12 months for Ixis, which has been appointed to the UK government’s G-Cloud I and II procurement programmes and secured significant contracts with the British Council and charities VSO and International Alert.

The deal will see Leafish directors Paul Byrne and Dylan Sides join the company as part of the acquisition to expand our Drupal hosting & development skillset with their combined experience of over 16 years dedicated to Drupal.

Dylan will be heading up our Drupal hosting infrastructure & QA process whilst Paul will be involved in the Drupal support, development and project management areas along with the rest of the team.

Paul Byrne, Leafish director, said:

“We are delighted to have this opportunity to work with Ixis and their clients. After a successful eight years developing and hosting websites at Leafish, we are keen to meet this new challenge and bring our business and experience to a new environment."

We are confident that the strengthened business can educate organisations about the benefits of adopting Drupal compared to proprietary models and become the provider of choice across a range of sectors.

Nov 05 2012
Nov 05

Ixis are delighted to be sponsoring the forthcoming DrupalCamp North West, which is being held at Salford University’s business school at MediaCity UK.

The event is the brainchild of the North West Drupal User Group, bringing together developers, themers and those using, or interested in, Drupal and will be held on 23-25 November.

Friday is entitled Drupal Means Business - giving delegates and businesses the opportunity to network with potential partners and learn from enthusiasts in the digital industry. The business day is aimed at Digital Managers, Marketing Managers and IT Procurement Managers.

Business sessions will include case studies from our clients including Epilepsy Action and the British Council sharing learning and knowledge on open source projects. 

Saturday will be a session-based format targeted at developers, with guest speakers and sessions held throughout the venue. Sunday gives the attendees the chance to decide the topics and formats of discussion. Popular proposed topics will be curated and groups will then enjoy a day of debate and collaborative discussions. Speakers confirmed include Morton DK and Josh Koenig, both active and prolific Drupal developers. Our very own Mike Carter will be hosting a session during the camp which is not to be missed!

We will also be recruiting during the weekend, so speak to us if you think you're right for Ixis, we are always looking for good developers with a strong PHP background, Linux Sys Admins and client services staff.

Ixis are supporting the event via sponsorship and also help with marketing and promotion of the event. Find out how you can spread the word too here.

You can book your place for the business day here and the developer days via the site here

We look forward to seeing you there!

Oct 01 2012
Oct 01

September saw our ‘Helping the Public Sector with Open Source’ campaign kick-off at the Efficient ICT 2012 conference in London.

As well as launching our hugely popular pocketsize guide to open source (OS) and Drupal, Ixis teamed up with the British Council to deliver a seminar detailing the view from both sides of the coin – the supplier and the public sector organisation switching from proprietary frameworks to OS.

Speaking with Alexei Paspalas, Head of Digital Technology from British Council, Ixis discussed its role in the Council’s web project, as well as providing key information organisations need to know when considering OS and which CMS is the right one for their project.

This was a great opportunity for the Ixis team to hear the questions and concerns about OS straight from the horses mouth.

And what was the main concern?

We were asked the question whether OS is more secure than proprietary systems. In an official statement from the Cabinet Office released in 2011, it addressed that all software has vulnerabilities and strengths and weaknesses in security characteristics, such as provenance, quality, support, and vulnerability management. The statement concluded that neither category is considered more or less secure than the other so therefore OS cannot be excluded for an options analysis for Government ICT.  Read more on the statement in this PDF on Open Source Software and Security

In addition, Drupal OS software has a community of more than 630,000 users and developers meaning people are constantly working to make sure it’s not only a cutting-edge OS platform but also that any security issues and bugs are fixed.

Our campaign trail continues this month at the forthcoming Think G-Cloud conference on 18th October at London’s Business Design Centre. We’d love to see you there and if you’d like to attend you can book your place here or drop us a line

Sep 10 2012
Sep 10

Ixis has taken delivery of a new batch of little ones being welcomed into the family as we speak.  

The new set of limited edition Druplicons are pantone matched to the Drupal logo and in my opinion, superior to their brothers. We will be distributing these along with our pocket sized guide to Drupal at our roadshow events, starting this Thursday at Efficient ICT 2012 in London.  

We will also be giving these little ones away at PHPNW 2012 conference in October and the recently announced NWDUG DrupalCamp North West in November.  We hope to see you at one of these events.  

Jun 06 2012
Jun 06

Recently we had to create a Drupal 7 multisite install that used subfolders of a single domain, rather than the standard sub domain approach.

This proved to be quite the challenge but we found the following solution courtesy of some lengthy discussion on the #drupal-uk IRC channel.

We were surprised about the lack of documentation or blog posts around such a set-up so for the future we're documenting the process.

Setting up each sub-site

In your Drupal document root create a symlink for each of your sites to the same document root folder:

cd /var/www/drupal
ln -s . site_1
ln -s . site_2
ln -s . site_3

Repeat for however many sites you wish to run as a sub folder, we've done it for 3 subsites.

Mapping to a Drupal site install

Each of your Drupal sub sites should be installed as normal with their own folder under the document root sites/ folder. Each directory containing its own settings.php pointing to a dedicated database etc.

Now within the new Drupal 7 global sites/sites.php mapping file use the following code:

<?php
$prefix 
$_SERVER['SERVER_NAME'];
$sites = array(
    
"8080.{$prefix}.site_1" => "site_1",
    
"8080.{$prefix}.site_2" => "site_2",
    
"8080.{$prefix}.site_3" => "site_3",
);
?>

Note: in the above code '8080' is pre-fixed to the path because our local development environment web server is running on port 8080. If you're using port 80 (standard) this bit can be removed. It's also worth noting that if your vhost has FollowSymLinks in it's options then this will not work, it has to be commented out.

Your Drupal sites should now be visible at http://SERVER_NAME/site_1 http://SERVER_NAME/site_2 http://SERVER_NAME/site_3

May 17 2012
May 17

Following on from last months initial release of the Cookie Control module for Drupal 7 there has been an increase in blog posts and talk both in and outside of the Drupal world about the implications and solutions to getting sites compliant in time for May 26th 2012.

This week the UK Government revealed that their own sites (some running on Drupal) will not be compliant in time for the deadline! While government websites do not carry advertising, cookies are still used to carry out various tasks, such as helping site administrators monitor levels of traffic.

If people listen to our advice and are prepared to take steps towards compliance there shouldn't be a problem," Dave Evans, the ICO's group manager for business and industry, told E-Consultancy last month. "However, if businesses deliberately stop short of total compliance, then there is a risk."

With under 10 days until the deadline we've been working on improving the Drupal Cookie Control module to provide a more flexible system and allow other Drupal modules which set cookies to be controlled by the central consent button in the Cookie Control pop-up.

Web Analytics now classed ‘Essential’ by UK Government?

As part of the latest Drupal Cookie Control release we've included a sub-module that disables Google Analytics tracking until a visitor gives consent to store cookies.

The other purpose of the Google Analytics sub module for Cookie Control was to demonstrate how to use the new callback integrations from 1.4+ and something I wanted to cover now to help get other modules conforming to the EU Directive too.

But it might not be needed now...

In a recent turn in the news it was suggested that analytics services may be exempt from the privacy directive after all. "It could, in some cases, be seen as an essential part of the relationship."

The UK’s Government Digital Service has taken a contrary stance on Analytics compared to the rest of the EU. It issued guidance to public sector websites that refers to analytics cookies as ‘minimally intrusive’ and ‘essential.’

The consensus was, especially in the case of first-party analytics cookies, these types of cookies are “minimally intrusive” (in line with the ICO guidance) and that the bulk of your efforts to rationalise your use of cookies should be focused on cookies classified as “moderately intrusive”.”

Integrating with Drupal Modules

Modules are most likely adding JavaScript code to the page using drupal_add_js() which in turn is adding cookies. This is how the Google Analytics module and AddThis module work. Other things modules often do is saving data in the global $_SESSION array.

To ensure your module isn't creating cookies on a users machine if they haven't given consent there's a few options. From the PHP side simply checking if the consent cookie has been set for the site works perfectly fine. Each site that has Cookie Control installed will use a unique cookie based on the site name. The cookie name can be obtained by calling the cookiecontrol_generatesitecookie() function. Wrap your code in some logic along the lines of:

<?php
if ($_COOKIE[cookiecontrol_generatesitecookie()]) {
  
$_SESSION['a_variable'] = '...';
}
?>

The same logic and $_COOKIE[] check can be done in a theme to wrap around snippets of JavaScript if a module is not being used.

It is worth noting that the use of server side cookie detection logic will be useless for pages which are cached for anonymous visitors. And as an authenticated user has already had to give consent to just login - consent can be assumed for any non cached page for a logged in user.

If the module is laying down some JavaScript on the client side there's more options. The first thing to do is wrap the existing JavaScript up inside a function. The function name can be anything, although a convention of prefixing it with 'cc' is suggested.

<?php
function ccAddAnalytics() { /* original script code goes in here */ }
?>

If you're trying to modify the script implemented by another module then hook_js_alter() is your friend.

Once the original script code is wrapped in a new function it needs to be added to the list of callback functions to be executed depending on the visitors interaction.

A visitor can choose to to agree to cookies being used, or close the pop-up - effectively not agreeing.

Cookie Control provides three callback list integration points:

  1. hook_cookieaccept_alter(&$callbacks)
    When a visitor clicks the consent button on the pop-up all functions registered here are executed and a consent cookie is created. The main use of this is to trigger analytics for the current page or to reveal on page content that may require cookies such as social widget buttons, Facebook Like buttons etc.
  2. hook_cookiesallowed_alter(&$callbacks)
    If a visitor has a consent cookie set already then whatever functions are registered in this callback list are executed on page load. This is where analytics code should be run. The JavaScript functions registered here can often be the same as used in the hook_cookieaccept_alter() list.
  3. hook_cookiesnotallowed_alter(&$callbacks)
    If a visitor has not given consent then any functions registered here are executed on each page load. This can be used to pop-up a reminder to the visitor about what they are missing, or redirect a user to an information page.

Registering a JavaScript function in any of the above hooks is simple:

<?php
function cookie_googleanalytics_cookieaccept_alter(&amp;$callbacks) {
  
$callbacks[] = 'ccAddAnalytics();';
}
?>

Clearing Up The Cookies

Use of the Drupal module is growing steadily over the past few weeks and is expected to continue growing as webmasters realise how close the deadline is getting.

Drupal still has a few challenges in complying as a CMS. Anonymous visitors are sometimes allocated a session cookie. Something which might not be possible to fix fully soon. However just by installing the Cookie Control module shows that you are take steps towards compliance - and that's what ICO (UK) are wanting to see.

As more sites begin using the module support for more varied cookie control blocking modules may be needed - so get in touch with Ixis if you need assistance in complying.

Cookies in the photo baked by Dries and Karlijn back when Drupal was 6 years old.

Apr 19 2012
Apr 19

As part of our commitment to supporting the Drupal Community, we sponsored a Drupal London community event on Monday evening, exploring how several UK government departments have been implementing open source software in their projects.

The agenda coming directly from Number 10 is to adopt open source software to deliver best value and Drupal is emerging as a clear favourite. The event was brilliantly attended by a wide range of Drupal agencies, Government Departments, public sector bodies and general Drupal enthusiasts.

We were treated to a rich and varied selection of presentations, including insights into the Government Open Source Action Plan from the Home Office; Running a Perfect Government Project delivered by Ofstead and a run down of how Data.gov.uk, one of the most famous Drupal projects was deployed from the Cabinet Office. Ixis have been working within this field for years and are currently putting the finishing touches to our latest work with the British Council digital project team.

We have recently implemented a Drupal content management system for the British Council, as well as managing the hosting and support for the website infrastructure.  The project will house the entire web infrastructure of the British Council international websites. Here at Ixis, we are excited by the future of open source within Government, particularly following our recent appointment onto the G-Cloud procurement framework. The presentations will be available shortly so you can share the learning from the event, so watch this space.  The first set of introductory videos can be viewed here

What was most interesting for me was how the Drupal community is committed to sharing best practise and collaboration on projects to work with Government and the Public Sector. Our objectives and passion are shared – to deliver excellent open source projects whilst sharing the code and learning from each other. In what other community does this happen in? The evening was topped off with the usual prizes for the best tweets and a social at the local pub after the presentations – well, it'd be rude not to! 

Apr 03 2012
Apr 03

Epilepsy Action is the largest member-led epilepsy organisation in Britain, acting as the voice for the UK’s estimated 600,000 people with epilepsy. Originally, Ixis provided support for the community site www.forum4e.com, which consists of user forums and blogs, before successfully winning the contract for the full site migration of www.epilepsy.org.uk to Drupal, as well as on-going hosting and support. 

The aim was to improve user experience and increase the range of information available to create more opportunities to encourage support. Ixis was recently commissioned to upgrade the site to Drupal 7.  The new site provides easy access to rich content and the ability for users to customise their experience, personalising their homepage and the content that is delivered to them.  The site is now fully responsive using the Omega base theme, delivering the content direct to mobile devises such as iPad and iPhone.

The site went live in time for Purple Day, the worldwide epilepsy awareness day, held annually on 26 March.  On that day, we turned the site purple for visitors! 

Mar 24 2012
Mar 24

The EU Privacy Directive was announced in May 2011 to much groans and disagreement. In 2012 we see the rule being enforced from May 26th - but are you prepared yet?

The UK government has updated the Privacy and Electronic Communications Regulations in response to the EU Privacy Directive but many UK websites have probably forgotten, or chosen to ignore, the upcoming changes with a risk of being fined up to £500,000 for a serious breach of the law.

What is the EU cookie directive?

The aim of this legislation is to increase online security and data privacy, giving users more control over what data can be held about them. It addresses how personal information is held and used.

The legislation forces websites to be transparent about how they are using cookies, detailing exactly what information each cookie holds and how long it will be held, and requires them to actively request permission from their users before cookies can be used.

Previously, the law dictated that websites had to explain how they were using cookies and how users can ‘opt out’. Most sites did so in their Privacy Policies, but this isn’t enough under the new law:  users now have to ‘opt in’, having been made fully aware of the implications of doing so.

How sites reacted to the new requirements has been mixed, and even more difficult for organisations who have had their site custom built long ago and no longer have access to the same developers to shoe horn in the changes.

Thankfully open source content management systems like Drupal bring a number of bonuses to the user:

  • You're not alone having to support the new changes as there's thousands in the same position as you on the same platform as you.
  • It's quite likely somebody has already done the research and solved the problem and can be used as a reference point for others.
  • Use of APIs make it easier to integrate new features around existing functionality on the website.

What cookies are used for

Popular cookies on a large number of sites come from a few sources:

  1. Web analytics software, such as Google Analytics, counts the number of visitors to each page of a website as well as how often the same person returns to a website.
  2. Banner advertising often employed to fund the websites content and development often use cookies.
  3. Having the ability for visitors to log in to the website and customise their experience.

There are lots more - from simple things like having a YouTube video or Google Map embedded on your site, sharing pages with social networking sites, to simple things like allowing the text on pages to be made bigger or smaller and remembering the selected size.

For more in-depth details read the different types of cookie usage and what "level" they fall in to blog post at cookielaw.org

To find out what cookies your site is setting enter your website url on the Cookie Cert database site. It can take as long as two hours for your site to be checked, so don't expect an instant result!

What others are doing

We've taken a look at some UK Government sites to see how they are implementing their own rules.

gov.uk - includes a 'beta warning' modal pop-up which includes a message "N.B. This site uses ‘cookies’ and Google Analytics. Closing this page sets a cookie so you don’t see it again. There’s more information on cookies at AboutCookies.org." Every page also contains a link in the footer pointing to their very clear and helpful cookie information page.

bt.com - this is a really slick and informative user experience for cookies. Click the 'change cookie settings' link in the footer to reveal a pop-up detailing all the cookies being set, their purpose, and a nifty slider to control how many cookies are used.

ico.gov.uk - displays a drab almost hacked in like message box at the top of their site with a consent tick box. A good example of how the cookie consent requirments could damage your sites nice design.

The Solution

cookie control pop-up user interfaceTo address the requirements of the cookie law we need to have consent from the site visitor before any cookies are set. Before consent is granted by a user they should be provided with information about what the cookies will be used for and your sites privacy policy.

The Cookie Control widget from Edinburgh based CivicUK appeared to be an elegant and consistent answer to the cookie requirements. The user interface provided a simple pop-up in the bottom corner of a visitors web browser with minimal options to complicate things.

The Cookie Control is added to any site using JavaScript along with some configuration options to fit with your site.

Drupal gets it easy

For Drupal 7 powered websites Ixis made it even easier with the development and release of the Cookie Control module on drupal.org to wrap up all the configuration options and Javascript code embedding in to a Drupal administration web page, easy!

For developers there's a few JavaScript callbacks to hook in to. These should be used to only execute JavaScript which uses cookies if the user has already given consent. Details can be found on the Cookie Control project page and in the README.txt file provided with the module. One example use for these functions is to only run the analytic tracking code when consent is granted.

CivicUK are working on some new additions to the Cookie Control project which we'll be integrating in to the Drupal module as soon as possible. In the mean time - if you're running a Drupal 7 site for EU visitors it would be well worth considering installing the module earlier than May 26th to ensure you comply.

To find out more about the Drupal module and download the code visit http://drupal.org/project/cookiecontrol

Feb 04 2012
Feb 04

There truly is never a dull moment at Ixis HQ and this week I have been taking part in an election process in a bid to be a director at large of the international Drupal Association board.

The Drupal Association is a non-profit organisation dedicated to helping the open-source Drupal CMS project flourish. They help the community with funding, infrastructure, events, promotion and distribution.  I jumped at the chance to be involved in the process and I am delighted that I decided to.

My motivation for joining was to give something back to the community that has welcomed me so warmly and to use my experience of marketing, fundraising and business development to develop new opportunities for Drupal.

My overall vision is to drive forward Drupals promotion as a CMS of choice for business, government and public sector organisations. From my initial research, the UK Government is pushing for Open Source Systems and in particular Drupal for implementation across all local authority sites. I think this is a great time to become involved in pushing forward this agenda and proving the value.

I am also keen to promote Drupal due to my first hand experience that it can lead to job creation and training opportunities, which is of particular importance in this economic climate.

Finally, I am keen to encourage more diverse groups, particularly women into the Drupal community. This may be through promoting programming as an alternative career choice and develop more female leaders and management within the community. I felt “qualified” to get involved as I have taken a variety of advisory and Board member positions throughout my career.

The Association organised 2 “meetings” for members of the community to pose questions to the nominees and get to know each candidates ideas.  The first took place at 1am UK time along with representatives from Australia, Brazil, the US and China.  It was a perfect opportunity for us to share our vision for participation on the board and to hone our Q and A skills.  I wasn’t too sharp at that time of the morning but I was wide awake at the end of the call, my head was buzzing with ideas and full of enthusiasm. 

The second meeting took place the next afternoon so I was in the office surrounded by the Ixis giddy kippers keen to get in on the action!  I enjoyed the second call equally as we had a chance to meet more of the community and field additional questions. 

I’ve highlighted below some of the discussion points and my responses:

  1. How do we reach out to new members?
    It's hard to engage people who are not necessarily already involved. The key is to reach out, especially to women and more diverse groups.
  2. What are the barriers to accessing the community?
    Not having access to support groups is a huge barrier, as is the size of the network that developers can rely on.  In terms of developers finding leverage they've got access to within the community, marketing materials, wealth of community without access to the community we are just like any other software so this is a strength we need to better promote.
  3. What’s the Drupal Associations role?
    To harness and to promote the successes of Drupal and therefore allowing others to shine as well rather than instead of just focusing on specific shops/companies My role would be to support and encourage the community and promote what we can do together.  From my experience with working with smaller organisations and 1 man/woman bands, any access to learning and support is always great for small shops.
  4. What Key Skill Can I bring?
    My previous experience in marketing/fundraising is key here as is my understanding of client requirements and community needs and how to bridge those two - understanding of clients’ requirements, developers’ needs and how to bridge those.

Whilst I realise I might be considered relatively unknown in the Drupal community (despite meeting and greeting 1500 of the community at DrupalCon London last summer) I am grateful for an opportunity to present my perspective on how we can strengthen the board and the future of the association. 

I’ve also been bowled over by the other candidates who have shown enthusiasm and dedication as well as encouraging each other in the process.

Whatever comes out of this process and who ever is taken onto the board, I know that I will continue to be a Drupal evangelist and support the future of open source.

Once you've an idea on who to vote for you can place your candidate preference on the voting form. After clicking through, you will be asked to rank each of the eligible voters, from 1st (top choice) to 10th (last choice).

Voting closes at Midnight Tuesday 7th February and you must already have a drupal.org username to vote.

Jan 19 2012
Jan 19

The Nuclear Skills Academy corporate website has grown over the years in to a commerce platform supporting their industry certification alongside their service and corporate information. 

As their virtual learning service has expanded Ixis were briefed to create a more flexible and improved foundation for the Nuclear Skills Academy web platform.

We decided on a combination of Drupal 7 and the Commerce project to re-create the site on.

The original site was based on Drupal 5 and Ubercart for the commerce transactions - along with organic groups with a lot of content and users. The virtual learning was and still is based on the Moodle system which is tightly integrated in to Drupal and Commerce with some custom integration modules.

The project involved a great deal of content and user migration along with discovering many gaps in the Drupal Commerce project. It was the first time we'd undertaken a Drupal 5 straight to Drupal 7 upgrade on a large site and it became apparent that the contributed modules can leave holes in the upgrade process which had to be manually filled with custom migration scripts.

The outcome was a vastly improved content administration environment for the Nuclear web editing team and a neatly integrated commerce system which will enable growth in the virtual learning space.

There are clearly still areas of Commerce which need to be polished off to make it a great viable product for clients to use to the fullest, but the beauty of Drupal allows us to work on and deploy the improvements over the coming weeks and months thanks to Features.

The Nuclear Skills Academy website is the first in a series of client upgrades to Drupal 7 for Ixis.

Oct 26 2011
Oct 26

Ixis was commissioned to create a website, TES Growing Ambitions, for the leading digital education business TSL Education, to support teachers helping young people discover the right career path.

At a time when the delivery of careers advice in schools is changing, TES Growing Ambitions will be a dedicated portal for the teaching profession to access a relevant and diverse collection of multimedia teaching resources for free, as well as being able to share information and best practice with peers and the wider professional careers community. 

Ixis was briefed to create a websites which would support the teaching profession and embed careers across the curriculum to ensure that young people have access to quality IAG as part of their learning. As part of this brief, Ixis was asked to provide for a range of features including an A-Z jobs search, multimedia resource platform, news and Q&A. The website can be found at growingambitions.tes.co.uk/

Stephanie Hosny said:

“We wanted to create a website which was as fresh and ambitious as the ethos behind the project. We have developed a Drupal based Content Management System to support the integration of media content and support teachers to search for the right information for their students. We hope the site will inspire and motivate young minds.”

Oct 25 2011
Oct 25

Let me introduce myself – I'm Steph, the newest member of the Ixis team. I've now completed 6 weeks in my new role as Business Development Manager at Ixis.

The directors have taken a leap of faith employing a non-techie to drive forward their thriving Drupal development, hosting and support business.

Although I was familiar with open source systems in a previous life I had only really dipped my toe into the world of Drupal. I must admit I was a little anxious that everyone would laugh at my amateur hour attempts to converse in Drupal speak, especially as my initiation test was to “go to London, set up and run an the Ixis sponsored exhibition stand at Drupalcon” – the national Drupal conference attended by 1500 people from 54 countries - an easy first day on the job then?!

However, I spent 4 days in the company of some of the most awe inspiring and creative people I could have hoped to meet, which was probably the best training I could have wished for and the whole event was a great success for the business and raising our brand profile.

I'm now picking up new techie terms every day, from Drush to Puppet and not forgetting the cURLS

Most recently, I've been working on proposals for our larger clients including the British Council, the British Epilepsy Association and launching TES Growing Ambitions.  

Its also been a hugely productive month in the hosting department, despite being a member of staff down there have been several new launches and lots of progress towards automating provisioning of new servers (hat tip to Peter L .)

Finally, I'm preparing the office ready for our new starters as the recent recruitment drive has been going well.  We are always on the look out for new staff members though so do keep in touch.

Here are some highlights of my first 6 weeks living in Drupal world:

  • Learning what its like to share an office with 8 techies

  • Meeting and speaking to our lovely clients

  • Seeing that despite the doom and gloom spread about the recession, that businesses who are ripe and ready for work can thrive

  • Meeting the lovely folk at Daresbury Science and Innovation Campus who constantly strive to support our businesses

  • Meeting Prince Andrew HRH who came to launch the new Enterprise Zone building Vanguard House

  • Attending T-Shirts and Suits coffee morning hosted by David Parish at the beautiful new Leaf Cafe on Bold Street

  • Seeing Mike C – one of our directors being interviewed for BBC Granada news

  • Rallying the team to take part in “organised fun” including quiz nights and our contribution to the British Epilepsy Association tea party charity fundraiser

  • Eating my body weight in biscuits whilst introducing the team to my weird and wonderful tea collection

I'm now looking forward to planning how I can continue to drive forward the Ixis business plan and get more enthusiastic developers, site technicians and linux sys administrators through the door as we try to keep up with demand.

So this time in 6 weeks whilst you are warming your mince pies by the fire getting ready for Santa to bring you one of our Druplicon Stress Balls, I'll still be dreaming of a Drupal Christmas.

Sep 12 2011
Sep 12

The Drupal Association Community Cultivation Grants handed out their first round of funding to produce a nice video short from the Brighton Area Drupal Association usergroup (UK).

The video gives a brief introduction to reasons people have chosen Drupal as their content management system of choice for business and pleasure.

If you're local to Brighton and fancy joining these guys each month - find out more about the meet-up on the BAD Ass website.

Aug 18 2011
Aug 18

Designed by Mark D, the Drupalcon London 2011 edition comes in 3 limited edition Union Flag colours and features the grinning Drupalicon on the front and our new branding on the reverse.

Come visit us at our 1st floor booth area where we'll have sofas for a sit down to re-charge your batteries and chat about our support, hosting and development services.

For anybody attending the conference we're also offering 12 months Drupal managed hosting for the price of 10 on our shared Rackspace platform.

The conference exhibitor area runs from August 23rd-25th 2011.

Jun 28 2011
Jun 28

At drupal7camp in Leeds I presented about how we develop and deploy our sites using Git and various Drupal tools.

I talked about the pros and cons of using Features and various different tips and tricks that we find really useful when developing.

Here is a copy of my presentation uploaded to Slideshare.

Overall the weekend was a huge success with a good attendance and session list. Thanks to all those involved in making it a great get together!

Jun 21 2011
Jun 21

Having been a Drupal exclusive business since inception 7 years ago we've seen Drupal evolve from version 4.6 in to the power of Drupal 7.

We've treated previous Drupal Camps and Drupalcon more as a staff training opportunity - especially for staff moving in to the Drupal field full-time. It's been a great success for learning about new ideas, techniques and even for crash courses on existing components we've not yet had chance to play with at the Ixis office. This year will be no different as we take the whole company down to London for a week of mixing with fellow Drupal users and businesses.

drupalcon gold sponsorFor the London 2011 conference we're a gold sponsor which brings with it booth space to promote our business services of developmentmanaged hosting and Drupal support. If you'd like to learn more about our services, meet us in person for a chat or to simply sit on our sofas and charge your laptop - then hunt us down at the conference.

We will be located at the top of the stairs on the mezzanine level this August.

See you there!

May 17 2011
May 17

On the 21st-22nd of May Leeds will be hosting the Drupal 7 Camp.

I'll be representing Ixis at the Camp. I'm hoping to be presenting about our internal Development/Deployment processes and possibly providing an introduction to Drupal 7 module development. Expect a full blog post on both presentations after the event.

The camp is a great opportunity to mix with other Drupal developers and people from the industry. If your around feel free to come and say hi.

With the promise of 144 pints of locally brewed beer it promises to be a great event! Tickets are still available - get yours here.

May 17 2011
May 17

The web game Quizible originated from the popular Microsoft Excel spreadsheet picture based games from the late 90's.

The Drupal powered site was developed as part of an in-house training month at Ixis. The mobile project came about as we looked for a new way to utilise the data Quizible had collected.

The Drupal Benefit

Drupal allowed us to offer users the ability to make their own quizzes and for everyone else to play them whilst achieving a score on the leaderboard. Over the next two years Quizible was featured on Techcrunch, it became popular on social sites such as Stumbleupon and Facebook - gaining over 20,000 users and hundreds of user generated quizzes.

The quiz data held in Drupal is built on the node system - each question is a single node and in turn a collection of nodes were attached to a quiz node by means of multiple node reference fields. The storage of quiz data like this meant it's very easy to re-cycle the data for other uses.

Over the years we noticed the Quizible website was being accessed using various Apple iOS devices using the Google Analytics module. Playing a quiz on the Apple iPad was reasonable but the iPhone was awkward to enter text in to the answer box due to the standard iPhone pop-over keyboard.

The Mobile Interface

When the mobile development began the alternative frameworks such as PhoneGap and Titanium Appcelerator were very rough and incomplete. So we opted for the traditional route of Apples Objective-C programming language.

The Apple user interface already has a common image browser control known as "cover flow" in the iPod application. We made use of this and populated it with the images for each question - simply using a different ImageCache preset to resize the original image to fit the small screen.

Taking on board the problems of text entry on a small screen we devised a new answer input system for the mobile device. The hangman style letter input cut the amount of key presses required to input the answer.

The Data Integration

The Quizible mobile game was to be distributed with no quiz data bundle - meaning users could pick what quiz they wanted to try before downloading all the image data. We needed to keep the over-the-air data transfer as small as possible to make it playable over 3G networks whilst players are on traveling to work or idling out in public.

The Drupal node data is served over HTTP using a combination of the Services package and Views with Views Datasource providing the JSON output style. The iOS application was given an authentication key used to authorise itself against the Quizible RPC server when requesting data. We chose the JSON-RPC server module for encoding the data during transit - this version of the JSON server was selected for its compatibility with the JSON framework for iOS.

As the Quizible website already has a number of Views configured to return latest quizzes, list of taxonomy categories, top rated quizzes etc we built on these by simply adding a new View Display with a JSON style formatter. These views provided the read-only data consumed by the mobile application to power all its main screen menus.

The RPC Services provides user login capability and access to actual quiz data - including the answers, hence the need for key based authentication for this data. Score submissions for the mobile leaderboard are also carried out with the RPC service.

The development of a custom RPC service module was relatively straight forward as it's nothing more than a hook_menu() like structure of methods and associated callback functions accepting arguments in hook_services() and processing data before returning it for encoding as JSON data and sending to the mobile device.

User Authentication

We'd built up a considerable Drupal user database over the year - so it was essential we allowed these users to login and use their existing account on the new mobile application. However we didn't have time to create a user registration process for the phone and we still wanted users to be able to record their scores on our leaderboard.

Facebook connect was the obvious integration, allowing us to provide new Quizible users with a quick way to get going without having to visit the quizible.com site to register for an account. When a user authenticated with Facebook we get their Facebook unique ID and send it to a custom login RPC web service which checked to see if the Facebook ID maps to a Drupal user ID. If a match is found the Drupal $user account object  is returned. If no association to a Drupal user is found, a new Drupal user account is created and a mapping to the Facebook ID is added. The login service then returns the Drupal $user account object back to the mobile application for use with the mobile leaderboard.

Keeping Score

Our mobile game didn't have the same user interface as the web based version so the scoring system had to work in a completely different manner. This meant providing a dedicated leaderboard for users to compete on.

Two types of score leadboard exist - the per quiz leaderboard, and the main global leaderboard. We also needed to keep the score per quiz-question as users could re-visit any quiz to see their previous progress - and the score per question was based on how many right and wrong letters they had guessed so far. We left the per question score calculation to the iPhone app as it would prove to expensive to be calculating this on the server during submission.

When scores were submitted back to the Drupal webservice it triggered a re-calculation of the users overall score for the global leaderboard, and cleared the leaderboard cache to show the new scores.

Keeping the leaderboard up to date posed some challenges for us. 

There was never a definitive end to a quiz, so when should the score be added to the leaderboard? We opted to cache the score in the iPhone application and resync when a user exited a quiz.

What if the user was off-line when they finished a quiz? The application stores a list of all quizes and scores. The network connection was checked regularly in the background and when a connection is made the scores are submitted back to the Drupal site via a custom webservice. The actual leaderboards are still visible whilst offline by using a HTML5 Manifest cache.

Performance

We've read many stories about how iPhone applications had exploded in popularity and killed servers due to traffic overload. The problem was in the back of our mind whilst building the Drupal backend of the iPhone application.

The main Quizible website is backed by Varnish caching for the pages - but this wasn't possible for the JSON web service calls to be cached so all webservice communication is carried out on a sub domain aliased to the same server. This allowed Varnish caching to be ommitted from the domain and gave us scope for moving the mobile API off to its own front end server in the future if traffic demanded it.

Launch

The approval process was mostly painless - the initial rejection from Apple was regarding user generated content of questionable copyright content - mainly Disney and Pixar movie stills (I wonder why? :-) )

Thankfully Drupal's CCK, Views & Rules tools made it easy to build a Quiz moderation queue, flagging to indicate the Quiz was "mobile friendly", and an email notification system to alert moderators that new Quizes were available for review! After showing our co-operation to Apple the app was approved for sale.

Our launch timing could not be more perfect - we hit the Apple AppStore Christmas eve deadline perfectly - meaning no other apps would be approved until after the Christmas holidays, giving us some opportunity to get noticed in the new releases ... maybe. 

Marketing

The app went live just before Christmas, we submitted a press release and posted various promo codes on popular forums such as Touch Arcade and twitter.

On the existing Quizible site we decided to redirect users using iPhones or iPads to the dedicated promo splash page by simply checking the useragent of the client. Little tweaks like this have helped to bring in additional sales each day.

After the initial launch we managed to hit the top 10 in the trivia games category which resulted in a large increase in sales, we also gave the application away for free for one day which saw it jump to number 1 in the free trivia section (and as a knock on effect we went to #1 in the trivia section in Taiwan!)

Quizible hasn't quite had the success of say Angry Birds but continues to make modest revenue which is nice given the project was initially a proof of concept in Drupal!

Sep 09 2010
Sep 09

Come and join our well established web development studio as the lead developer of our PHP based team.

We build interactive sites of no fixed type, so every project can be very different to keep you on your toes! You'll be involved with a number of projects at any one time ranging from international brands to charities and start-ups. We also run several internal projects which you'll have chance to provide input and development for - this is where we often experiment with new ideas first.

Ideally you'll have several years (3+) PHP experience, but also the knowledge of best practices when working as a team on projects. It will be your responsibility to take the lead on technical decisions for projects.

Key Skills we're looking for are:

  • PHP 5, OOP.
  • Javascript & frameworks such as jQuery.
  • SQL & query performance analysis.
  • Development methodologies & code control for iterative release processes. We're moving to Git for SCM.
  • Performance and scalability techniques.
  • Strong problem solving and debugging skills
  • Excellent verbal and written communication skills for client interaction.
  • Excellent organisational skills, including the ability to work under pressure in a fast paced environment across multiple small and large scale projects.

Your work will cover a variety of areas, including:

  • Building new applications and sites in PHP from the ground up with the Drupal framework.
  • Extending existing Drupal projects which may have been developed outside of the company previously.
  • Providing email and phone technical support around Drupal projects we've built for clients.
  • Experimental mobile application development using web technologies.

As you may notice, we love Drupal! So previous experience of the technology and API is helpful but everybody in the team is on hand to provide help when needed. There's a chance to play with the latest technologies and contribute back to the Drupal open source community as part of your daily work.

The majority of your work will be carried out on-site in our office1 in Daresbury, Cheshire, UK, but there will occasionally be some travel involved for client meetings in London and anywhere in-between. Therefore this position will only be suitable for UK residents who are already located in the North West of England or who are willing to relocate to the area.

Why Work For Ixis?

We've been in the Drupal business for over 6 years, it's all we do. We think it keeps our staff focused and highly skilled by avoiding the distraction of Wordpress, Magento or Joomla.

We run a fun, friendly, and relaxed office environment as well as beer in the evening opportunities outside of work. Our office hours are flexible to work around busy traffic periods.

We aim to give all staff members as much opportunity for suitable training as possible. This year we took the whole development team to the Drupal conference in Copenhagen for a week. PHP5 Zend certification is also available.

Salary: £negotiable - Tell us what you're worth and why.
Hours: 8 hours a day to be taken between 8:30am and 6pm.
Apply: Submit a short cover letter, salary expectation and a copy of your CV via our online application form.

1 ability to play Pool is not a requirement for the job.
NOTE: Job agencies need not apply. We're not interested. Thanks.

Job position filled

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web