Upgrade Your Drupal Skills
We trained 1,000+ Drupal Developers over the last decade.
See Advanced Courses NAH, I know EnoughCodeception modules for Drupal acceptance testing
Over the last couple of years, we have been using Codeception at Ixis for running automated acceptance tests during development work. Over this time we attempted to distil some of the ideas and abstract custom code into Codeception modules, which are all available on GitHub.
Adopting Codeception for automated acceptance testing has been quite the learning process: getting used to the Codeception system, best practices and underlying code; facing challenges when writing tests for unusual or difficult situations; moving from the cURL-based PhpBrowser to the more advanced and complex WebDriver; problems with underlying components such as Selenium and PhantomJS; lack of integration with Drupal... phew! And that's not before we've considered processes, methods and best practices for running automating testing itself!
The latter, however, is a story for perhaps another time. This post summarises the Codeception modules we've worked on, why they might be useful and some lessons learned.
Drupal Content Type Registry
Drupal Content Type Registry is a module to provide a set of classes that encapsulate Drupal content types. This makes it much easier to quickly test standard Drupal functionality relating to content types, taking into account how they exist on your site. It enables testing of things such as the content types admin page, the 'manage fields' page for each content type, and provides createNode()
and deleteNode()
methods that can be used to quickly create test nodes where you can provide the test data using specific values, random values, or a range of values where one is picked at random.
Once enabled, a contentTypes.yml file should be created which defines:
- GlobalFields - fields that will be used across all of the content types on the site. This is useful for things like title and body fields, to save you having to redefine the exact same field on every content type.
- ContentTypes - content types on the site and the fields they have (global or otherwise).
Here's the example from the README:
GlobalFields: body: machineName: body label: Body type: Long text and summary selector: "#edit-body-und-0-value" widget: Text area with a summary required: true title: machineName: title label: Title type: Node module element selector: "#edit-title" ContentTypes: news: humanName: News machineName: news fields: globals: - title - body field_image: machineName: field_image label: Image type: Image selector: "#edit-field-image" widget: Media file selector required: true testData: "image1.png" field_icon: machineName: field_icon label: Icon type: Text selector: "#edit-field-icon" widget: Text field skipRoles: - editor - publisher testData: - smiley - grumpy - happy - wacky preSteps: - ["click", ["#button"]] - ["fillField", ["#the-field", "the-value"]] postSteps: - ["waitForJs", ["return jQuery.active == 0;"]] submit: "#edit-submit-me-please"
The definition of fields comprises the machine name, label, field type, widget and CSS ID or selector used when filling in node edit forms. Similarly, the definition of content types comprises the machine name, human-readable name and a list of field definitions. The module also provides a set of classes representing most field widgets provided by Drupal 7 core.
There are other features too, such as: preSteps
and postSteps
, which allow optional steps to run before and after filling the field; testData
, which allows specific test data (literals or random text) to be used for each field; skippedRoles
, which skips certain fields for specific user roles when creating nodes; and "Extras", which simulate the user clicking things on the node edit form that are not actually fields, like set the sticky status or the publication status of a node. For more detailed information on the module's configuration, see the README file.
This module (in combination with Drupal User Registry) also provides enough to create close to generic tests that will check your Drupal 7 site for all expected content types and fields. See this example Gist.
Credit must go to Chris Cohen for his original work on the Content Type Registry.
Drupal Drush
Drupal Drush allows the running of Drush commands in acceptance tests. It also allows the use of the following statements in tests:
// Execute "drush cc all" $I->getDrush("cc", array("all"))->mustRun();
The getDrush()
method returns and instance of Symfony\Component\Process\Process so you can read stdout, get the exit code, etc. The ->mustRun()
method is useful as a Symfony\Component\Process\Exception\ProcessFailedException exception will be thrown if the command fails, meaning your test will automatically fail.
Drupal Mail System
Drupal Mail System allows the testing of the Drupal mail system.
// Test to see expected number of emails sent. $I->seeNumberOfEmailsSent(1); // Clear emails from queue. $I->clearSentEmails(); // Check email fields contains text $I->seeSentEmail(array( "body" => "body contains this text", "subject" => "subject contains this text", ));
Relies on TestingMailSystem class which stores the emails in a Drupal system variable.
Drupal Pages
Drupal Pages is a dependency of some other modules listed here and contains several PageObjects for "generic" Drupal 7 pages, based on "vanilla Drupal 7' and the default Bartik front-end or Seven administration themes. These PageObject classes can be extended to override any static properties as required for the site being tested.
Drupal User Registry
Drupal User Registry is a Codeception module for managing test users when running acceptance tests with Codeception. Once configured, it can automatically create and delete test Drupal users at the beginning and end of a test suite run. These users can then be used during acceptance tests to login and test elements of the project specific to that role.
The module is configured in the suite configuration:
class_name: AcceptanceTester modules: enabled: - PhpBrowser - DrupalUserRegistry config: PhpBrowser: url: 'http://localhost/myapp/' DrupalUserRegistry: defaultPass: "foobar" users: administrator: name: administrator email: admin@example.com pass: "foo%^&&" roles: [ administrator, editor ] root: true editor: name: editor email: editor@example.com roles: [ editor, sub-editor ] "sub editor": name: "sub editor" email: "[email protected]" roles: [ sub-editor ] authenticated: name: authenticated email: authenticated@example.com roles: [ "authenticated user" ] create: true delete: true drush-alias: '@mysite.local'
- defaultPass - use this password for all created user accounts, unless they have one individually specified.
- users - a list of test user accounts to create, complete with username. email, password and a list of roles.
- create and delete - whether to create and delete users at the start and end of a run.
- drush-alias - the Drush alias to use when managing users via
DrushTestUserManager
.
Once configured, the module also allows the use of the following statements in tests:
// Returns a DrupalTestUser object representing the test user available for // this role. $user = $I->getUserByRole($roleName); // Returns a DrupalTestUser object representing the test user available for // exactly these roles. $user = $I->getUserByRole([$roleName1, $roleName2]); // Returns a DrupalTestUser object representing the user, or false if no users // were found. Note this will only return a user defined and managed by this // module, it will not return information about arbitrary accounts on the site // being tested. $user = $I->getUser($userName); // Returns an indexed array of configured roles, for example: // array( // 0 => 'administrator', // 1 => 'editor', // 2 => ... // ); $roles = $I->getRoles(); // Returns a DrupalTestUser object representing the "root" user (account with // uid 1), if credentials are configured: $rootUser = $I->getRootUser();
This module is used in any acceptance test suite we create in order to test specific elements where being logged in as a user with specific roles is necessary. However, there are some limitations and improvements that could be made.
The user registry can be used in combination with codeception Step Objects and Drupal Pages to provide login and logout functions to enable testing as an authenticated user. For example:$ php codecept.phar generate:stepobject acceptance AuthenticatedSteps
class AuthenticatedSteps extends \AcceptanceTester { /** * Log in. * * @param DrupalTestUser $person * The Drupal Person to log in. */ public function login(DrupalTestUser $person) { $I = $this; $I->amOnPage(UserAccountPage::route('login')); $I->expectTo('not be redirected due to already being logged in'); $I->seeCurrentUrlEquals('/' . UserAccountPage::route('login')); $I->expectTo('see various elements of the login page'); $I->seeElement(UserAccountPage::$loginFormUsernameSelector); $I->seeElement(UserAccountPage::$loginFormPasswordSelector); $I->seeElement(UserAccountPage::$loginFormSubmitSelector); $I->amGoingTo('fill in the login form'); $I->fillField(UserAccountPage::$loginFormUsernameSelector, $person->name); $I->fillField(UserAccountPage::$loginFormPasswordSelector, $person->pass); $I->click(UserAccountPage::$loginFormSubmitSelector); $I->expectTo('log in successfully'); $I->dontSee(UserAccountPage::$loginFormCredentialsErrorMessage, Page::$drupalErrorMessageSelector); } /** * Log out. */ public function logout() { $I = $this; $I->amGoingTo('log out'); $I->amOnPage(UserAccountPage::route('logout')); $I->expectTo('be redirected to the front page'); $I->seeCurrentUrlEquals('/'); } }
/** * Test pages. * * @guy AcceptanceTester\AuthenticatedSteps */ class ThePagesCest { /** * Test the page as an authenticated user. * * @param AuthenticatedSteps $I */ public function testThePage(AuthenticatedSteps $I) { $I->login($I->getRootUser()); // ... $I->logout(); } }
There's potential for integrating the login()
, logout()
and other related functionality into the Drupal User Registry module itself, but still allow the test suite author to override or specify these procedures themselves.
Drupal Variable
Drupal Variable allows us to test Drupal system variables, for example:
// Assert that the target site has variable "clean_url" set to 1 $I->seeVariable("clean_url", 1); // Set a variable. $I->haveVariable("clean_url", 0); // Delete a variable. $I->dontHaveVariable("clean_url"); // Retrieve a variable value. $value = $I->getVariable("clean_url");
This module was the key to working out different "connections" to Drupal itself. Previously, we were restricted by the assumption that everything needed to be done via the browser in acceptance testing (see the lessons learned around creating test content via the test browser, in Drupal Content Type Registry above).
Drupal Variable can connect to a Drupal site using one of three methods:
- Bootstrapped - if the site is accessible on a locally mounted file system, fully bootstrapping the site is possible. In this case we use the
Codeception\Module\Drupal\Variable\VariableStorage\Bootstrapped
class and ensure we set thedrupal_root
configuration setting. - Direct connection - if the site is remote but we have access via an open MySQL connection, we use the
Codeception\Module\Drupal\Variable\VariableStorage\DirectConnection
class and set adsn
,user
andpassword
. - Drush - if the site is remote but access via a Drush alias is available, we use the
Codeception\Module\Drupal\Variable\VariableStorage\Drush
class and ensure we set thedrush_alias
setting in the module's configuration.
In review
Problems with creating test content in the browser
Creating lots of test content in the browser with Drupal Content Type Registry has proved problematic in places and in hindsight not the best way to achieve our goal. Many of our development projects and supported sites have additional elements that make things that little more difficult too, such as alternative publishing workflows with Workbench or WYSIWYG editing and media management with CKEditor and the Media suite of modules.
Whilst these are problems that must be tackled when explicitly testing the UI and user flow to verify a particular feature, it's not necessary to effectively repeatedly run this test in order to set up test data and content. Not only is it in places complex, but running a full suite (in multiple environments, i.e. in different browsers) is slow and a lot of that time is (obvious facepalm incoming) setting up test data.
Once we had a more 'useful' connection to Drupal with Drush, database or a fully bootstrapped connection as implemented in the Drupal Variable module, it became clear that there could be a better (or at least faster) way of setting up test content. We're not 100% what this will involve yet, perhaps custom code, or integration with modules such as Migrate or Node export.
Refactoring and relating the modules
Most of these module sprang from solving different problems whilst testing different projects. Whilst we have one or two suites using all if not most of them, the modules function together but may benefit from refactoring any shared code. For example, Drupal User Registry will currently check for, create and delete test users using Drush and aliases and the original intention was to have other ways of doing so via other connections. As mentioned above, Drupal Variable implements a different method but already has three types of connection, including Drush, which Drupal User Registry could utilise.
Behat?
Fairly recently Codeception introduced Gherkin and natural language features which is up there at the top of a list of advantages that Behat may have over Codeception - second only to it's much more extensive integration with Drupal, of course! Behat is being considered as an alternative or supplemental method for testing our Drupal sites.
About Drupal Sun
Drupal Sun is an Evolving Web project. It allows you to:
- Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
- Facet based on tags, author, or feed
- Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
- View the entire article text inline, or in the context of the site where it was created
See the blog post at Evolving Web