Jan 08 2019
Jan 08

It’s always hard to pick up the most useful Drupal 8 modules because it depends on the site you will create or manage. But there are some really helpful modules you can use in almost every situation.

In this post, I will share some modules that I use almost all the time in my Drupal 8 projects, they are not related to a particular type of site but they are always helpful, both in development or production environment.

1. Admin Toolbar

(D8) - https://www.drupal.org/project/admin_toolbar

drupal-admin-toolbar

The Admin Toolbar module will greatly save your time. By having a drop-down menu and extending the original Drupal menu, it helps to perform various admin actions faster and easier.

The module works on the top of the default toolbar core module, therefore it is a very light module and keeps all the toolbar functionalities (shortcut / media responsive).

The module also provides a submodule called "Admin Toolbar Extra Tools" adding extra links similar to Admin Menu module for Drupal 7 (flush caches, run cron, etc...).

2. Module Filter

(D7/D8) - https://www.drupal.org/project/module_filter

drupal-module-filter

The modules list page can become quite big when dealing with a fairly large site. To simplify your modules administration, you can install the Module Filter. This module provides you a tab for each package, giving you two alternative ways (with the default filter textfield) to quickly find and re-configure your modules.

3. Shield

(D7/D8) - https://www.drupal.org/project/shield

drupal-module-shield

This module helps you protect your development or staging site with HTTP authentication. Anonymous visitors and search engines won’t be able to reach your test environment, but we and our customers will.

4. Content Lock

(D7/D8) - https://www.drupal.org/project/content_lock

drupal-module-content-lock

This module prevents users from editing the same node. If another user is trying to edit the same node, he will be notified that the content is already being edited.

The other great feature of this module is that it prevents users from leaving an edit form page without first saving it, they will be notified when attempting to close the browser window/tab, clicking on a link, etc. If the user confirms that he wants to leave without saving the node, the edit lock gets automatically removed.

5. Environment Indicator

(D7/D8) - https://www.drupal.org/project/environment_indicator

drupal-module-environment-indicator

This module will help you stay sane while working on your different environments by adding a configurable color bar to each one of your environments.

The Environment Indicator module adds a colored bar on the site, informing you which environment you're currently in (Development, Staging, Production, etc.). This is incredibly useful if you have multiple environments for each of your sites, and are prone to forgetting which version of the site you are currently looking at.

6. reCAPTCHA

(D7/D8) - https://www.drupal.org/project/recaptcha

drupal-module-recaptcha

reCaptcha is a module built on top of the captcha module that implements the google captcha service to protect your site from spam. This web service shows a checkbox “I’m not a robot” at the bottom of your form. The service gives a challenge based on choosing images related to a subject.

Another option for protecting your site from spam is the Honeypot module.

7. Block Class

(D7/D8)  - https://www.drupal.org/project/block_class

drupal-module-block-class

This module allows users to add CSS classes to any block through the block configuration interface. So we don’t have to add classes to our twig templates. We just need to add our new classes to our CSS file.

8. Configuration Split

(D8) - https://www.drupal.org/project/config_split

drupal-module-config-split

This module filters the import/export of the configuration. It allows to define sets of configurations that will get exported to separate directories when exporting from different environments.  The canonical example for this is to have the devel module enabled or having a few block placements or views in the development environment and then not export them into the set of configurations to be deployed, yet still being able to share the development configuration with colleagues.

9. RoleAssign

(D7/D8) -  https://www.drupal.org/project/roleassign

drupal-rol-asign

This module allows you, as a site administrator, to delegate the assignment of roles to another user without giving him the Administer permissions permission.

The module creates a new permission called Assign roles. Users with this permission are able to assign selected roles to other users. On the other hand, users with the Administer permissions permission may select which roles are available for assignment through this module.

For larger sites with multiple levels of administrators or whenever you need finer-grained control over which role can assign which other role, check out Role Delegation.

10. Delete all

(D7/D8) - https://www.drupal.org/project/delete_all

drupal-module-delete-all

This is the module that I always install on local or dev environment in conjunction with the Devel Generate module. It allows to delete all content and/or users from a site with just one click.

This particularly handy on a test site the client was using for a period of time, and he must clean it up before starting with real data or when we test import or migration with thousand of nodes. Really helpful.

Usage statistics of these modules.

Here you can find the usage statistics for those modules across all versions.

drupal-usage-statistics

Which Drupal 8 modules do you consider helpful?

I wish to hear about which Drupal 8 modules you consider helpful for your projects. Leave a comment and share it with the community !

Nov 27 2018
Nov 27

Batch processing is usually an important aspect of any Drupal project and even more when we need to process huge amounts of data.

The main advantage of using batch is that it allows large amounts of data to be processed into small chunks or page requests that run without any manual intervention. Rather than use a single page load to process lots of data, the batch API allows the data to be processed as number of small page requests.

Using batch processing, we divide a large process into small pieces, each one of them executed as a separate page request, thereby avoiding stress on the server. This means that we can easily process 10,000 items without using up all of the server resources in a single page load.

This helps to ensure that the processing is not interrupted due to PHP timeouts, while users are still able to receive feedback on the progress of the ongoing operations.

Some uses of the batch API in Drupal:

  • Import or migrate data from an external source
  • Clean-up internal data
  • Run an action on several nodes
  • Communicate with an external API

Normally, batch jobs are launched by a form. However, what can we do if we want a *nix crontab to launch them on a regular basis? One of the best solutions is to use an external command like a custom Drush command and launch it from this crontab.

In this post we are going to create a custom Drush 9 command that loads all the nodes of a content type passed in as an argument (page, article ...). Then a batch process will simulate a long operation on each node. Next, we'll see how to run this drush command from crontab.

You can find the code for this module here: https://github.com/KarimBoudjema/Drupal8-ex-batch-with-drush9-command

This is the tree of the module:

tree web/modules/custom/ex_batch_drush9/
web/modules/custom/ex_batch_drush9/
|-- composer.json
|-- drush.services.yml
|-- ex_batch_drush9.info.yml
|-- README.txt
`-- src
    |-- BatchService.php
    `-- Commands
        `-- ExBatchDrush9Commands.php

We are going to proceed in three steps:

  1. Create a class to host our two main callback methods for batch processing (BatchService.php).
  2. Create our custom Drush 9 command to retrieve nodes, to create and process the batch sets (ExBatchDrush9Commands.php).
  3. Create a crontab task to run automatically the Drush command at scheduled times.

1. Create a BatchService class for the batch operations

A batch process is made of two main callbacks functions, one for processing each batch and the other for post-processing operations.

So in our class will have two methods, processMyNode() for processing each batch and processMyNodeFinished() to be launched when the batch processing is finished.

It's best practice to store the callback functions in their own file. This keeps them separate from anything else that your module might be doing. In this case I prefer to store them in a class that we can reuse later as a service.

Here is the code of BatchService.php

<?php
namespace Drupal\ex_batch_drush9;
/**
 * Class BatchService.
 */
class BatchService {
  /**
   * Batch process callback.
   *
   * @param int $id
   *   Id of the batch.
   * @param string $operation_details
   *   Details of the operation.
   * @param object $context
   *   Context for operations.
   */
  public function processMyNode($id, $operation_details, &$context) {
    // Simulate long process by waiting 100 microseconds.
    usleep(100);
    // Store some results for post-processing in the 'finished' callback.
    // The contents of 'results' will be available as $results in the
    // 'finished' function (in this example, processMyNodeFinished()).
    $context['results'][] = $id;
    // Optional message displayed under the progressbar.
    $context['message'] = t('Running Batch "@id" @details',
      ['@id' => $id, '@details' => $operation_details]
    );
  }
  /**
   * Batch Finished callback.
   *
   * @param bool $success
   *   Success of the operation.
   * @param array $results
   *   Array of results for post processing.
   * @param array $operations
   *   Array of operations.
   */
  public function processMyNodeFinished($success, array $results, array $operations) {
    $messenger = \Drupal::messenger();
    if ($success) {
      // Here we could do something meaningful with the results.
      // We just display the number of nodes we processed...
      $messenger->addMessage(t('@count results processed.', ['@count' => count($results)]));
    }
    else {
      // An error occurred.
      // $operations contains the operations that remained unprocessed.
      $error_operation = reset($operations);
      $messenger->addMessage(
        t('An error occurred while processing @operation with arguments : @args',
          [
            '@operation' => $error_operation[0],
            '@args' => print_r($error_operation[0], TRUE),
          ]
        )
      );
    }
  }
}

In the processMyNode()method we are going to process each element of our batch. As you can see, in this method we just simulate a long operation with the usleep() PHP function. Here we could load each node or connect with an external API. We also grab some information for post-processing.

In the processMyNodeFinished() method, we display relevant information to the user and we can even save the unprocessed operations for a later process.

2. Create the custom Drush 9 command to launch the batch

This is the most important part of our module. With this Drush command, we'll retrieve the data and fire the batch processing on those data.

Drush commands are now based on classes and the Annotated Command format. This will change the fundamental structure of custom Drush commands. This is great because we can now inject services in our command class and take advantage of all the OO power of Drupal 8.

A Drush command is composed of three files:

drush.services.yml - This is the file where our Drush command definition goes into. This is a Symfony service definition. Do not use your module's regular services.yml as you may have done in Drush 8 or else you will confuse the legacy Drush, which will lead to a PHP error.

You'll can see that in our example we inject two core services in our command class: entity_type.manager to access the nodes to process and logger.factory to log some pre-process and post-process information.

services:
  ex_batch_drush9.commands:
    class: \Drupal\ex_batch_drush9\Commands\ExBatchDrush9Commands
    tags:
      - { name: drush.command }
    arguments: ['@entity_type.manager', '@logger.factory']

composer.json - This is where we declare the location of the Drush command file for each version of Drush by adding the extra.drush.services section to the composer.json file of the implementing module. This is now optional, but will be required for Drush 10.

{
    "name": "org/ex_batch_drush9",
    "description": "This extension provides new commands for Drush.",
    "type": "drupal-drush",
    "authors": [
        {
            "name": "Author name",
            "email": "[email protected]"
        }
    ],
    "require": {
        "php": ">=5.6.0"
    },
    "extra": {
        "drush": {
            "services": {
                "drush.services.yml": "^9"
            }
        }
    }
}

MyModuleCommands.php - (src/Commands/ExBatchDrush9Commands.php in our case) It's in this class that we are going to define the custom Drush commands of our module. This class uses the Annotated method for commands, which means that each command is now a separate function with annotations that define its name, alias, arguments, etc. This class can also be used to define hooks with @hook annotation.

Some of the annotations available for use are:

@command: This annotation is used to define the Drush command. Make sure that you follow Symfony’s module:command structure for all your commands.
@aliases: An alias for your command.
@param: Defines the input parameters. For example, @param: integer $number
@option: Defines the options available for the commands. This should be an associative array where the name of the option is the key and the value could be - false, true, string, InputOption::VALUE_REQUIRED, InputOption::VALUE_OPTIONAL or an empty array.
@default: Defines the default value for options.
@usage: Demonstrates how the command should be used. For example, @usage: mymodule:command --option
@hook: Defines a hook to be fired. The default format is @hook type target, where type determines when the hook is called and target determines where the hook is called.

For a complete list of all the hooks available and their usage, refer to: https://github.com/consolidation/annotated-command

Here is the code for our Drush command.

<?php
namespace Drupal\ex_batch_drush9\Commands;
use Drupal\Core\Entity\EntityTypeManagerInterface;
use Drupal\Core\Logger\LoggerChannelFactoryInterface;
use Drush\Commands\DrushCommands;
/**
 * A Drush commandfile.
 *
 * In addition to this file, you need a drush.services.yml
 * in root of your module, and a composer.json file that provides the name
 * of the services file to use.
 *
 * See these files for an example of injecting Drupal services:
 *   - http://cgit.drupalcode.org/devel/tree/src/Commands/DevelCommands.php
 *   - http://cgit.drupalcode.org/devel/tree/drush.services.yml
 */
class ExBatchDrush9Commands extends DrushCommands {
  /**
   * Entity type service.
   *
   * @var \Drupal\Core\Entity\EntityTypeManagerInterface
   */
  private $entityTypeManager;
  /**
   * Logger service.
   *
   * @var \Drupal\Core\Logger\LoggerChannelFactoryInterface
   */
  private $loggerChannelFactory;
  /**
   * Constructs a new UpdateVideosStatsController object.
   *
   * @param \Drupal\Core\Entity\EntityTypeManagerInterface $entityTypeManager
   *   Entity type service.
   * @param \Drupal\Core\Logger\LoggerChannelFactoryInterface $loggerChannelFactory
   *   Logger service.
   */
  public function __construct(EntityTypeManagerInterface $entityTypeManager, LoggerChannelFactoryInterface $loggerChannelFactory) {
    $this->entityTypeManager = $entityTypeManager;
    $this->loggerChannelFactory = $loggerChannelFactory;
  }
  /**
   * Update Node.
   *
   * @param string $type
   *   Type of node to update
   *   Argument provided to the drush command.
   *
   * @command update:node
   * @aliases update-node
   *
   * @usage update:node foo
   *   foo is the type of node to update
   */
  public function updateNode($type = '') {
    // 1. Log the start of the script.
    $this->loggerChannelFactory->get('ex_batch_drush9')->info('Update nodes batch operations start');
    // Check the type of node given as argument, if not, set article as default.
    if (strlen($type) == 0) {
      $type = 'article';
    }
    // 2. Retrieve all nodes of this type.
    try {
      $storage = $this->entityTypeManager->getStorage('node');
      $query = $storage->getQuery()
        ->condition('type', $type)
        ->condition('status', '1');
      $nids = $query->execute();
    }
    catch (\Exception $e) {
      $this->output()->writeln($e);
      $this->loggerChannelFactory->get('ex_batch_drush9')->warning('Error found @e', ['@e' => $e]);
    }
    // 3. Create the operations array for the batch.
    $operations = [];
    $numOperations = 0;
    $batchId = 1;
    if (!empty($nids)) {
      foreach ($nids as $nid) {
        // Prepare the operation. Here we could do other operations on nodes.
        $this->output()->writeln("Preparing batch: " . $batchId);
        $operations[] = [
          '\Drupal\ex_batch_drush9\BatchService::processMyNode',
          [
            $batchId,
            t('Updating node @nid', ['@nid' => $nid]),
          ],
        ];
        $batchId++;
        $numOperations++;
      }
    }
    else {
      $this->logger()->warning('No nodes of this type @type', ['@type' => $type]);
    }
    // 4. Create the batch.
    $batch = [
      'title' => t('Updating @num node(s)', ['@num' => $numOperations]),
      'operations' => $operations,
      'finished' => '\Drupal\ex_batch_drush9\BatchService::processMyNodeFinished',
    ];
    // 5. Add batch operations as new batch sets.
    batch_set($batch);
    // 6. Process the batch sets.
    drush_backend_batch_process();
    // 6. Show some information.
    $this->logger()->notice("Batch operations end.");
    // 7. Log some information.
    $this->loggerChannelFactory->get('ex_batch_drush9')->info('Update batch operations end.');
  }
}

In this class, we first inject our two core services in the __construct() method: entity_type.manager and logger.factory.

Next, in the updateNode() annotated method we define our command with three annotations:

@param string $type - Defines the input parameter, the content type in our case.
@command update:node - Defines the name of the Drush command. In this case it's "update:node" so we would launch the command with: drush update:node
@aliases update-node - Defines an alias for the command

The main part of this command is the creation of the operations array for our batch processing (See points 3,4,5 and 6). Nothing strange here, we just define our operations array (3 and 4) pointing to the two callback functions that are located in our BatchService.php class.

Once the batch operations are added as new batch sets (5), we process the batch sets with the function drush_backend_batch_process(). This is a drop in replacement for the existing batch_process() function of Drupal. It will process a Drupal batch by spawning multiple Drush processes.

Finally, we show information to the user and log some information for a later use.

That's it! We can now test our brand new Drush 9 custom command!

To do so, just clear the cache with drush cr and launch the command with drush update:node

3. Bonus: Run the Drush command from crontab

As noted before, we want to run this custom command from crontab to perform the update automatically at the scheduled times.

The steps taken may vary based on your server's operating system. With Mac, Linux and Unix servers, we manage scheduled tasks by creating and editing crontabs that execute cron jobs at specified intervals.

1. Open a terminal window on your computer and enter the following command to edit your crontab - this will invoke your default editor (usually a flavor of vi):

crontab -e

2. Enter the following scheduled task with our Drush command (where [docroot_path] is your server's docroot):

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
*/5 * * * * drush --root=[docroot_path] update:node

We need to add an environment path in our cron table first line (cron commands run one after anonther).
This command will run our custom Drush command every five minutes.

3. Restart the cron service with the following command:

systemctl restart cron

4. Check if the command is running
We can check the log of our Drupal application every five minutes since we logged some information with the command itself, but we can also use another way with the following command:

sudo tail -f /var/mail/root

Recap.

- We created a class to host our two main callback methods for batch processing (BatchService.php).
- We created a custom Drush 9 command to retrieve nodes, to create and process the batch sets, injecting core services, like entity_type.manager to access the nodes to process and logger.factory to log some pre-process and post-process information.
- We created a crontab task to run automatically the Drush command at scheduled times.

This way, we can now run heavy processes, on a regular basis without putting undue stress on the server.

Oct 13 2018
Oct 13

Creating forms is part of the day to day live of a Drupal programmer.  Forms are represented as nested arrays in both Drupal 7 and Drupal 8. But they are different in that in Drupal 7 you define your form arrays in functions and in Drupal 8 you create a form class.

In this post we’ll create a custom form with two fields, a text field and a checkbox field, we’ll validate those fields, display them in a message and finally redirect the user to the home page.

You can find the code of this module here.

Here is a screenshot of the form:

drupal 8 custom form

You can see the folder structure of this module below:

web/modules/custom/ex81
|-- ex81.info.yml
|-- ex81.routing.yml
`-- src
    `-- Form
        `-- HelloForm.php

2 directories, 3 files

First we’ll create the form step by step manually, then we’ll use Drupal Console to do it in a easier way.

1. Create the module’s skeleton
2. Create the form’s class and assign a name to the form
3. Build the form
4. Validate the form
5. Process the collected values
6. Add a route to your form
7. Do it easier with Drupal Console

In Drupal 8, each form is defined by a controller class that implements the interface \Drupal\Core\Form\FormInterface which defines four basic methods:

getFormId() - Defines the unique ID for the form
buildForm() - Triggered when the user requests the form. Builds the basic $form array with all the fields of the form.
validateForm() -  Triggered when the form is submitted. It’s used to check the values collected by the form and optionally raise errors.
submitForm() -  Used to carry out the form submission process, if the form has passed validation with no errors.

1. Create the module’s skeleton

As always, we use the Drupal Console to create the module. Don’t forget to install the module after it has been generated.

We use the following command:

drupal generate:module  --module="ex81" \
--machine-name="ex81" \
--module-path="modules/custom" \
--description="Example of a simple custom form" \
--core="8.x" \
--package="Custom" \
--uri="http://default" \
--no-interaction

Once created we can install it with drupal module:install ex81 command

2. Create the form’s class

We create a new php file called HelloForm.php in the src/Form folder. In this file we create a new class called HelloForm that extends the abstract class FormBase (which implements the FormInterface interface).

In this class we create the getformId() method to set a unique string identifying the form.

<?php

namespace Drupal\ex81\Form;

use Drupal\Core\Form\FormBase;
use Drupal\Core\Form\FormStateInterface;

class HelloForm extends FormBase {

  /**
   * Returns a unique string identifying the form.
   *
   * The returned ID should be a unique string that can be a valid PHP function
   * name, since it's used in hook implementation names such as
   * hook_form_FORM_ID_alter().
   *
   * @return string
   *   The unique string identifying the form.
   */
  public function getFormId() {
    return 'ex81_hello_form';
  }

}

3. Build the form

Now we add to this class a new method called buildForm() which defines the text and the checkbox fields of our form. We also add a field description to add a small message on top of the form.
At the end of the method we add a submit field and return the form array.

  /**
   * Form constructor.
   *
   * @param array $form
   *   An associative array containing the structure of the form.
   * @param \Drupal\Core\Form\FormStateInterface $form_state
   *   The current state of the form.
   *
   * @return array
   *   The form structure.
   */
  public function buildForm(array $form, FormStateInterface $form_state) {

    $form['description'] = [
      '#type' => 'item',
      '#markup' => $this->t('Please enter the title and accept the terms of use of the site.'),
    ];

    
$form['title'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Title'),
      '#description' => $this->t('Enter the title of the book. Note that the title must be at least 10 characters in length.'),
      '#required' => TRUE,
    ];

    $form['accept'] = array(
      '#type' => 'checkbox',
      '#title' => $this
        ->t('I accept the terms of use of the site'),
      '#description' => $this->t('Please read and accept the terms of use'),
    );


    // Group submit handlers in an actions element with a key of "actions" so
    // that it gets styled correctly, and so that other modules may add actions
    // to the form. This is not required, but is convention.
    $form['actions'] = [
      '#type' => 'actions',
    ];

    // Add a submit button that handles the submission of the form.
    $form['actions']['submit'] = [
      '#type' => 'submit',
      '#value' => $this->t('Submit'),
    ];

    return $form;

  }

4. Validate the form

To validate the form we just need to implement the validateForm() method from \Drupal\Core\Form\FormInterface in our HelloForm class.

In our example we’ll raise an error if the length of the title is less than 10 and also if the checkbox is not checked. User-submitted values can be accessed via $form_state->getValue('key') where "key" is the name of the element whose value we would like to retrieve.

We raise an error thanks to the setErrorByName() method of the $form_state object.

  /**
   * Validate the title and the checkbox of the form
   * 
   * @param array $form
   * @param \Drupal\Core\Form\FormStateInterface $form_state
   * 
   */
  public function validateForm(array &$form, FormStateInterface $form_state) {
    parent::validateForm($form, $form_state);

    $title = $form_state->getValue('title');
    $accept = $form_state->getValue('accept');

    if (strlen($title) < 10) {
      // Set an error for the form element with a key of "title".
      $form_state->setErrorByName('title', $this->t('The title must be at least 10 characters long.'));
    }

    if (empty($accept)){
      // Set an error for the form element with a key of "accept".
      $form_state->setErrorByName('accept', $this->t('You must accept the terms of use to continue'));
    }

  }

5. Process the collected values

Now it’s time to process the values collected by our form. We do this with the submitForm() submission handler method.

Here we can save the data to the database, send them to an external API or use them in a service. In our case we’ll just display those values and redirect the user to the home page. To redirect the user, we’ll use the setRedirect() method of the $form_state object.

Remember that if you need to save those values in the config system, you should extend the ConfigFormBase as we saw earlier in another post.

  /**
   * Form submission handler.
   *
   * @param array $form
   *   An associative array containing the structure of the form.
   * @param \Drupal\Core\Form\FormStateInterface $form_state
   *   The current state of the form.
   */
  public function submitForm(array &$form, FormStateInterface $form_state) {

    // Display the results.
    
    // Call the Static Service Container wrapper
    // We should inject the messenger service, but its beyond the scope of this example.
    $messenger = \Drupal::messenger();
    $messenger->addMessage('Title: '.$form_state->getValue('title'));
    $messenger->addMessage('Accept: '.$form_state->getValue('accept'));

    // Redirect to home
    $form_state->setRedirect('<front>');

  } 

6. Add a route to your form

Now that we have added a controller to your custom module, we are almost done. The last thing is to add a route in our routine file in order to allow users to access our form.

If we haven’t created this file before, we create the ex81.routing.yml file in the root of the module. In this file we add the route that points to the form controller (HelloForm) we created above. In this case we won’t be using the _controller key, but the _form key, so Drupal will use the form builder service when serving this request.

ex81.hello_form:
  path: '/ex81/helloform'
  defaults:
    _form: 'Drupal\ex81\Form\HelloForm'
    _title: 'Simple custom form example'
  requirements:
    _permission: 'access content'

The above code adds a route that maps the path /ex81/helloform to our form controller Drupal\ex81\Form\HelloForm.

Now we can navigate to our form and test it!

Recap.

1. We created the module with the Drupal Console command: drupal generate:module
2. We created the form controller class HelloForm in the src\Form folder. This class extends the abstract class FormBase that implements the interface FormInterface.
3. We set a unique string ID for our form with the getFormId() method.
4. We defined the fields of the form with the buildForm() method.
5. To validate the form, we used the validateForm() method and use the setErrorByName() method of the $form_state object.
6. We processed the values of the form with the submitForm() method and redirect the user to the home page with the setRedirect() method of the $form_state object.
7. We created a route that points to our form using the specific key _form

7. Bonus: Create a custom form with Drupal Console

We’ll create the same form as above but in another module called 'ex82'.

drupal generate:module  --module="ex81" \
--machine-name="ex81" \
--module-path="modules/custom" \
--description="Example of a simple custom form" \
--core="8.x" \
--package="Custom" \
--uri="http://default" \
--no-interaction

Now we’ll create the form with a simple Drupal Console command: drupal generate:form

drupal generate:form  \
--module="ex82" \
--class="HelloForm" \
--form-id="ex82_hello_form" \
--services='messenger' \
--inputs='"name":"description", "type":"item", "label":"Description", "options":"", "description":"Please enter the title and accept the terms of use of the site.", "maxlength":"", "size":"", "default_value":"", "weight":"0", "fieldset":""' \
--inputs='"name":"title", "type":"textfield", "label":"Title", "options":"", "description":"Enter the title of the book. Note that the title must be at least 10 characters in length.", "maxlength":"64", "size":"64", "default_value":"", "weight":"0", "fieldset":""' \
--inputs='"name":"accept", "type":"checkbox", "label":"I accept the terms of use of the site", "options":"", "description":"Please read and accept the terms of use", "maxlength":"", "size":"", "default_value":"", "weight":"0", "fieldset":""' \
--path="/ex82/helloform" \
--uri="http://default" \
--no-interaction

This will generate two files, HelloForm.php and ex82.routing.yml, as you can see Drupal Console generate almost the entire form.

Let’s take a look at the new generated HelloForm.php

<?php

namespace Drupal\ex82\Form;

use Drupal\Core\Form\FormBase;
use Drupal\Core\Form\FormStateInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;
use Drupal\Core\Messenger\MessengerInterface;

/**
 * Class HelloForm.
 */
class HelloForm extends FormBase {

  /**
   * Drupal\Core\Messenger\MessengerInterface definition.
   *
   * @var \Drupal\Core\Messenger\MessengerInterface
   */
  protected $messenger;
  /**
   * Constructs a new HelloForm object.
   */
  public function __construct(
    MessengerInterface $messenger
  ) {
    $this->messenger = $messenger;
  }

  public static function create(ContainerInterface $container) {
    return new static(
      $container->get('messenger')
    );
  }


  /**
   * {@inheritdoc}
   */
  public function getFormId() {
    return 'ex82_hello_form';
  }

  /**
   * {@inheritdoc}
   */
  public function buildForm(array $form, FormStateInterface $form_state) {
    $form['description'] = [
      '#type' => 'item',
      '#title' => $this->t('Description'),
      '#description' => $this->t('Please enter the title and accept the terms of use of the site.'),
    ];
    $form['title'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Title'),
      '#description' => $this->t('Enter the title of the book. Note that the title must be at least 10 characters in length.'),
      '#maxlength' => 64,
      '#size' => 64,
    ];
    $form['accept'] = [
      '#type' => 'checkbox',
      '#title' => $this->t('I accept the terms of use of the site'),
      '#description' => $this->t('Please read and accept the terms of use'),
    ];
    $form['submit'] = [
      '#type' => 'submit',
      '#value' => $this->t('Submit'),
    ];

    return $form;
  }

  /**
   * {@inheritdoc}
   */
  public function validateForm(array &$form, FormStateInterface $form_state) {
    parent::validateForm($form, $form_state);
  }

  /**
   * {@inheritdoc}
   */
  public function submitForm(array &$form, FormStateInterface $form_state) {
    // Display result.
    foreach ($form_state->getValues() as $key => $value) {
      drupal_set_message($key . ': ' . $value);
    }

  }

}

First you can see that we voluntary inject the messenger service in the Drupal Console command, this allows us to avoid calling the Static Service Container wrapper \Drupal::messenger(). We’ll use it in the buildForm() method later.

Now, we’ll need to add the code for the validation and the submit process with:
- validateForm() method
- submitForm() method

You can copy the same code of the validateForm() method described above in this post in the section 4.

  public function validateForm(array &$form, FormStateInterface $form_state) {
    parent::validateForm($form, $form_state);

    $title = $form_state->getValue('title');
    $accept = $form_state->getValue('accept');

    if (strlen($title) < 10) {
      // Set an error for the form element with a key of "title".
      $form_state->setErrorByName('title', $this->t('The title must be at least 10 characters long.'));
    }

    if (empty($accept)){
      // Set an error for the form element with a key of "accept".
      $form_state->setErrorByName('accept', $this->t('You must accept the terms of use to continue'));
    }

  }

For the buildForm() method we’ll use the same code as above but with a small change since we injected the messenger service

 public function submitForm(array &$form, FormStateInterface $form_state) {

    $this->messenger->addMessage('Title: '.$form_state->getValue('title'));
    $this->messenger->addMessage('Accept: '.$form_state->getValue('accept'));

    // Redirect to home
    $form_state->setRedirect('<front>');
    return;
  }

As simple as that. YES! Drupal Console is definitively a great tool to create custom forms!!!

Sep 24 2018
Sep 24

In Drupal 7, to manage system variables, we used variable_get() / variable_set() /  variable_del() calls and stored those variables in the variable table of the database.

In Drupal 8 we now use the Configuration system which provides a central place for modules to store configuration data. This system allows to store information that can be synchronized between development and production sites. This information is often created during site building and is not typically generated by regular users during normal site operation.
In Drupal 8 configuration is still stored in the database,  but it can now be synced with YML files on the disk for deployment purposes aswell.

In this post we will create a configuration form that stores a configuration value in our system. In this case it will be a form that stores the value of an external API key. Next, we will retrieve this value and use it in a very simple controller.

1. Create the module
2. Create the form
3. Check the form
4. Retrieve the config information in a controller

1. Create the module skeleton

We use the following Drupal Console command.

drupal generate:module  \
--module="ex08" \
--machine-name="ex08" \
--module-path="modules/custom" \
--description="An simple example of a config form" \
--core="8.x" \
--package="Custom" \
--module-file \
--uri="http://default" \
--no-interaction

2. Create the form

To create the config form we use Drupal Console with the drupal generate:form:config command.

drupal generate:form:config  \
--module="ex08" \
--class="ExternalApiKeyForm" \
--form-id="external_api_key_form" \
--config-file \
--inputs='"name":"your_external_api_key", "type":"textfield", "label":"Your external API Key", "options":"", "description":"Enter your external API Key", "maxlength":"64", "size":"64", "default_value":"none", "weight":"0", "fieldset":""' \
--path="/admin/config/ex08/externalapikey" \
--uri="http://default" \
--no-interaction

This Drupal Console command will:
- create the config form file ExternalApiKeyForm.php under the src/form folder
- update or create the ex08.routing.yml file with a route to the config form to access it.

Now let’s see what this code is doing in detail.

3. The config form

If we now open the ExternalApiKeyForm.php file we see the following code.

<?php

namespace Drupal\ex08\Form;

use Drupal\Core\Form\ConfigFormBase;
use Drupal\Core\Form\FormStateInterface;

/**
 * Class ExternalApiKeyForm.
 */
class ExternalApiKeyForm extends ConfigFormBase {

  /**
   * {@inheritdoc}
   */
  protected function getEditableConfigNames() {
    return [
      'ex08.externalapikey',
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function getFormId() {
    return 'external_api_key_form';
  }

  /**
   * {@inheritdoc}
   */
  public function buildForm(array $form, FormStateInterface $form_state) {
    $config = $this->config('ex08.externalapikey');
    $form['your_external_api_key'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Your external API Key'),
      '#description' => $this->t('Store the external API Key'),
      '#maxlength' => 64,
      '#size' => 64,
      '#default_value' => $config->get('your_external_api_key'),
    ];
    return parent::buildForm($form, $form_state);
  }

  /**
   * {@inheritdoc}
   */
  public function validateForm(array &$form, FormStateInterface $form_state) {
    parent::validateForm($form, $form_state);
  }

  /**
   * {@inheritdoc}
   */
  public function submitForm(array &$form, FormStateInterface $form_state) {
    parent::submitForm($form, $form_state);

    $this->config('ex08.externalapikey')
      ->set('your_external_api_key', $form_state->getValue('your_external_api_key'))
      ->save();
  }

}

Fist we see that the class ExternalApiKeyForm extends ConfigFormBase abstract class which also extends the FormBase abstract class.

The ExternalApiKeyForm class has four basic methods:

getEditableConfigNames() - gets the configuration object name (The name of the configuration environment where we store or get the values). In our case we give it the name: 'ex08'.externalapikey'.

getFormId() - returns the form's unique ID. This will be the name of our form or its ID. In our case, we named it 'external_api_key_form'.

buildForm() - returns the form array. This method creates the form and in this case one item named 'your_external_api_key'. This item is a text field. We will also use this name for
the key of the key/value pair stored in the configuration.

We also get the current value for the key 'your_external_api_key' from the configuration object to set it as a default value for this field:   '#default_value' => $config->get('your_external_api_key'),
 
validateForm() - validates the form.

submitForm() - processes the form submission. In this step, we will set and save the value of the key 'your_external_api_key' in the configuration system.

4. The route to the form

Drupal Console also updates or creates the ex08.routing.yml file with a route to the config form to access it.

ex08.external_api_key_form:
  path: '/admin/config/ex08/externalapikey'
  defaults:
    _form: '\Drupal\ex08\Form\ExternalApiKeyForm'
    _title: 'ExternalApiKeyForm'
  requirements:
    _permission: 'access administration pages'
  options:
    _admin_route: TRUE


Nothing strange here. With path we define the URL to access the form. With _form we indicate the class which generates the form and with _title we define the title of the form.  With _permission we give access to users who have permission to access administration pages (user access to the administration path).

Now that we have a form and a route, we can access the form from the URL /admin/config/ex08/externalapikey and test it.

Drupal 8 config form

5. Debug the configuration object.

We can also debug the configuration object with Drupal Console command debug:config (dc) which give us a list configuration objects names and single configuration object:

Drupal console debug config

 

6. Retrieve the value in a controller class.

To get or set the value of a configuration object in a class, we will need a service. In fact we need the core service 'config.factory'.

We can find it with the Drupal Console commands: drupal debug:container or drupal debug:container config.factory

This service has a method that allows to get the value by its key of a configuration object.

So, when we create the controller, well inject this service to use later.

Lets create the controller and inject the service at the same time with Drupal Console.

drupal generate:controller  --module="ex08" \
--class="ExternalApiKeyController" \
--routes='"title":"External API Key", "name":"ex08.external_api_key_controller_ShowKey", "method":"ShowKey", "path":"/ex08/ShowKey"' \
--services='config.factory' \
--uri="http://default" \
--no-interaction

This command creates the file ExternalApiKeyController.php under /src/Controller and update the ex08.routing.yml file with a new route.

The updated ex08.routing.yml file:

ex08.external_api_key_form:
  path: '/admin/config/ex08/externalapikey'
  defaults:
    _form: '\Drupal\ex08\Form\ExternalApiKeyForm'
    _title: 'ExternalApiKeyForm'
  requirements:
    _permission: 'access administration pages'
  options:
    _admin_route: TRUE

ex08.external_api_key_controller_ShowKey:
  path: '/ex08/ShowKey'
  defaults:
    _controller: '\Drupal\ex08\Controller\ExternalApiKeyController::ShowKey'
    _title: 'External API Key'
  requirements:
    _permission: 'access content'

The new ExternalApiKeyController.php:

<?php

namespace Drupal\ex08\Controller;

use Drupal\Core\Controller\ControllerBase;
use Symfony\Component\DependencyInjection\ContainerInterface;
use Drupal\webprofiler\Config\ConfigFactoryWrapper;

/**
 * Class ExternalApiKeyController.
 */
class ExternalApiKeyController extends ControllerBase {

  /**
   * Drupal\webprofiler\Config\ConfigFactoryWrapper definition.
   *
   * @var \Drupal\webprofiler\Config\ConfigFactoryWrapper
   */
  protected $configFactory;

  /**
   * Constructs a new ExternalApiKeyController object.
   */
  public function __construct(ConfigFactoryWrapper $config_factory) {
    $this->configFactory = $config_factory;
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container) {
    return new static(
      $container->get('config.factory')
    );
  }

  /**
   * Showkey.
   *
   * @return array
   *
   */
  public function ShowKey() {

    return [
      '#type' => 'markup',
      '#markup' => $this->t('The ShowKey() action')
    ];
  }

}

We can see that the config.factory service has been retrieved from the service container  in the create() method and sent back (dependency injection) to the class itself through the __construct() method. Thank you, Drupal Console!!!

Now, in the ShowKey() method, we need to access the value of our ex08.externalapikey configuration object and get the value of the key your_external_api_key. This is what we can see in the following code:

  public function ShowKey() {

    // Get the configuration object
    $config = $this->configFactory->get('ex08.externalapikey');
    // Get the value of the key 'your_external_api_key' 
    $key = $config->get('your_external_api_key');

    return [
      '#type' => 'markup',
      '#markup' => $this->t('The External API Key is: @key',['@key'=>$key])
    ];
  }

Done!!!

Recap.

1. We created a configuration form, and a route to this form, with the DC command drupal generate:form:config. This form generated a configuration object ('ex08.externalapikey') and a set of key/value pair. In our case, we have just one key/value pair and the key is: 'your_external_api_key'

2. We debugged the key/value pair of the configuration object with the DC command drupal debug:config  <name-of-config-object>

3. We created a controller and injected the service we need (config.factory) to access the config system.
In the controller, we called the service object to get the value of a particular key of the configuration object.

As we can see, today Drupal Console is a must in Drupal 8.

7. Bonus: Retrieve the config value in a hook

Sometimes we need set or get configuration values in a hook, in this case we can't inject the service config.factory by dependency injection.

We’ll use the \Drupal static service container wrapper to do so.

function ex08_form_alter(&$form, \Drupal\Core\Form\FormStateInterface $form_state, $form_id){
        // Get the config object
        $config = \Drupal::config('ex08.externalapikey');
        // Get the key value
        $key = $config->get('your_external_api_key');
}
Aug 07 2018
Aug 07

Queues are particularly important when we need to stash some tasks for later processing. To do so, we are going to put some tasks or data in a queue (create the queue) and later we will process those tasks with a QueueWorker plugin (process the queue), usually triggered by cron.

There are several ways to create a queue:
- With a form class
- With a controller class
- With a hook_cron() function
 
To process the queue, we also have different options:
- As a cron process with a QueueWorker plugin
- As a batch process also with QueueWorker plugin but extending a base plugin
- As a batch process claiming each item of the queue in a service or in a controller

Today we'll create the queue with a controller and process it with a QueueWorker plugin when cron runs or manually with Drupal Console.

Creating the queue with a controller has the advantage of allowing us to use an external crontab (e.g., a Linux crontab), so we won't need Drupal’s cron at all to run the controller, allowing us to launch it when we need it. This is also a more reliable method because it uses fewer resources and it is independent of any page request. Remember that the Drupal cron is 'poor man’s cron', as it depends on page requests for its execution even if there is a way to execute it with a linux crontab.

In this example module we'll generate a queue with a controller, importing the title and the description tags form the Drupal Planet RSS file. Next, when Cron runs, , we'll create a node page with a QueueWorker plugin for each item in the queue. You can download the code here: https://github.com/KarimBoudjema/Drupal8-ex-queue-api-01.

This module has two main parts:

1. A controller class src/Controller/ExQueueController.php with its corresponding route in exqueue01.routing.yml and two main methods:

getData() - Get external data and insert the data in the queue 'exqueue_import'.
deleteTheQueue() - Delete all item in the queue.

2. A Cron QueueWorker plugin src/Plugin/QueueWorker/ExQueue01.php that will process each item in the queue.

You can see the tree module here:

web/modules/custom/exqueue01/
|-- exqueue01.info.yml
|-- exqueue01.module
|-- exqueue01.permissions.yml
|-- exqueue01.routing.yml
|-- README.txt
`-- src
    |-- Controller
    |   `-- ExQueueController.php
    `-- Plugin
        `-- QueueWorker
            `-- ExQueue01.php

4 directories, 7 files

That's not too difficult, is it?

1. Create the queue with a controller.

Here is the code of the controller. You can also get it here: https://github.com/KarimBoudjema/Drupal8-ex-queue-api-01/blob/master/src...

<?php
namespace Drupal\exqueue01\Controller;
use Drupal\Core\Controller\ControllerBase;
use Drupal\Core\Queue\QueueFactory;
use Symfony\Component\DependencyInjection\ContainerInterface;
use Drupal\Core\Messenger\MessengerInterface;
use GuzzleHttp\ClientInterface;
use GuzzleHttp\Exception\RequestException;
/**
 * Class ExQueueController.
 *
 * Demonstrates the use of the Queue API
 * There is two routes.
 * 1) \Drupal\exqueue01\Controller\ExQueueController::getData
 * The getData() methods allows to load external data and
 * for each array element create a queue element
 * Then on Cron run, we create a page node for each element with
 * 2) \Drupal\exqueue01\Controller\ExQueueController::deleteTheQueue
 * The deleteTheQueue() methods delete the queue "exqueue_import"
 * and all its elements
 * Once the queue is created with tha data, on Cron run
 * we create a new page node for each item in the queue with the QueueWorker
 * plugin ExQueue01.php .
 */
class ExQueueController extends ControllerBase {
  /**
   * Drupal\Core\Messenger\MessengerInterface definition.
   *
   * @var \Drupal\Core\Messenger\MessengerInterface
   */
  protected $messenger;
  /**
   * Symfony\Component\DependencyInjection\ContainerAwareInterface definition.
   *
   * @var \Symfony\Component\DependencyInjection\ContainerAwareInterface
   */
  protected $queueFactory;
  /**
   * GuzzleHttp\ClientInterface definition.
   *
   * @var \GuzzleHttp\ClientInterface
   */
  protected $client;
  /**
   * Inject services.
   */
  public function __construct(MessengerInterface $messenger, QueueFactory $queue, ClientInterface $client) {
    $this->messenger = $messenger;
    $this->queueFactory = $queue;
    $this->client = $client;
  }
  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container) {
    return new static(
      $container->get('messenger'),
      $container->get('queue'),
      $container->get('http_client')
    );
  }
  /**
   * Delete the queue 'exqueue_import'.
   *
   * Remember that the command drupal dq checks first for a queue worker
   * and if it exists, DC suposes that a queue exists.
   */
  public function deleteTheQueue() {
    $this->queueFactory->get('exqueue_import')->deleteQueue();
    return [
      '#type' => 'markup',
      '#markup' => $this->t('The queue "exqueue_import" has been deleted'),
    ];
  }
  /**
   * Getdata from external source and create a item queue for each data.
   *
   * @return array
   *   Return string.
   */
  public function getData() {
    // 1. Get data into an array of objects
    // 2. Get the queue and the total of items before the operations
    // 3. For each element of the array, create a new queue item
    // 1. Get data into an array of objects
    // We can choose between two methods
    // getDataFromRSS() or getFakeData()
    $data = $this->getDataFromRss();
    // $data = $this->getFakeData();
    if (!$data) {
      return [
        '#type' => 'markup',
        '#markup' => $this->t('No data found'),
      ];
    }
    // 2. Get the queue and the total of items before the operations
    // Get the queue implementation for 'exqueue_import' queue.
    $queue = $this->queueFactory->get('exqueue_import');
    // Get the total of items in the queue before adding new items.
    $totalItemsBefore = $queue->numberOfItems();
    // 3. For each element of the array, create a new queue item.
    foreach ($data as $element) {
      // Create new queue item.
      $queue->createItem($element);
    }
    // 4. Get the total of item in the Queue.
    $totalItemsAfter = $queue->numberOfItems();
    // 5. Get what's in the queue now.
    $tableVariables = $this->getItemList($queue);
    $finalMessage = $this->t('The Queue had @totalBefore items. We should have added @count items in the Queue. Now the Queue has @totalAfter items.',
      [
        '@count' => count($data),
        '@totalAfter' => $totalItemsAfter,
        '@totalBefore' => $totalItemsBefore,
      ]);
    return [
      '#type' => 'table',
      '#caption' => $finalMessage,
      '#header' => $tableVariables['header'],
      '#rows' => $tableVariables['rows'],
      '#attributes' => $tableVariables['attributes'],
      '#sticky' => $tableVariables['sticky'],
      'empty' => $this->t('No items.'),
    ];
  }
  /**
   * Generate an array of objects to simulate getting data from an RSS file.
   *
   * @return array
   *   Return an array of data
   */
  protected function getFakeData() {
    // We should get the XML content and convert it to an array of item objects
    // We use now an example array of item object.
    $content = [];
    for ($i = 1; $i <= 10; $i++) {
      $item = new \stdClass();
      $item->title = 'Title ' . $i;
      $item->body = 'Body ' . $i;
      $content[] = $item;
    }
    return $content;
  }
  /**
   * Generate an array of objects from an external RSS file.
   *
   * @return array|bool
   *   Return an array or false
   */
  protected function getDataFromRss() {
    // 1. Try to get the data form the RSS
    // URI of the XML file.
    $uri = 'https://www.drupal.org/planet/rss.xml';
    // 1. Try to get the data form the RSS.
    try {
      $response = $this->client->get($uri, ['headers' => ['Accept' => 'text/plain']]);
      $data = (string) $response->getBody();
      if (empty($data)) {
        return FALSE;
      }
    }
    catch (RequestException $e) {
      return FALSE;
    }
    // 2. Retrieve data in a simple xml object.
    $data = simplexml_load_string($data);
    // 3. Transform in a array of object
    // We could transform in array
    // $data = json_decode(json_encode($data));
    // Look at all children of the channel child.
    $content = [];
    foreach ($data->children()->children() as $child) {
      if (!empty($child->title)) {
        // Create an object.
        $item = new \stdClass();
        $item->title = $child->title->__toString();
        $item->body = $child->description->__toString();
        // Place the object in an array.
        $content[] = $item;
      }
    }
    if (empty($content)) {
      return FALSE;
    }
    return $content;
    /*
    // Using simplexml_load_file
    $xml = simplexml_load_file($uri);
    $data = json_decode(json_encode($xml));
    ksm($data);
     */
  }
  /**
   * Get all items of queue.
   *
   * Next place them in an array so we can retrieve them in a table.
   *
   * @param object $queue
   *   A queue object.
   *
   * @return array
   *   A table array for rendering.
   */
  protected function getItemList($queue) {
    $retrieved_items = [];
    $items = [];
    // Claim each item in queue.
    while ($item = $queue->claimItem()) {
      $retrieved_items[] = [
        'data' => [$item->data->title, $item->item_id],
      ];
      // Track item to release the lock.
      $items[] = $item;
    }
    // Release claims on items in queue.
    foreach ($items as $item) {
      $queue->releaseItem($item);
    }
    // Put the items in a table array for rendering.
    $tableTheme = [
      'header' => [$this->t('Title'), $this->t('ID')],
      'rows'   => $retrieved_items,
      'attributes' => [],
      'caption' => '',
      'colgroups' => [],
      'sticky' => TRUE,
      'empty' => $this->t('No items.'),
    ];
    return $tableTheme;
  }
}

First we injected the QueueFactory service into our controller. We also injected two others services like messenger to display Drupal messages to the user and the http_client service to retrieve the Drupal Planet RSS file.

Next in the getData() method we'll do three processes:

a- Get the data from an external RSS file
b- Get or create the queue and populate it
c- Retrieve all the items in the queue for information

a -  Get the data form the Drupal Planet RSS file in an array of objects with the custom method getDataFromRss()

$data = $this->getDataFromRss();

Note that there is also another custom method getFakeData() that you can use in local. It simulates the results of getting data from a external RSS file.

b – Get or in fact create the queue 'exqueue_import', get the total of item in the at this time for information purpose and populate the queue with the $data array.

     $queue = $this->queueFactory->get('exqueue_import');
        $totalItemsBefore = $queue->numberOfItems();
        foreach ($data as $element) {
          // Create new queue item.
          $queue->createItem($element);
        }

Here comes the most interesting parts.
$queue = $this->queueFactory->get('exqueue_import');
This creates an instance of the default QueueInterface and the name of the queue we want to use or to create.

$totalItemsBefore = $queue->numberOfItems();
This give us the total of items in the queue, currenty for information purposes only.

$queue->createItem($element);
This simply adds an item to the queue. As the $data array is made of objects (see the getDataFromRss() method), we are wrapping the data as a good practice.

c- Retrieve all the items in the queue for information with the getItemList() custom method.

In this method we first claim each item of the queue with the claimItem() method. This method also locks or lease the item for a period of time of one hour by default. So we'll need to release those items later.
We save each item in the  $retrieved_items array that will use to show the information later to the user.

    // Claim each item in queue.    
    while ($item = $queue->claimItem()) {
      $retrieved_items[] = [
        'data' => [$item->data->title, $item->item_id],
      ];
      // Track item to release the lock.
      $items[] = $item;
    }


We also save the item information to the $items array to release each item in the next loop.

    // Release claims on items in queue.
    foreach ($items as $item) {
      $queue->releaseItem($item);
    }

That's it. You can see now how easy it is to create a queue and populating it.

You can now visit the page (/exqueue01/getData) and see how the queue is populated.

To debug the queue, you can use the Drupal Console command drupal debug:queue , but it will work only if you have a QueueWorker plugin associated with this queue.

You can also see the table queue in the database and see if it has been populated with the data of your queue.  

In a next post we'll see how to create a queue with a form and with a hook_cron

2. Create a Cron QueueWorker plugin to process the queue

As we've seen, create a queue is quite simple. Now we have to code a QueueWorker to process each element in the queue.

This QueueWorker is responsible for processing each element of the queue when cron runs. In our case, the process consists in creating a page node with the data of each item.

<?php
namespace Drupal\exqueue01\Plugin\QueueWorker;
use Drupal\Core\Entity\EntityTypeManagerInterface;
use Drupal\Core\Logger\LoggerChannelFactoryInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;
use Drupal\Core\Plugin\ContainerFactoryPluginInterface;
use Drupal\Core\Queue\QueueWorkerBase;
/**
 * Save queue item in a node.
 *
 * To process the queue items whenever Cron is run,
 * we need a QueueWorker plugin with an annotation witch defines
 * to witch queue it applied.
 *
 * @QueueWorker(
 *   id = "exqueue_import",
 *   title = @Translation("Import Content From RSS"),
 *   cron = {"time" = 5}
 * )
 */
class ExQueue01 extends QueueWorkerBase implements ContainerFactoryPluginInterface {
  /**
   * Drupal\Core\Entity\EntityTypeManagerInterface definition.
   *
   * @var \Drupal\Core\Entity\EntityTypeManagerInterface
   */
  private $entityTypeManager;
  /**
   * Drupal\Core\Logger\LoggerChannelFactoryInterface definition.
   *
   * @var \Drupal\Core\Logger\LoggerChannelFactoryInterface
   */
  private $loggerChannelFactory;
  /**
   * {@inheritdoc}
   */
  public function __construct(array $configuration,
                              $plugin_id,
                              $plugin_definition,
                              EntityTypeManagerInterface $entityTypeManager,
                              LoggerChannelFactoryInterface $loggerChannelFactory) {
    parent::__construct($configuration, $plugin_id, $plugin_definition);
    $this->entityTypeManager = $entityTypeManager;
    $this->loggerChannelFactory = $loggerChannelFactory;
  }
  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition) {
    return new static(
      $configuration,
      $plugin_id,
      $plugin_definition,
      $container->get('entity_type.manager'),
      $container->get('logger.factory')
    );
  }
  /**
   * {@inheritdoc}
   */
  public function processItem($item) {
    // Save the queue item in a node
    // Check the values of $item.
    $title = isset($item->title) && $item->title ? $item->title : NULL;
    $body = isset($item->body) && $item->body ? $item->body : NULL;
    try {
      // Check if we have a title and a body.
      if (!$title || !$body) {
        throw new \Exception('Missing Title or Body');
      }
      $storage = $this->entityTypeManager->getStorage('node');
      $node = $storage->create(
        [
          'type' => 'page',
          'title' => $item->title,
          'body' => [
            'value' => $item->body,
            'format' => 'basic_html',
          ],
        ]
      );
      $node->save();
      // Log in the watchdog for debugging purpose.
      $this->loggerChannelFactory->get('debug')
        ->debug('Create node @id from queue %item',
          [
            '@id' => $node->id(),
            '%item' => print_r($item, TRUE),
          ]);
    }
    catch (\Exception $e) {
      $this->loggerChannelFactory->get('Warning')
        ->warning('Exception trow for queue @error',
          ['@error' => $e->getMessage()]);
    }
  }
}

First let's have a look at the annotation of this plugin

 * @QueueWorker(
 *   id = "exqueue_import",
 *   title = @Translation("Import Content From RSS"),
 *   cron = {"time" = 5}
 * )

id = "exqueue_import"
Indicates the ID of the queue we will process
cron = {"time" = 5}
Indicates that the plugin should be used by the cron system. The time key indicates that we allocate 5 seconds to process all the items in this queue. If we couldn't process some items in the allocated time, they will be processed at the next cron run.

But all the action happens in the processItem() method from the QueueWorkerInterface. It's in this method that we process each item. It's quite a straightforward process. Nothing new here.

Note about Exception

Note that on cron run, the processQueues() method of the cron object in cron.php will be fired. You can find the code for this method below.

/**
 * Processes cron queues.
 */
protected function processQueues() {
  // Grab the defined cron queues.
  foreach ($this->queueManager->getDefinitions() as $queue_name => $info) {
    if (isset($info['cron'])) {
      // Make sure every queue exists. There is no harm in trying to recreate
      // an existing queue.
      $this->queueFactory->get($queue_name)->createQueue();
      $queue_worker = $this->queueManager->createInstance($queue_name);
      $end = time() + (isset($info['cron']['time']) ? $info['cron']['time'] : 15);
      $queue = $this->queueFactory->get($queue_name);
      $lease_time = isset($info['cron']['time']) ?: NULL;
      while (time() < $end && ($item = $queue->claimItem($lease_time))) {
        try {
          $queue_worker->processItem($item->data);
          $queue->deleteItem($item);
        }
        catch (RequeueException $e) {
          // The worker requested the task be immediately requeued.
          $queue->releaseItem($item);
        }
        catch (SuspendQueueException $e) {
          // If the worker indicates there is a problem with the whole queue,
          // release the item and skip to the next queue.
          $queue->releaseItem($item);
          watchdog_exception('cron', $e);
          // Skip to the next queue.
          continue 2;
        }
        catch (\Exception $e) {
          // In case of any other kind of exception, log it and leave the item
          // in the queue to be processed again later.
          watchdog_exception('cron', $e);
        }
      }
    }
  }
}

You can see in the processQueues() method that the processItem() method is in a ‘try catch’ block, so if we throw an Exception in our QueueWorker it will be catched by the processQueues() method, the item won't be deleted and the Exception will be logged. So the item would stay in the queue if we throw an exception in the processItem() method. That's a good design, as we can examine the log and see what's happened and do something with this item later.

You can also see that you could throw a RequeueException or a SuspendQueueException. In the first case the item is requeued and in the second the process is suspended for the whole queue.

But if we want, like in our case, to delete the item from our queue if there is an error, we need to catch the Exception in a ‘try catch’ block in our QueueWorker itself. In this case, we won't return an Exception to the processQueues() method and then our item will be deleted. That's a choice. This means that if we don't have a title or a description, the data is useless for us and we can delete it from the queue. This also shows that we accepted useless data to be queued in an earlier process.  

3. Process the queue

Now we can process the queue with:

cron – In my testing, I was unable to process a queue with drush cron or drupal cron:excecute because they only invoke modules with cron handlers implementing hook_cron. You'll have to launch cron manually in the drupal interface.

Drupal Console – You can use the Drupal Console drupal queue:run <name-of-the-queue> command. In our case it would be drupal queue:run  exqueue_import

Recap

We created a controller ExQueueController.php with a getData() method.
We get the data from the Drupal Planet RSS file with the getDataFromRss() method into an array of objects.
We get or created an instance of the default QueueInterface with $this->queueFactory->get('exqueue_import').
We created each item in the queue with the $queue->createItem() method
Once the queue is created, we set up a Cron QueueWorker plugin where we processed each item in the queue with the processItem() method.

Jun 04 2018
Jun 04

Mon, June 4, 2018

Runing composer install or composer update on your Drupal 8 installation to install or update modules and themes could be sometimes frustrating because it can be very slow. Too slow in fact. But it doesn't have to be that way. Here are some tips to speed your Composer working with Drupal.

Install Prestissimo

Prestismo is a global composer plugin that that enables parallel installations and it's very fast! It can be more than 2x faster. But Prestissimo requires cURL, which may not work behind certain firewalls or proxies.

To install prestissimo follow these steps:

composer self-update

composer global require hirak/prestissimo


Once it is installed, Composer should install a new project much faster than before.

You can configure how many connections you would prefer, but I have found that the default of 6 seems to be pretty good.

To uninstall:

composer global remove hirak/prestissimo


Disable Xdebug?

Before Composer 1.3., Xdebug slowed down Composer substantially, sometimes making installs take 2-4x longer.

But with the newer versions of Composer (> 1.2.) this issue has been fixed. So be sure to update your Composer to the latest version with the command composer self-update

Related Content

Jun 03 2018
Jun 03

Today composer is the recommended approach to install (o more precisely to download)  Drupal 8. This is true for the core but also for contributed modules and themes.

So now, to start a new Drupal 8 project, we need to download it via composer and not as we did before with drush or drupal console. Of course, we'll still use drush or drupal console to install contrib modules or themes, but not for downloading them.

The main benefit of using composer is that it allows us to systematically manage a sprawling list of dependencies (and their subsidiary dependencies) and ensure that the right versions for each package are used or updated.

An official project composer template has been created for installing Drupal with composer. We will create our project directly using this template, which is also available on Packagist.

Installing composer

First you need to install composer. Please see Getting Started on getcomposer.org to install Composer itself.

Download Drupal core using Composer

To create a new project based on the official composer template we can run the following Composer command:

composer create-project drupal-composer/drupal-project:8.x-dev my_project_name_dir --stability dev --no-interaction

Don't forget to change 'my_project_name_dir' with the name of the directory where you want to install Drupal.

This will download the drupal-composer/drupal-project project from Packagist into a folder named 'my_project_name_dir'. It also automatically executes composer install which will download Drupal 8 and all its dependencies.

In fact, this create-project composer command is the equivalent of doing a git clone https://github.com/drupal-composer/drupal-project.git  my_project_name_dir followed by a composer install of the vendors.

# create-project is equivalent of a git clone and a composer install
git clone https://github.com/drupal-composer/drupal-project.git my_project_name_dir
composer install

What does the template do?

When installing the given composer.json some tasks are taken care of:

  • Drupal will be installed in the web-directory.
  • Autoloader is implemented to use the generated composer autoloader in vendor/autoload.php, instead of the one provided by Drupal (web/vendor/autoload.php).
  • Modules (packages of type drupal-module) will be placed in web/modules/contrib/
  • Theme (packages of type drupal-theme) will be placed in web/themes/contrib/
  • Profiles (packages of type drupal-profile) will be placed in web/profiles/contrib/
  • Creates default writable versions of settings.php and services.yml.
  • Creates web/sites/default/files-directory.
  • Latest version of drush is installed locally for use at vendor/bin/drush.
  • Latest version of DrupalConsole (yes !!!) is installed locally for use at vendor/bin/drupal.
  • Creates environment variables based on your .env file. See .env.example.

Please note that Drupal will be installed in 'my_project_name_dir/web/'

The project template also comes with a .gitignore file that keeps Drupal core and all the contributed packages outside of Git, similar to the regular vendor/ packages. So based on an updated composer.json file, we can maintain a smaller Git repository and recreate our project any time. A lot of the benefits of Drush Make have now been incorporated into a Composer flow.

What can I do if I don't want the install Drupal in the web/ directory?

Imagine that we need to install Drupal in the docroot/ directory instead of the web/ directory.

In this case we'll do the following:

# 1. Download the project template with git
git clone https://github.com/drupal-composer/drupal-project.git my_project_name_dir

# 2. Change the 'installer-paths' in the 'extra' section of the new downloaded composer.json
cd my_project_name_dir
vi composer.json

# 3. Run composer install 
composer install

Install your project

Now that we have downloaded the Drupal project and all its dependencies, we can install our Drupal project by running the DrupalConsole command drupal site:install or go to your local website (localhost://) to install the site.

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web