Apr 24 2019
Apr 24
video drupal tutorials

After three months of hard work, mostly at night and during the weekends, we finally launched the first minimal viable version of VideoDrupal.org (beta 1.0), a curated mashup of Drupal video tutorials published on Youtube to make them more accessible for personal, educational and professional purposes.

For 2019, I've set up three main goals: improve my English, find a "real" Drupal Team to work with and launch this website. So, I've reached one of my goals. Checked! To be honest, I think it was the easiest one. Improving my English and finding a community-oriented Drupal agency will be harder. But I'm working on it!

Why VideoDrupal.org?

During the past 10 years, I have organized or co-organized several DrupalCamps and Drupal meet-ups here in Bolivia but also in Peru, Panama, Mexico and Belgium (Belgium is where I was born and I lived until I was 27 years old). People organizing those kinds of events know that this represents a huge effort during five or six months to motivate the local community, find the venue, setup the website, find the sponsors, plan and launch promotional activities, organize and schedule the sessions and so on... But the event itself is also really stressful and self-demanding with many different needs to satisfy.

When the camp is over, we usually feel completely exhausted, all those efforts for only three days of knowledge sharing, fun and parties with new and old friends? So many weeks of preparation to reach 300 to 500 people each time? Is it worth it? Yes it is! Really... For yourself but also for the community.

But I have always been wondering if we could give something back to the community that would last for longer (not just three days) and something that people could use or take advantage of more often.

On the other hand, as a Drupal trainer, I know how hard it could be to learn Drupal for newcomers. I'm not speaking about the well-known "learning curve", but just about how to begin learning Drupal, to find the right documentation, training or video to do so. There are a lot of very good paid resources for that, but also a ton of free resources on drupal.org, in blogs and free video platforms like Youtube. Many videos on Youtube are real gems, and not only for beginners. But these resources are very difficult to find for the average user. Is there something we could do to help people to find Drupal videos on Youtube that fit their needs in a better way?

It's with those two questions in mind, giving something back to the community that could last longer and helping people find Drupal videos on Youtube in a faster and easier way, that the idea of VideoDrupal.org came to me a few years ago. But due to my professional activities I've never found the time to go on with this project until last October, when I decided to make a switch in my career and to become a Drupal freelancer. Working from home gave me the opportunity to better organize my schedule and have more time to dedicate to this project.  

What is VideoDrupal.org?

Like I said before, VideoDrupal.org is a curated mashup of Drupal videos and tutorials published on Youtube to make them more accessible for personal, educational and professional purposes.

The main goals of the site are:

1. Be a central point to easily find Drupal videos or tutorials (Beta 1.0)
2. Allow to discover trending topics (Beta 1.0)
3. Help beginners and advanced drupalists to learn Drupal (Beta 1.0)
4. Allow the community to tag videos to improve search results (Beta 2.0 - coming soon)
5. Spread personal or community efforts to promote Drupal (Beta 1.0)

As this website is aimed to serve the Drupal community, it's free and will stay free forever.

How to learn Drupal with VideoDrupal.org?

Aside from an advanced search solution, we set up a dedicated page for people who want to learn Drupal. This page is currently divided in two sections. The first one is focused on learning the basics of Drupal site building and learning Drupal theming with a curated set of amazing playlists and standalone videos.

The other section focuses on specific Drupal learning topics, like learning Git, learning Phpstorm, discovering the new Cache API of Drupal 8 or learning Gatsby and Drupal 8 for more advanced users. Each week I will add a new topic to this section.

How do we curate the videos?

We select the videos based on their overall quality. To do this, we first start by analyzing the channel.
If it's a community channel with more than ten videos, we gather all the videos and playlists. We trust in our community, so there is nothing to curate in that case.
If it's not a community channel, we first check if the channel has a learning Drupal dedicated playlist. If so, we gather all the videos of this playlist. If not, we quickly analyze each Drupal video with the following key factors in mind:

- Is the video in English? (Yes, this sounds weird, but we want to keep the project simple at this time. We're looking for a sustainable multilingual solution to implement in the beta 4.0)
- Is the sound quality good enough to understand clearly what the speaker is saying?
- Is the video about learning or promoting Drupal?
- Is the subject addressed in a clear and understandable way for the end user?

We do NOT modify the content of the video in any way.

We strongly believe that maintaining a small set of high-quality Drupal videos will help to make this project a success. You can help submitting a channel, a playlist or a video by clicking here.

Who is behind VideoDrupal.org?

At this time, we are just two: Santiago Rico (rasaric) and myself. When I finished the first round of site building and development, I proposed Santiago, a young and talented designer and Drupal frontend developer, to help me with the design and the theming of the site. He has done a really stunning job so far, hasn’t he?

karim-sandiSantiago and Karim. Happy to finally launch VideoDrupal.org

We generally work on this site late in the afternoon, at night or during the weekends. That's not always easy but we are so excited to do that for the community which has given us so much during the last ten years.

Are we competing with paid Drupal learning solutions?

Absolutely not! Those paid solutions are vital for the community thanks to their high quality, we don't want to compete with them. We strongly believe that VideoDrupal.org is like an accelerator that will push new people to have a deeper interest in Drupal and later, choose a paid solution to improve their Drupal skills. Thinking this way, we are sure that VideoDrupal.org will act at the top of the sales funnel (awareness and interest) of those paid solutions companies and bring them more customers to the consideration step.   

In fact, we don't want to compete with anybody. We just wish to offer another way to gather and keep more people to and within the Drupal community. See number 5 of the site's purposes above.

Acknowledgment

First, I would like to thank Joaquin Bravo who gave me the confidence to start this project and mentored me at the very first start. Those two or three hours of mentoring where the best Drupal gift I've ever had. Thank you so much Joaquin!

On the other hand, I would like to thank Platform.sh for hosting VideoDrupal.org for free. You may know that I'm not a devops guy, so I was looking for a Drupal hosting service that allow us to focus on what best we can do: theming and developing Drupal websites. I've been delighted by Platform.sh for two main raisons:
- As a Drupal community oriented company, they accepted immediately, without asking any questions.
- The deployment work flow was surprisingly easy, just a few clicks and boom! The site was online with a fully functional dev. and prod. environment. I believe I'm going to love doing more devops tasks now.
Thank you so much for supporting us guys!

Finally, I would like to thank my wife Alejandra and my two young adolescents, Salim and Amélie who support me in this never-ending journey. Without their continuous and understanding support, this project would never have been undertaken.

What's next?

We are now planing the beta 2.0 version, a version only focused on improving the search results and the related video block. For this version we plan to:

- Fine tuning Search Api and Solr
- Fine tuning the MLT video block
- Allow the community to tag videos to improve search results and MLT block. The current video tags are too vague and often refer to the channel or event itself. We plan to add community tags like:

  • type (site building - theming - development - devops - arquitecture - community building - marketing - self-improvement - case study)
  • level (beginner - intermediate - advanced)
  • kind (tutorial - conference - meeting)
  • subject (free tagging)

- Find a way to access the video's captions and process them through an auto tagging tool like OpenCalais so we could discover automatically the most relevant tags.
- Your idea here?

The beta 3.0 will be centered on the user experience, for this version we plan to:

- Improve the mobile experience
- Allow the user to create their own playlists
- Allow the community to create learning playlists
- Enable comments in the learning nodes
- Give the user the possibility to know which video he watched and when
- Improve the discovery of trending topics and videos
- Your idea here?

It's your turn to help the community

If you are a Drupal agency you could perhaps accelerate the process, funding some development. Even if it's not a priority for us, we work on this site on a a pro-bono basis, this kind of support may help to finish those beta versions earlier. Just contact us if you're interested.

If you are a Drupal fan like us, we would love to have your feedback to better serve the community. What could we improve on this project? Do you think that we are on the right tracks? Are there other needs that we didn't consider? This project is also yours, so leave us a comment and if you like it, share it with your friends.

Mar 15 2019
Mar 15

In this post we’ll see how to save temporarily values from a form and how to retrieve or process them later in a controller. To do that, we’ll use the Form API and the private tempstore (the temporary store storage system of Drupal 8).

The use case is the following: we need to build a simple RSS reader (a form) where the user could introduce the URL of an RSS file and the number of items to retrieve from that file.  Next, in a new page (a controller), the application should display the list of items with a link to each syndicated page .

The easiest way to achieve it would be to retrieve the values in our buildForm() method, process them and display the result thought a specific field of the form. But that’s not our use case.

To process the form’s values and to display the results in another page, we’ll first need to store the form's values and retrieve them later in a controller. But how and where to store and retrieve those values?

Long story short: Store and retrieve data with the Private Tempstore System of Drupal 8

Drupal 8 has a powerful key/value system to temporarily store user-specific data and to keep them available between multiple requests even if the user is not logged in. This is the Private Tempstore system.

// 1. Get the private tempstore factory, inject this in your form, controller or service.
$tempstore = \Drupal::service('tempstore.private');
// Get the store collection. 
$store = $tempstore->get('my_module_collection');
// Set the key/value pair.
$store->set('key_name', $value);

// 2. Get the value somewhere else in the app.
$tempstore = \Drupal::service('tempstore.private');
// Get the store collection. 
$store = $tempstore->get('my_module_collection');
// Get the key/value pair.
$value = $store->get('key_name');

// Delete the entry. Not mandatory since the data will be removed after a week.
$store->delete('key_name');

Fairly simple, isn't it? As you can see, the Private Tempstore is a simple key/value pair storage organized with collections (we usually give the name of our module to the collection) to maintain data available temporarily across multiple page requests for a specific user.

Now, that you have the recipe, we can go back to our use case where we are now going to figure out, how and where we could store and retrieve the values of our form.

Long story: Illustrate the Private Tempstore system in Drupal 8

For the long story part, we are going to create a module with a form and a controller to first get the data form the user, store the form’s data in a private tempstore and redirect the form to a controller where we’ll retrieve the data from the tempstore and process them.

This is the form.

drupal8-privatetempstore-form

And this is the controller.

drupal8-privatetempstore-controller

You can find and download the code of this companion module here.

Types of data storage in Drupal 8

In Drupal 8 we have various APIs for data storage:

Database API  - To interact directly with the database.
State API - A key/value storage to store data related to the state or individual environment (dev., staging, production) of a Drupal installation, like external API keys or the last time cron was run.
UserData API - A storage to store data related to an individual environment but specific to a given user, like a flag or some user preferences.
TempStore API - A key/value storage to keep temporary data (private o shared) across several page requests.
Entity API - Used to store content (node, user, comment …) or configuration (views, roles, …) data.
TypedData API - A low level API to create and describe data in a consistent way.    

Our use case refers to user-specific data we need for a short period of time, data that is also not specific to one environment, so the best fit is certainly the TempStore API but in its private flavour because the data and the results are different for each user.

The only difference between private and shared tempstore is that the private tempstore entries strictly belong to a single user whereas with shared tempstore, the entries can be shared between several users.

Storing the form's values with the Private TempStore storage

Our goal is to build a form where user can introduce an RSS file URL and a number of items to retrieve from that file, next we need to store temporarily those values to retrieve them later in a controller.

Let’s now take a closer look at how we could store (or save) our form’s values. As we get the data (url and items to retrieve) from a form, it’s clear that we are going to store the data in the submitForm() method since this method is called when the form is validated and submitted.

Here is the code of our form.

<?php
namespace Drupal\ex_form_values\Form;
use Drupal\Core\Form\FormBase;
use Drupal\Core\Form\FormStateInterface;
// DI.
use Symfony\Component\DependencyInjection\ContainerInterface;
use Drupal\Core\Messenger\MessengerInterface;
use Drupal\Core\Logger\LoggerChannelFactoryInterface;
use Drupal\Core\TempStore\PrivateTempStoreFactory;
/**
 * Class WithControllerForm.
 *
 * Get the url of a RSS file and the number of items to retrieve
 * from this file.
 * Store those two fields (url and items) to a PrivateTempStore object
 * to use them in a controller for processing and displaying
 * the information of the RSS file.
 */
class WithStoreForm extends FormBase {
  /**
   * Drupal\Core\Messenger\MessengerInterface definition.
   *
   * @var \Drupal\Core\Messenger\MessengerInterface
   */
  protected $messenger;
  /**
   * Drupal\Core\Logger\LoggerChannelFactoryInterface definition.
   *
   * @var \Drupal\Core\Logger\LoggerChannelFactoryInterface
   */
  protected $loggerFactory;
  /**
   * Drupal\Core\TempStore\PrivateTempStoreFactory definition.
   *
   * @var \Drupal\Core\TempStore\PrivateTempStoreFactory
   */
  private $tempStoreFactory;
  /**
   * Constructs a new WithControllerForm object.
   */
  public function __construct(
    MessengerInterface $messenger,
    LoggerChannelFactoryInterface $logger_factory,
    PrivateTempStoreFactory $tempStoreFactory
  ) {
    $this->messenger = $messenger;
    $this->loggerFactory = $logger_factory;
    $this->tempStoreFactory = $tempStoreFactory;
  }
  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container) {
    return new static(
      $container->get('messenger'),
      $container->get('logger.factory'),
      $container->get('tempstore.private')
    );
  }
  /**
   * {@inheritdoc}
   */
  public function getFormId() {
    return 'with_state_form';
  }
  /**
   * {@inheritdoc}
   */
  public function buildForm(array $form, FormStateInterface $form_state) {
    $form['url'] = [
      '#type' => 'url',
      '#title' => $this->t('Url'),
      '#description' => $this->t('Enter the url of the RSS file'),
      '#default_value' => 'https://www.drupal.org/planet/rss.xml',
      '#weight' => '0',
    ];
    $form['items'] = [
      '#type' => 'select',
      '#title' => $this->t('# of items'),
      '#description' => $this->t('Enter the number of items to retrieve'),
      '#options' => [
        '5' => $this->t('5'),
        '10' => $this->t('10'),
        '15' => $this->t('15'),
      ],
      '#default_value' => 5,
      '#weight' => '0',
    ];
    $form['actions'] = [
      '#type' => 'actions',
      '#weight' => '0',
    ];
    // Add a submit button that handles the submission of the form.
    $form['actions']['submit'] = [
      '#type' => 'submit',
      '#value' => $this->t('Submit'),
    ];
    return $form;
  }
  /**
   * Submit the form and redirect to a controller.
   *
   * 1. Save the values of the form into the $params array
   * 2. Create a PrivateTempStore object
   * 3. Store the $params array in the PrivateTempStore object
   * 4. Redirect to the controller for processing.
   *
   * {@inheritdoc}
   */
  public function submitForm(array &$form, FormStateInterface $form_state) {
    // 1. Set the $params array with the values of the form
    // to save those values in the store.
    $params['url'] = $form_state->getValue('url');
    $params['items'] = $form_state->getValue('items');
    // 2. Create a PrivateTempStore object with the collection 'ex_form_values'.
    $tempstore = $this->tempStoreFactory->get('ex_form_values');
    // 3. Store the $params array with the key 'params'.
    try {
      $tempstore->set('params', $params);
      // 4. Redirect to the simple controller.
      $form_state->setRedirect('ex_form_values.simple_controller_show_item');
    }
    catch (\Exception $error) {
      // Store this error in the log.
      $this->loggerFactory->get('ex_form_values')->alert(t('@err', ['@err' => $error]));
      // Show the user a message.
      $this->messenger->addWarning(t('Unable to proceed, please try again.'));
    }
  }
}

In the submitForm() method we can see two lines where we deal with the private tempstore in step 2 and 3.

$tempstore = $this->tempStoreFactory->get('ex_form_values');
//...
$tempstore->set('params', $params);

In the first line we call the PrivateTempStoreFactory to instantiate a new PrivateTempStore object trough its get() method. As you can see, we get the factory by DI. We also define the name of our collection (ex_form_values) with the same name as our module (by convention) and pass it to the get() method. So, at this time we created a new PrivateTempStore object for a collection named "ex_form_values".

In the second line we use the set() method of the PrivateTempStore object which will store the key/value pair we need to save, in our case, the key is 'params' and the value is the $params array that contains our form’s values.

The PrivateTempStoreFactory uses a storage based on the KeyValueStoreExpirableInterface, this storage is a key/value storage with an expiration date that allows automatic removal of old entities and this storage uses the default DatabaseStorageExpirable storage to save the entries in the key_value_expire table.

Here is the structure of this table:

key_value_expire-strucuture

The PrivateTempStore object has the following protected properties, all passed to the form by DI:
$storage - The key/value storage object used for this data.
$lockBackend - The lock object used for this data.
$currentUser - The current user who owned the data. If he's anonymous, the session ID will be used.
$requestStack - Service to start a session for anonymous user
$expire - The time to live for the entry in seconds. By default, entries are stored for one week (604800 seconds) before expiring as defined in the KeyValueStoreExpirableInterface.

We can’t set any of those values, they are all set by the KeyValueStoreExpirableInterface when we instantiate the new PrivateTempStore object.

The PrivateTempStore object method we are interested at this time is the set() method which will store the key/value we need to save, in our case, the key is 'params' and the value is the $params array that contains our form’s values. You can find the code of this method here and below, the beginning of this method:

public function set($key, $value) {

  // Ensure that an anonymous user has a session created for them, as
  // otherwise subsequent page loads will not be able to retrieve their
  // tempstore data.
  if ($this->currentUser
    ->isAnonymous()) {

    // @todo when https://www.drupal.org/node/2865991 is resolved, use force
    //   start session API rather than setting an arbitrary value directly.
    $this
      ->startSession();
    $this->requestStack
      ->getCurrentRequest()
      ->getSession()
      ->set('core.tempstore.private', TRUE);
  }
  // ....
}

As you can see at the beginning of this method, we ensure that an anonymous user has a session created for him, thanks to this, we can also retrieve a session ID that we will use to identify him. If it’s an authenticated user, we will use his UID. You can confirm that in the getOwner() method of the PrivateTempStore object.

When we submit the form, thanks to these two lines, we’ll save the form's values in the database. If you look at the key_value_expire table, you can see that there are other records, most of them with update information of our app., but also new records with the collection value of tempstore.private.ex_form_values. Here is a query on this collection when an anonymous and the admin user use the form.

key_value_expire-drupal8

We didn’t output the column 'value' because it’s too large. But you can see now how our form's values are stored in the database and particularly for anonymous users with their session ID.

At this time, we know how to store temporarily the values of a form in the privatetempstore of Drupal. This was not so hard, wasn’t it? Yes, Drupal is a great framework and it saves us a lot of time when developing a web application.

Now let’s see how to retrieve and process the values in a controller.

Retrieve our form's values from the privatetempstore

To retrieve the data, process them and display the results, we’ll need to redirect our form to a controller. Here is the code of this controller.

<?php
namespace Drupal\ex_form_values\Controller;
use Drupal\Core\Controller\ControllerBase;
// DI.
use Drupal\Core\Messenger\MessengerInterface;
use Drupal\ex_form_values\MyServices;
use Symfony\Component\DependencyInjection\ContainerInterface;
use Drupal\Core\TempStore\PrivateTempStoreFactory;
use GuzzleHttp\ClientInterface;
// Other.
use Drupal\Core\Url;
/**
 * Target controller of the WithStoreForm.php .
 */
class SimpleController extends ControllerBase {
  /**
   * Tempstore service.
   *
   * @var \Drupal\Core\TempStore\PrivateTempStoreFactory
   */
  protected $tempStoreFactory;
  /**
   * GuzzleHttp\ClientInterface definition.
   *
   * @var \GuzzleHttp\ClientInterface
   */
  protected $clientRequest;
  /**
   * Messenger service.
   *
   * @var \\Drupal\Core\Messenger\MessengerInterface
   */
  protected $messenger;
  /**
   * Custom service.
   *
   * @var \Drupal\ex_form_values\MyServices
   */
  private $myServices;
  /**
   * Inject services.
   */
  public function __construct(PrivateTempStoreFactory $tempStoreFactory,
                              ClientInterface $clientRequest,
                              MessengerInterface $messenger,
                              MyServices $myServices) {
    $this->tempStoreFactory = $tempStoreFactory;
    $this->clientRequest = $clientRequest;
    $this->messenger = $messenger;
    $this->myServices = $myServices;
  }
  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container) {
    return new static(
      $container->get('tempstore.private'),
      $container->get('http_client'),
      $container->get('messenger'),
      $container->get('ex_form_values.myservices')
    );
  }
  /**
   * Target method of the the WithStoreForm.php.
   *
   * 1. Get the parameters from the tempstore for this user
   * 2. Delete the PrivateTempStore data from the database (not mandatory)
   * 3. Display a simple message with the data retrieved from the tempstore
   * 4. Get the items from the rss file in a renderable array
   * 5. Create a link back to the form
   * 6. Render the array.
   *
   * @return array
   *   An render array.
   */
  public function showRssItems() {
    // 1. Get the parameters from the tempstore.
    $tempstore = $this->tempStoreFactory->get('ex_form_values');
    $params = $tempstore->get('params');
    $url = $params['url'];
    $items = $params['items'];
    // 2. We can now delete the data in the temp storage
    // Its not mandatory since the record in the key_value_expire table
    // will expire normally in a week.
    // We comment this task for the moment, so we can see the values
    // stored in the key_value_expire table.
    /*
    try {
      $tempstore->delete('params');
    }
    catch (\Exception $error) {
      $this->loggerFactory->get('ex_form_values')->alert(t('@err',['@err' => $error]));
    }
    */
    // 3. Display a simple message with the data retrieved from the tempstore
    // Set a cache tag, so when an anonymous user enter in the form
    // we can invalidate this cache.
    $build[]['message'] = [
      '#type' => 'markup',
      '#markup' => t("Url: @url - Items: @items", ['@url' => $url, '@items' => $items]),
    ];
    // 4. Get the items from the rss file in a renderable array.
    if ($articles = $this->myServices->getItemFromRss($url, $items)) {
      // Create a render array with the results.
      $build[]['data_table'] = $this->myServices->buildTheRender($articles);
    }
    // 5. Create a link back to the form.
    $build[]['back'] = [
      '#type' => 'link',
      '#title' => 'Back to the form',
      '#url' => URL::fromRoute('ex_form_values.with_store_form'),
    ];
    // Prevent the rendered array from being cached.
    $build['#cache']['max-age'] = 0;
    // 6. Render the array.
    return $build;
  }
}

First we inject all the services we need and also a custom service MyServices.php used to retrieve the RSS file and the required number of items.

This controller has a simple method called showRssItems(). This method performs six tasks:

1. Get the data we need from the PrivateTempstore
2. Delete the PrivateTempStore data from the database (not mandatory)
3. Display a simple message with the data retrieved from the tempstore
4. Get the items from the rss file in a render array
5. Create a link back to the form
6. Render the array.

The tasks that interest us are the first and the second one, where we deal with the tempstore.

$tempstore = $this->tempStoreFactory->get('ex_form_values');

This line is now straightforward for us, since we know that the get() method of the PrivateTempStoreFactory instantiate a new PrivateTempStore object for the collection 'ex_form_values'. Nothing new here.

$params = $tempstore->get('params');

In the line above, we just retrieve the value of the key 'params' from the storage collection into the variable $params. Remember that we are using the privatetempstorage, as we retrieve the value, we’ll also check if it's the same user with the getOwner() method of the PrivateTempStore object.

The second task is deleting the data from the tempstore, this is not mandatory since we are working with a temporary store where the data will be deleted automatically after a week by default. But if we expect a heavy use of our form, it could be a good idea since we won’t normally need the data anymore.

try {
  $tempstore->delete('params');
}
catch (\Exception $error) {
  $this->loggerFactory->get('ex_form_values')->alert(t('@err',['@err' => $error]));
}

As the delete() method of our privatetempstore object can trow an error, we’ll catch it in a 'try catch' block. For learning purpose I’ll suggest to comment those lines to see how the data are stored in the key_value_expire table.

The rest of the controller is pretty straightforward, we are going to pass the data to small methods of our custom service to retrieve the RSS file and the number of items we need, build a render array with the results as you can see in task number four. Next, we’ll create a simple link back to our form and render the entire array. Nothing strange here.

Recap

We wanted, trough a form, to get a RSS file URL, the number of items to retrieve from, and display this information in a controller. To do so, we needed to store the information of the form, retrieve it and process it later in a controller.

We decided to use the Private TempSore system of Drupal because:
- The information doesn’t belong to a specific environment or a particular state of our application (see State API for that case)
- The information is not specific to a user and doesn’t need to be stored with its profile (see UserData API for that case). On the other hand, anonymous users should also have access and use the form
- We need the information temporarily, for a short period of time
- It has to be private because the information belongs to each user (anonymous or authenticated)

To store the data, we used the submit() method of our form and the following lines:

$tempstore = $this->tempStoreFactory->get('ex_form_values') - to instantiate a new PrivateTempStore object with a collection name 'ex_form_values' that refers to our module in this case.

$tempstore->set('params', $params) - to save the value of the variable $params in the database with the key 'params'. Remember that as we use a private storage, Drupal will retrieve the user ID or its session ID if anonymous, and concatenate it with the key value in the 'name' row of the 'key_value_expire' table.

To retrieve the information in our controller’s showRssItems() method, we used the following code:

$tempstore = $this->tempStoreFactory->get('ex_form_values') - to instantiate a new PrivateTempStore object with a collection name 'ex_form_values' that refers to our module in this case.

$params = $tempstore->get('params') - to retrieve the value stored in the keyed 'params' record of this collection and for this particular user.

Yes! Once again, through this simple example, we can see how powerful Drupal is, and how its framework is easy to use.

And you? In which situation do you think we could use the privatetempstore? Please, share your ideas with the community.

Jan 09 2019
Jan 09

It’s not easy to create a list of the most useful Drupal 8 modules because it really depends on the site you will create or manage. But there are some really helpful modules that you can use in almost all cases.

In this post, I will share a list of modules that I use on almost all my Drupal 8 projects. They aren’t specifically for a particular type of site, but I find that they are always helpful, both in development and production environments.

1. Admin Toolbar

(D8) - https://www.drupal.org/project/admin_toolbar

drupal-admin-toolbar

The Admin Toolbar module will save you a great deal of time. It provides a drop-down menu that extends Drupal built in menu. It allows you to perform various admin actions faster and easier.

The module works on the top of the default toolbar core module. It’s a very light module and keeps all the toolbar functionalities (shortcut / media responsive).

The module also provides a submodule called "Admin Toolbar Extra Tools," which adds extra links similar to Admin Menu module for Drupal 7 (flush caches, run cron, etc...).

2. Module Filter

(D7/D8) - https://www.drupal.org/project/module_filter

drupal-module-filter

The modules list page can become quite long and unwieldy as your site grows. To simplify your module administration, you can install the Module Filter module, which, as shown above, provides you a separate tab for each module / module package. This gives you two different ways (with the default filter textfield/by package) to quickly find and re-configure your modules.

3. Shield

(D7/D8) - https://www.drupal.org/project/shield

drupal-module-shield

This module helps you protect your development or staging site with HTTP authentication. Anonymous visitors and search engines won’t be able to reach your test environment, but you and your site users will.

4. Content Lock

(D7/D8) - https://www.drupal.org/project/content_lock

drupal-module-content-lock

This module prevents users from editing the same node. If another user is trying to edit the same node, he or she will be notified that the content is currently being edited.

The other great feature of this module is that it prevents users from leaving an edit form page without first saving it. They will be notified when attempting to close the browser window/tab, clicking on a link, etc. If the user confirms that he wants to leave without saving the node, the edit lock is automatically removed.

5. Environment Indicator

(D7/D8) - https://www.drupal.org/project/environment_indicator

drupal-module-environment-indicator

This module will help you stay sane while working in your different environments by adding a configurable color bar to each.

The Environment Indicator module adds a colored bar on the site that informs you which environment you're currently in (Development, Staging, Production, etc.). This is incredibly useful if you have multiple environments for each of your sites and are prone to forgetting which version of the site you are currently looking at.

6. reCAPTCHA

(D7/D8) - https://www.drupal.org/project/recaptcha

drupal-module-recaptcha

reCaptcha is a module built on top of the captcha module that implements the Google Captcha service to protect your site from spam. This web service shows a checkbox “I’m not a robot” at the bottom of your form. The service presents a challenge to the user where the user has to choose a particular set of images related to a subject.

Another option for protecting your site from spam is the Honeypot module.

7. Block Class

(D7/D8)  - https://www.drupal.org/project/block_class

drupal-module-block-class

This module allows users to add CSS classes to any block through the block configuration interface instead of to our twig templates. We just need to add our new classes to our CSS file.

8. Configuration Split

(D8) - https://www.drupal.org/project/config_split

drupal-module-config-split

This module filters the import/export of the configuration. It allows you to define sets of configurations that will get exported to separate directories when exporting from different environments. For example, you may need to have the devel module enabled or have a few blocks or views placed a certain way in the development environment, but not want that configuration to be exported to production. Using configuration split, you can specify both and thus easily share the development configuration with colleagues.

This module is also really useful with a multi-site installation.

9. RoleAssign

(D7/D8) -  https://www.drupal.org/project/roleassign

drupal-rol-asign

This module allows you, as a site administrator, to delegate the assignment of roles to another user without giving him or her the “Administer permissions” permission.

The module creates a new permission called “Assign roles”. Users with this permission are able to assign selected roles to other users. On the other hand, users with the “Administer permissions” permission may select which roles are available for assignment through this module.

For larger sites with multiple levels of administrators or whenever you need finer-grained control over which role can assign which other roles, check out Role Delegation.

10. Delete all

(D7/D8) - https://www.drupal.org/project/delete_all

drupal-module-delete-all

This is the module that I always install on my local or dev environment in conjunction with the Devel Generate module. It allows you to delete all content and/or users from a site with just one click.

This was particularly handy on a test site that one of my clients was using for a period of time when we needed to clean it up before starting with real data or when I tested imports and migrations of thousands of nodes. Really helpful!

Usage statistics of these modules.

The table below provides the usage statistics for these modules across all versions.

drupal-usage-statistics

Which Drupal 8 modules do you consider helpful?

I would love to hear about your “must install” Drupal 8 modules that are most helpful for your projects. Please leave a comment and share it with the community!

Nov 27 2018
Nov 27

Batch processing is usually an important aspect of any Drupal project and even more when we need to process huge amounts of data.

The main advantage of using batch is that it allows large amounts of data to be processed into small chunks or page requests that run without any manual intervention. Rather than use a single page load to process lots of data, the batch API allows the data to be processed as number of small page requests.

Using batch processing, we divide a large process into small pieces, each one of them executed as a separate page request, thereby avoiding stress on the server. This means that we can easily process 10,000 items without using up all of the server resources in a single page load.

This helps to ensure that the processing is not interrupted due to PHP timeouts, while users are still able to receive feedback on the progress of the ongoing operations.

Some uses of the batch API in Drupal:

  • Import or migrate data from an external source
  • Clean-up internal data
  • Run an action on several nodes
  • Communicate with an external API

Normally, batch jobs are launched by a form. However, what can we do if we want a *nix crontab to launch them on a regular basis? One of the best solutions is to use an external command like a custom Drush command and launch it from this crontab.

In this post we are going to create a custom Drush 9 command that loads all the nodes of a content type passed in as an argument (page, article ...). Then a batch process will simulate a long operation on each node. Next, we'll see how to run this drush command from crontab.

You can find the code for this module here: https://github.com/KarimBoudjema/Drupal8-ex-batch-with-drush9-command

This is the tree of the module:

tree web/modules/custom/ex_batch_drush9/
web/modules/custom/ex_batch_drush9/
|-- composer.json
|-- drush.services.yml
|-- ex_batch_drush9.info.yml
|-- README.txt
`-- src
    |-- BatchService.php
    `-- Commands
        `-- ExBatchDrush9Commands.php

We are going to proceed in three steps:

  1. Create a class to host our two main callback methods for batch processing (BatchService.php).
  2. Create our custom Drush 9 command to retrieve nodes, to create and process the batch sets (ExBatchDrush9Commands.php).
  3. Create a crontab task to run automatically the Drush command at scheduled times.

1. Create a BatchService class for the batch operations

A batch process is made of two main callbacks functions, one for processing each batch and the other for post-processing operations.

So in our class will have two methods, processMyNode() for processing each batch and processMyNodeFinished() to be launched when the batch processing is finished.

It's best practice to store the callback functions in their own file. This keeps them separate from anything else that your module might be doing. In this case I prefer to store them in a class that we can reuse later as a service.

Here is the code of BatchService.php

<?php
namespace Drupal\ex_batch_drush9;
/**
 * Class BatchService.
 */
class BatchService {
  /**
   * Batch process callback.
   *
   * @param int $id
   *   Id of the batch.
   * @param string $operation_details
   *   Details of the operation.
   * @param object $context
   *   Context for operations.
   */
  public function processMyNode($id, $operation_details, &$context) {
    // Simulate long process by waiting 100 microseconds.
    usleep(100);
    // Store some results for post-processing in the 'finished' callback.
    // The contents of 'results' will be available as $results in the
    // 'finished' function (in this example, processMyNodeFinished()).
    $context['results'][] = $id;
    // Optional message displayed under the progressbar.
    $context['message'] = t('Running Batch "@id" @details',
      ['@id' => $id, '@details' => $operation_details]
    );
  }
  /**
   * Batch Finished callback.
   *
   * @param bool $success
   *   Success of the operation.
   * @param array $results
   *   Array of results for post processing.
   * @param array $operations
   *   Array of operations.
   */
  public function processMyNodeFinished($success, array $results, array $operations) {
    $messenger = \Drupal::messenger();
    if ($success) {
      // Here we could do something meaningful with the results.
      // We just display the number of nodes we processed...
      $messenger->addMessage(t('@count results processed.', ['@count' => count($results)]));
    }
    else {
      // An error occurred.
      // $operations contains the operations that remained unprocessed.
      $error_operation = reset($operations);
      $messenger->addMessage(
        t('An error occurred while processing @operation with arguments : @args',
          [
            '@operation' => $error_operation[0],
            '@args' => print_r($error_operation[0], TRUE),
          ]
        )
      );
    }
  }
}

In the processMyNode()method we are going to process each element of our batch. As you can see, in this method we just simulate a long operation with the usleep() PHP function. Here we could load each node or connect with an external API. We also grab some information for post-processing.

In the processMyNodeFinished() method, we display relevant information to the user and we can even save the unprocessed operations for a later process.

2. Create the custom Drush 9 command to launch the batch

This is the most important part of our module. With this Drush command, we'll retrieve the data and fire the batch processing on those data.

Drush commands are now based on classes and the Annotated Command format. This will change the fundamental structure of custom Drush commands. This is great because we can now inject services in our command class and take advantage of all the OO power of Drupal 8.

A Drush command is composed of three files:

drush.services.yml - This is the file where our Drush command definition goes into. This is a Symfony service definition. Do not use your module's regular services.yml as you may have done in Drush 8 or else you will confuse the legacy Drush, which will lead to a PHP error.

You'll can see that in our example we inject two core services in our command class: entity_type.manager to access the nodes to process and logger.factory to log some pre-process and post-process information.

services:
  ex_batch_drush9.commands:
    class: \Drupal\ex_batch_drush9\Commands\ExBatchDrush9Commands
    tags:
      - { name: drush.command }
    arguments: ['@entity_type.manager', '@logger.factory']

composer.json - This is where we declare the location of the Drush command file for each version of Drush by adding the extra.drush.services section to the composer.json file of the implementing module. This is now optional, but will be required for Drush 10.

{
    "name": "org/ex_batch_drush9",
    "description": "This extension provides new commands for Drush.",
    "type": "drupal-drush",
    "authors": [
        {
            "name": "Author name",
            "email": "[email protected]"
        }
    ],
    "require": {
        "php": ">=5.6.0"
    },
    "extra": {
        "drush": {
            "services": {
                "drush.services.yml": "^9"
            }
        }
    }
}

MyModuleCommands.php - (src/Commands/ExBatchDrush9Commands.php in our case) It's in this class that we are going to define the custom Drush commands of our module. This class uses the Annotated method for commands, which means that each command is now a separate function with annotations that define its name, alias, arguments, etc. This class can also be used to define hooks with @hook annotation.

Some of the annotations available for use are:

@command: This annotation is used to define the Drush command. Make sure that you follow Symfony’s module:command structure for all your commands.
@aliases: An alias for your command.
@param: Defines the input parameters. For example, @param: integer $number
@option: Defines the options available for the commands. This should be an associative array where the name of the option is the key and the value could be - false, true, string, InputOption::VALUE_REQUIRED, InputOption::VALUE_OPTIONAL or an empty array.
@default: Defines the default value for options.
@usage: Demonstrates how the command should be used. For example, @usage: mymodule:command --option
@hook: Defines a hook to be fired. The default format is @hook type target, where type determines when the hook is called and target determines where the hook is called.

For a complete list of all the hooks available and their usage, refer to: https://github.com/consolidation/annotated-command

Here is the code for our Drush command.

<?php
namespace Drupal\ex_batch_drush9\Commands;
use Drupal\Core\Entity\EntityTypeManagerInterface;
use Drupal\Core\Logger\LoggerChannelFactoryInterface;
use Drush\Commands\DrushCommands;
/**
 * A Drush commandfile.
 *
 * In addition to this file, you need a drush.services.yml
 * in root of your module, and a composer.json file that provides the name
 * of the services file to use.
 *
 * See these files for an example of injecting Drupal services:
 *   - http://cgit.drupalcode.org/devel/tree/src/Commands/DevelCommands.php
 *   - http://cgit.drupalcode.org/devel/tree/drush.services.yml
 */
class ExBatchDrush9Commands extends DrushCommands {
  /**
   * Entity type service.
   *
   * @var \Drupal\Core\Entity\EntityTypeManagerInterface
   */
  private $entityTypeManager;
  /**
   * Logger service.
   *
   * @var \Drupal\Core\Logger\LoggerChannelFactoryInterface
   */
  private $loggerChannelFactory;
  /**
   * Constructs a new UpdateVideosStatsController object.
   *
   * @param \Drupal\Core\Entity\EntityTypeManagerInterface $entityTypeManager
   *   Entity type service.
   * @param \Drupal\Core\Logger\LoggerChannelFactoryInterface $loggerChannelFactory
   *   Logger service.
   */
  public function __construct(EntityTypeManagerInterface $entityTypeManager, LoggerChannelFactoryInterface $loggerChannelFactory) {
    $this->entityTypeManager = $entityTypeManager;
    $this->loggerChannelFactory = $loggerChannelFactory;
  }
  /**
   * Update Node.
   *
   * @param string $type
   *   Type of node to update
   *   Argument provided to the drush command.
   *
   * @command update:node
   * @aliases update-node
   *
   * @usage update:node foo
   *   foo is the type of node to update
   */
  public function updateNode($type = '') {
    // 1. Log the start of the script.
    $this->loggerChannelFactory->get('ex_batch_drush9')->info('Update nodes batch operations start');
    // Check the type of node given as argument, if not, set article as default.
    if (strlen($type) == 0) {
      $type = 'article';
    }
    // 2. Retrieve all nodes of this type.
    try {
      $storage = $this->entityTypeManager->getStorage('node');
      $query = $storage->getQuery()
        ->condition('type', $type)
        ->condition('status', '1');
      $nids = $query->execute();
    }
    catch (\Exception $e) {
      $this->output()->writeln($e);
      $this->loggerChannelFactory->get('ex_batch_drush9')->warning('Error found @e', ['@e' => $e]);
    }
    // 3. Create the operations array for the batch.
    $operations = [];
    $numOperations = 0;
    $batchId = 1;
    if (!empty($nids)) {
      foreach ($nids as $nid) {
        // Prepare the operation. Here we could do other operations on nodes.
        $this->output()->writeln("Preparing batch: " . $batchId);
        $operations[] = [
          '\Drupal\ex_batch_drush9\BatchService::processMyNode',
          [
            $batchId,
            t('Updating node @nid', ['@nid' => $nid]),
          ],
        ];
        $batchId++;
        $numOperations++;
      }
    }
    else {
      $this->logger()->warning('No nodes of this type @type', ['@type' => $type]);
    }
    // 4. Create the batch.
    $batch = [
      'title' => t('Updating @num node(s)', ['@num' => $numOperations]),
      'operations' => $operations,
      'finished' => '\Drupal\ex_batch_drush9\BatchService::processMyNodeFinished',
    ];
    // 5. Add batch operations as new batch sets.
    batch_set($batch);
    // 6. Process the batch sets.
    drush_backend_batch_process();
    // 6. Show some information.
    $this->logger()->notice("Batch operations end.");
    // 7. Log some information.
    $this->loggerChannelFactory->get('ex_batch_drush9')->info('Update batch operations end.');
  }
}

In this class, we first inject our two core services in the __construct() method: entity_type.manager and logger.factory.

Next, in the updateNode() annotated method we define our command with three annotations:

@param string $type - Defines the input parameter, the content type in our case.
@command update:node - Defines the name of the Drush command. In this case it's "update:node" so we would launch the command with: drush update:node
@aliases update-node - Defines an alias for the command

The main part of this command is the creation of the operations array for our batch processing (See points 3,4,5 and 6). Nothing strange here, we just define our operations array (3 and 4) pointing to the two callback functions that are located in our BatchService.php class.

Once the batch operations are added as new batch sets (5), we process the batch sets with the function drush_backend_batch_process(). This is a drop in replacement for the existing batch_process() function of Drupal. It will process a Drupal batch by spawning multiple Drush processes.

Finally, we show information to the user and log some information for a later use.

That's it! We can now test our brand new Drush 9 custom command!

To do so, just clear the cache with drush cr and launch the command with drush update:node

3. Bonus: Run the Drush command from crontab

As noted before, we want to run this custom command from crontab to perform the update automatically at the scheduled times.

The steps taken may vary based on your server's operating system. With Mac, Linux and Unix servers, we manage scheduled tasks by creating and editing crontabs that execute cron jobs at specified intervals.

1. Open a terminal window on your computer and enter the following command to edit your crontab - this will invoke your default editor (usually a flavor of vi):

crontab -e

2. Enter the following scheduled task with our Drush command (where [docroot_path] is your server's docroot):

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
*/5 * * * * drush --root=[docroot_path] update:node

We need to add an environment path in our cron table first line (cron commands run one after anonther).
This command will run our custom Drush command every five minutes.

3. Restart the cron service with the following command:

systemctl restart cron

4. Check if the command is running
We can check the log of our Drupal application every five minutes since we logged some information with the command itself, but we can also use another way with the following command:

sudo tail -f /var/mail/root

Recap.

- We created a class to host our two main callback methods for batch processing (BatchService.php).
- We created a custom Drush 9 command to retrieve nodes, to create and process the batch sets, injecting core services, like entity_type.manager to access the nodes to process and logger.factory to log some pre-process and post-process information.
- We created a crontab task to run automatically the Drush command at scheduled times.

This way, we can now run heavy processes, on a regular basis without putting undue stress on the server.

Oct 13 2018
Oct 13

Creating forms is part of the day to day live of a Drupal programmer.  Forms are represented as nested arrays in both Drupal 7 and Drupal 8. But they are different in that in Drupal 7 you define your form arrays in functions and in Drupal 8 you create a form class.

In this post we’ll create a custom form with two fields, a text field and a checkbox field, we’ll validate those fields, display them in a message and finally redirect the user to the home page.

You can find the code of this module here.

Here is a screenshot of the form:

drupal 8 custom form

You can see the folder structure of this module below:

web/modules/custom/ex81
|-- ex81.info.yml
|-- ex81.routing.yml
`-- src
    `-- Form
        `-- HelloForm.php

2 directories, 3 files

First we’ll create the form step by step manually, then we’ll use Drupal Console to do it in a easier way.

1. Create the module’s skeleton
2. Create the form’s class and assign a name to the form
3. Build the form
4. Validate the form
5. Process the collected values
6. Add a route to your form
7. Do it easier with Drupal Console

In Drupal 8, each form is defined by a controller class that implements the interface \Drupal\Core\Form\FormInterface which defines four basic methods:

getFormId() - Defines the unique ID for the form
buildForm() - Triggered when the user requests the form. Builds the basic $form array with all the fields of the form.
validateForm() -  Triggered when the form is submitted. It’s used to check the values collected by the form and optionally raise errors.
submitForm() -  Used to carry out the form submission process, if the form has passed validation with no errors.

1. Create the module’s skeleton

As always, we use the Drupal Console to create the module. Don’t forget to install the module after it has been generated.

We use the following command:

drupal generate:module  --module="ex81" \
--machine-name="ex81" \
--module-path="modules/custom" \
--description="Example of a simple custom form" \
--core="8.x" \
--package="Custom" \
--uri="http://default" \
--no-interaction

Once created we can install it with drupal module:install ex81 command

2. Create the form’s class

We create a new php file called HelloForm.php in the src/Form folder. In this file we create a new class called HelloForm that extends the abstract class FormBase (which implements the FormInterface interface).

In this class we create the getformId() method to set a unique string identifying the form.

<?php

namespace Drupal\ex81\Form;

use Drupal\Core\Form\FormBase;
use Drupal\Core\Form\FormStateInterface;

class HelloForm extends FormBase {

  /**
   * Returns a unique string identifying the form.
   *
   * The returned ID should be a unique string that can be a valid PHP function
   * name, since it's used in hook implementation names such as
   * hook_form_FORM_ID_alter().
   *
   * @return string
   *   The unique string identifying the form.
   */
  public function getFormId() {
    return 'ex81_hello_form';
  }

}

3. Build the form

Now we add to this class a new method called buildForm() which defines the text and the checkbox fields of our form. We also add a field description to add a small message on top of the form.
At the end of the method we add a submit field and return the form array.

  /**
   * Form constructor.
   *
   * @param array $form
   *   An associative array containing the structure of the form.
   * @param \Drupal\Core\Form\FormStateInterface $form_state
   *   The current state of the form.
   *
   * @return array
   *   The form structure.
   */
  public function buildForm(array $form, FormStateInterface $form_state) {

    $form['description'] = [
      '#type' => 'item',
      '#markup' => $this->t('Please enter the title and accept the terms of use of the site.'),
    ];

    
$form['title'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Title'),
      '#description' => $this->t('Enter the title of the book. Note that the title must be at least 10 characters in length.'),
      '#required' => TRUE,
    ];

    $form['accept'] = array(
      '#type' => 'checkbox',
      '#title' => $this
        ->t('I accept the terms of use of the site'),
      '#description' => $this->t('Please read and accept the terms of use'),
    );


    // Group submit handlers in an actions element with a key of "actions" so
    // that it gets styled correctly, and so that other modules may add actions
    // to the form. This is not required, but is convention.
    $form['actions'] = [
      '#type' => 'actions',
    ];

    // Add a submit button that handles the submission of the form.
    $form['actions']['submit'] = [
      '#type' => 'submit',
      '#value' => $this->t('Submit'),
    ];

    return $form;

  }

4. Validate the form

To validate the form we just need to implement the validateForm() method from \Drupal\Core\Form\FormInterface in our HelloForm class.

In our example we’ll raise an error if the length of the title is less than 10 and also if the checkbox is not checked. User-submitted values can be accessed via $form_state->getValue('key') where "key" is the name of the element whose value we would like to retrieve.

We raise an error thanks to the setErrorByName() method of the $form_state object.

  /**
   * Validate the title and the checkbox of the form
   * 
   * @param array $form
   * @param \Drupal\Core\Form\FormStateInterface $form_state
   * 
   */
  public function validateForm(array &$form, FormStateInterface $form_state) {
    parent::validateForm($form, $form_state);

    $title = $form_state->getValue('title');
    $accept = $form_state->getValue('accept');

    if (strlen($title) < 10) {
      // Set an error for the form element with a key of "title".
      $form_state->setErrorByName('title', $this->t('The title must be at least 10 characters long.'));
    }

    if (empty($accept)){
      // Set an error for the form element with a key of "accept".
      $form_state->setErrorByName('accept', $this->t('You must accept the terms of use to continue'));
    }

  }

5. Process the collected values

Now it’s time to process the values collected by our form. We do this with the submitForm() submission handler method.

Here we can save the data to the database, send them to an external API or use them in a service. In our case we’ll just display those values and redirect the user to the home page. To redirect the user, we’ll use the setRedirect() method of the $form_state object.

Remember that if you need to save those values in the config system, you should extend the ConfigFormBase as we saw earlier in another post.

  /**
   * Form submission handler.
   *
   * @param array $form
   *   An associative array containing the structure of the form.
   * @param \Drupal\Core\Form\FormStateInterface $form_state
   *   The current state of the form.
   */
  public function submitForm(array &$form, FormStateInterface $form_state) {

    // Display the results.
    
    // Call the Static Service Container wrapper
    // We should inject the messenger service, but its beyond the scope of this example.
    $messenger = \Drupal::messenger();
    $messenger->addMessage('Title: '.$form_state->getValue('title'));
    $messenger->addMessage('Accept: '.$form_state->getValue('accept'));

    // Redirect to home
    $form_state->setRedirect('<front>');

  } 

6. Add a route to your form

Now that we have added a controller to your custom module, we are almost done. The last thing is to add a route in our routine file in order to allow users to access our form.

If we haven’t created this file before, we create the ex81.routing.yml file in the root of the module. In this file we add the route that points to the form controller (HelloForm) we created above. In this case we won’t be using the _controller key, but the _form key, so Drupal will use the form builder service when serving this request.

ex81.hello_form:
  path: '/ex81/helloform'
  defaults:
    _form: 'Drupal\ex81\Form\HelloForm'
    _title: 'Simple custom form example'
  requirements:
    _permission: 'access content'

The above code adds a route that maps the path /ex81/helloform to our form controller Drupal\ex81\Form\HelloForm.

Now we can navigate to our form and test it!

Recap.

1. We created the module with the Drupal Console command: drupal generate:module
2. We created the form controller class HelloForm in the src\Form folder. This class extends the abstract class FormBase that implements the interface FormInterface.
3. We set a unique string ID for our form with the getFormId() method.
4. We defined the fields of the form with the buildForm() method.
5. To validate the form, we used the validateForm() method and use the setErrorByName() method of the $form_state object.
6. We processed the values of the form with the submitForm() method and redirect the user to the home page with the setRedirect() method of the $form_state object.
7. We created a route that points to our form using the specific key _form

7. Bonus: Create a custom form with Drupal Console

We’ll create the same form as above but in another module called 'ex82'.

drupal generate:module  --module="ex81" \
--machine-name="ex81" \
--module-path="modules/custom" \
--description="Example of a simple custom form" \
--core="8.x" \
--package="Custom" \
--uri="http://default" \
--no-interaction

Now we’ll create the form with a simple Drupal Console command: drupal generate:form

drupal generate:form  \
--module="ex82" \
--class="HelloForm" \
--form-id="ex82_hello_form" \
--services='messenger' \
--inputs='"name":"description", "type":"item", "label":"Description", "options":"", "description":"Please enter the title and accept the terms of use of the site.", "maxlength":"", "size":"", "default_value":"", "weight":"0", "fieldset":""' \
--inputs='"name":"title", "type":"textfield", "label":"Title", "options":"", "description":"Enter the title of the book. Note that the title must be at least 10 characters in length.", "maxlength":"64", "size":"64", "default_value":"", "weight":"0", "fieldset":""' \
--inputs='"name":"accept", "type":"checkbox", "label":"I accept the terms of use of the site", "options":"", "description":"Please read and accept the terms of use", "maxlength":"", "size":"", "default_value":"", "weight":"0", "fieldset":""' \
--path="/ex82/helloform" \
--uri="http://default" \
--no-interaction

This will generate two files, HelloForm.php and ex82.routing.yml, as you can see Drupal Console generate almost the entire form.

Let’s take a look at the new generated HelloForm.php

<?php

namespace Drupal\ex82\Form;

use Drupal\Core\Form\FormBase;
use Drupal\Core\Form\FormStateInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;
use Drupal\Core\Messenger\MessengerInterface;

/**
 * Class HelloForm.
 */
class HelloForm extends FormBase {

  /**
   * Drupal\Core\Messenger\MessengerInterface definition.
   *
   * @var \Drupal\Core\Messenger\MessengerInterface
   */
  protected $messenger;
  /**
   * Constructs a new HelloForm object.
   */
  public function __construct(
    MessengerInterface $messenger
  ) {
    $this->messenger = $messenger;
  }

  public static function create(ContainerInterface $container) {
    return new static(
      $container->get('messenger')
    );
  }


  /**
   * {@inheritdoc}
   */
  public function getFormId() {
    return 'ex82_hello_form';
  }

  /**
   * {@inheritdoc}
   */
  public function buildForm(array $form, FormStateInterface $form_state) {
    $form['description'] = [
      '#type' => 'item',
      '#title' => $this->t('Description'),
      '#description' => $this->t('Please enter the title and accept the terms of use of the site.'),
    ];
    $form['title'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Title'),
      '#description' => $this->t('Enter the title of the book. Note that the title must be at least 10 characters in length.'),
      '#maxlength' => 64,
      '#size' => 64,
    ];
    $form['accept'] = [
      '#type' => 'checkbox',
      '#title' => $this->t('I accept the terms of use of the site'),
      '#description' => $this->t('Please read and accept the terms of use'),
    ];
    $form['submit'] = [
      '#type' => 'submit',
      '#value' => $this->t('Submit'),
    ];

    return $form;
  }

  /**
   * {@inheritdoc}
   */
  public function validateForm(array &$form, FormStateInterface $form_state) {
    parent::validateForm($form, $form_state);
  }

  /**
   * {@inheritdoc}
   */
  public function submitForm(array &$form, FormStateInterface $form_state) {
    // Display result.
    foreach ($form_state->getValues() as $key => $value) {
      drupal_set_message($key . ': ' . $value);
    }

  }

}

First you can see that we voluntary inject the messenger service in the Drupal Console command, this allows us to avoid calling the Static Service Container wrapper \Drupal::messenger(). We’ll use it in the buildForm() method later.

Now, we’ll need to add the code for the validation and the submit process with:
- validateForm() method
- submitForm() method

You can copy the same code of the validateForm() method described above in this post in the section 4.

  public function validateForm(array &$form, FormStateInterface $form_state) {
    parent::validateForm($form, $form_state);

    $title = $form_state->getValue('title');
    $accept = $form_state->getValue('accept');

    if (strlen($title) < 10) {
      // Set an error for the form element with a key of "title".
      $form_state->setErrorByName('title', $this->t('The title must be at least 10 characters long.'));
    }

    if (empty($accept)){
      // Set an error for the form element with a key of "accept".
      $form_state->setErrorByName('accept', $this->t('You must accept the terms of use to continue'));
    }

  }

For the buildForm() method we’ll use the same code as above but with a small change since we injected the messenger service

 public function submitForm(array &$form, FormStateInterface $form_state) {

    $this->messenger->addMessage('Title: '.$form_state->getValue('title'));
    $this->messenger->addMessage('Accept: '.$form_state->getValue('accept'));

    // Redirect to home
    $form_state->setRedirect('<front>');
    return;
  }

As simple as that. YES! Drupal Console is definitively a great tool to create custom forms!!!

Sep 24 2018
Sep 24

In Drupal 7, to manage system variables, we used variable_get() / variable_set() /  variable_del() calls and stored those variables in the variable table of the database.

In Drupal 8 we now use the Configuration system which provides a central place for modules to store configuration data. This system allows to store information that can be synchronized between development and production sites. This information is often created during site building and is not typically generated by regular users during normal site operation.
In Drupal 8 configuration is still stored in the database,  but it can now be synced with YML files on the disk for deployment purposes aswell.

In this post we will create a configuration form that stores a configuration value in our system. In this case it will be a form that stores the value of an external API key. Next, we will retrieve this value and use it in a very simple controller.

1. Create the module
2. Create the form
3. Check the form
4. Retrieve the config information in a controller

1. Create the module skeleton

We use the following Drupal Console command.

drupal generate:module  \
--module="ex08" \
--machine-name="ex08" \
--module-path="modules/custom" \
--description="An simple example of a config form" \
--core="8.x" \
--package="Custom" \
--module-file \
--uri="http://default" \
--no-interaction

2. Create the form

To create the config form we use Drupal Console with the drupal generate:form:config command.

drupal generate:form:config  \
--module="ex08" \
--class="ExternalApiKeyForm" \
--form-id="external_api_key_form" \
--config-file \
--inputs='"name":"your_external_api_key", "type":"textfield", "label":"Your external API Key", "options":"", "description":"Enter your external API Key", "maxlength":"64", "size":"64", "default_value":"none", "weight":"0", "fieldset":""' \
--path="/admin/config/ex08/externalapikey" \
--uri="http://default" \
--no-interaction

This Drupal Console command will:
- create the config form file ExternalApiKeyForm.php under the src/form folder
- update or create the ex08.routing.yml file with a route to the config form to access it.

Now let’s see what this code is doing in detail.

3. The config form

If we now open the ExternalApiKeyForm.php file we see the following code.

<?php

namespace Drupal\ex08\Form;

use Drupal\Core\Form\ConfigFormBase;
use Drupal\Core\Form\FormStateInterface;

/**
 * Class ExternalApiKeyForm.
 */
class ExternalApiKeyForm extends ConfigFormBase {

  /**
   * {@inheritdoc}
   */
  protected function getEditableConfigNames() {
    return [
      'ex08.externalapikey',
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function getFormId() {
    return 'external_api_key_form';
  }

  /**
   * {@inheritdoc}
   */
  public function buildForm(array $form, FormStateInterface $form_state) {
    $config = $this->config('ex08.externalapikey');
    $form['your_external_api_key'] = [
      '#type' => 'textfield',
      '#title' => $this->t('Your external API Key'),
      '#description' => $this->t('Store the external API Key'),
      '#maxlength' => 64,
      '#size' => 64,
      '#default_value' => $config->get('your_external_api_key'),
    ];
    return parent::buildForm($form, $form_state);
  }

  /**
   * {@inheritdoc}
   */
  public function validateForm(array &$form, FormStateInterface $form_state) {
    parent::validateForm($form, $form_state);
  }

  /**
   * {@inheritdoc}
   */
  public function submitForm(array &$form, FormStateInterface $form_state) {
    parent::submitForm($form, $form_state);

    $this->config('ex08.externalapikey')
      ->set('your_external_api_key', $form_state->getValue('your_external_api_key'))
      ->save();
  }

}

Fist we see that the class ExternalApiKeyForm extends ConfigFormBase abstract class which also extends the FormBase abstract class.

The ExternalApiKeyForm class has four basic methods:

getEditableConfigNames() - gets the configuration object name (The name of the configuration environment where we store or get the values). In our case we give it the name: 'ex08'.externalapikey'.

getFormId() - returns the form's unique ID. This will be the name of our form or its ID. In our case, we named it 'external_api_key_form'.

buildForm() - returns the form array. This method creates the form and in this case one item named 'your_external_api_key'. This item is a text field. We will also use this name for
the key of the key/value pair stored in the configuration.

We also get the current value for the key 'your_external_api_key' from the configuration object to set it as a default value for this field:   '#default_value' => $config->get('your_external_api_key'),
 
validateForm() - validates the form.

submitForm() - processes the form submission. In this step, we will set and save the value of the key 'your_external_api_key' in the configuration system.

4. The route to the form

Drupal Console also updates or creates the ex08.routing.yml file with a route to the config form to access it.

ex08.external_api_key_form:
  path: '/admin/config/ex08/externalapikey'
  defaults:
    _form: '\Drupal\ex08\Form\ExternalApiKeyForm'
    _title: 'ExternalApiKeyForm'
  requirements:
    _permission: 'access administration pages'
  options:
    _admin_route: TRUE


Nothing strange here. With path we define the URL to access the form. With _form we indicate the class which generates the form and with _title we define the title of the form.  With _permission we give access to users who have permission to access administration pages (user access to the administration path).

Now that we have a form and a route, we can access the form from the URL /admin/config/ex08/externalapikey and test it.

Drupal 8 config form

5. Debug the configuration object.

We can also debug the configuration object with Drupal Console command debug:config (dc) which give us a list configuration objects names and single configuration object:

Drupal console debug config

 

6. Retrieve the value in a controller class.

To get or set the value of a configuration object in a class, we will need a service. In fact we need the core service 'config.factory'.

We can find it with the Drupal Console commands: drupal debug:container or drupal debug:container config.factory

This service has a method that allows to get the value by its key of a configuration object.

So, when we create the controller, well inject this service to use later.

Lets create the controller and inject the service at the same time with Drupal Console.

drupal generate:controller  --module="ex08" \
--class="ExternalApiKeyController" \
--routes='"title":"External API Key", "name":"ex08.external_api_key_controller_ShowKey", "method":"ShowKey", "path":"/ex08/ShowKey"' \
--services='config.factory' \
--uri="http://default" \
--no-interaction

This command creates the file ExternalApiKeyController.php under /src/Controller and update the ex08.routing.yml file with a new route.

The updated ex08.routing.yml file:

ex08.external_api_key_form:
  path: '/admin/config/ex08/externalapikey'
  defaults:
    _form: '\Drupal\ex08\Form\ExternalApiKeyForm'
    _title: 'ExternalApiKeyForm'
  requirements:
    _permission: 'access administration pages'
  options:
    _admin_route: TRUE

ex08.external_api_key_controller_ShowKey:
  path: '/ex08/ShowKey'
  defaults:
    _controller: '\Drupal\ex08\Controller\ExternalApiKeyController::ShowKey'
    _title: 'External API Key'
  requirements:
    _permission: 'access content'

The new ExternalApiKeyController.php:

<?php

namespace Drupal\ex08\Controller;

use Drupal\Core\Controller\ControllerBase;
use Symfony\Component\DependencyInjection\ContainerInterface;
use Drupal\webprofiler\Config\ConfigFactoryWrapper;

/**
 * Class ExternalApiKeyController.
 */
class ExternalApiKeyController extends ControllerBase {

  /**
   * Drupal\webprofiler\Config\ConfigFactoryWrapper definition.
   *
   * @var \Drupal\webprofiler\Config\ConfigFactoryWrapper
   */
  protected $configFactory;

  /**
   * Constructs a new ExternalApiKeyController object.
   */
  public function __construct(ConfigFactoryWrapper $config_factory) {
    $this->configFactory = $config_factory;
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container) {
    return new static(
      $container->get('config.factory')
    );
  }

  /**
   * Showkey.
   *
   * @return array
   *
   */
  public function ShowKey() {

    return [
      '#type' => 'markup',
      '#markup' => $this->t('The ShowKey() action')
    ];
  }

}

We can see that the config.factory service has been retrieved from the service container  in the create() method and sent back (dependency injection) to the class itself through the __construct() method. Thank you, Drupal Console!!!

Now, in the ShowKey() method, we need to access the value of our ex08.externalapikey configuration object and get the value of the key your_external_api_key. This is what we can see in the following code:

  public function ShowKey() {

    // Get the configuration object
    $config = $this->configFactory->get('ex08.externalapikey');
    // Get the value of the key 'your_external_api_key' 
    $key = $config->get('your_external_api_key');

    return [
      '#type' => 'markup',
      '#markup' => $this->t('The External API Key is: @key',['@key'=>$key])
    ];
  }

Done!!!

Recap.

1. We created a configuration form, and a route to this form, with the DC command drupal generate:form:config. This form generated a configuration object ('ex08.externalapikey') and a set of key/value pair. In our case, we have just one key/value pair and the key is: 'your_external_api_key'

2. We debugged the key/value pair of the configuration object with the DC command drupal debug:config  <name-of-config-object>

3. We created a controller and injected the service we need (config.factory) to access the config system.
In the controller, we called the service object to get the value of a particular key of the configuration object.

As we can see, today Drupal Console is a must in Drupal 8.

7. Bonus: Retrieve the config value in a hook

Sometimes we need set or get configuration values in a hook, in this case we can't inject the service config.factory by dependency injection.

We’ll use the \Drupal static service container wrapper to do so.

function ex08_form_alter(&$form, \Drupal\Core\Form\FormStateInterface $form_state, $form_id){
        // Get the config object
        $config = \Drupal::config('ex08.externalapikey');
        // Get the key value
        $key = $config->get('your_external_api_key');
}
Aug 07 2018
Aug 07

Queues are particularly important when we need to stash some tasks for later processing. To do so, we are going to put some tasks or data in a queue (create the queue) and later we will process those tasks with a QueueWorker plugin (process the queue), usually triggered by cron.

There are several ways to create a queue:
- With a form class
- With a controller class
- With a hook_cron() function
 
To process the queue, we also have different options:
- As a cron process with a QueueWorker plugin
- As a batch process also with QueueWorker plugin but extending a base plugin
- As a batch process claiming each item of the queue in a service or in a controller

Today we'll create the queue with a controller and process it with a QueueWorker plugin when cron runs or manually with Drupal Console.

Creating the queue with a controller has the advantage of allowing us to use an external crontab (e.g., a Linux crontab), so we won't need Drupal’s cron at all to run the controller, allowing us to launch it when we need it. This is also a more reliable method because it uses fewer resources and it is independent of any page request. Remember that the Drupal cron is 'poor man’s cron', as it depends on page requests for its execution even if there is a way to execute it with a linux crontab.

In this example module we'll generate a queue with a controller, importing the title and the description tags form the Drupal Planet RSS file. Next, when Cron runs, , we'll create a node page with a QueueWorker plugin for each item in the queue. You can download the code here: https://github.com/KarimBoudjema/Drupal8-ex-queue-api-01.

This module has two main parts:

1. A controller class src/Controller/ExQueueController.php with its corresponding route in exqueue01.routing.yml and two main methods:

getData() - Get external data and insert the data in the queue 'exqueue_import'.
deleteTheQueue() - Delete all item in the queue.

2. A Cron QueueWorker plugin src/Plugin/QueueWorker/ExQueue01.php that will process each item in the queue.

You can see the tree module here:

web/modules/custom/exqueue01/
|-- exqueue01.info.yml
|-- exqueue01.module
|-- exqueue01.permissions.yml
|-- exqueue01.routing.yml
|-- README.txt
`-- src
    |-- Controller
    |   `-- ExQueueController.php
    `-- Plugin
        `-- QueueWorker
            `-- ExQueue01.php

4 directories, 7 files

That's not too difficult, is it?

1. Create the queue with a controller.

Here is the code of the controller. You can also get it here: https://github.com/KarimBoudjema/Drupal8-ex-queue-api-01/blob/master/src...

<?php
namespace Drupal\exqueue01\Controller;
use Drupal\Core\Controller\ControllerBase;
use Drupal\Core\Queue\QueueFactory;
use Symfony\Component\DependencyInjection\ContainerInterface;
use Drupal\Core\Messenger\MessengerInterface;
use GuzzleHttp\ClientInterface;
use GuzzleHttp\Exception\RequestException;
/**
 * Class ExQueueController.
 *
 * Demonstrates the use of the Queue API
 * There is two routes.
 * 1) \Drupal\exqueue01\Controller\ExQueueController::getData
 * The getData() methods allows to load external data and
 * for each array element create a queue element
 * Then on Cron run, we create a page node for each element with
 * 2) \Drupal\exqueue01\Controller\ExQueueController::deleteTheQueue
 * The deleteTheQueue() methods delete the queue "exqueue_import"
 * and all its elements
 * Once the queue is created with tha data, on Cron run
 * we create a new page node for each item in the queue with the QueueWorker
 * plugin ExQueue01.php .
 */
class ExQueueController extends ControllerBase {
  /**
   * Drupal\Core\Messenger\MessengerInterface definition.
   *
   * @var \Drupal\Core\Messenger\MessengerInterface
   */
  protected $messenger;
  /**
   * Symfony\Component\DependencyInjection\ContainerAwareInterface definition.
   *
   * @var \Symfony\Component\DependencyInjection\ContainerAwareInterface
   */
  protected $queueFactory;
  /**
   * GuzzleHttp\ClientInterface definition.
   *
   * @var \GuzzleHttp\ClientInterface
   */
  protected $client;
  /**
   * Inject services.
   */
  public function __construct(MessengerInterface $messenger, QueueFactory $queue, ClientInterface $client) {
    $this->messenger = $messenger;
    $this->queueFactory = $queue;
    $this->client = $client;
  }
  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container) {
    return new static(
      $container->get('messenger'),
      $container->get('queue'),
      $container->get('http_client')
    );
  }
  /**
   * Delete the queue 'exqueue_import'.
   *
   * Remember that the command drupal dq checks first for a queue worker
   * and if it exists, DC suposes that a queue exists.
   */
  public function deleteTheQueue() {
    $this->queueFactory->get('exqueue_import')->deleteQueue();
    return [
      '#type' => 'markup',
      '#markup' => $this->t('The queue "exqueue_import" has been deleted'),
    ];
  }
  /**
   * Getdata from external source and create a item queue for each data.
   *
   * @return array
   *   Return string.
   */
  public function getData() {
    // 1. Get data into an array of objects
    // 2. Get the queue and the total of items before the operations
    // 3. For each element of the array, create a new queue item
    // 1. Get data into an array of objects
    // We can choose between two methods
    // getDataFromRSS() or getFakeData()
    $data = $this->getDataFromRss();
    // $data = $this->getFakeData();
    if (!$data) {
      return [
        '#type' => 'markup',
        '#markup' => $this->t('No data found'),
      ];
    }
    // 2. Get the queue and the total of items before the operations
    // Get the queue implementation for 'exqueue_import' queue.
    $queue = $this->queueFactory->get('exqueue_import');
    // Get the total of items in the queue before adding new items.
    $totalItemsBefore = $queue->numberOfItems();
    // 3. For each element of the array, create a new queue item.
    foreach ($data as $element) {
      // Create new queue item.
      $queue->createItem($element);
    }
    // 4. Get the total of item in the Queue.
    $totalItemsAfter = $queue->numberOfItems();
    // 5. Get what's in the queue now.
    $tableVariables = $this->getItemList($queue);
    $finalMessage = $this->t('The Queue had @totalBefore items. We should have added @count items in the Queue. Now the Queue has @totalAfter items.',
      [
        '@count' => count($data),
        '@totalAfter' => $totalItemsAfter,
        '@totalBefore' => $totalItemsBefore,
      ]);
    return [
      '#type' => 'table',
      '#caption' => $finalMessage,
      '#header' => $tableVariables['header'],
      '#rows' => $tableVariables['rows'],
      '#attributes' => $tableVariables['attributes'],
      '#sticky' => $tableVariables['sticky'],
      'empty' => $this->t('No items.'),
    ];
  }
  /**
   * Generate an array of objects to simulate getting data from an RSS file.
   *
   * @return array
   *   Return an array of data
   */
  protected function getFakeData() {
    // We should get the XML content and convert it to an array of item objects
    // We use now an example array of item object.
    $content = [];
    for ($i = 1; $i <= 10; $i++) {
      $item = new \stdClass();
      $item->title = 'Title ' . $i;
      $item->body = 'Body ' . $i;
      $content[] = $item;
    }
    return $content;
  }
  /**
   * Generate an array of objects from an external RSS file.
   *
   * @return array|bool
   *   Return an array or false
   */
  protected function getDataFromRss() {
    // 1. Try to get the data form the RSS
    // URI of the XML file.
    $uri = 'https://www.drupal.org/planet/rss.xml';
    // 1. Try to get the data form the RSS.
    try {
      $response = $this->client->get($uri, ['headers' => ['Accept' => 'text/plain']]);
      $data = (string) $response->getBody();
      if (empty($data)) {
        return FALSE;
      }
    }
    catch (RequestException $e) {
      return FALSE;
    }
    // 2. Retrieve data in a simple xml object.
    $data = simplexml_load_string($data);
    // 3. Transform in a array of object
    // We could transform in array
    // $data = json_decode(json_encode($data));
    // Look at all children of the channel child.
    $content = [];
    foreach ($data->children()->children() as $child) {
      if (!empty($child->title)) {
        // Create an object.
        $item = new \stdClass();
        $item->title = $child->title->__toString();
        $item->body = $child->description->__toString();
        // Place the object in an array.
        $content[] = $item;
      }
    }
    if (empty($content)) {
      return FALSE;
    }
    return $content;
    /*
    // Using simplexml_load_file
    $xml = simplexml_load_file($uri);
    $data = json_decode(json_encode($xml));
    ksm($data);
     */
  }
  /**
   * Get all items of queue.
   *
   * Next place them in an array so we can retrieve them in a table.
   *
   * @param object $queue
   *   A queue object.
   *
   * @return array
   *   A table array for rendering.
   */
  protected function getItemList($queue) {
    $retrieved_items = [];
    $items = [];
    // Claim each item in queue.
    while ($item = $queue->claimItem()) {
      $retrieved_items[] = [
        'data' => [$item->data->title, $item->item_id],
      ];
      // Track item to release the lock.
      $items[] = $item;
    }
    // Release claims on items in queue.
    foreach ($items as $item) {
      $queue->releaseItem($item);
    }
    // Put the items in a table array for rendering.
    $tableTheme = [
      'header' => [$this->t('Title'), $this->t('ID')],
      'rows'   => $retrieved_items,
      'attributes' => [],
      'caption' => '',
      'colgroups' => [],
      'sticky' => TRUE,
      'empty' => $this->t('No items.'),
    ];
    return $tableTheme;
  }
}

First we injected the QueueFactory service into our controller. We also injected two others services like messenger to display Drupal messages to the user and the http_client service to retrieve the Drupal Planet RSS file.

Next in the getData() method we'll do three processes:

a- Get the data from an external RSS file
b- Get or create the queue and populate it
c- Retrieve all the items in the queue for information

a -  Get the data form the Drupal Planet RSS file in an array of objects with the custom method getDataFromRss()

$data = $this->getDataFromRss();

Note that there is also another custom method getFakeData() that you can use in local. It simulates the results of getting data from a external RSS file.

b – Get or in fact create the queue 'exqueue_import', get the total of item in the at this time for information purpose and populate the queue with the $data array.

     $queue = $this->queueFactory->get('exqueue_import');
        $totalItemsBefore = $queue->numberOfItems();
        foreach ($data as $element) {
          // Create new queue item.
          $queue->createItem($element);
        }

Here comes the most interesting parts.
$queue = $this->queueFactory->get('exqueue_import');
This creates an instance of the default QueueInterface and the name of the queue we want to use or to create.

$totalItemsBefore = $queue->numberOfItems();
This give us the total of items in the queue, currenty for information purposes only.

$queue->createItem($element);
This simply adds an item to the queue. As the $data array is made of objects (see the getDataFromRss() method), we are wrapping the data as a good practice.

c- Retrieve all the items in the queue for information with the getItemList() custom method.

In this method we first claim each item of the queue with the claimItem() method. This method also locks or lease the item for a period of time of one hour by default. So we'll need to release those items later.
We save each item in the  $retrieved_items array that will use to show the information later to the user.

    // Claim each item in queue.    
    while ($item = $queue->claimItem()) {
      $retrieved_items[] = [
        'data' => [$item->data->title, $item->item_id],
      ];
      // Track item to release the lock.
      $items[] = $item;
    }


We also save the item information to the $items array to release each item in the next loop.

    // Release claims on items in queue.
    foreach ($items as $item) {
      $queue->releaseItem($item);
    }

That's it. You can see now how easy it is to create a queue and populating it.

You can now visit the page (/exqueue01/getData) and see how the queue is populated.

To debug the queue, you can use the Drupal Console command drupal debug:queue , but it will work only if you have a QueueWorker plugin associated with this queue.

You can also see the table queue in the database and see if it has been populated with the data of your queue.  

In a next post we'll see how to create a queue with a form and with a hook_cron

2. Create a Cron QueueWorker plugin to process the queue

As we've seen, create a queue is quite simple. Now we have to code a QueueWorker to process each element in the queue.

This QueueWorker is responsible for processing each element of the queue when cron runs. In our case, the process consists in creating a page node with the data of each item.

<?php
namespace Drupal\exqueue01\Plugin\QueueWorker;
use Drupal\Core\Entity\EntityTypeManagerInterface;
use Drupal\Core\Logger\LoggerChannelFactoryInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;
use Drupal\Core\Plugin\ContainerFactoryPluginInterface;
use Drupal\Core\Queue\QueueWorkerBase;
/**
 * Save queue item in a node.
 *
 * To process the queue items whenever Cron is run,
 * we need a QueueWorker plugin with an annotation witch defines
 * to witch queue it applied.
 *
 * @QueueWorker(
 *   id = "exqueue_import",
 *   title = @Translation("Import Content From RSS"),
 *   cron = {"time" = 5}
 * )
 */
class ExQueue01 extends QueueWorkerBase implements ContainerFactoryPluginInterface {
  /**
   * Drupal\Core\Entity\EntityTypeManagerInterface definition.
   *
   * @var \Drupal\Core\Entity\EntityTypeManagerInterface
   */
  private $entityTypeManager;
  /**
   * Drupal\Core\Logger\LoggerChannelFactoryInterface definition.
   *
   * @var \Drupal\Core\Logger\LoggerChannelFactoryInterface
   */
  private $loggerChannelFactory;
  /**
   * {@inheritdoc}
   */
  public function __construct(array $configuration,
                              $plugin_id,
                              $plugin_definition,
                              EntityTypeManagerInterface $entityTypeManager,
                              LoggerChannelFactoryInterface $loggerChannelFactory) {
    parent::__construct($configuration, $plugin_id, $plugin_definition);
    $this->entityTypeManager = $entityTypeManager;
    $this->loggerChannelFactory = $loggerChannelFactory;
  }
  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition) {
    return new static(
      $configuration,
      $plugin_id,
      $plugin_definition,
      $container->get('entity_type.manager'),
      $container->get('logger.factory')
    );
  }
  /**
   * {@inheritdoc}
   */
  public function processItem($item) {
    // Save the queue item in a node
    // Check the values of $item.
    $title = isset($item->title) && $item->title ? $item->title : NULL;
    $body = isset($item->body) && $item->body ? $item->body : NULL;
    try {
      // Check if we have a title and a body.
      if (!$title || !$body) {
        throw new \Exception('Missing Title or Body');
      }
      $storage = $this->entityTypeManager->getStorage('node');
      $node = $storage->create(
        [
          'type' => 'page',
          'title' => $item->title,
          'body' => [
            'value' => $item->body,
            'format' => 'basic_html',
          ],
        ]
      );
      $node->save();
      // Log in the watchdog for debugging purpose.
      $this->loggerChannelFactory->get('debug')
        ->debug('Create node @id from queue %item',
          [
            '@id' => $node->id(),
            '%item' => print_r($item, TRUE),
          ]);
    }
    catch (\Exception $e) {
      $this->loggerChannelFactory->get('Warning')
        ->warning('Exception trow for queue @error',
          ['@error' => $e->getMessage()]);
    }
  }
}

First let's have a look at the annotation of this plugin

 * @QueueWorker(
 *   id = "exqueue_import",
 *   title = @Translation("Import Content From RSS"),
 *   cron = {"time" = 5}
 * )

id = "exqueue_import"
Indicates the ID of the queue we will process
cron = {"time" = 5}
Indicates that the plugin should be used by the cron system. The time key indicates that we allocate 5 seconds to process all the items in this queue. If we couldn't process some items in the allocated time, they will be processed at the next cron run.

But all the action happens in the processItem() method from the QueueWorkerInterface. It's in this method that we process each item. It's quite a straightforward process. Nothing new here.

Note about Exception

Note that on cron run, the processQueues() method of the cron object in cron.php will be fired. You can find the code for this method below.

/**
 * Processes cron queues.
 */
protected function processQueues() {
  // Grab the defined cron queues.
  foreach ($this->queueManager->getDefinitions() as $queue_name => $info) {
    if (isset($info['cron'])) {
      // Make sure every queue exists. There is no harm in trying to recreate
      // an existing queue.
      $this->queueFactory->get($queue_name)->createQueue();
      $queue_worker = $this->queueManager->createInstance($queue_name);
      $end = time() + (isset($info['cron']['time']) ? $info['cron']['time'] : 15);
      $queue = $this->queueFactory->get($queue_name);
      $lease_time = isset($info['cron']['time']) ?: NULL;
      while (time() < $end && ($item = $queue->claimItem($lease_time))) {
        try {
          $queue_worker->processItem($item->data);
          $queue->deleteItem($item);
        }
        catch (RequeueException $e) {
          // The worker requested the task be immediately requeued.
          $queue->releaseItem($item);
        }
        catch (SuspendQueueException $e) {
          // If the worker indicates there is a problem with the whole queue,
          // release the item and skip to the next queue.
          $queue->releaseItem($item);
          watchdog_exception('cron', $e);
          // Skip to the next queue.
          continue 2;
        }
        catch (\Exception $e) {
          // In case of any other kind of exception, log it and leave the item
          // in the queue to be processed again later.
          watchdog_exception('cron', $e);
        }
      }
    }
  }
}

You can see in the processQueues() method that the processItem() method is in a ‘try catch’ block, so if we throw an Exception in our QueueWorker it will be catched by the processQueues() method, the item won't be deleted and the Exception will be logged. So the item would stay in the queue if we throw an exception in the processItem() method. That's a good design, as we can examine the log and see what's happened and do something with this item later.

You can also see that you could throw a RequeueException or a SuspendQueueException. In the first case the item is requeued and in the second the process is suspended for the whole queue.

But if we want, like in our case, to delete the item from our queue if there is an error, we need to catch the Exception in a ‘try catch’ block in our QueueWorker itself. In this case, we won't return an Exception to the processQueues() method and then our item will be deleted. That's a choice. This means that if we don't have a title or a description, the data is useless for us and we can delete it from the queue. This also shows that we accepted useless data to be queued in an earlier process.  

3. Process the queue

Now we can process the queue with:

cron – In my testing, I was unable to process a queue with drush cron or drupal cron:excecute because they only invoke modules with cron handlers implementing hook_cron. You'll have to launch cron manually in the drupal interface.

Drupal Console – You can use the Drupal Console drupal queue:run <name-of-the-queue> command. In our case it would be drupal queue:run  exqueue_import

Recap

We created a controller ExQueueController.php with a getData() method.
We get the data from the Drupal Planet RSS file with the getDataFromRss() method into an array of objects.
We get or created an instance of the default QueueInterface with $this->queueFactory->get('exqueue_import').
We created each item in the queue with the $queue->createItem() method
Once the queue is created, we set up a Cron QueueWorker plugin where we processed each item in the queue with the processItem() method.

Jun 04 2018
Jun 04

Mon, June 4, 2018

Runing composer install or composer update on your Drupal 8 installation to install or update modules and themes could be sometimes frustrating because it can be very slow. Too slow in fact. But it doesn't have to be that way. Here are some tips to speed your Composer working with Drupal.

Install Prestissimo

Prestismo is a global composer plugin that that enables parallel installations and it's very fast! It can be more than 2x faster. But Prestissimo requires cURL, which may not work behind certain firewalls or proxies.

To install prestissimo follow these steps:

composer self-update

composer global require hirak/prestissimo


Once it is installed, Composer should install a new project much faster than before.

You can configure how many connections you would prefer, but I have found that the default of 6 seems to be pretty good.

To uninstall:

composer global remove hirak/prestissimo


Disable Xdebug?

Before Composer 1.3., Xdebug slowed down Composer substantially, sometimes making installs take 2-4x longer.

But with the newer versions of Composer (> 1.2.) this issue has been fixed. So be sure to update your Composer to the latest version with the command composer self-update

Related Content

Jun 03 2018
Jun 03

Today composer is the recommended approach to install (o more precisely to download)  Drupal 8. This is true for the core but also for contributed modules and themes.

So now, to start a new Drupal 8 project, we need to download it via composer and not as we did before with drush or drupal console. Of course, we'll still use drush or drupal console to install contrib modules or themes, but not for downloading them.

The main benefit of using composer is that it allows us to systematically manage a sprawling list of dependencies (and their subsidiary dependencies) and ensure that the right versions for each package are used or updated.

An official project composer template has been created for installing Drupal with composer. We will create our project directly using this template, which is also available on Packagist.

Installing composer

First you need to install composer. Please see Getting Started on getcomposer.org to install Composer itself.

Download Drupal core using Composer

To create a new project based on the official composer template we can run the following Composer command:

composer create-project drupal-composer/drupal-project:8.x-dev my_project_name_dir --stability dev --no-interaction

Don't forget to change 'my_project_name_dir' with the name of the directory where you want to install Drupal.

This will download the drupal-composer/drupal-project project from Packagist into a folder named 'my_project_name_dir'. It also automatically executes composer install which will download Drupal 8 and all its dependencies.

In fact, this create-project composer command is the equivalent of doing a git clone https://github.com/drupal-composer/drupal-project.git  my_project_name_dir followed by a composer install of the vendors.

# create-project is equivalent of a git clone and a composer install
git clone https://github.com/drupal-composer/drupal-project.git my_project_name_dir
composer install

What does the template do?

When installing the given composer.json some tasks are taken care of:

  • Drupal will be installed in the web-directory.
  • Autoloader is implemented to use the generated composer autoloader in vendor/autoload.php, instead of the one provided by Drupal (web/vendor/autoload.php).
  • Modules (packages of type drupal-module) will be placed in web/modules/contrib/
  • Theme (packages of type drupal-theme) will be placed in web/themes/contrib/
  • Profiles (packages of type drupal-profile) will be placed in web/profiles/contrib/
  • Creates default writable versions of settings.php and services.yml.
  • Creates web/sites/default/files-directory.
  • Latest version of drush is installed locally for use at vendor/bin/drush.
  • Latest version of DrupalConsole (yes !!!) is installed locally for use at vendor/bin/drupal.
  • Creates environment variables based on your .env file. See .env.example.

Please note that Drupal will be installed in 'my_project_name_dir/web/'

The project template also comes with a .gitignore file that keeps Drupal core and all the contributed packages outside of Git, similar to the regular vendor/ packages. So based on an updated composer.json file, we can maintain a smaller Git repository and recreate our project any time. A lot of the benefits of Drush Make have now been incorporated into a Composer flow.

What can I do if I don't want the install Drupal in the web/ directory?

Imagine that we need to install Drupal in the docroot/ directory instead of the web/ directory.

In this case we'll do the following:

# 1. Download the project template with git
git clone https://github.com/drupal-composer/drupal-project.git my_project_name_dir

# 2. Change the 'installer-paths' in the 'extra' section of the new downloaded composer.json
cd my_project_name_dir
vi composer.json

# 3. Run composer install 
composer install

Install your project

Now that we have downloaded the Drupal project and all its dependencies, we can install our Drupal project by running the DrupalConsole command drupal site:install or go to your local website (localhost://) to install the site.

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web