Solr Search with Drupal 8

Search is an important facet of any large website these days. We’d talked previously about why you want to take full control of your site search. Bombarding your users with a mess of links won’t do anyone any favors. One of our favorite solutions for this problem is Apache Solr and recently we had the opportunity to set it up on Drupal 8. Let’s take a moment to go through a bit of what that solution looked like and some thoughts along the way.

Setting the stage

Before we dive too far into the how, we really ought to give a bit of time to the why. More specifically we need to cover why we didn’t simply use one of the existing modules contributed to Drupal by the community. At the time, there was one prominent module group for implementing Solr search in Drupal 8. The Search API module in tandem with the Search API Solr Search module were really the most direct way to implement this advanced search on your site. These are great modules and for a different situation would have worked just fine. Unfortunately, the requirements we were working with for the project were more specific than these modules were equipped to handle.

There were three key things that we needed control over and we aren’t keen on hacking a module to get something like this done. We needed to have specific control over what was indexed into Solr. The Search API module allows for you to generically specify how fields are translated to the Solr index, but if you need some different handling you would either need multiple indexes or you would need to sacrifice some of that customization. The site also needed to make use of a fairly complicated feature of Solr, the more like this query. (Warning, incoming search jargon!) This query allows you to search the index for content relevant to another indexed piece of content. This relevancy is determined by fields you specify in the query and results can be limited to content that meets a certain relevancy score threshold.

The last thing we had to have in this was the ability to manage how often content was indexed. The existing modules allowed for this action to happen on a periodic cron, but wasn’t able to have the index updated as soon as changes were made to content. This project was going to have a lot of content updated each day and that meant we couldn’t afford to wait for things to be indexed and updated. With these three things creating hurdles to getting Solr implemented in this project it seemed like we were going to have to go another way, but after looking at some documentation we determined that creating our own implementation would not be so difficult.

Solr search with Solarium

Before we get too far ahead of ourselves, we should note that this wasn’t done with a contributable module in mind. That isn’t because we don’t like giving back the the community, we totally do, it was because it was created for a very specific client need. There will likely be a more generic version of this coming out down the road if demand is high enough. Also, we are under the impression that most use cases are covered by the modules mentioned above, so that would be where most would start. Enough with the disclaimers; let’s talk Solarium.

We went with Solarium as the Solr client to use for this. That is what most of the existing Drupal modules use and it seemed to be the most direct way to do this with PHP. Installing Solarium is pretty simple with Composer and Drupal 8. (If you aren’t using Composer yet, you really should be.) Using a client for communicating with a Solr instance isn’t specifically required. Ultimately, the requests are just simple HTTP calls, but the client saves you from having to memorize all of the admittedly confusing query language that comes with using Solr.

Installing Solarium can be done as simply as composer install "solarium/solarium". You could also do this by adding a line to your composer.json file in the require section for "solarium/solarium": “3.6.0”. Your approach on this part may vary, but this should be done from the root of your Drupal site so that this library goes into the global dependencies for the project. These instructions are also detailed a bit more in the official docs for Solarium, here. The official docs also have a bunch of example code that will help if you dive into this like we did.

For this implementation, we opted to create a Solr PHP class to do the heavy lifting and made use of a Drupal service for calls to it from the rest of the app.

namespace Drupal\my_module\Solr;
use Solarium\Core\Client\Client;


class SolrExample {

  /**
   * Connection to the solr server
   *
   * @var Client
   */

  protected $solr;
}

The heart of the class is going to be the connection to Solr which is done through the Solarium client. We will make use of this client in our constructor by setting it up with the credentials and default settings for connection to our Solr instance. In our case, we used a config form to get the connection details and are passing those to the client. We wanted to use the configuration management system so that we could keep those settings consistent between environments. This allowed more accurate testing and fewer settings for developers to keep track of.

/**
* Solr constructor.
*/
public function __construct() {
 $config = \Drupal::config(‘example.solr_config’); //Normally we’d inject this, but for this example we’ll ignore that
 $settings = [
   'endpoint' => [
     'default' => [
       'host' => $config->get('host'),
       'port' => $config->get('port'),
       'path' => $config->get('path'),
       'scheme' => $config->get('protocol'),
       'http_method' => 'AUTO',
       'site_hash' => TRUE,
      ]
    ]
  ];
 $this->solr = new Client($settings);
}

We are doing this in the constructor so that we don’t have to create a new client connection multiple times during a given call. In our case, we ended up using this as a Drupal service which allows us to only have the Client object created once per call and gives a simple way to use this class throughout the app.

The next part is the actual search method. This does a lot and may not be clear from the code below. In this method, we take parameters passed in and build a Solr query. We have a helper function in this that does some specific formatting of the search terms to put it in the right query syntax. For most sites, this code would serve fine for doing generic searching of the whole index or having multiple versions for searching with specific filters

/**
* General Search functionality
*
* @param array $params
*
* @return mixed
*/
public function search($params = []) {

 $query = $this->solr->createSelect();

 $default_params = [
   'start' => 0,
   'rows' => 20,
   'sort' => 'score',
   'sort_direction' => 'DESC',
   'search' => '*:*',
   'time' => '*'
 ];

 $params = array_merge($default_params, $params);

// Building a proper solr search query with the search params
 $search_string = $this->getTextSearchString($params['search'], $params['time']);

 $query->setQuery($params['search']);
 $query->setStart($params['start'])->setRows($params['rows']);
 $query->addSort($params['sort'], $params['sort_direction'] == 'ASC' ? $query::SORT_ASC : $query::SORT_DESC);

 try {
   $results = $this->solr->select($query);
   return ['status' => 1, 'docs' => $results->getData()['response']['docs']];
 }
 catch (HttpException $e) {
   \Drupal::logger('custom_solr')->warning('Error connecting to solr while searching content. Message: @message',['@message' => $e->getMessage()]);
   return ['status' => 0, 'docs' => [], 'message' => 'Unable to reach search at this time. Try again later.'];
 }
}

The code we’ve presented so far isn’t breaking new ground and for the most part does a similar job to the existing search modules available from the Drupal community. What really made us do something custom was the more like this feature of Solr. At the time that we were implementing this, we found that piece to be not quite working in one module and impossible to figure out in another, so we put our own together. 

Thankfully with Solarium, this was a pretty simple query to tackle and we were able to have related content on the site without much other setup. We can create a new more like this query and submit an id so Solr knows which content to compare against for similarity. The rest of it behaves very similar to the search method presented previously. The results are still returned the same and we are able to do some other filtering to change the minimum relevancy score or number of rows.

$query = $this->solr->createMoreLikeThis();
$helper = $query->getHelper();
$query->setQuery('id:' . $id);
$query->setRows($params['rows']);
$query->setMltFields($params['mltfields']);
$query->setMinimumDocumentFrequency(1);
$query->setMinimumTermFrequency(1);
$query->createFilterQuery('status')->setQuery($params['queryfields']);

We didn’t share all of the code used for this here, obviously. The point of this post isn’t to help others create an exact duplicate of this custom implementation of Solarium in Drupal 8. At the time of this writing, it seems that the existing Solr modules might be in great shape for most use cases. We wanted to point out that if you have to dip into code for something like this, it can certainly be done and without an insane amount of custom code.

MIKE OUT