Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Oct 08 2020
Oct 08

If you've followed Pantheon's Build Tool Instructions and used Pantheon's Example Drops 8 Composer as the base to take advantage of CircleCI then you've encountered Behat tests. Behat tests are extremely valuable for testing site functionality before new code goes to production or a shared code stream. However, on a custom site these default tests provided by Pantheon are likely to fail if you've made even a benign change like deleting Tags vocabulary on a Drupal standard installation.

The first few times I encountered these tests and inevitable failures, I would summarily delete or comment out the Behat test and proceed with development as normal. "We'll come back to those later", I always said. One of the barriers to taking full advantage of the Behat scaffolding that Pantheon provided was that there wasn't obvious guidance on how to run the tests locally before they caused failures in CircleCI builds. 

However, with just a few file changes to your Lando-based project, you can have these tests running locally. Good test coverage can give developers confidence when pushing code and reduces the chance of breaking basic existing functionality. An investment in tests early in a project can save time and headaches later on and build confidence in the development team.

(These instructions assume Drops 8 2.3.1 composer-based installation.)

  1. Add a drush alias file with url and db-url settings to drush/lando.aliases.drushrc.php.1

    <?php
    
    $aliases['epihc'] = array(
      'uri' => 'https://epihc.lndo.site/',
      'db-url' => 'mysql://pantheon:[email protected]:3306/pantheon',
    );
    
  2. Add lando-specific behat config to tests/behat/behat-lando.yml.

    default:
      suites:
        default:
          contexts:
            - FeatureContext
            - Drupal\DrupalExtension\Context\DrupalContext
            - Drupal\DrupalExtension\Context\MinkContext
            - Drupal\DrupalExtension\Context\MessageContext
            - Drupal\DrupalExtension\Context\DrushContext
            - FailAid\Context\FailureContext
      extensions:
        Drupal\MinkExtension:
          goutte: ~
          base_url: https://epihc.lndo.site/  # Replace with your site's URL
        Drupal\DrupalExtension:
          blackbox: ~
          api_driver: 'drush'
          drush:
            alias: 'lando.epihc'
    
  3. Add Behat tooling to the Lando config and restart/rebuild lando to .lando.yml.

    tooling:
      behat:
        description: Run behat tests.
        cmd:
          - appserver: /app/vendor/bin/behat --config=/app/tests/behat/behat-lando.yml
    
  4. Run lando behat in terminal to run the tests.

The final task will be to update the Behat tests for your particular site's needs. Your main test file is at tests/behat/features/content.feature. Luckily Behat uses natural language and is pretty easy to write

Other things to know:

  • Check out your dev dependencies for the packages that are working together to make the tests Drupal friendly. The Behat Drupal Extension page has some good references.
  • We use Pantheon, Drops-8, and Lando as a starting point for many of our projects but other approaches are equally valid. 
  • If you need to debug the actual build process on CircleCI, go to .ci/test/behat/run
  • Behat tests aren't appropriate for every use case. Visual regression tests, linters, code sniffers, and/or unit test have their place as well. 

Hopefully these instructions will help you get started on your test-driven way!

1 Replace pantheon with drupal8 when using the drupal8 Lando recipe. This database setting is required for Behat to run drush commands. 

Oct 01 2020
Oct 01

Recently, I came across an issue where private files in media entities, that were embedded in a paragraph, were accessible by anonymous users. While a user could not get access to the page, access was allowed via direct URL to the file.

In Drupal 8/9, when a file is attached to an entity, it receives some assumptions regarding its general access. That access generally follows the permission access rules of the entity that it's attached to. While this has its benefits, and certainly made sense back in the days when files were directly attached to nodes, it can create a situation where a private file can be directly accessed because its parent entity type is publicly accessible. 

In my case, a paragraph type that allowed for embedding different media entities was set to be viewable by all. Even though the parent node of the paragraph was set to a private access control, the extra embed levels of paragraphs and media items created an access override.

After trying out many different setups and modules that added more permission levels and controls, I finally came across a simple hook based fix, for making sure that any file that comes from the private file directory gets it's access checked before downloading. That hook is hook_file_download. I prefer simple code fixes and setups, rather than adding more modules and config to try and manage.

The hook

hook_file_download

This hook allows modules to enforce permissions on file downloads whenever Drupal is handling file download, as opposed to the web server bypassing Drupal and returning the file from a public directory. 

Basically, what this means is that as long as a file is not being publicly served up through the public file directory, we can use this as an extra check point to make sure that private items stay private. More on this hook here.

The code

I'm including 2 ways to setup some basic control. Hopefully one will help out or guide you to the solution you need.

The first way (# 1) just assumes that all anonymous users should not be able to view any files from the private stream wrapper. The second way (# 2) checks if the current user has a specific permission set. For our case, the custom permission is called `access private files`. If the current user does not have this custom permission, the file is blocked.
 


use Drupal\Core\StreamWrapper\StreamWrapperManager;

/**
 * Implements hook_file_download().
 */
function MODULE_file_download($uri) {
        # Check if the file is coming from the private stream wrapper
        if (StreamWrapperManager::getScheme($uri) == 'private') {
                
                # 1: Block anonymous users.
                if (Drupal::currentUser()->isAnonymous()) {
                return -1;
        }
        
        # 2: The user does not have the permission "access private files".
        if (!\Drupal::currentUser()->hasPermission('access private files')) {
                return -1;
        }
        }
                
        return NULL;
}

For the custom permission, whichever module you place your hook in, you need to add a permission YML file (MYMODULE.permissions.yml) in the module directory, with the following snippet:


access private files:
  title: 'Access private files'
  description: 'View privately stored files from their direct URL path'

Once the permission is in place, you can assign what roles can access private files as an ultimate access control.

The happy ending

The use case for this setup may be unique for most Drupal sites, but I know I have come across this particular situation on more than one occasion. Node types that leverage the power of paragraphs and media entities, for creating unique page experiences, can easily find themselves in this "Private file access overridden" situation. So make sure to always test private file access by visiting the direct URL access to those files. You never want to be caught exposing your private files on the internet!

For more information about this issue, please see https://www.drupal.org/project/drupal/issues/2984093 
and https://www.drupal.org/node/2904842.

Sep 16 2020
Sep 16

Gatsby Image, along with gatsby-transformer-sharp and gatsby-plugin-sharp, is a common way to handle images in a Gatsby project.  It gives us the power to process source images at build time to create a 'srcSet' for  '<img>' or '<picture>' tags, or create versions in grayscale, duotone, cropped, rotated, etc.

When Gatsby builds from a Drupal source, it downloads the source images and processes them to create these image variations which are stored in the public directory. The ability to optimize and art-direct images at build time is great, but build performance suffers while these assets are generated. As a site grows in images, the time it takes to build grows as well. Image processing can take hours, while the rest of the build takes mere minutes.

Drupal has the built in ability to generate its own derivative images on demand and cache them, so why not just build our Gatsby components in a way that leverages Drupal Image Styles and dramatically speed up the Gatsby build process? Gatsby can just query for the image urls and let Drupal serve them directly.

For this experiment, I didn’t use Gatsby Image at all, though you could format your image data in a way that the Gatsby Image component expects, and still pass that in. My goal was to explore my options for getting image variants from Drupal via JSON:API and GraphQL, and to pass that to a custom React component, so that’s what I’ll demonstrate in this post.

I’ll show examples using both the core JSON:API module, as well as the GraphQL contrib module, which gives us a different developer experience.

Drupal Modules Used

Core: 
 - jsonapi
 - rest

Contrib:
 - graphql - An alternative to jsonapi with a developer experience that, so far, I prefer.
 - jsonapi_image_styles - required to access image styles with jsonapi, not required with graphql module.

Gatsby Configuration


{ 
  resolve: "gatsby-source-graphql", 
  options: { 
    typeName: "Drupal", 
    fieldName: "drupal", 
    url: `http://backend.lndo.site/graphql/` 
  }, 
}, 
{ 
  resolve: `gatsby-source-drupal`, 
  options: { 
    baseUrl: `http://backend.lndo.site/`, 
    apiBase: `jsonapi`, // optional, defaults to `jsonapi` 
    skipFileDownloads: true, 
  }, 
},

Here I’m configuring both the graphql and the drupal (jsonapi) sources. As this is only a local experiment, I’m not concerned with authentication and I’m using Lando/Docker for local development. In a real project, I’d extract the relevant parts to environment variables so they can be changed for the production server. I’d also only use one or the other, and not both.

In the graphql config, you’ll notice 'fieldName:' is set to 'drupal', which is what we’ll use to access the GraphQL API, as opposed to the JSON API.

In the 'gatsby-source-drupal' config I’ve set 'skipFileDownloads:' to 'true' because we will not need them for local processing or delivery. This skips all file downloads, but there are ways to be more specific using filters. https://www.gatsbyjs.com/plugins/gatsby-source-drupal/#filters

Option 1: GraphQL

Query Against the GraphQL API


query($id: String!) {
  drupal {
    nodeById(id: $id) {
      ... on Drupal_NodeArticle {
        title: entityLabel
        author: entityOwner {
          authorName: entityLabel
        }
        body {
          processed
        }
        fieldImage {
          alt
          xxl: derivative (style: XXL) {
            width
            url
          }
          xl: derivative (style: XL) {
            width
            url
          }
          large: derivative (style: LARGE) {
            width
            url
          }
          medium: derivative (style: MEDIUM) {
            width
            url
          }
          small: derivative (style: SMALL) {
            width
            url
          }
        }
      }
    }
  }
}


This query is for a specific article node in Drupal that contains a field named 'field_image'.  With the GraphQL module in Drupal, we have access to the 'derivative' function that allows us to query for the values of a specific image style.

In this case we’ve set up 5 image styles in Drupal which scale an image to different widths:

  • Small: 300
  • Medium: 600
  • Large: 1200
  • XL: 1800
  • XXL: 2400

The sizes are meant to cover a “full width” image at various breakpoints or screen resolutions.  Of course, you can set up Image Styles more suited to your needs.

We can alias each derivative by prefixing it with the style name and a colon, allowing us to have multiple derivatives in the same query, setting the style argument to the appropriate style name.

The result of the query includes all the values we need for our Image component.


        "fieldImage": {
          "alt": "Arduino",
          "xxl": {
            "width": 2400,
            "url": "http://backend.lndo.site/sites/default/files/styles/xxl/public/2020-09/arduino-harrison-broadbent_0.jpg?itok=EyvwD2ta"
          },
          "xl": {
            "width": 1800,
            "url": "http://backend.lndo.site/sites/default/files/styles/xl/public/2020-09/arduino-harrison-broadbent_0.jpg?itok=0gub18uQ"
          },
          "large": {
            "width": 1200,
            "url": "http://backend.lndo.site/sites/default/files/styles/large/public/2020-09/arduino-harrison-broadbent_0.jpg?itok=eNUhJHb6"
          },
          "medium": {
            "width": 600,
            "url": "http://backend.lndo.site/sites/default/files/styles/medium/public/2020-09/arduino-harrison-broadbent_0.jpg?itok=Rvsmw0an"
          },
          "small": {
            "width": 300,
            "url": "http://backend.lndo.site/sites/default/files/styles/small/public/2020-09/arduino-harrison-broadbent_0.jpg?itok=sNDME4Ju"
          }
        }

We can see that the url for each style is specific to the path Drupal will use to serve that version of the image. We’re letting Drupal do the work.

Next: An Article Component

Here we put it all together to render a node.


import React from 'react';
import { Link, graphql } from 'gatsby';
import Layout from '../components/layout';
import Image from '../components/image';

export const query = graphql`
query($id: String!) {
  drupal {
    nodeById(id: $id) {
      ... on Drupal_NodeArticle {
        title: entityLabel
        author: entityOwner {
          authorName: entityLabel
        }
        body {
          processed
        }
        fieldImage {
          alt
          xxl: derivative (style: XXL) {
            width
            url
          }
          xl: derivative (style: XL) {
            width
            url
          }
          large: derivative (style: LARGE) {
            width
            url
          }
          medium: derivative (style: MEDIUM) {
            width
            url
          }
          small: derivative (style: SMALL) {
            width
            url
          }
        }
      }
    }
  }
}`

const ArticleTemplate = ({ data: { drupal: { nodeById: article } }}) => {

  return (
    <Layout>
      <div className="article">
        <h1>{article.title}</h1>
        <p className="author">Posted by {article.author.authorName}</p>
        <Image
          image={article.fieldImage}
          sizes="90vw"
        />
        <div dangerouslySetInnerHTML={{ __html: article.body.processed}}></div>
        <Link to='/'>&larr; Back  to all articles</Link>
      </div>
    </Layout>
  );
};

export default ArticleTemplate;

This 'fieldImage' object is passed to the Image component as the first prop named 'image'. Our second prop is named 'sizes' and this is set manually where we call our Image component. It gives us some control over which version of the image is rendered given what we know about how wide it will be in the layout at certain breakpoints.

Next: An Image Component

I want to create a responsive image with a 'srcSet' similar to what we would get with gatsby-image.


import React from 'react';

const Image = ({image, sizes}) => {

  const derivatives = [];

  for (const derivative in image) {
    if (image[derivative].url) {
      derivatives.push(`${image[derivative].url} ${image[derivative].width}w`);
    }
  }

  const srcSet = derivatives.join();

  return (
    <img
      srcSet={srcSet}
      sizes={sizes}
      src={image.small.url}
      alt={image.alt}
    />
  );
};

export default Image;

The '<img>' tag will need 'srcSet', 'sizes', 'src' and 'alt' values. The query to the GraphQL API in Drupal collects all this information, returning it in the 'fieldImage' object that we passed to the 'image' prop. The 'for' loop iterates over the derivatives and concatenates each image URL with its width, pushing each to an array to create a set. The array is later joined to create the full value of the 'srcSet' image attribute. 
The 'src' attribute can have a value of any of the derivative urls. In this case 'image.small.url'.

Finally, the 'alt' attribute comes as part of the query, at 'image.alt'.

Option 2: JSON:API Version 

If you’re more comfortable with, or more invested in using the JSON:API rather than the GraphQL module in Drupal, you can achieve the same results with the addition of the 'jsonapi_image_styles' module, which exposes the image styles to the JSON:API.

In this case the query would look more like this:


  query($id: String!) {
    nodePage(id: { eq: $id }) {
      title
      field_image {
        alt
      }
      relationships {
        uid {
          display_name
        }
        field_image {
          image_style_uri {
            small
            medium
            large
            xl
            xxl
          }
        }
      }
      body {
        processed
      }
    }
  }

However, the results are a bit unexpected.


        "field_image": {
          "image_style_uri": [
            {
              "small": null,
              "medium": null,
              "large": "http://backend.lndo.site/sites/default/files/styles/large/public/2020-09/engin-akyurt-girl-blond-hair-unsplash.jpg?itok=MrcdTwGV",
              "xl": null,
              "xxl": null
            },
            {
              "small": null,
              "medium": "http://backend.lndo.site/sites/default/files/styles/medium/public/2020-09/engin-akyurt-girl-blond-hair-unsplash.jpg?itok=vQx__izk",
              "large": null,
              "xl": null,
              "xxl": null
            },
            {
              "small": "http://backend.lndo.site/sites/default/files/styles/small/public/2020-09/engin-akyurt-girl-blond-hair-unsplash.jpg?itok=hKO4dR34",
              "medium": null,
              "large": null,
              "xl": null,
              "xxl": null
            },
            {
              "small": null,
              "medium": null,
              "large": null,
              "xl": "http://backend.lndo.site/sites/default/files/styles/xl/public/2020-09/engin-akyurt-girl-blond-hair-unsplash.jpg?itok=MPph1lH4",
              "xxl": null
            },
            {
              "small": null,
              "medium": null,
              "large": null,
              "xl": null,
              "xxl": "http://backend.lndo.site/sites/default/files/styles/xxl/public/2020-09/engin-akyurt-girl-blond-hair-unsplash.jpg?itok=sz-ymDam"
            }
          ]
        }
      },

We get an object for each image style that contains keys for all the image styles queried, with 'null' values, except for the style corresponding to that part of the query. I’m not sure why this is, but we have to process our results differently to build the 'srcSet'.  Let’s illustrate with a Picture component so we don’t have to change the Image component.


import React from 'react';

const Picture = ({image, alt, sizes}) => {

  const derivatives = [];
  let src = "";

  image.forEach(derivative => {
    if (derivative.small) {
      src = derivative.small
      derivatives.push(`${derivative.small} 300w`)
    }
    if (derivative.medium) {
      derivatives.push(`${derivative.medium} 600w`)
    }
    if (derivative.large) {
      derivatives.push(`${derivative.large} 1200w`)
    }
    if (derivative.xl) {
      derivatives.push(`${derivative.xl} 1800w`)
    }
    if (derivative.xxl) {
      derivatives.push(`${derivative.xxl} 2400w`)
    }
  })

  const srcSet = derivatives.join();

  return (
    <picture>
      <source srcSet={srcSet} sizes={sizes}/>
      <img src={src}
        alt={alt}
        />
    </picture>
  );
};

export default Picture;

Notice that we also have an 'alt' prop because the 'alt' data comes from a different part of the query. It can’t be accessed from 'relationships.field_image' like the image styles are. 

Conclusion

Whichever API type you use on the backend, I hope I’ve shown that we can still leverage the power of Drupal to process images as they’re needed instead of having to download and process them during the Gatsby build, bloating the build time and the size of the build artifact. 

Combine this with the 'image_effects' module in Drupal and create image styles to meet many different needs. You can have your fast Gatsby build times and responsive images too!
 

"Code like nobody is watching... but then refactor. ;)"

Sep 16 2020
Sep 16

Gatsby Image, along with gatsby-transformer-sharp and gatsby-plugin-sharp, is a common way to handle images in a Gatsby project.  It gives us the power to process source images at build time to create a 'srcSet' for  '' or '' tags, or create versions in grayscale, duotone, cropped, rotated, etc.

When Gatsby builds from a Drupal source, it downloads the source images and processes them to create these image variations which are stored in the public directory. The ability to optimize and art-direct images at build time is great, but build performance suffers while these assets are generated. As a site grows in images, the time it takes to build grows as well. Image processing can take hours, while the rest of the build takes mere minutes.

Sep 04 2020
Sep 04

Oftentimes projects need a way to serve multiple domains from the same installation or from the same codebase. In Drupal we have several ways to accomplish this, and this post will describe some architecture options that make this possible, as well as the relevant factors that can help you decide which option provides the best ‘fit’ for the implementation.

We’ll be looking at four distinct implementation choices: Classic Drupal Multisite, Domain Access or Mega Site, Distribution Profile, and an offshoot of the distribution profile called Custom Upstream.

Classic Drupal Multisite

In a classic Drupal multisite architecture, the individual websites share code (the Drupal core, modules, and themes), but each website has its own database so they do not share content, configuration, or settings. Each website can also have some extra modules / themes under its own subdirectory in the same codebase.

Advantages
  • Single codebase:
    • Onboarding developers and administrative tasks are easier with a single codebase.
    • Code and library updates only need to be done once.
  • Some degree of customization is possible, because each site has its own configuration, and it can have it’s own custom theme and modules.
  • Shared features can be accomplished by using Feature modules: features modules can be shared across the sites, and then individual sites can import the configuration from it. Another option would be to use the Config Split module to segment what configuration is shared across sites and what should be overridden per site.
  • Less hosting costs: a single hosting plan that supports all of the traffic from all of the sites might be cheaper than hosting per individual site.
Disadvantages
  • If a site decides to customize functionality provided by a Features module, then it would lose any further updates to the module, as they have deviated from it. This is one of the greatest risks of a multisite setup, because if sites start diverging in functionality, it makes no sense to keep maintaining a single codebase.
  • All developers working in the team have access to the codebase that affects all of the sites. This implies a high level of trust in the whole team, and does not allow for separation of concerns, meaning that while team members could be assigned to work only in a particular site, the codebase architecture implies it cannot be enforced on a code level.
  • Heavy maintenance:
    • Configuration and updates still need to be run once per site and saved back into code configuration (in each site’s config folder).
    • Deployments in this architecture need to happen “all at once” for all the sites, which can potentially be chaotic (there is no simple “revert” a commit for a single site, for example).
    • Scripts are a frequent solution that comes up to automate maintenance tasks across the sites, which means more devops time.
  • Single point of failure:
    • Since there is a single server hosting all of the sites, a traffic spike can affect all of the other sites. This is mitigated if most of the content is static (for anonymous users), because it can be cached by a CDN which can deliver the (cached) pages even if the server is down.
    • A code error has the potential to affect all of the sites.
  • As a note, this setup is not supported by Pantheon, because they don’t allow multiple databases per hosting plan.
Conclusion

A shared codebase setup would be most beneficial if all of the sites are using similar modules and settings. “Snowflake sites”, i.e. sites that need lots of special customization to code or config, can rapidly increase the risk of technical debt in an architecture that was meant to maintain similar sites.

Mega Site (Domain Access)

This architecture consists of a single codebase and single database/installation. The Domain Access suite of modules provide functionality that allows serving multiple sites (domains) from this setup, and specifying which pieces of content belong to a particular site, or are shared across some or all of the sites.

Advantages
  • Maintaining a single Drupal installation.
    • Updating a single site is faster and easier to test (vs once per site in a multisite setup).
    • Maintaining configuration and functionality is straightforward (no need for Feature modules).
  • Admin users can effortlessly manage all content from one platform.
  • Content and users can be shared across multiple sites (if needed).
  • Allows sharing the same features and functionality to all affiliate sites.
Disadvantages
  • Single point of failure. A single hosting plan for all of the sites. This is mitigated if most of the content is static (for anonymous users), because it can be cached by a CDN which can deliver the (cached) pages even if the server is down.
  • Some contrib modules, and custom or complex functionality might need custom development to make sure they are compatible with the Domain Access module. We call this “glue code”, and a lot might be needed depending on the complexity of the project.
  • If a site needs to be “taken away” into its own hosting plan, it’s not straightforward. (note: here we're referring to scenarios where a site might be spun off to another owner because it needs to pay for its own hosting for funding or other organizational reasons.)
  • Requires strict governance enforcing a policy of no-exceptions (or extremely few). An exception would be a functionality or config on a site that doesn’t follow the rest. Each “exception” adds complexity to the site, and if left unchecked, can quickly become a maintenance nightmare.
Conclusion

A mega-site setup makes sense if sites will share content and/or users, and if all of the sites will have the same functionality, content types/data structures, and will only differ in limited configurations or theme variations, as in a family of affiliated sites.

Distribution Profile

A Drupal distribution can also be created and maintained. A distribution typically contains a declaration of the required modules and libraries, custom theme and custom modules, and configuration that gets imported during install, maybe even default content. Updates to configuration can also be provided via Features modules as described in the multisite section. Under this use case, each site has its own codebase that includes the distribution profile, and is hosted under separate hosting accounts.

Some well-known distributions are Commerce Kickstart, Lightning, Open Social, etc. While these are large projects that aim to provide a generic starting point for projects, a custom distribution profile can also be created to address the needs of a collection of sites with similar functionality, passing the main burden of development to a centralized team of devs in charge of providing updates and new features.

Advantages
  • Sites are independent: Code errors or traffic spikes on one site don’t affect other sites.
  • Flexibility: They can be customized more easily (as long as they don’t override configuration provided by the base distribution, so they could still receive functionality updates).
Disadvantages
  • Again, sites overriding features provided by the distribution means they won’t be able to update to the latest bug fixes or new functionality provided in the profile.
  • This architecture involves maintaining the distribution codebase, plus each of the sites.
  • Maintaining a distribution is also hard work. Even more work and attention need to be dedicated if the distribution will be published on drupal.org, as aspects of dealing with security updates and bug reports are more relevant.
Conclusion

A distribution profile works well if the sites are all starting out the same, but they are expected to grow in different directions. Maintenance work is expected and independent for each of the sites. A typical scenario is a university distribution profile, where each department can have its own independent site.

Custom Upstream

This involves maintaining what is essentially a distribution profile directly from a code repository, instead of making it available in drupal.org. Advantages and disadvantages are similar to the distribution profile as well.

Conclusion

A custom upstream can be useful for companies that maintain a base installation profile, and expect to do bespoke development work on each site in order to accommodate different user requirements, transactions or other scenarios from site to site.

For further reading:

Aug 31 2020
Aug 31

What We're Doing Here: Webforms and Third Party Integration

In Drupal 8, the Webform module can do a lot out of the box. It can do even more with the multitude of its contributed modules. But every now and then a situation arises where you need to get into the guts of create some custom webform magic to get the job done.

Overview: Submitting Results to a Third Party Site as a Query

We will look at one way to integrate the power of webforms with some third party integration and custom code. We are going to have a webform modify the submitted results into a query string, and redirect the user to a third party url with the query attached. We will use a remote submission handler to tell Drupal that when the form is submitted, it also needs to submit those values to a remote, third party URL. In our case, the third party site is bookdirect.com, the client is a Destination Marketing Organization (DMO) and we need to format the submitted values to work with their jackrabbit api. Ultimately, we are using a Drupal webform, to send the user to Bookdirect/Jackrabbit with their lodging search filled in.

Our remote submission handler will define our base URL, but outside of that, the handler doesn't give us much else to do what we need to do in the UI. And while we wanted to give our client the freedom to add more fields to the webform in the future, we wanted to keep it simple for them so they would only have to add the correct base URL and not have to worry about any code. 

For our form, it wasn't necessary to save any submitted results, however, we are going to use a webform api hook that fires right after a submission would normally be saved. That hook is hook_webform_handler_invoke_alter and hook_webform_handler_invoke_METHOD_NAME_alter.

More specifically, since we want to act after saving and everything else has ran, the hook we want is: 
hook_webform_handler_invoke_post_save_alter.

Even if the results are set to not save, webforms will still trigger this "post save" hook. Versions of these hooks are: 

  • hook_webform_handler_invoke_pre_create_alter
    
  • hook_webform_handler_invoke_post_create_alter
    
  • hook_webform_handler_invoke_post_load_alter
    
  • hook_webform_handler_invoke_pre_delete_alter
    
  • hook_webform_handler_invoke_post_delete_alter
    
  • hook_webform_handler_invoke_pre_save_alter
    
  • hook_webform_handler_invoke_post_save_alter

Looking at the method name in these hooks, you can see when they act on a webform submission. For example, if you need to alter a value before the submission is initially saved, you can use hook_webform_handler_invoke_pre_save_alter.

How We're Doing It: 10 Steps

Even though this example is based on working with Bookdirect, I will try to keep the code somewhat generalized so that you can hopefully adjust it to your needs.

1. Create a simple webform

For our form, we required a date form item with the name of start_date, a date form item with the name of end_date, and a taxonomy select form item with the name lodging_category, where we choose from a list of terms. Use any form items you need and adjust the code as needed. I will show you ways to alter those field values before passing them into our remote url query.

2. Add a remote submission handler to your webform

Once you have your webform, you need to add a remote submission handler. Go to your forms Settings tab, then click on Emails/Handlers. Click on Add Handler. Select Remote post handler from the modal and fill in the Completed URL as the base URL that we will be submitting to. Remember to Save your handler. You can also select or deselect what values will be submitted. For our case we only need the 3 form items. We will add some of our own values in the submission query later.

3. Make a module to hold your code

Create a new module or use an existing custom module. The hook and code will need to go into the MYMODULE.module file of your module. For instructions on how to create a module, please see the Drupal.org manual page on module creation

4. Declare your hook

In your MYMODULE.module file, add our hook snippet

/**
 *  Implements hook_webform_handler_invoke_post_save_alter().
 */
function MYMODULE_webform_handler_invoke_post_save_alter(\Drupal\webform\Plugin\WebformHandlerInterface $handler, array &$args) {

}

Make sure to adjust the MYMODULE name with the actual name of your module

5. Setting up some variables

To make things a little easier to play with, we will create some variables to help define our data and info that we will work with. Inside of your new function add:

$webform = $handler->getWebform();

$webform_submission = $handler->getWebformSubmission();

$webform_id = $webform->id();

$handler_id = $handler->getHandlerId();

6. Narrowing down our code

And since we want to make sure we only run on our desired webform and with our remote submission handler, add the following snippet after the variables:

if ($webform_id == 'FILL_IN_WITH_THE_ID_OF_YOUR_WEBFORM' && $handler_id == 'FILL_IN_WITH_THE_ID_OF_YOUR_REMOTE_HANDLER') {
    // Our magic goes here  
}

(make sure to update the IDs above with the actual ID's of your webforms.

7. Get your handler configuration

To get the configuration setup of your handler, use:

$configuration = $handler->getConfiguration();

To pull the settings out of that configuration, step into the settings array and look for the completed_url value.

$bookdirect_url = $configuration['settings']['completed_url'];

8. Alter your data

We need to alter the start_date data from what webform submits (MM-DD-YYYY), to what bookdirect requires (MM/DD/YYY). We will use the getElementData() on the webform submission data to pull out the initial submitted value. We will make our change to that submitted data, and then apply the altered value back into the submission array using the setElements() method. The same alteration needs to be done to the end_date field. 

$start_date = $webform_submission->getElementData('start_date');
$start_date = explode('-', $start_date);
$start_date = [$start_date[1], $start_date[2], $start_date[0]];
$start_date = implode('/', $start_date);
$webform_submission->setElementData('start_date', $start_date);

Another alteration we need to make for our example bookdirect form is to take the taxonomy term value and switch it out with a field value on that term. Our taxonomy terms have term id's that the webform uses in the select widget, but we need to submit a bookdirect ID that corresponds to those taxonomy terms. You can use this concept to switch out field values with different field values from different entities on your webform submissions.

// First check if a value was submitted.
if ($lodging_term_id = $webform_submission->getElementData('lodgingID')) {
    
    // Load up our term to test if it has a book direct value
    $lodging_term = \Drupal\taxonomy\Entity\Term::load($lodging_term_id);
    if ($lodging_term->hasField('field_bookdirect_id') && $bookdirect_id = $lodging_term->get('field_bookdirect_id')->first()->getValue()) {

        // If it has a bookdirect field and a value, switch out our tid value with our bookdirect ID value.
        $webform_submission->setElementData('lodgingID', $bookdirect_id['value']);
    };
};

9. Build your query

Get our submitted data.

$query = $webform_submission->getData();

Alter values into another key. Maybe the fields were created before you knew what the required name should have been for the final destination. In this example, start_date is now supposed to be checkin, but we can't easily change it on our webform.

$query['checkin'] = $query['start_date'];
unset($query['start_date']);
$query['checkout'] = $query['end_date'];
unset($query['end_date']);

Add some static values to our query that were not part of our submitted webform values. Since this is a bookdirect example, these are values that bookdirect requires, but we don't need or want to add to our form directly.

$query['widget_id'] = 'WIDGET_ID_HERE';
$query['campaign'] = 'CAMPAIGN_ID_HERE';

10. Build the URL and redirect the user

Build your URL with our built up query and send our user to the URL with a 301 redirect.

$url = \Drupal\Core\Url::fromUri($bookdirect_url);
$url->setOptions(array('query' => $query));
$destination = $url->toString();

$response = new RedirectResponse($destination, 301);
$response->send();

 

Final Code and Related Case Study

You can find the final code here on github. And you can read more about the client website here: Visit Rapid City Case Study.

Aug 26 2020
Aug 26

The creation and delivery of ethical medical systems is an urgent concern for healthcare professionals and systems around the globe. A worldwide pandemic has only heightened the stakes for those involved in ensuring that delivery of patient care is driven by ethical and transparent decision-making across the international, private healthcare sector. To help drive the adoption of ethical systems, the International Finance Corporation (IFC) partnered with the World Bank to create the EPiHC (Ethical Principles in Health Care) initiative, which seeks to work with partners around the globe to sign onto a set of 10 principles to guide the behavior of healthcare providers, payors, and investors.

Chapter Three worked with the IFC’s programmatic and communications teams to provide strategy, design and development for this global initiative. We’re especially proud of these key accomplishments for this project:

  • Efficient project work: from kickoff to completion of development in under 2 months. The client had critical, short deadlines that needed to be met in order to announce a more public phase of this effort to recruit new organizations to sign on to EPiHC.
  • A completely remote collaboration: with the client. Given that this was one of the first new projects established during the current pandemic, we turned what might have been in-person strategy sessions into a series of smaller online workshops.
  • A detailed Signatory form and backend workflow: that allows the IFC team to vet organizations interested in joining the EPiHC initiative.
  • Built on Drupal 8: providing the IFC team a platform to build a robust Signatory community via user profiles (with engagement that will be expanded over time).
  • Content strategy: we suggested refactoring content previously locked away in a PDF and turning that into “chapters” incorporated on the left side of the Principles page
EPiHC Principles Page

The Principles Page at the new website provides an easy-to-use "chapters" style interface to quickly access each of the 10 principles that make up the EPiHC initiative.

The EPiHC initiative was previously housed deep within the IFC website. This setup was suitable as an incubator while the content and program was being built up, but over time became increasingly impractical from both a visibility and architectural perspective. As shown below, the navigation structure of the previous mini-site looked like it all related to the EPiHC project but in fact those links ("Thought Leadership", "Case Studies" and "Publications") referred to IFCs other topical content, not specifically content for EPiHC. This was confusing and necessitated moving the pages to their own, uniquely branded site.

EPiHC Old Minisite

The old EPiHC minisite inherited navigation (including "Thought Leadership", "Case Studies" and "Publications") from the main IFC website and not directly related to EPiHC. Though temporary, this caused confusion and necessitated a move to a standalone site.

To make EPiHC visible, sustainable and marketable to organizations interested in signing on, we created a clean, elegant design that provides easy to understand pathways to their essential content: the “EPiHC Principles” and the “Become a Signatory” form.

EPiHC New Website by Chapter Three

The new EPiHC website designed and built by Chapter Three shows clean design, prominent highlight of the Signatory Form and elegant branding between imagery and the project's logo. 

The EPiHC team has already secured dozens of Signatories to this effort and expects to recruit hundreds more. The recruitment effort will rely heavily on the clear content architecture and clean design of the new EPiHC website, powered by Drupal 8 and built quickly as a result of close collaboration between the IFC and Chapter Three.

For more information on this project, or to discuss yours, please contact us here

Jul 11 2020
Jul 11

External Media module for Drupal 8 and 9 is a successor of the File Chooser Field module that was bulit for Drupal 7. The same functionality was implemented for Wordpress too. The module provides custom form element.

The new form element would allow you to add external services such as Google Drive, Dropbox, Box, OneDrive, AWS, Unsplash, Instagram, Pixabay, Pexels and other services, in addition to regular file choose field. 

External Media form element.

Implementation is very simple. Just like you would do it with managed_file form element you would change it to external_media and you are all set. To make sure the module is supported in your custom modules I would suggest to add the following:

'#type' => (\Drupal::moduleHandler()->moduleExists('external_media'))
  ? 'external_media' : 'managed_file',

If you would like to force your module to use External Media add dependency to the module's .info file:

dependencies:
  - drupal:external_media

The module supports both File and Image field types and allows control of visibility of each plugin individually in addition to permissions. File extension and cardinality field settings are respected in popup widgets when supported.

External Media form display

Download External Media

Jun 15 2020
Jun 15

This post will document how to get Gatsby to talk to Drupal when Drupal is running in Lando.

For whatever reason, trying to get Gatsby to access Drupal in a Lando container doesn't work. I'm not sure if it's localhost or localhost:PORT that Gatsby has trouble with, but we can work around that limitation by using ngrok. ngrok lets us access localhost URLs from a public URL. I installed it on my Mac using Homebrew (cask), but you can visit the downloads page for other methods of installation. You'll also have to register on the ngrok website, but their free account will work just fine.

Once you have ngrok installed, set up Lando as usual. In my particular instance, I used the pantheon recipe, but any recipe should work.

Start up your Lando app (lando start). Once it has successfully started, Lando will provide you with a handful of URLs you can use to access Drupal within Lando. I've found the best results when using the non https localhost:PORT URL. As a side note, this port changes every time your Lando app is restarted.

Open up a terminal, and type in

ngrok http http://localhost:PORT

where PORT equals your Lando app's port as noted above.

Once ngrok has started, you'll get a few more URLs, but the one you want is labeled "Forwarding" and starts with http://.

If using gatsby-source-drupal, put this URL in the baseUrl option in gatsby-config.js (or use a .env.development file):

...
plugins: [
  {
    resolve: `gatsby-source-drupal`,
    options: {
      baseUrl: `http://xxxxxxxxxxxx.ngrok.io`,
      apiBase: `jsonapi`,
    },
  },
]
...

You can now start Gatsby (npm run develop).

One little tip: in your Gatsby package.json, add this to your scripts section:

"scripts": {
  "refresh": "curl -X POST http://localhost:8000/__refresh",
}

You will need to create a .env.development file at the root of your Gatsby project if you don't already have one, and add the following:

ENABLE_GATSBY_REFRESH_ENDPOINT=true

This will allow you to grab new content from Drupal without having to stop and restart Gatsby:

npm run refresh
Mar 13 2020
Mar 13

Drupal 9 release date has been pinned for June 3, 2020, and it's coming up super fast. What does that mean for your site?

First of all, don't panic. Drupal 7 and 8 end of life are scheduled until November 2021, so there is plenty of time to upgrade. However it is always good to plan ahead with time and take advantage of the new features and security releases with the new version.

If you are on D7

Moving to Drupal 9 will be very similar as moving to Drupal 8, and in fact, there is no reason to wait, and the recommendation is to move to D8 as soon as possible, incorporating the tools described in the next section to search for possible incompatibilities.

Coming from D7, the greatest challenge might be the availability (or not) of the modules you already have installed, and finding and implementing replacements wherever needed. Take this as a chance to audit your site and plan a migration with a new architecture that fits your needs.

Also try out the Upgrade Status module, as it "checks the list of projects you have installed and shows their availability for newer versions of Drupal core".

If you are on Drupal 8

At its core, Drupal 9 will be the same as the latest release of Drupal 8, minus the deprecated components removed, and third party dependencies updated. This means that an upgrade from D8 should be fairly easy as it only involves making sure your codebase isn't making use of deprecated code.
Checking your site for readiness is simple using the mglaman/drupal-check utility. It is a simple CLI tool to generate a report of deprecation errors. Install it as a development package on your site and use:

# Install:
composer require mglaman/drupal-check --dev

# Run on a directory:
drupal-check web/modules
Drupal Check CLI tool example

Some things to keep in mind while checking deprecation notices:

  • Update all modules to the latest development version, to ensure testing against the latest code.
  • Don't just check the contrib modules, run it against themes, profiles and custom code.
  • If your project has continuous integration, aim to incorporate this tool into the workflow to verify readiness and avoid regressions.
  • Don't run this tool in a production environment :)

If CLI tools aren't your fancy or would like a nicer UI to show to project managers and clients, the Upgrade Status module will provide a nice dashboard with a summary and detailed information for each module of your site. It uses drupal-check as it's underlying tool.

Upgrade Status module report

Fixing deprecations

Now that you've got a report of deprecated code usage, it's time to fix it. The deprecation notices should state clearly what is deprecated, and suggested changes. I also like to look at the source code of the deprecated function and see what Drupal core is using inside it, as it shows unequivocally what needs to be done.

Let's take an example:

Call to deprecated constant REQUEST_TIME: Deprecated in drupal:8.3.0 and is removed from drupal:9.0.0. Use \Drupal::time()->getRequestTime();

Fixing the error can be as simple as replacing REQUEST_TIME with:

\Drupal::time()->getRequestTime()

In fact, a tool called drupal-rector is under development to help automate this process. A handy list of deprecation fixes can be found in the drupal/check documentation as well.

However be aware that \Drupal calls should be avoided in classes whenever possible, and dependency injection used instead. So for the example above, if REQUEST_TIME was used inside a service class, we'd inject the 'datetime.time' service into it (the service returned by \Drupal::time()) and then call getRequestTime() on it. For more in-depth information on how to call services using dependency injection, read Accessing services from the drupal.org docs.

Mark your modules as D9 ready

If you have fixed all deprecation notices, and are a module maintainer or have custom modules in your site, mark them as compatible with Drupal 8 and 9 in the info.yml file:

name: My Module
type: module
core_version_requirement: ^8.8 || ^9

A note on third party dependencies

Drupal 9 will have it's third party dependencies updated, most notably Symfony 4.4 components. Be sure to test your site in a D9 beta using these dependencies to avoid potential conflicts when D9 is released. Make sure you are running with the recommended dependencies versions by using the 9.0.x branch of drupal/core-recommended.

Finally, if you are starting a new build

Start with the latest D8 release! As mentioned, Drupal 9 is D8 at its core, so it is safe to start development with Drupal 8 and wait for the release date to upgrade. Just keep an eye out on module deprecation notices using the suggested tools from above.

Mar 05 2020
Mar 05

This post is part of our Drupal 9 upgrade series. If you're on Drupal 8 already, there's lots of useful advice below; we'll also have a post just for you soon.

Drupal 7 will be unsupported past 2021. Consider this post your 18 month friendly nudge  :-)

With end-of-life for Drupal 7 security support set for the end of 2021 and the release of Drupal 9 this summer, it's time to make a plan.

You've likely already been thinking about what to do with your current Drupal 7 site. Starting with a formal review that we outline below now (ideally by sometime in Q2 2020), gives you time to audit your existing site, evaluate needs and options, and create a budget and process plan for upgrading to Drupal 9 before D7's security updates reach end-of-life status.

And since you've been on D7 for a few years, your site's design, content and features may be due for a refresh. It's a good time to assess all these things -- and create a plan that is manageable for your team and doesn't leave frustrated and stressed in 2021. 

And here's a brief overview (our cheat sheet) of an 18 month plan: 

  • Q2 - Q3 2020: Technical audit and resource / scope planning

    • Document the Drupal infrastructure (modules, content types, features) that constitute your current site.

    • Understand what modules are core v. contributed and their equivalents in D8 / D9

    • Do a qualitative assessment of whether the site is still "working" for you. Are there current site features that have fallen out of use? Is there new functionality that you'd like to build?

    • Do we want to just replatform to Drupal 9, or take the opportunity to redesign our site as well?

    • What's a realistic budget for a redesign / replatform? 

    • Who will be on your project team when you move forward? 

  • Q3 - Q4 2020: Finalize the scope of your project, your available budget, and get started

    • Now that you know what you want, secure the appropriate resources for the scope of work.

    • Remember, major technology projects like these take time. So if possible, start your replatform / redesign sooner rather than later. This will give you time to make adjustments to your new site's features, design and content without the threat of delaying your transition off of Drupal 7.

    • Our *general* guidance is that a replatform / redesign can take anywhere from 4-6 months or more, depending on complexity and your team's availability to contribute actively to the project. 

  • Q1 - Q2 2021: Design & development is well underway

    • Depending on project start and scope, you'll likely have design finalized and development well underway in Q1.
  • Q2 - Q3 2021: Launch your new site

    • Our recommendation is that your new site launch no later than Q3 2021. You'll give yourself enough wiggle room in case anything takes more time than you thought it would. ;-)
Drupal 7 to Drupal 9 Quick Guidance Drupal.org: If you start now, we'll be moving to Drupal 8 first to allow a seamless upgrade to Drupal 9. If the project starts next year, we're likely to start building directly in D9.

Am I going to Drupal 9 directly or Drupal 8 first? 

Well, it depends on two things: when you start your rebuild project and where Drupal 9's release schedule stands. If you were to start your move now, we'd migrate your site to 8.8 first, and then transition you to Drupal 9. If your project starts next year, we'll likely build the site directly in D9. (note: the D9 release date depends on when beta requirements are met. Current plan is to release on June 3, 2020, but it could be as late as December (https://www.drupal.org/core/release-cycle-overview).

Whether replatform or redesign, the initial audit and further evaluation of your features and functions for the new site will be critical to how and when we start the build in D8 or D9. The overall level of effort in either case is directly related to how complex your current site is -- and what new specs your team wants. Whatever the case, the Chapter Three team are experts at Drupal 7 to Drupal 8 migrations (we’ve done well over a hundred!) and we'll help your team navigate the process from 7 to 8 or 9. We're active contributors to Drupal 9 and are closely monitoring progress of core and contributed modules for all of our clients. 

The best way to get your upgrade done is to work with us! Chapter Three's contributions to Drupal 8 helped build much of the infrastructure that Drupal users see today. We know how to manage an effective transition. Contact us today and let's get your customized 18 month plan in motion.  

Dec 17 2019
Dec 17

Developer portals need three kinds of great content to succeed. 

  1. API Documentation

  2. Technical Documentation

  3. Marketing Content

Content is being consumed differently now than it was in the past. Dense long-form, bespoke technical communication is going away. Clear, highly visual , easy to read content is taking the lead.  When building a developer portal, content quality and organization needs to be a high priority for your team. .   

API Documentation

Clear API Documentation is a critical component of success.  Your developers rely on this documentation during development.   It needs to be easy to read and understand, and most importantly it needs to be consistent across all your APIs; this consistency starts with good planning upfront. This is content strategy with a particular focus on technical documentation and will result in a writing guide for API docs to help foster usability and readability. Templates and writing guides provide consistency that will increase your developer community’s happiness and productivity.  

Technical Documentation

Technical Documentation is your API program’s supporting content. These include practical use case scenarios, SDK documentation, Frequently Asked Questions, and other technical topic content. Paying attention to the quality of this content is important.  Rambling, bespoke technical documentation will not be consumed by your developer community and will slow the growth of your API acceptance. We help companies develop content governance strategy plans that create a consistent tone, tenor and taxonomy for your technical documentation. Consistent, easy to read, well tagged technical content is a winning combination. 

Marketing Content

Marketing Content. Digital transformation is being powered by APIs; the company API developer portal is becoming the place where these initiatives are being communicated. The ‘customer’ is the developer who comes to your site and consumes your API’s in order to build out new products, services and transactions. Growing adoption of API programs along these channels requires solid marketing material. This content communicates the business value of using your API products and should include all stakeholders in the process as you gather input from across the organization. Work with writers who will create great copy and content to sell your API program. 

The Apigee developer portal is a powerful content management system built using Drupal 8. It is the perfect platform to manage these three types of content.   

Chapter Three is a digital agency that helps organizations develop amazing Apigee developer portal experiences by providing API program strategic planning, content strategy, technical writing, development, training and support.

We look forward to partnering with you. 

Oct 28 2019
Oct 28

Recently one of our clients asked us to come up with a better language detection and redirection solution for their multilingual Drupal 8 site. Most out of the box solutions do not provide a great user experience and IP based detection does not always work as expected. Browser-based redirection is also not an ideal option since at some point a visitor might want to manually choose which language they would like to see.

Having this issue in hand I started looking into possible solutions, I looked at a number of multilingual Drupal and non-Drupal sites and could not find anything that would work for our client. I thought what if we ask a visitor what to do by showing them a box with browser detected language. This is just like Chrome's translation prompt that asks you if you would like to translate the site. The prompt that is very simple and not as annoying as some auto-redirect solutions.

So this is what I came up with. A simple to use Drupal 8 module with few configurations needed. I decided to call it Language Suggestion.

The module supports auto redirects based on visitor selection and custom prompt messages per language. Here is the full list of configurable options:

  • Container CSS element. This is where you would need to specify the main page container. You may need this when you have some additional message boxes at the top of the page, e.g. Cookie usage etc.. By default this is set to "body" HTML tag.
  • Always redirect. This option adds auto-redirect to the website based on previously suggested language selection.
  • Redirect to [langcode].domain.com, domain.com/[langcode] or custom domains.
  • Language switch CSS element. This option overrides the auto language redirect. For instance when used after auto redirect still wants to switch to another language they will be able to do so. Make sure to specify language switch element class name or ID.
  • Language suggestion delay. This option allows the language suggestion box to appear with delay. The value is in seconds.
  • Dismiss delay. This option indicates how long language suggestion box should be hidden and when to reappear.
  • The browser language mapping is the mapping where you specify when to show language suggestions.
  • Browser-based/HTTP header-based (Experimental) language detection.

Language mapping page:

Language Suggestion Drupal 8 mapping page

Language settings page:

Language Suggestion Drupal 8 settings page

This is how language suggestion box appears on the site:

Language Suggestion Drupal 8 suggestion box

Configuration page is located at (Administration > Configuration > Regional and language > Language Suggestion) /admin/config/regional/language-suggestion

To test the module please make sure you have only one language in your browser settings. The language should be different from your current website language. And make sure that language is enabled in your Drupal 8 site.

I tried to keep visitor's interaction annoyance level with the language suggestion prompt to its minimum. I hope this solution could be useful for you and your clients or give you some good ideas.

Download Language Suggestion

or composer require drupal/language_suggestion

May 10 2019
May 10

On several occasions clients have come to us asking for a form with a gated resource. They want the  resource (.pdf, .doc, .xls etc.) to be available to download after the user fills out a form. In exchange for the visitor's valuable information, we provide them a file.

This is easily achievable in Drupal 8 with the Webform and Token modules.  At the time of creation of this blog post, I used Drupal version 8.7.0.

gated resource webform gif demo

Install modules:

Install the modules as usual. If you need more information on this topic, you can find it here.

Enable modules:

Enable Webform UI, Webform Node (which comes with webform) and the Token module.

  • Webform UI is not really required if you are creating your form via the yml editor, but provides a nice UI to generate and configure our form elements.
  • Webform Node provides a bridge between webform submissions and nodes. The description of the module reads: "Provides a Webform content type which allows webforms to be integrated into a website as nodes."
    For this example I will not be using the new Webform content type that the Webform Node module provides,I am using it because it allow me to bridge our webform submission with the node field tokens.
  • Token is going to get the gated file URL and name.

Configuring the content type:

For this tutorial I will use the Article content type, add a file field that we will name Gated file.

gated form add file field

I hide the field from the default display or any display you might use.

disable the field from the node display

By default the file will be disabled/not shown, therefore not accessible to site visitors.

Creating a webform:

I am going to create a basic webform with only two form elements: one for name and another one for email with standard validation.

* Important to add the form via: /admin/structure/webform, as a webform entity not as the new webform content type, so you can reference the form from different nodes.

Here is a screenshot of my simple form:

gated resource gated form screenshot

Referenced webform:

In this example I will be using an entity reference field to reference the webform we have just created, via the new field called Gated form (field_gated_form):

create a new entity reference to a webform

Then I create a node of type Article. 

You can see the gated file field and the gated form field highlighted in the following screenshot, also you can notice that I am referencing our previously created webform in the Gated form select

creating a node of type article with the gated resource and the form referenced

Creating a confirmation modal with the file exposure:

Because the file is not visible to the user, we will use the webform confirmation modal to expose a link to the file.

image displaying the settings for the webform

if you browse the available tokens at the bottom of the Confirmation message you'll see the following tokens:

If you need a deeper nesting look at /admin/help/token page, where you'll see all of the token values available per field.

example of token values available per field inside node

For the confirmation message I added the following HTML:


Here is your file:
 <a href="https://www.chapterthree.com/blog/gated-resources-forms-webform-and-token/[webform_submission:node:field_gated_file:entity:url]" target="_blank">
  [webform_submission:node:field_gated_file:entity:name]
 </a>

We are using tokens to get the name and the URL of the field_gated_field, which is the machine name of our file field. Then we form a simple link (HTML a tag) that allow the user to download the file.

Webform in Drupal 8 is super flexible, easy to use and powerful. This is just a simple example of what can be achieved with a little bit of site building and a bunch of clicks.

Disclaimer:
This is not a secure way to protect files, anyone with the original link to the document will be able to see it and download it, but for some use cases is good enough.

Decoupling Pattern Lab from your theme a City of San Francisco project.

Nov 06 2018
Nov 06
Nov 06 2018
Nov 06

Pattern Lab (PL), a commonly known pattern library, is an open-source project to generate a design system for your site. In the last two years it has gotten a lot of attention in the Drupal community. It's a great way to implement a design system into your front-end workflow.

The following post describes how our client (the City and County of San Francisco) began to implement a pattern library that will eventually be expanded upon and re-used for other agency websites across the SF.gov ecosystem.

USWDS

Using the U.S. Web Design System (USWDS), until their own pattern library was ready for prime time, was a client requirement.

USWDS uses a pattern library system called Fractal. I think Fractal is a great idea, but it lacked support for Twig, Twig is the template engine used by Drupal. Fractal out of the box uses Handlebars (templating engine in JavaScript), and thought the template language in Fractal can be customized I wasn’t able to make it work with Twig macros and iterations even with the use of twig.js

Creating the Pattern Lab

Ultimately, I decided to start from scratch. In addition to the USWDS requirement, the client also needed to be able to reuse this pattern library on other projects. I used the Pattern Lab Standard Edition for Twig, among other things this means that you need PHP in the command line in order to "compile" or generate the pattern library.

I added a gulpfile that was in charge of watching for changes in the PL source folders. Once a Twig, Sass or JavaScript file was changed, the pattern library was re-generated.

Generating the Pattern Library

I also needed Gulp to watch the file changes.

The following is a SIMPLE example of the Gulp task that generates the PL watching the folders. the following code snippet shows the config object containing an array of the folder directories.


{
  "css": {
    "file" : "src/sass/_all.scss",
    "src": [
      "pattern-lab/source/_patterns/*.scss",
      "pattern-lab/source/_patterns/**/*.scss",
      "pattern-lab/source/scss/*.scss"
    ],
    "pattern_lab_destination": "pattern-lab/public/css",
    "dist_folder": "dist/css"
  },
  "js": {
    "src": [
      "pattern-lab/source/js/*.js",
      "pattern-lab/source/js/**/*.js"
    ]
  }
}

And the following one is a common watcher in gulp:


gulp.task('watch', function () {
    gulp.watch(config.js.src, ['legacy:js']);
    gulp.watch(config.css.src, ['pl:css']);
    gulp.watch(config.pattern_lab.src, ['generate:pl']);
    gulp.watch(config.pattern_lab.javascript.src, ['generate:pl']);
});


The following task is in charge of generating the pattern library with PHP:



gulp.task('pl:php', shell.task('php pattern-lab/core/console --generate'));


Please NOTE that this is an oversimplified example.

Sass

Having generated the Pattern Library, I figured out that in order to use this Pattern Lab into my Drupal theme, I needed to generate a single CSS file and single JavaScript (JS) file.
The main Sass file imports all Sass code from USWDS by using the `@import` statement.
I imported the source Sass code from USWDS which I required with npm and imported  the source file directly from the node_modules folder:




//pattern-lab/source/scss/components.scss

// Styles basic HTML elements
@import '../../../node_modules/uswds/src/stylesheets/elements/buttons';
@import '../../../node_modules/uswds/src/stylesheets/elements/embed';

Then I imported the scss files that were inside my pattern elements:

// Styles inside patterns.
@import "../_patterns/00-protons/*.scss";
@import "../_patterns/01-atoms/**/*.scss";
@import "../_patterns/02-molecules/**/*.scss";
@import "../_patterns/03-organisms/**/*.scss";
@import "../_patterns/04-templates/**/*.scss";
@import "../_patterns/05-pages/**/*.scss";



All the styles were dumped into a single file called components.css

Having this single CSS file created I was able to use USWDS CSS classes along with the new ones.
I had to add a /dist folder where the transpiled Sass would live and be committed for later use in the Drupal theme.

JavaScript

I did something similar for JavaScript. The biggest challenge was to compile the USWDS JavaScript files exactly as they were. I resorted to copying all the source for the JavaScript into the src folder of the pattern library and set a watcher specifically for the USWDS JavaScript, and added another watcher for the new Pattern Lab JavaScript:

Example:

In the following example I compile all the JS that lives inside the components into a single file.

Then the resulting file is copied  to: ./pattern-lab/public/js which is the folder that reads the Pattern Lab when working on Pattern Lab only.
The other copy of the file goes to the distribution folder ./dist/pl/js which is the one I use in my Drupal theme.



// Component JS.
// -------------------------------------------------------------------- //
// The following task concatenates all the JavaScript files inside the
// _patterns folder, if new patterns need to be added the config.json array
// needs to be edited to watch for more folders.

gulp.task('pl:js', () => {
    return gulp.src(config.pattern_lab.javascript.src)
        .pipe(sourcemaps.init())
        .pipe(babel({
            presets: ['es2015']
        }))
        .pipe(concat("components.js"))
        .pipe(sourcemaps.write())
        .pipe(gulp.dest('./pattern-lab/public/js'))
        .pipe(gulp.dest('./dist/pl/js'));
});


The resulting files:



--/dist/
--/dist/css/components.css
--/dist/js/components.js


Were included the HEAD of my Pattern Lab by editing pattern-lab/source/_meta/_00-head.twig 

I included the following lines:



<link rel="stylesheet" href="https://www.chapterthree.com/blog/decoupling-pattern-lab-from-your-theme-a-city-of-san-francisco-project/../../css/components.css" media="all">
<script src="https://www.chapterthree.com/blog/decoupling-pattern-lab-from-your-theme-a-city-of-san-francisco-project/../../js/dist/uswds.min.js"></script>
<script src="https://www.chapterthree.com/blog/decoupling-pattern-lab-from-your-theme-a-city-of-san-francisco-project/../../js/dist/components.js"></script>


Please refer to the repo if you need the details of the integration: GitHub - SFDigitalServices/sfgov-pattern-lab: SFGOV Pattern Lab

Integrating the Pattern Lab with Drupal.

Composer and libraries:

I used the following plugin:



composer require oomphinc/composer-installers-extender


This plugin allowed me to put the pattern library in a folder different than vendor
Then I added some configuration to the composer.json

Under extra I specified where composer should install the repository of type github:



"extra": {
        "installer-paths": {
            "web/libraries/{$name}": ["type:github"],
          }


Then under repositories I set the type:github



"repositories": {
        "github": {
            "type": "package",
            "package": {
                "name": "sf-digital-services/sfgov-pattern-lab",
                "version": "master",
                "type": "drupal-library",
                "source": {
                    "url": "https://github.com/SFDigitalServices/sfgov-pattern-lab.git",
                    "type": "git",
                    "reference": "master"
                }
            }
        }
    }


and required the package under require: As you can see the name matches the name in the previously declared github repo:



"require": {
   "sf-digital-services/sfgov-pattern-lab": "dev-master",
}


A composer update should clone the github repo and place the Pattern Lab inside relative to the Drupal web folder:

/web/libraries/sfgov-pattern-lab

Components Libraries

The Component Libraries module was especially important because it allowed me to map the Pattern Lab components easily into my theme.

Then I had to map my Pattern Lab components with the Drupal theme:

The Drupal Theme:

I created a standard Drupal theme:

The sfgovpl.info.yml file:

In the following part of the sfgovpl.info.yml file I connected the Pattern Lab Twig files to Drupal:



component-libraries:
  protons:
    paths:
      - ../../../libraries/sfgov-pattern-lab/pattern-lab/source/_patterns/00-protons
  atoms:
    paths:
      - ../../../libraries/sfgov-pattern-lab/pattern-lab/source/_patterns/01-atoms
  molecules:
    paths:
      - ../../../libraries/sfgov-pattern-lab/pattern-lab/source/_patterns/02-molecules
  organisms:
    paths:
      - ../../../libraries/sfgov-pattern-lab/pattern-lab/source/_patterns/03-organisms
  templates:
    paths:
      - ../../../libraries/sfgov-pattern-lab/pattern-lab/source/_patterns/04-templates
  pages:
    paths:
      - ../../../libraries/sfgov-pattern-lab/pattern-lab/source/_patterns/05-pages

libraries:
  - sfgovpl/sfgov-pattern-lab


The sfgovpl.libraries.yml file:

In the last line of the previous code example, you can see that I required sfgov-pattern-lab library,  which will include the files compiled by Gulp into my Pattern Lab.



sfgov-pattern-lab:
  css:
    base:
      '/libraries/sfgov-pattern-lab/dist/css/components.css': {}
  js:
    '/libraries/sfgov-pattern-lab/dist/pl/js/components.js': {}


Using the Twig templates in our theme:

The following is an example of how to use a molecule from the pattern library into the Drupal theme:

You can include the @molecules/08-search-results/03-topic-search-result.twig twig like this:

Pattern Lab twig:

node--topic--search-index.html.twig



<div class="topic-search-result">

  <div class="topic-search-result--container">

    <div class="content-type"><i class="sfgov-icon-filefilled"></i><span>{{ content_type }}</span></div>

    <a class="title-url" href="https://www.chapterthree.com/blog/decoupling-pattern-lab-from-your-theme-a-city-of-san-francisco-project/{{ url }}"><h4>{{ title }}</h4></a>

    <p class="body">{{ body|striptags('<a>')|raw }}</p>

  </div>

</div>

Drupal template:

The following example calls the Pattern lab molecule originally located at: web/libraries/sfgov-pattern-lab/pattern-lab/source/_patterns/02-molecules/08-search-results/03-topic-search-result.twig but thanks to the Components module we just call it as: @molecules/08-search-results/03-topic-search-result.twig



{# Set variables to use in the component. #}
{% set url = path('entity.node.canonical', {'node': elements['#node'].id()  }) %}
{% set description = node.get('field_description').getValue()[0]['value'] %}
{% set type = node.type.entity.label %} {# content type #}
{# Icluding the molecule in our Pattern Lab.#}

{% include "@molecules/08-search-results/03-topic-search-result.twig" with {
  "content_type": type,
  "url": url,
  "title": elements['#node'].get('title').getString(),
  "body": description
} %}


Recommendations

SFGOV Pattern Lab, Initial development was made in large part for Chapter Three and this post is intended to show the core concepts of decoupling your Pattern Lab from your Drupal theme.

You can find the full code implementation for the Pattern library and Drupal in the following Urls:

SFDigitalServices/sfgov and the Pattern Lab here: SFDigitalServices/sfgov-pattern-lab

You should try the value module, it is great for extracting values in the Drupal twig templates and connect them with your Pattern Lab twig templates.

Give a try to the UI Patterns module, looks promising and a great solution for decoupled Pattern Libraries.

Introducing React Comments

May 14 2018
May 14
May 14 2018
May 14

Commenting system giant Disqus powers reader conversations on millions of sites, including large publishers like Rolling Stone and the Atlantic. So when Disqus quietly introduced ads into their free plans last year, there was some understandable frustration.

Why did @disqus just add a bunch of ads to my site without my permission? https://t.co/CzXTTuGs67 pic.twitter.com/y2QbFFzM8U

— Harry Campbell (@TheRideshareGuy) February 1, 2017

I might be late but now getting tacky ads injected into my comments on my site so saying goodbye @disqus . I refuse to pay for an ad-free experience when it has been free all this time. #baitedandswitched

— KenyaRae (@IamKenyaRae) April 18, 2018 React comments is responsive and slick on mobile devices, as shown here on an iPhone X.

Around this time, one of our clients, Military.com, decided to move their content out of Disqus, not only to avoid licensing fees to avoid ads, but also so they could own their comment content. Aside from those two issues, Military.com was satisfied with the user experience of Disqus. An interactive component like comments presented a perfect opportunity to progressively decouple using React.js. So we began work on the React Comments module.

React Comments is a drop-in replacement for Drupal Core’s Comment module. It has all the same features, but with a few added benefits. Because it was built using React, user interactions appear on the page immediately, with no page refresh needed.

Editors can moderate comments in context from the node’s front-end display, rather than digging through the Drupal Admin UI. And regular users can flag comments as inappropriate, too, so you can identify repeat offenders and block them from commenting. Coupled with the Comment Admin Notify module, site administrators can get emails about new comments on their site, so they can take action right away.

When looking at the front-end, anyone can flag a comment as inappropriate. Then, through the same interface, privileged users will be able to delete problematic comments as needed.

And, of course, there are no ads.

Military.com has been using this module on their site since launch in December 2017. With over 10 million active users, they amassed over two thousand comments within just a couple weeks of launch.

Our site uses it as well, as you can see right on this blog post!

In the future, we’re looking to add the ability to improve the theming experience by including an option to wrap the comments section in an iframe with an optional custom stylesheet. Also, setting up notifications for commenters who receive replies is on the horizon.

Try the React Comments module on your site and see what you think. We’d love to get a comment from you with your thoughts on it!

Recruit more effectively with Taleo-integrated Drupal

Apr 26 2018
Apr 26
Apr 26 2018
Apr 26

Most corporate websites have a career section where people can find out what jobs are open. These sections can range from just a few standing openings to hundreds of vacant positions across multiple job types that open and close frequently. Larger companies traditionally use HR software systems to manage this ebb and flow.

One popular and powerful HR system is Taleo. Many companies simply link offsite to the taleo.com listing, which serves as the company’s only place to see and apply to openings. Other times, that same listing gets embedded on the company’s career section. As you can see in the user journey below for our client MemorialCare, this got the job done, but didn’t really convey a company’s brand or culture.

Step 1: Candidate finds the careers section of MemorialCare’s website. Step 2: Candidate is linked to the unbranded MemorialCare Taleo job listing.

 

Step 3: Candidate sees the job description on the unbranded Taleo site, and decides whether or not to apply.

You need a delightful candidate user journey to attract amazing candidates, and this begins with a branded careers section. So, we are giving Taleo customers the ability to display their job postings within their Drupal site, and branding them to match.

MemorialCare saw a dramatic improvement when we redesigned their careers section as a separate website and built an integration with Taleo. The result is a beautiful candidate journey that captures MemorialCare’s brand and culture.

Step 1: Candidates can begin searching by location and job title right away, without clicking additional obscure links. Step 2: Searching will produce a list of jobs. Step 3: A detail page gives you an easy to read overview of what to expect and a clear link to apply.

With this Taleo integration, the HR department has full control of the look and feel for their careers content. If they desire a change, there’s no need to contact Taleo– their developers (or Chapter Three, if they prefer!) can make the change right away. When job descriptions or details are updated in Taleo, it automatically gets updated on the new careers site, too! And, Taleo meta-tags give Drupal job listings an extra SEO boost. Jobs listed on the new site are ready to be ingested by a variety of job aggregators.

The way you present your organization matters to prospective talent. Building a Taleo integration with your Drupal site and making it look beautiful will give your organization a competitive edge with quality candidates. If this is something your company is currently struggling with, please feel free to contact us to find out how we can help. 

Presentation: A Drupal City

Apr 11 2018
Apr 11
Apr 11 2018
Apr 11

I presented at DrupalCon Nashville about working with the City of San Francisco to make a better transaction experience for residents. Moving beyond a simple content site where we tell users how to do things, we are now developing a brand new city website in Drupal 8, where residents can actually do those things online. The presentation covers how to run an agile project of this scale in a government environment, what we did as a part of discovery, where we're going, and how our foray into design & development is progressing so far.

Here are the slides to digest. Don't hesitate to reach out if you have any questions!

Access Control Modules for Drupal 8

Mar 19 2018
Mar 19
Mar 19 2018
Mar 19

With all the changes in Drupal 8, it’s no wonder the landscape for access control modules is adapting. While the port of Organic Groups has started, there are several major issues to resolve before it’s ready for use. There are, however, a couple promising new options.

Permissions By Term

If you don’t need group management, Permissions by Term is the Drupal 8 substitute for the Taxonomy Access Control module. It controls node visibility based on the how a node is tagged. Term access is granted by role, and individual users can be whitelisted for term access permissions.

Each term allows you to select by role who can access it from the taxonomy term edit page.

 

Moderation

If your content has drafts or other moderation states, this module will play nicely. Permissions by Term handles access control to published nodes, while moderation modules control access to unpublished moderation states.

Multilingual

This module works with translations if all translations have the same "access control" (they are tagged with same terms). If not, the latest translation of the node gets saved to the node access table, and the other translations get lost. There is work in progress on this issue. 

Shortcomings

  • At present, this module does not control edit and delete permissions since it only works for published content. 
  • This is not the right module if content and users need to be organized by groups. It can be used, but inefficiently, as it would require each user group to be a role, resulting in duplication of "term group A" and role "group A". A workaround could be to whitelist the users to the term, but that might create overhead in the node_access table. If this is a requirement, the Group module is a better choice. 

Technical Overview

Setup is very straightforward: Enable it and configure role access for each taxonomy term on the term edit page. This module uses the node grants API and has 2 custom tables. The codebase looks modern and contains tests.  Node grants built for the current user object list every node the user can access, which could be thousands, and does not seem efficient. This might require attention if your site will has many nodes.

Group

What it does

Group allows content to be organized into groups that users can join. Access is controlled by group roles. The default roles are anonymous, outsider (logged-in but not member), and member, administrators can create additional group roles. An optional module provides out of the box CRUD access control for nodes. 

Group provides an interface to show you what belongs to it.

 

Moderation

The core Content Moderation module permissions take precedence for editing (not for viewing/deleting) so site editors can edit all content even if they are not members or editors of a group. This potentially unexpected behavior presents a moderately complex issue that probably has a custom solution for each site’s unique set of business rules. This, and other similar conflicts, can be solved with a custom module defining the permissions priorities.

Multilingual

This works module works ok with translations. All translations automatically belong to the same groups as the original node and this cannot be changed.

Shortcomings

  • We didn’t find a straightforward way to add a node to a group through the node/add form. Instead, we needed to create the node through a "create content" link in the group page, or manually associate it on another form.
  • If groups are required to control access to nodes, this module works as-is. Associating non-node entities with groups for access control is possible, but requires additional work.

Technical Overview

This module comes with good documentation for site builders and developers. It’s user interface looks similar to Organic Groups, but seems more intuitive. Site administrators can create multiple “group types,” and all groups, group types, user/group memberships, and content/group relationships are fieldable entities (meaning they get all the entity goodness!). Group roles and permissions are configurable per group type, and the module uses the node grants API.

Summary

Drupal 8’s architecture changes mean we may say goodbye to some beloved (and not-so-beloved) module versions. This creates an opportunity for our community to revisit how we solve problems, and better serve our users. Permissions by Term and Group are another indicator that the Drupal 8 community is alive, well and, perhaps most importantly, adapting to change.

Dreditor Refresh: A New User Style for Dreditor

Jan 29 2018
Jan 29
Jan 29 2018
Jan 29

Dreditor is a beloved and indispensable tool, in the form of a browser extension, that enhances project issue pages on Drupal.org. When it comes to reviewing patches, it turns what would just be a plain text file into a feature rich interface for reviewing patches, allowing users to easily select and comment on lines of code, which then get pasted into the comment form, as properly formatted HTML.

I've been lucky to have some extra community time here at Chapter Three over the past couple of weeks. While perusing the core issue queue, I decided to resurrect an old User Style I created for Dreditor back in 2009, and give Dreditor a little refresh. You can install the Dreditor Refresh style with Stylish here. Make sure you also have Dreditor installed, and have logged into Drupal.org.

Happy contributing! Enjoy!

Screenshot of Dreditor Refresh while reviewing a patch

UPDATE: Work is happing to implement the enhancements Dreditor brings to Drupal.org directly, which is great.  You can follow along and help out here: https://www.drupal.org/node/1673278

Oct 16 2017
Oct 16

Last week at DrupalCamp Quito, I presented an updated, Spanish-language version of my DrupalCon session. If you would like to view the presentation in English, you can find it on my DrupalCon blog post.

Las estructuras orientadas a objetos han reemplazado a nuestros queridos "hooks" que nos permitían extender Drupal con nueva funcionalidad sin necesidad de hackear core (u otros módulos de contrib). Pero, ¿cómo funciona esto? En esta charla revisamos cómo extender un módulo para implementar single sign-on (SSO), y al hacerlo nos adentramos a cómo la programación orientada a objetos hace magia en nuestros módulos, haciéndolos más fáciles de escribir, entender y depurar. Adicionalmente, se describen algunos de los patrones de diseño de Drupal, cómo utilizar event listeners, sobreescribir rutas y otras herramientas.

[embedded content]

Slides: https://goo.gl/QgskXw
Código de ejemplo: https://github.com/arlina-espinoza/openid_fb

How to Alter Node URL Alias Based on Taxonomy Term in Drupal 8

Sep 07 2017
Sep 07
Sep 07 2017
Sep 07

Recently I had to generate term-specific aliases (aliases that are different from the default alias pattern set for Article entities). This is how to do it:

1. Enable the Pathauto module
2. Set the default URL alias pattern for your content type in order to fire the hook
3. Implement hook_pathauto_alias_alter() in your .module file.

Example module structure:

mymodule/
  - mymodule.info.yml
  - mymodule.module
  - src/
    - ArticlePathAlias.php

I like to keep .module clean and simple and because of that I store the main logic in src/ArticlePathAlias.php file.

The mymodule.info.yml this is just a regular .info file.

4. Add the following to your mymodule.module file:

use Drupal\mymodule\ArticlePathAlias;

/**
 * Implements hook_pathauto_alias_alter().
 */
function mymodule_pathauto_alias_alter(&$alias, array &$context) {
  if ($new_alias = (new ArticlePathAlias())->generate($context)) {
    $alias = $new_alias;
  }
}

5. Add the following to your src/ArticlePathAlias.php file:

<?php

namespace Drupal\mymodule;

use Drupal\Component\Utility\Html;
use Drupal\taxonomy\Entity\Term;

/**
 * Generate URL aliases for articles.
 */
class ArticlePathAlias {

  protected $terms = [
    'Term name 1' => 'custom/alias',
    'Term name 2' => 'custom2/alias',
    'Term name 3' => 'custom3/alias',
  ];
  protected $pattern = '/%term%/%year%/%monthnum%/%day%/%postname%';

  public function generate($context) {
    if ($context['bundle'] === 'article' && ($context['op'] == 'insert' || $context['op'] == 'update')) {
      return $this->assembleAlias($context['data']['node']);
    }
  }

  protected function assembleAlias($entity) {
    $date = new \DateTime(date('c', $entity->getCreatedTime()));
    $parameters = [
      '%year%'     => $date->format('Y'),
      '%monthnum%' => $date->format('m'),
      '%day%'      => $date->format('d'),
      '%postname%' => Html::cleanCssIdentifier($entity->getTitle()),
      '%term%'     => $this->findTermAlias($entity),
    ];
    if (!empty($parameters['%term%'])) {
      return str_replace(array_keys($parameters), array_values($parameters), $this->pattern);
    }
  }

  protected function findTermAlias($entity) {
    // Make sure to change `field_keywords` to the field you would like to check.
    if ($keywords = $entity->get('field_keywords')->getValue()) {
      foreach ($keywords as $data) {
        $term = Term::load($data['target_id']);
        $name = $term->getName();
        if (in_array($name, array_keys($this->terms))) {
          return $this->terms[$name];
        }
      }
    }
  }

}

The code above will generate /%term%/%year%/%monthnum%/%day%/%postname% alias or (/custom/alias/2017/07/21/test-title) depending on the term.

Make sure you change field_keywords to your own taxonomy term reference field. Also change $context['bundle'] === 'article' to entity type that will trigger custom alias.

How to Alter Entity Autocomplete Results in Drupal 8

Aug 26 2017
Aug 26
Aug 26 2017
Aug 26

Sometimes you might want to display additional data in the autocomplete results, for instance add content language next to the title, or display entity type or any other related data. In this blog post I will demonstrate how to alter suggestions in autocomplete fields in Drupal 8. The project is available for download from github, see the link at the bottom of the page.

Link autocomplete results

Here is the module structure I will be using:

alter_entity_autocomplete/
  - alter_entity_autocomplete.info.yml
  - alter_entity_autocomplete.services.yml
  - src/
    - EntityAutocompleteMatcher.php
    - Controller/
      - EntityAutocompleteController.php
    - Routing/
      - AutocompleteRouteSubscriber.php

Contents of the alter_entity_autocomplete.info.yml file:

name: Alter Entity Autocomplete
description: The module alters entity autocomplete suggestion list.
type: module
core: 8.x

Contents of the alter_entity_autocomplete.services.yml file:

services:

  alter_entity_autocomplete.route_subscriber:
    class: Drupal\alter_entity_autocomplete\Routing\AutocompleteRouteSubscriber
    tags:
      - { name: event_subscriber }

  alter_entity_autocomplete.autocomplete_matcher:
    class: Drupal\alter_entity_autocomplete\EntityAutocompleteMatcher
    arguments: ['@plugin.manager.entity_reference_selection']

Contents of the src/EntityAutocompleteMatcher.php file. This is the file where you would change the output of the sugesstions/autocomplete results:

<?php

namespace Drupal\alter_entity_autocomplete;

use Drupal\Component\Utility\Html;
use Drupal\Component\Utility\Tags;

class EntityAutocompleteMatcher extends \Drupal\Core\Entity\EntityAutocompleteMatcher {

  /**
   * Gets matched labels based on a given search string.
   */
  public function getMatches($target_type, $selection_handler, $selection_settings, $string = '') {

    $matches = [];

    $options = [
      'target_type'      => $target_type,
      'handler'          => $selection_handler,
      'handler_settings' => $selection_settings,
    ];

    $handler = $this->selectionManager->getInstance($options);

    if (isset($string)) {
      // Get an array of matching entities.
      $match_operator = !empty($selection_settings['match_operator']) ? $selection_settings['match_operator'] : 'CONTAINS';
      $entity_labels = $handler->getReferenceableEntities($string, $match_operator, 10);

      // Loop through the entities and convert them into autocomplete output.
      foreach ($entity_labels as $values) {
        foreach ($values as $entity_id => $label) {

          $entity = \Drupal::entityTypeManager()->getStorage($target_type)->load($entity_id);
          $entity = \Drupal::entityManager()->getTranslationFromContext($entity);

          $type = !empty($entity->type->entity) ? $entity->type->entity->label() : $entity->bundle();
          $status = '';
          if ($entity->getEntityType()->id() == 'node') {
            $status = ($entity->isPublished()) ? ", Published" : ", Unpublished";
          }

          $key = $label . ' (' . $entity_id . ')';
          // Strip things like starting/trailing white spaces, line breaks and tags.
          $key = preg_replace('/\s\s+/', ' ', str_replace("\n", '', trim(Html::decodeEntities(strip_tags($key)))));
          // Names containing commas or quotes must be wrapped in quotes.
          $key = Tags::encode($key);
          $label = $label . ' (' . $entity_id . ') [' . $type . $status . ']';
          $matches[] = ['value' => $key, 'label' => $label];
        }
      }
    }

    return $matches;
  }

}

Contents of the src/Controller/EntityAutocompleteController.php file:

<?php

namespace Drupal\alter_entity_autocomplete\Controller;

use Drupal\Core\KeyValueStore\KeyValueStoreInterface;
use Drupal\alter_entity_autocomplete\EntityAutocompleteMatcher;
use Symfony\Component\DependencyInjection\ContainerInterface;

class EntityAutocompleteController extends \Drupal\system\Controller\EntityAutocompleteController {

  /**
   * The autocomplete matcher for entity references.
   */
  protected $matcher;

  /**
   * {@inheritdoc}
   */
  public function __construct(EntityAutocompleteMatcher $matcher, KeyValueStoreInterface $key_value) {
    $this->matcher = $matcher;
    $this->keyValue = $key_value;
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container) {
    return new static(
      $container->get('alter_entity_autocomplete.autocomplete_matcher'),
      $container->get('keyvalue')->get('entity_autocomplete')
    );
  }

}

Here is contents of the src/Routing/AutocompleteRouteSubscriber.php file:

<?php

namespace Drupal\alter_entity_autocomplete\Routing;

use Drupal\Core\Routing\RouteSubscriberBase;
use Symfony\Component\Routing\RouteCollection;

class AutocompleteRouteSubscriber extends RouteSubscriberBase {

  public function alterRoutes(RouteCollection $collection) {
    if ($route = $collection->get('system.entity_autocomplete')) {
      $route->setDefault('_controller', '\Drupal\alter_entity_autocomplete\Controller\EntityAutocompleteController::handleAutocomplete');
    }
  }

}

This module was developed with help of my co-workers and I hope you will find this useful.

Download from Github

Aug 25 2017
Aug 25

Sometimes you might want to display additional data in the autocomplete results, for instance add content language next to the title, or display entity type or any other related data. In this blog post I will demonstrate how to alter suggestions in autocomplete fields in Drupal 8. The project is available for download from github, see the link at the bottom of the page.

Link autocomplete results

Here is the module structure I will be using:

alter_entity_autocomplete/
  - alter_entity_autocomplete.info.yml
  - alter_entity_autocomplete.services.yml
  - src/
    - EntityAutocompleteMatcher.php
    - Controller/
      - EntityAutocompleteController.php
    - Routing/
      - AutocompleteRouteSubscriber.php

Contents of the alter_entity_autocomplete.info.yml file:

name: Alter Entity Autocomplete
description: The module alters entity autocomplete suggestion list.
type: module
core: 8.x

Contents of the alter_entity_autocomplete.services.yml file:

services:

  alter_entity_autocomplete.route_subscriber:
    class: Drupal\alter_entity_autocomplete\Routing\AutocompleteRouteSubscriber
    tags:
      - { name: event_subscriber }

  alter_entity_autocomplete.autocomplete_matcher:
    class: Drupal\alter_entity_autocomplete\EntityAutocompleteMatcher
    arguments: ['@plugin.manager.entity_reference_selection']

Contents of the src/EntityAutocompleteMatcher.php file. This is the file where you would change the output of the sugesstions/autocomplete results:

<?php

namespace Drupal\alter_entity_autocomplete;

use Drupal\Component\Utility\Html;
use Drupal\Component\Utility\Tags;

class EntityAutocompleteMatcher extends \Drupal\Core\Entity\EntityAutocompleteMatcher {

  /**
   * Gets matched labels based on a given search string.
   */
  public function getMatches($target_type, $selection_handler, $selection_settings, $string = '') {

    $matches = [];

    $options = [
      'target_type'      => $target_type,
      'handler'          => $selection_handler,
      'handler_settings' => $selection_settings,
    ];

    $handler = $this->selectionManager->getInstance($options);

    if (isset($string)) {
      // Get an array of matching entities.
      $match_operator = !empty($selection_settings['match_operator']) ? $selection_settings['match_operator'] : 'CONTAINS';
      $entity_labels = $handler->getReferenceableEntities($string, $match_operator, 10);

      // Loop through the entities and convert them into autocomplete output.
      foreach ($entity_labels as $values) {
        foreach ($values as $entity_id => $label) {

          $entity = \Drupal::entityTypeManager()->getStorage($target_type)->load($entity_id);
          $entity = \Drupal::entityManager()->getTranslationFromContext($entity);

          $type = !empty($entity->type->entity) ? $entity->type->entity->label() : $entity->bundle();
          $status = '';
          if ($entity->getEntityType()->id() == 'node') {
            $status = ($entity->isPublished()) ? ", Published" : ", Unpublished";
          }

          $key = $label . ' (' . $entity_id . ')';
          // Strip things like starting/trailing white spaces, line breaks and tags.
          $key = preg_replace('/\s\s+/', ' ', str_replace("\n", '', trim(Html::decodeEntities(strip_tags($key)))));
          // Names containing commas or quotes must be wrapped in quotes.
          $key = Tags::encode($key);
          $label = $label . ' (' . $entity_id . ') [' . $type . $status . ']';
          $matches[] = ['value' => $key, 'label' => $label];
        }
      }
    }

    return $matches;
  }

}

Contents of the src/Controller/EntityAutocompleteController.php file:

<?php

namespace Drupal\alter_entity_autocomplete\Controller;

use Drupal\Core\KeyValueStore\KeyValueStoreInterface;
use Drupal\alter_entity_autocomplete\EntityAutocompleteMatcher;
use Symfony\Component\DependencyInjection\ContainerInterface;

class EntityAutocompleteController extends \Drupal\system\Controller\EntityAutocompleteController {

  /**
   * The autocomplete matcher for entity references.
   */
  protected $matcher;

  /**
   * {@inheritdoc}
   */
  public function __construct(EntityAutocompleteMatcher $matcher, KeyValueStoreInterface $key_value) {
    $this->matcher = $matcher;
    $this->keyValue = $key_value;
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container) {
    return new static(
      $container->get('alter_entity_autocomplete.autocomplete_matcher'),
      $container->get('keyvalue')->get('entity_autocomplete')
    );
  }

}

Here is contents of the src/Routing/AutocompleteRouteSubscriber.php file:

<?php

namespace Drupal\alter_entity_autocomplete\Routing;

use Drupal\Core\Routing\RouteSubscriberBase;
use Symfony\Component\Routing\RouteCollection;

class AutocompleteRouteSubscriber extends RouteSubscriberBase {

  public function alterRoutes(RouteCollection $collection) {
    if ($route = $collection->get('system.entity_autocomplete')) {
      $route->setDefault('_controller', '\Drupal\alter_entity_autocomplete\Controller\EntityAutocompleteController::handleAutocomplete');
    }
  }

}

This module was developed with help of my co-workers and I hope you will find this useful.

Download from Github

How to Fix HTML Content Issues During Migration in Drupal 8

Aug 18 2017
Aug 18
Aug 18 2017
Aug 18

In this post I will show you a technique to fix HTML issues, import images or perform content operations during migrations.

We have to fix source content before most content migrations. This can be challenging if there are many entries in the source database. The powerful Drupal 8 Migration API provides elegant ways to solve this type of problem.

To solve HTML issues, I always create my own process plugin. Here is an example how you would call your own process plugin to fix HTML issues in the body field:

  'field_body/value':
    -
      plugin: fix_html_issues
      images_source: '/minnur/www/source-images'
      images_destination: 'public://body-images/'
      source: post_content
    -
      plugin: skip_on_empty
      method: row

As you can see, I am piling up several process plugins for field_body/value field migration. You may also pass custom parameters to your process plugin (in my example, params are: images_source and images_destination ). You may add any number of process plugins depending on your needs.

Now let's view the plugin code. Please note all of the process plugins are stored in the src/Plugin/migrate/process directory in your migration module.

The plugin imports images into Drupal as media entities and replaces <img> tags with Drupal entity embed tags <drupal-entity data-embed-button="embed_image"></drupal-entity>. Below is the source code of the plugin:

<?php

namespace Drupal\wp_migration\Plugin\migrate\process;

use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\file\FileInterface;
use Drupal\migrate\Row;
use Drupal\media_entity\Entity\Media;
use Drupal\Core\Database\Database;
use Drupal\Component\Utility\Unicode;

/**
 * @MigrateProcessPlugin(
 *   id = "fix_html_issues"
 * )
 */
class FixHTMLissues extends ProcessPluginBase {

  /**
   * {@inheritdoc}
   */
  public function transform($html, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {

    // Values for the following variables are specified in the YAML file above.
    $images_source = $this->configuration['images_source'];
    $destination = $this->configuration['images_destination'];

    preg_match_all('/<img[^>]+>/i', $html, $result);

    if (!empty($result[0])) {

      foreach ($result as $img_tags) {
        foreach ($img_tags as $img_tag) {

          preg_match_all('/(alt|title|src)=("[^"]*")/i', $img_tag, $tag_attributes);

          $filepath = str_replace('"', '', $tag_attributes[2][1]);

          if (!empty($tag_attributes[2][1])) {

            // Create file object from a locally copied file.
            $filename = basename($filepath);

            if (file_prepare_directory($destination, FILE_CREATE_DIRECTORY)) {

              if (filter_var($filepath, FILTER_VALIDATE_URL)) { 
                $file_contents = file_get_contents($filepath);
              }
              else {
                $file_contents = file_get_contents($images_source . $filepath);
              }
              $new_destination = $destination . '/' . $row->getSourceProperty('id') . '-' . $filename;

              if (!empty($file_contents)) {

                if ($file = file_save_data($file_contents, $new_destination, FILE_EXISTS_REPLACE)) {

                  // Create media entity using saved file.
                  $media = Media::create([
                    'bundle'      => 'image',
                    'uid'         => \Drupal::currentUser()->id(),
                    'langcode'    => \Drupal::languageManager()->getDefaultLanguage()->getId(),
                    'status'      => Media::PUBLISHED,
                    'field_image' => [
                      'target_id' => $file->id(),
                      'alt'       => !empty($tag_attributes[2][0]) ? Unicode::truncate(str_replace('"', '', $tag_attributes[2][0]), 512) : '',
                      'title'     => !empty($tag_attributes[2][0]) ? Unicode::truncate(str_replace('"', '', $tag_attributes[2][0]), 1024) : '',
                    ],
                  ]);

                  $media->save();
                  $uuid = $this->getMediaUuid($file);
                  $html = str_replace($img_tag, '<p><drupal-entity
                    data-embed-button="embed_image" 
                    data-entity-embed-display="entity_reference:media_thumbnail"
                    data-entity-embed-display-settings="{"image_style":"large","image_link":""}"
                    data-entity-type="media"
                    data-entity-uuid="' . $uuid . '"></drupal-entity>></p>', $html);
                }

              }

            }
          }
        }
      }
    }
    return $html;
  }

  /**
   * Get Media UUID by File ID.
   */
  protected function getMediaUuid(FileInterface $file) {
    $query = db_select('media__field_image', 'f', ['target' => 'default']);
    $query->innerJoin('media', 'm', 'm.mid = f.entity_id');
    $query->fields('m', ['uuid']);
    $query->condition('f.field_image_target_id', $file->id());
    $uuid = $query->execute()->fetchField();
    return $uuid;
  }

}

The process plugin code can get really nasty, and that's fine. Since this could be just a small portion of the overall migration, you don't want spend time to make it look nice and optimized. The best way to improve your code is to write more migrations and optimize it over time.

I hope this was helpful and I would love to hear about your techniques and solutions for content migration issues.

How to Prevent Duplicate Terms During a Drupal 8 Migration

Aug 03 2017
Aug 03
Aug 03 2017
Aug 03

In this post I will show a custom process plugin that I created to migrate taxonomy terms. The plugin handles the creation of new terms and prevents duplicates.

Below is a portion of the migration template. In the example, I am migrating new terms into keywords vocabulary via field_keywords field.

  field_keywords:
    -
      plugin: existing_term
      # Destination (Drupal) vocabulary name
      vocabulary: keywords
      # Source query should return term name
      source: term_name
    -
      plugin: skip_on_empty
      method: row

This is the source code for the process plugin.

<?php

namespace Drupal\my_module\Plugin\migrate\process;

use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\MigrateSkipProcessException;
use Drupal\migrate\Row;
use Drupal\taxonomy\Entity\Term;

/**
 * Check if term exists and create new if doesn't.
 *
 * @MigrateProcessPlugin(
 *   id = "existing_term"
 * )
 */
class ExistingTerm extends ProcessPluginBase {

  /**
   * {@inheritdoc}
   */
  public function transform($term_name, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    $vocabulary = $this->configuration['vocabulary'];
    if (empty($term_name)) {
      throw new MigrateSkipProcessException();
    }
    if ($tid = $this->getTidByName($term_name, $vocabulary)) {
      $term = Term::load($tid);
    }
    else {
      $term = Term::create([
        'name' => $term_name, 
        'vid'  => $vocabulary,
      ])->save();
      if ($tid = $this->getTidByName($term_name, $vocabulary)) {
        $term = Term::load($tid);
      }
    }
    return [
      'target_id' => is_object($term) ? $term->id() : 0,
    ];
  }

  /**
   * Load term by name.
   */
  protected function getTidByName($name = NULL, $vocabulary = NULL) {
    $properties = [];
    if (!empty($name)) {
      $properties['name'] = $name;
    }
    if (!empty($vocabulary)) {
      $properties['vid'] = $vocabulary;
    }
    $terms = \Drupal::entityManager()->getStorage('taxonomy_term')->loadByProperties($properties);
    $term = reset($terms);
    return !empty($term) ? $term->id() : 0;
  }

}

The logic of the plugin is very simple. Please let me know in comments or questions. Also, please share any problems you've had during your content migrations and how you solved them.

How to Migrate Posts from Wordpress to Drupal 8

Jul 28 2017
Jul 28
Jul 28 2017
Jul 28

WordPress to Drupal 8

In this post I will show you how to migrate thumbnail content from Wordpress to Drupal 8. My goals are to help you better understand the content migration process, give you starting point for future migrations, and teach you how to write process plugins and migration sources. Taxonomy terms and users migration is more straightforward so I won't cover it here.

This migration example contains templates to migrate thumbnails content. For this post, I assume the image/thumbnail field is using the Media module field. I will be using the Migrate drush module to run migrations.

First, make sure to configure your connection in your settings.php file. Add the following with proper credentials:

$databases['migrate']['default'] = [
    'driver'   => 'mysql',
    'database' => 'wordpress_dbname',
    'username' => 'wordpress_dbuser',
    'password' => 'wordpress_dbpassowrd',
    'host'     => '127.0.0.1',
];

Here is the module structure I will be using:

wp_migration/
  - wp_migration.info.yml
  - wp_migration.module
  - migration_templates/
    - wp_content.yml
    - wp_thumbnail.yml
    - wp_media.yml
  - src/
    - Plugin/
      - migrate/
        - process/
          - AddUrlAliasPrefix.php
          - DateToTimestamp.php
        - source/
          - SqlBase.php
          - Content.php
          - Thumbnail.php

Contents of the wp_wordpress.info.yml file:

name: Wordpress Migration
type: module
description: Migrate Wordpress content into Drupal 8.
core: 8.x
dependencies:
  - migrate
  - migrate_drupal
  - migrate_drush

Contents of the migration_templates/wp_thumbnail.yml file:

id: wp_thumbnail
label: 'Thumbnails'
migration_tags:
  - Wordpress
source:
  plugin: wordpress_thumbnail
  # This is WP table prefix  (custom variable)
  # DB table example: [prefix]_posts
  table_prefix: wp
  constants:
    # This path should point ot WP uploads directory.
    source_base_path: '/path/to/source/wp/uploads'
    # This is directory name in Drupal where to store migrated files
    uri_file: 'public://wp-thumbnails'
process:
  filename: filename
  source_full_path:
    -
      plugin: concat
      delimiter: /
      source:
        - constants/source_base_path
        - filepath
    -
      plugin: urlencode
  uri_file:
    -
      plugin: concat
      delimiter: /
      source:
        - constants/uri_file
        - filename
    -
      plugin: urlencode
  uri:
    plugin: file_copy
    source:
      - '@source_full_path'
      - '@uri_file'
  status: 
    plugin: default_value
    default_value: 1
  changed: 
    plugin: date_to_timestamp
    source: post_date
  created: 
    plugin: date_to_timestamp
    source: post_date
  uid: 
    plugin: default_value
    default_value: 1
destination:
  plugin: 'entity:file'
migration_dependencies:
  required: {}
  optional: {}

Contents of the migration_templates/wp_media.yml file:

id: wp_media
label: 'Media'
migration_tags:
  - Wordpress
source:
  plugin: wordpress_thumbnail
  # This is WP table prefix  (custom variable)
  # DB table example: [prefix]_posts
  table_prefix: wp
  constants:
    bundle: image
process:
  bundle: 'constants/bundle'
  langcode:
    plugin: default_value
    default_value: en
  'field_image/target_id':
    -
      plugin: migration
      migration: wp_thumbnail
      source: post_id
    -
      plugin: skip_on_empty
      method: row
destination:
  plugin: 'entity:media'
migration_dependencies:
  required: {}
  optional: {}

Contents of the migration_templates/wp_content.yml file:

id: wp_content
label: 'Content'
migration_tags:
  - Wordpress
source:
  plugin: wordpress_content
  # Wordpress post type (custom variable)
  post_type: post
  # This is WP table prefix  (custom variable)
  # DB table example: [prefix]_posts
  table_prefix: wp
process:
  type:
    plugin: default_value
    default_value: article
  'path/pathauto':
    plugin: default_value
    default_value: 0
  'path/alias':
    # This will add the following to URL aliases in Drupal
    plugin: add_url_alias_prefix
    # url/alias/prefix/2017/07/21/[post-title]
    prefix: url/alias/prefix
    source: path_alias
  promote: 
    plugin: default_value
    default_value: 0
  sticky: 
    plugin: default_value
    default_value: 0
  langcode:
    plugin: default_value
    default_value: en
  status: 
    plugin: default_value
    default_value: 1
  title: post_title
  created: 
    plugin: date_to_timestamp
    source: post_date
  changed: 
    plugin: date_to_timestamp
    source: post_modified
  field_image:
    -
      plugin: migration
      migration: wp_media
      source: thumbnail
    -
      plugin: skip_on_empty
      method: row
  'body/summary': post_excerpt
  'body/format':
    plugin: default_value
    default_value: full_html
  'body/value': post_content
destination:
  plugin: 'entity:node'
migration_dependencies:
  required: {}
  optional: {}

Contents of the src/Plugin/migrate/process/AddUrlAliasPrefix.php file:

<?php

namespace Drupal\wp_migration\Plugin\migrate\process;

use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\Row;

/**
 * Add prefix to URL aliases.
 *
 * @MigrateProcessPlugin(
 *   id = "add_url_alias_prefix"
 * )
 */
class AddUrlAliasPrefix extends ProcessPluginBase {

  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    $prefix = !empty($this->configuration['prefix']) ? '/' . $this->configuration['prefix'] : '';
    return $prefix . $value;
  }

}

Contents of the src/Plugin/migrate/process/DateToTimestamp.php file:

<?php

namespace Drupal\wp_migration\Plugin\migrate\process;

use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\Row;

/**
 * Date to Timetamp conversion.
 *
 * @MigrateProcessPlugin(
 *   id = "date_to_timestamp"
 * )
 */
class DateToTimestamp extends ProcessPluginBase {

  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    return strtotime($value . ' UTC');
  }

}

I like to keep my module code clean and organized so I use base classes that I later extend in individual migration source files.

Here is contents of the src/Plugin/migrate/source/SqlBase.php file:

<?php

namespace Drupal\wp_migration\Plugin\migrate\source;

use Drupal\migrate_drupal\Plugin\migrate\source\DrupalSqlBase;
use Drupal\migrate\Row;

class SqlBase extends DrupalSqlBase {

  /**
   * Get database table prefix from the migration template.
   */
  protected function getPrefix() {
    return !empty($this->configuration['table_prefix']) ? $this->configuration['table_prefix'] : 'wp';
  }

  /**
   * Get Wordpress post type from the migration template.
   */
  protected function getPostType() {
    return !empty($this->configuration['post_type']) ? $this->configuration['post_type'] : 'post';
  }

  /**
   * Generate path alias via pattern specified in `permalink_structure`.
   */
  protected function generatePathAlias(Row $row) {
    $prefix = $this->getPrefix();
    $permalink_structure = $this->select($prefix . '_options', 'o', ['target' => 'migrate'])
      ->fields('o', ['option_value'])
      ->condition('o.option_name', 'permalink_structure')
      ->execute()
      ->fetchField();
    $date = new \DateTime($row->getSourceProperty('post_date'));
    $parameters = [
      '%year%'     => $date->format('Y'),
      '%monthnum%' => $date->format('m'),
      '%day%'      => $date->format('d'),
      '%postname%' => $row->getSourceProperty('post_name'),
    ];
    $url = str_replace(array_keys($parameters), array_values($parameters), $permalink_structure);
    return rtrim($url, '/');
  }

  /**
   * Get post thumbnail.
   */
  protected function getPostThumbnail(Row $row) {
    $prefix = $this->getPrefix();
    $query = $this->select($prefix . '_postmeta', 'pm', ['target' => 'migrate']);
    $query->innerJoin($prefix . '_postmeta', 'pm2', 'pm2.post_id = pm.meta_value');
    $query->fields('pm', ['post_id'])
      ->condition('pm.post_id', $row->getSourceProperty('id'))
      ->condition('pm.meta_key', '_thumbnail_id')
      ->condition('pm2.meta_key', '_wp_attached_file');
    return $query->execute()->fetchField();
  }

}

Contents of the src/Plugin/migrate/source/Thumbnail.php file:

<?php

namespace Drupal\wp_migration\Plugin\migrate\source;

use Drupal\migrate\Row;

/**
 * Extract content thumbnails.
 *
 * @MigrateSource(
 *   id = "wordpress_thumbnail"
 * )
 */
class Thumbnail extends SqlBase {

  /**
   * {@inheritdoc}
   */
  public function query() {
    $prefix = $this->getPrefix();
    $query = $this->select($prefix . '_postmeta', 'pm', ['target' => 'migrate']);
    $query->innerJoin($prefix . '_postmeta', 'pm2', 'pm2.post_id = pm.meta_value');
    $query->innerJoin($prefix . '_posts', 'p', 'p.id = pm.post_id');
    $query->fields('pm', ['post_id']);
    $query->fields('p', ['post_date']);
    $query->addField('pm2', 'post_id', 'file_id');
    $query->addField('pm2', 'meta_value', 'filepath');
    $query
      ->condition('pm.meta_key', '_thumbnail_id')
      ->condition('pm2.meta_key', '_wp_attached_file')
      ->condition('p.post_status', 'publish')
      ->condition('p.post_type', 'post');
    return $query;
  }

  /**
   * {@inheritdoc}
   */
  public function fields() {
    return [
      'post_id'   => $this->t('Post ID'),
      'post_date' => $this->t('Media Uploaded Date'),
      'file_id'   => $this->t('File ID'),
      'filepath'  => $this->t('File Path'),
      'filename'  => $this->t('File Name'),
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function getIds() {
    return [
      'post_id' => [
        'type'  => 'integer',
        'alias' => 'pm2',
      ],
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function prepareRow(Row $row) {
    $row->setSourceProperty('filename', basename($row->getSourceProperty('filepath')));
  }

}

Contents of the src/Plugin/migrate/source/Content.php file:

<?php

namespace Drupal\wp_migration\Plugin\migrate\source;

use Drupal\migrate\Row;

/**
 * Extract content from Wordpress site.
 *
 * @MigrateSource(
 *   id = "wordpress_content"
 * )
 */
class Content extends SqlBase {

  /**
   * {@inheritdoc}
   */
  public function query() {
    $prefix = $this->getPrefix();
    $query = $this->select($prefix . '_posts', 'p');
    $query
      ->fields('p', [
        'id',
        'post_date',
        'post_title',
        'post_content',
        'post_excerpt',
        'post_modified',
        'post_name'
      ]);
    $query->condition('p.post_status', 'publish');
    $query->condition('p.post_type', $this->getPostType());
    return $query;
  }

  /**
   * {@inheritdoc}
   */
  public function fields() {
    return [
      'id'            => $this->t('Post ID'),
      'post_title'    => $this->t('Title'),
      'thumbnail'     => $this->t('Post Thumbnail'),
      'post_excerpt'  => $this->t('Excerpt'),
      'post_content'  => $this->t('Content'),
      'post_date'     => $this->t('Created Date'),
      'post_modified' => $this->t('Modified Date'),
      'path_alias'    => $this->t('URL Alias'),
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function getIds() {
    return [
      'id' => [
        'type'  => 'integer',
        'alias' => 'p',
      ],
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function prepareRow(Row $row) {
    // This will generate path alias using WP alias settings.
    $row->setSourceProperty('path_alias', $this->generatePathAlias($row));
    // Get thumbnail ID and pass it to the wp_media migration plugin.
    $row->setSourceProperty('thumbnail', $this->getPostThumbnail($row));
  }

}

IMPORTANT: You must run migrations in their proper order. In this example you have to run wp_thumbnail first, wp_media second and wp_content last.

Comments Not Loading?

Due to some temporarily SSL cert issue please refresh the page using this link in order to be able to leave comments.

Jul 27 2017
Jul 27

WordPress to Drupal 8

In this post I will show you how to migrate thumbnail content from Wordpress to Drupal 8. My goals are to help you better understand the content migration process, give you starting point for future migrations, and teach you how to write process plugins and migration sources. Taxonomy terms and users migration is more straightforward so I won't cover it here.

This migration example contains templates to migrate thumbnails content. For this post, I assume the image/thumbnail field is using the Media module field. I will be using the Migrate drush module to run migrations.

First, make sure to configure your connection in your settings.php file. Add the following with proper credentials:

$databases['migrate']['default'] = [
    'driver'   => 'mysql',
    'database' => 'wordpress_dbname',
    'username' => 'wordpress_dbuser',
    'password' => 'wordpress_dbpassowrd',
    'host'     => '127.0.0.1',
];

Here is the module structure I will be using:

wp_migration/
  - wp_migration.info.yml
  - wp_migration.module
  - migration_templates/
    - wp_content.yml
    - wp_thumbnail.yml
    - wp_media.yml
  - src/
    - Plugin/
      - migrate/
        - process/
          - AddUrlAliasPrefix.php
          - DateToTimestamp.php
        - source/
          - SqlBase.php
          - Content.php
          - Thumbnail.php

Contents of the wp_wordpress.info.yml file:

name: Wordpress Migration
type: module
description: Migrate Wordpress content into Drupal 8.
core: 8.x
dependencies:
  - migrate
  - migrate_drupal
  - migrate_drush

Contents of the migration_templates/wp_thumbnail.yml file:

id: wp_thumbnail
label: 'Thumbnails'
migration_tags:
  - Wordpress
source:
  plugin: wordpress_thumbnail
  # This is WP table prefix  (custom variable)
  # DB table example: [prefix]_posts
  table_prefix: wp
  constants:
    # This path should point ot WP uploads directory.
    source_base_path: '/path/to/source/wp/uploads'
    # This is directory name in Drupal where to store migrated files
    uri_file: 'public://wp-thumbnails'
process:
  filename: filename
  source_full_path:
    -
      plugin: concat
      delimiter: /
      source:
        - constants/source_base_path
        - filepath
    -
      plugin: urlencode
  uri_file:
    -
      plugin: concat
      delimiter: /
      source:
        - constants/uri_file
        - filename
    -
      plugin: urlencode
  uri:
    plugin: file_copy
    source:
      - '@source_full_path'
      - '@uri_file'
  status: 
    plugin: default_value
    default_value: 1
  changed: 
    plugin: date_to_timestamp
    source: post_date
  created: 
    plugin: date_to_timestamp
    source: post_date
  uid: 
    plugin: default_value
    default_value: 1
destination:
  plugin: 'entity:file'
migration_dependencies:
  required: {}
  optional: {}

Contents of the migration_templates/wp_media.yml file:

id: wp_media
label: 'Media'
migration_tags:
  - Wordpress
source:
  plugin: wordpress_thumbnail
  # This is WP table prefix  (custom variable)
  # DB table example: [prefix]_posts
  table_prefix: wp
  constants:
    bundle: image
process:
  bundle: 'constants/bundle'
  langcode:
    plugin: default_value
    default_value: en
  'field_image/target_id':
    -
      plugin: migration
      migration: wp_thumbnail
      source: post_id
    -
      plugin: skip_on_empty
      method: row
destination:
  plugin: 'entity:media'
migration_dependencies:
  required: {}
  optional: {}

Contents of the migration_templates/wp_content.yml file:

id: wp_content
label: 'Content'
migration_tags:
  - Wordpress
source:
  plugin: wordpress_content
  # Wordpress post type (custom variable)
  post_type: post
  # This is WP table prefix  (custom variable)
  # DB table example: [prefix]_posts
  table_prefix: wp
process:
  type:
    plugin: default_value
    default_value: article
  'path/pathauto':
    plugin: default_value
    default_value: 0
  'path/alias':
    # This will add the following to URL aliases in Drupal
    plugin: add_url_alias_prefix
    # url/alias/prefix/2017/07/21/[post-title]
    prefix: url/alias/prefix
    source: path_alias
  promote: 
    plugin: default_value
    default_value: 0
  sticky: 
    plugin: default_value
    default_value: 0
  langcode:
    plugin: default_value
    default_value: en
  status: 
    plugin: default_value
    default_value: 1
  title: post_title
  created: 
    plugin: date_to_timestamp
    source: post_date
  changed: 
    plugin: date_to_timestamp
    source: post_modified
  field_image:
    -
      plugin: migration
      migration: wp_media
      source: thumbnail
    -
      plugin: skip_on_empty
      method: row
  'body/summary': post_excerpt
  'body/format':
    plugin: default_value
    default_value: full_html
  'body/value': post_content
destination:
  plugin: 'entity:node'
migration_dependencies:
  required: {}
  optional: {}

Contents of the src/Plugin/migrate/process/AddUrlAliasPrefix.php file:

<?php

namespace Drupal\wp_migration\Plugin\migrate\process;

use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\Row;

/**
 * Add prefix to URL aliases.
 *
 * @MigrateProcessPlugin(
 *   id = "add_url_alias_prefix"
 * )
 */
class AddUrlAliasPrefix extends ProcessPluginBase {

  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    $prefix = !empty($this->configuration['prefix']) ? '/' . $this->configuration['prefix'] : '';
    return $prefix . $value;
  }

}

Contents of the src/Plugin/migrate/process/DateToTimestamp.php file:

<?php

namespace Drupal\wp_migration\Plugin\migrate\process;

use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\Row;

/**
 * Date to Timetamp conversion.
 *
 * @MigrateProcessPlugin(
 *   id = "date_to_timestamp"
 * )
 */
class DateToTimestamp extends ProcessPluginBase {

  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    return strtotime($value . ' UTC');
  }

}

I like to keep my module code clean and organized so I use base classes that I later extend in individual migration source files.

Here is contents of the src/Plugin/migrate/source/SqlBase.php file:

<?php

namespace Drupal\wp_migration\Plugin\migrate\source;

use Drupal\migrate_drupal\Plugin\migrate\source\DrupalSqlBase;
use Drupal\migrate\Row;

class SqlBase extends DrupalSqlBase {

  /**
   * Get database table prefix from the migration template.
   */
  protected function getPrefix() {
    return !empty($this->configuration['table_prefix']) ? $this->configuration['table_prefix'] : 'wp';
  }

  /**
   * Get Wordpress post type from the migration template.
   */
  protected function getPostType() {
    return !empty($this->configuration['post_type']) ? $this->configuration['post_type'] : 'post';
  }

  /**
   * Generate path alias via pattern specified in `permalink_structure`.
   */
  protected function generatePathAlias(Row $row) {
    $prefix = $this->getPrefix();
    $permalink_structure = $this->select($prefix . '_options', 'o', ['target' => 'migrate'])
      ->fields('o', ['option_value'])
      ->condition('o.option_name', 'permalink_structure')
      ->execute()
      ->fetchField();
    $date = new \DateTime($row->getSourceProperty('post_date'));
    $parameters = [
      '%year%'     => $date->format('Y'),
      '%monthnum%' => $date->format('m'),
      '%day%'      => $date->format('d'),
      '%postname%' => $row->getSourceProperty('post_name'),
    ];
    $url = str_replace(array_keys($parameters), array_values($parameters), $permalink_structure);
    return rtrim($url, '/');
  }

  /**
   * Get post thumbnail.
   */
  protected function getPostThumbnail(Row $row) {
    $prefix = $this->getPrefix();
    $query = $this->select($prefix . '_postmeta', 'pm', ['target' => 'migrate']);
    $query->innerJoin($prefix . '_postmeta', 'pm2', 'pm2.post_id = pm.meta_value');
    $query->fields('pm', ['post_id'])
      ->condition('pm.post_id', $row->getSourceProperty('id'))
      ->condition('pm.meta_key', '_thumbnail_id')
      ->condition('pm2.meta_key', '_wp_attached_file');
    return $query->execute()->fetchField();
  }

}

Contents of the src/Plugin/migrate/source/Thumbnail.php file:

<?php

namespace Drupal\wp_migration\Plugin\migrate\source;

use Drupal\migrate\Row;

/**
 * Extract content thumbnails.
 *
 * @MigrateSource(
 *   id = "wordpress_thumbnail"
 * )
 */
class Thumbnail extends SqlBase {

  /**
   * {@inheritdoc}
   */
  public function query() {
    $prefix = $this->getPrefix();
    $query = $this->select($prefix . '_postmeta', 'pm', ['target' => 'migrate']);
    $query->innerJoin($prefix . '_postmeta', 'pm2', 'pm2.post_id = pm.meta_value');
    $query->innerJoin($prefix . '_posts', 'p', 'p.id = pm.post_id');
    $query->fields('pm', ['post_id']);
    $query->fields('p', ['post_date']);
    $query->addField('pm2', 'post_id', 'file_id');
    $query->addField('pm2', 'meta_value', 'filepath');
    $query
      ->condition('pm.meta_key', '_thumbnail_id')
      ->condition('pm2.meta_key', '_wp_attached_file')
      ->condition('p.post_status', 'publish')
      ->condition('p.post_type', 'post');
    return $query;
  }

  /**
   * {@inheritdoc}
   */
  public function fields() {
    return [
      'post_id'   => $this->t('Post ID'),
      'post_date' => $this->t('Media Uploaded Date'),
      'file_id'   => $this->t('File ID'),
      'filepath'  => $this->t('File Path'),
      'filename'  => $this->t('File Name'),
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function getIds() {
    return [
      'post_id' => [
        'type'  => 'integer',
        'alias' => 'pm2',
      ],
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function prepareRow(Row $row) {
    $row->setSourceProperty('filename', basename($row->getSourceProperty('filepath')));
  }

}

Contents of the src/Plugin/migrate/source/Content.php file:

<?php

namespace Drupal\wp_migration\Plugin\migrate\source;

use Drupal\migrate\Row;

/**
 * Extract content from Wordpress site.
 *
 * @MigrateSource(
 *   id = "wordpress_content"
 * )
 */
class Content extends SqlBase {

  /**
   * {@inheritdoc}
   */
  public function query() {
    $prefix = $this->getPrefix();
    $query = $this->select($prefix . '_posts', 'p');
    $query
      ->fields('p', [
        'id',
        'post_date',
        'post_title',
        'post_content',
        'post_excerpt',
        'post_modified',
        'post_name'
      ]);
    $query->condition('p.post_status', 'publish');
    $query->condition('p.post_type', $this->getPostType());
    return $query;
  }

  /**
   * {@inheritdoc}
   */
  public function fields() {
    return [
      'id'            => $this->t('Post ID'),
      'post_title'    => $this->t('Title'),
      'thumbnail'     => $this->t('Post Thumbnail'),
      'post_excerpt'  => $this->t('Excerpt'),
      'post_content'  => $this->t('Content'),
      'post_date'     => $this->t('Created Date'),
      'post_modified' => $this->t('Modified Date'),
      'path_alias'    => $this->t('URL Alias'),
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function getIds() {
    return [
      'id' => [
        'type'  => 'integer',
        'alias' => 'p',
      ],
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function prepareRow(Row $row) {
    // This will generate path alias using WP alias settings.
    $row->setSourceProperty('path_alias', $this->generatePathAlias($row));
    // Get thumbnail ID and pass it to the wp_media migration plugin.
    $row->setSourceProperty('thumbnail', $this->getPostThumbnail($row));
  }

}

IMPORTANT: You must run migrations in their proper order. In this example you have to run wp_thumbnail first, wp_media second and wp_content last.

Comments Not Loading?

Due to some temporarily SSL cert issue please refresh the page using this link in order to be able to leave comments.

How to Implement AppNexus ads in Drupal 8

Jul 18 2017
Jul 18

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web