Sep 29 2015
Sep 29

While I couldn’t make it to Drupalcon Barcelona last week, I insisted on giving a presentation somewhere. Forum One let me do it as a webinar: Planning and Building Salesforce Integration. There are some truly wonderful tools for integrating Salesforce with Drupal out there, but the tricky part is planning, documenting, and estimating the task.

My webinar presentation doesn’t cover the basics of Salesforce or Drupal. Rather, it is tailored for a certain (basic) level of knowledge about both systems: what they are, how to set them up, and the basic data models. It covers the toolset you need, its strengths and weaknesses, how to plan an integration, and what kind of amazing cool stuff you can do if you’re smart about it.


[embedded content]

Salesforce is an extremely powerful CRM platform, and that includes options for external integrations. There are two different APIs available, but basically all you need to do is wrap requests in an OAuth2 session, and query the Salesforce data with their own query language, SOQL.

That said, I would never send you off to go and write totally custom integration code. There’s already a fantastic suite of modules written for Drupal that provide this base functionality and more: the Salesforce Suite. The Suite is actually a set of 5 modules, providing an OAuth2 wrapper and SOQL query builder, two different ways to connect to Salesforce, an interface to map Salesforce objects to Drupal objects, and separate modules for both pushing data into Salesforce, and pulling data from it. It’s a pretty complete package, and it is an excellent base for even highly-customized integrations.

Out of the box, the Salesforce Suite also does an excellent job of direct, 1:1 mappings between Salesforce Objects and Drupal entities. It schedules and queues synchronizations well, and offers a lot of hooks to help with your custom development. When you’re planning your integration, you have to look out for “red flags” that will help you estimate:

  • Conceptual objects in Drupal that are actually made of two or more Entities, e.g., users with their profile2 objects.
  • Modules that don’t store their data in entities, e.g., the Webform module.
  • A wide variety of different types of fields. Many fields actually need some translation between Drupal and Salesforce, and you will have to provide that yourself.

Very often, custom code is the best way to augment the synchronization that you’ve built in the Drupal UI. The following are some of my favorite examples:

  • Salesforce stores country names in a non-ISO format. If you’re syncing to a Drupal location field, you will have to do a little bit of translating.
  • Salesforce booleans (checkboxes) are stored differently from Drupal booleans. Again, a little bit of translation is required.
  • Salesforce doesn’t have the same validation rules as Drupal, e.g., Salesforce Contacts are not required to have an email address, but Drupal users are.

All this by way of saying that custom code is almost always required for an integration to go smoothly. It doesn’t have to be complicated code, but it is there.

With all this in mind, I recommend planning your integration around use cases and user stories for both systems. We paint a picture of the people who will be using both the Drupal and Salesforce sides of this, and what information they want to have readily available. In most cases it boils down to a long list of objects and fields to integrate. You can use this list to estimate based on the number of different field types, number of 1:many object mappings, etc. You can also use it to consider large-scale architectural options, like the decision to install the Redhen Drupal CRM to map all your data into these entities, and then sync from there.

Another important question (brought up by a participant during the live broadcast) is whether to keep your data customizations in custom Salesforce fields, or in Drupal custom code. Salesforce allows you to create custom fields that are filled automatically according to rules that you configure. So that country name problem I mentioned above could be solved by syncing Drupal with a custom Salesforce field that translates the names for you. That saves you the trouble of coding in Drupal, which otherwise seems like it would be hard to maintain. As a general policy I prefer to keep “code-like” customizations in the site’s codebase, where I can then easily find them in case of a problem or change. It can save serious headaches down the road! For those of you who are more comfortable in Salesforce, however, the same reasoning might push you towards the opposite decision. You should take the route that seems the most maintainable to you, and not worry about what some guy on the Internet said in a blog post ????

The webinar also includes a quick walkthrough of a simple integration between Drupal and Salesforce, synchronizing users and contacts. We right away run into some of those very predictable problems that our estimation process was designed to root out, and we talk about how to solve them.

My favorite part of the presentation (if I do say so myself!) is the “blue sky” section near the end. Salesforce and Drupal are each incredibly powerful, flexible platforms on their own, and so I really enjoy coming up with cool ways to have them magnify each others’ power. Some fun ones:

  • If you include Salesforce data in Drupal user objects, you can do A/B testing in Drupal based on peoples’ engagement in other channels.
  • You can tag users in your analytics package based on their Salesforce profiles. Imagine a special analytics report of how your donors behave on the site, or simply people who opened the last newsletter, or whatever other user group you can imagine.
  • You can track user engagements from your website in Salesforce. For example, including a user’s comments on blog posts as a part of their Salesforce history.
  • If your site is a user community with points, referrals, and other rewards for interaction, those should definitely be reflected in Salesforce profiles.
  • (My #1 favorite) You can integrate information from social media, email blasts, human contact, and the website to provide little surprises and personalizations for users. Imagine a message like, “we saw you shared our blog post on Facebook – thanks!,” or, “have you checked out the newest version of our product yet?”

We had some great questions come up in the Q&A section, as well. We talked a bit about using machine learning (and Salesforce’s new Predictive Decisions feature) to drive website content decisions, and potential future areas for development in the Salesforce suite around the Salesforce metadata API. Cool stuff, but you’ll have to watch the video to get it.

The Salesforce suite offers an excellent foundation for building Salesforce integrations. As always though, the really exciting stuff is on the cutting edge of what the industry offers. To take advantage of that, you’ll need more than just a small custom module, and you’ll also need serious strategic and developer leadership. I’m looking forward to having that conversation with you soon!

Previous Post

Coding vs Clicking and Building Layouts from 7 to 8 – More Fun at DrupalCon Barcelona

May 14 2015
May 14

My colleague Adam Juran and I just finished with our session, Zero to MVP in 40 minutes: Coder and Themer Get Rich Quick in Silicon Valley, at DrupalCon LA. This one was a real journey to prepare, and through it we learned a lot of dirty truths about Drupal 8, Javascript frameworks, and the use cases where the two make sense together.

The live coding challenge in our session proposal seemed simple: create a web application that ingests content from an external API, performs content management tasks (data modelling, relationships, etc.) through the Drupal 8 interface, and deliver it all to an AngularJS front-end. This is exactly the “headless Drupal” stuff that everyone has been so excited about for the last year, so doing it in a 40 minute head-to-head code battle seemed like an entertaining session.

Ingesting content from an external API

The first hard truth we discovered was the limitations of the still-nascent Drupal 8. Every monthly release of a new Drupal 8 beta includes a series of “change records,” defining all the system-wide changes that will have to be accounted for everywhere else. For example, one change record notes that a variable we often use in Drupal forms is now a different kind of object. This change breaks every single form, everywhere in Drupal.

The frequency of this kind of change record is a problem for anyone who tries to maintain a contributed module. No one can keep up with their code breaking every month, so most don’t. The module works when they publish it as “stable”, but two or three months later, it’s fundamentally broken. changes like this currently happen 10-15 times every month. Any module we were hoping to use as a part of this requirement – Twitter, Oauth, Facebook – were broken when we started testing.

We finally settled on using Drupal’s robust Migrate module to bring in external content. After all, Drupal 7 Migrate can import content from almost any format! Turns out that this isn’t the case with Drupal 8 core’s Migrate module. It’s limited to the basic framework you need for all migrations. Importers for various file types and sources simply haven’t been written yet.

No matter which direction we turned, we were faced with the fact that Drupal 8 needed work to perform the first requirement in our challenge. We chose to create a CSV Source plugin ourselves (with much help from mikeryan and chx) just to be able to meet this requirement. This was not something we could show in the presentation; it was only a prerequisite. Phew!

Displaying it All in Angular

Building an AngularJS based front end for this presentation involved making decisions about architecture, which ended up as the critical focus of our talk. AngularJS is a complete framework, which normally handles the entire application: data ingestion, manipulation, and display. Why would you stick Drupal in there? And what would an Angular application look like architecturally, with Drupal 8 inside?

You always have a choice of what to do and where to do it. Either system can ingest data, and either system can do data manipulation. Your decision should be based on which tool does each job the best, in your particular use case: a catch-all phrase that includes factors like scalability and depth of functionality, but also subtler elements like the expertise of your team. If you have a shop full of AngularJS people and a simple use case, you should probably build the entire app in Angular!

Given that perspective, Drupal really stands out as a data ingestion and processing engine. Even when you have to write a new Migration source plugin, the Entity model, Drupal’s “plug-ability”, and Views make data crunching extremely easy. This is a strong contrast to data work in Angular, where you have to write everything from scratch.

We feel that the best combination of Drupal and Angular is with Drupal ingesting content, manipulating it, and spitting it out in a ready-to-go format for AngularJS to consume. This limits the Angular application to its strengths: layout, with data from a REST back-end, and only simple logic.

The Session

[embedded content]

In the session, we talked a bit about the larger concepts involved, and moved fairly quickly into the technical demonstration. First, Adam demonstrated the flexibility of the decoupled front-end, using bower libraries to build an attractive layout without writing a single line of custom CSS.  Then I demonstrated importing data from CSV sources into Drupal 8, along with the simplicity of configuring Drupal Views to output JSON. Taken together, the videos are 37 minutes long – not bad for a totally custom RESTful endpoint and a nice looking front-end!

Here is Adam’s screencast, showing off the power of the bootstrap-material-design library to build a good looking site without any custom CSS at all:

Here is my screencast, demonstrating how easy it is to create Migrate module importers and REST exports in Drupal 8.

And here is the final screencast, quickly showing the changes we made in AngularJS to have it call the two Drupal Services.

Want to learn of Forum One’s Drupal development secrets? Check out our other Drupalcon blog posts, or visit our booth (#107) and talk with our tech wizards in person!

Previous Post

DrupalCon LA Day 1!

Next Post

Hacking the Feds: Forum One Among the Winners at GSA Hack-a-Thon

May 13 2015
May 13

DrupalCon day one was a great start to this year’s North American Drupal conference! Forum One is well represented this year, giving seven presentations this week.

The Con started off with the traditional “pre-note” show in the early morning. The pre-note is a session designed to get people out of their seats and into the feeling of this big, welcoming community. Jam McGuire, Robert Douglass,

via <a href=

via Mendel: Forum One’s Kristina Bjoran leads the Prenote finale. From left: Jeffrey McGuire, Larry Garfield, Campbell Vertesi, Adam Juran, Dries Buytaert, Ronai Brumett, Aaron Porter

Forum One’s Adam Juran and I have been putting these together for a few years now, and for DrupalCon LA we wrote a Disney musical about Drupal. From Ariel’s song “Part of My Site” to our own version of Into the Woods’ “Agony,” the show got a lot of laughs with its parody lyrics. One high point was Dries, the founder of Drupal, entering the stage with top hat and cane to perform, “When you install Drupal 8? to the tune of “When You Wish Upon a Star” – ending prematurely with a fatal error! This was followed by “Someday D8 Will Come”, and a lot of laughs. The prenote ended with Forum One’s Kristina Bjoran leading the audience in a DrupalCon version of “Let It Go” from Frozen. After all the laughter, it was a nice moment to hear the audience cheer in unison: “we come for code, but we stay for community.”

[embedded content]

Dries’ keynote came next. This year he didn’t talk so much about the great new features of Drupal 8 – we’ve been talking about that for four years now! Instead, he focused on the history of Drupal as a platform, starting in his dorm room in 2001. Once we got to the present day, he switched to the coming challenges in the web sector. The Internet is becoming less and less about browser-based interaction, according to Dries. Increasingly people access data using tailored apps or devices, which means there is a great need for a data back-end like Drupal that can provide for all of these end points. Consumers demand more and more customized and predictive content, and Drupal 8 is a strong platform for that capability.

The day was filled with interesting sessions, but a few stuck out to us. There was Amitai and Josh Koenig’s Decoupled Drupal talk, where they demonstrated an automatic headless Drupal site generator. There were a couple of interesting sessions about long form content: the technical side by Murray Woodman and Jeff Eaton, and the strategic side by Forum One’s Kristina Bjoran and Courtney Clark. Courtney had a double-header day: she also presented about Forum One’s work on content strategy for Drupal.org. I got to present with Adam Juran and Jam McGuire about headless Drupal, building a simple Drupal 8 backed AngularJS demonstration in 40 minutes. We learned a lot about various prototyping tools, and were surprised to find no clear consensus on a standard toolkit for this important problem. Forum One resources were asked a lot of questions about how we use Pattern Lab in this space. Forum One’s Daniel Ferro and Dan Mouyard have a session about Pattern Lab on Thursday.

[embedded content]

[embedded content]

[embedded content]

Be sure to keep checking back for more of our takeaways and recaps of DrupalCon LA.

The cast of the prenote: Dries Buytaert, Aaron Porter, Ben Finklea, Robert Douglass, Adam Juran, Campbell Vertesi, Jeremy Thorson, Kristina Bjoran, Ronai Brumei, Larry Garfield, Jeffrey

via Mendel: The cast of the prenote, from top left: Dries Buytaert, Aaron Porter, Ben Finklea, Robert Douglass, Adam Juran, Campbell Vertesi, Jeremy Thorson, Kristina Bjoran, Ronai Brumett, Larry Garfield, Jeffrey McGuire

Previous Post

Google to Non-Mobile sites: ‘You’re Dead to Me’

Next Post

Zero to MVP in 40 Minutes: What We learned Building Headless Drupal 8 for DrupalCon LA

Sep 30 2014
Sep 30

image05

Today started out bright and early for our Forum One team, setting up for our part in the famous DrupalCon Prenote. This is one of the best-known “secrets” of DrupalCon. As Drupal founder Dries Buytaert puts it, “If you only get up early once during DrupalCon, this is the morning to do it.” In past years we’ve taught the audience how to pour beer (DrupalCon Munich), conducted the crowd in the “Drupal Opera” (DrupalCon Prague), and explored the funny and strange talents of the Drupal community (DrupalCon Portland). Of course, no one could forget our famous Coder/Themer Wonder Twins appearance at the Drupal Superheroes Prenote from DrupalCon Austin!

This year, the Prenote theme was Drupal memories. We heard from many of the famous Drupal core contributors about how they became involved in the community and how it ultimately changed their lives. A beautiful highlight was Nancy Beers sharing the romantic video her husband sent her from Drupal Camp in Seville, shortly after they met at DrupalCon in London. After showing the video, Nancy got down on one knee on stage and proposed!

Adam and I got to re-enact the founding of Acquia, one of Drupal’s biggest service providers. We re-enacted that first partnership between Dries Buytaert and Jay Batson in a great Star Wars-themed parody. “Join me, and together we can rule the Internets as CEO and CTO,” intoned Jay in a Darth Vader mask. The audience loved it, and, of course, Adam and I thoroughly enjoyed our parts as well.

image04

Campbell and Bryn Vertesi sing Drupal MemoriesAt the end of the reminiscing, we directed the audience to stand up and take “selfies” of themselves with the stage in the background, while the core contributors up front took their own “selfies” to match. Then I took the microphone with my opera singing, Drupalist wife, Bryn Vertesi, to sing a Drupal-lyrics version of “Memories”, from the musical CATS. “Once we’re Beta, you’ll understand what happiness is,” became the catchphrase for the day!

DrupalCon Selfies

The Dries keynote was exciting as well, mostly because of the announcement that Drupal 8 is going to Beta at the end of the convention! This is great news for developers and clients alike, as the Drupal 8 API brings enormous improvements in flexibility, scalability, and usability. Forum One’s own Kalpana Goel has been hard at work, not just helping to write Drupal 8, but mentoring others as well. She spent her day in the sprint room, where the core contributors mixed celebrating the milestone with planning sessions for the next development phase.

image03

Today I also got to try out a new session, introducing the fundamental layout concepts in Drupal 7 and 8, and teaching people how to combine them for the best effect. Panels, Display Suite, and Context – oh my! ran overtime with a full room, and finally I decided we had to move the discussion to a “Birds of a Feather” workshop, tomorrow. I’m looking forward to it!

This was a long and eventful day for us here at DrupalCon Amsterdam. We’ll finish it off with a well-deserved beer at one of Holland’s famous breweries, hopefully somewhere along one of the many beautiful canals that dot this city. We’ll report back with more tomorrow!

DrupalCon Amsterdam

Read our other updates for DrupalCon:
DrupalCon Amsterdam, Day 3: Drupal 8 Beta Released
DrupalCon Amsterdam, Day 4: Our Kung fu is more powerful than yours!
DrupalCon Amsterdam, Day 1: Signs, Signs Everywhere Signs

Previous Post

DrupalCon Amsterdam, Day 1: Signs, Signs Everywhere Signs

Next Post

DrupalCon Amsterdam, Day 3: Drupal 8 Beta Released

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web