Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough

Consuming the new Twitter 1.1 API with Feeds and friends

Update: This post now contains a feature that you can import in D7 to see the Twitter feed in action.

The new Twitter 1.1 API kicked in recently, which meant a new cycle of maintenance for anyone consuming their data programmatically. My own Feeds + Views demo site streams #drupal, using Feeds and complementary modules. I had to make a few changes to the importer to adapt to the new API:

  • Authorization using OAuth
  • Parsing JSON instead of XML

The new Twitter 1.1 API requires OAuth authentication/authorization for every request. Fortunately, I had already written Feeds OAuth to solve for feeds that require OAuth, so I just had to plug this in. Well, not "just", because it took choosing among the several authorization options that Twitter provides, and fixing a couple of bugs in the module itself.

Twitter provides several options for authorization, depending on the needs of the consumer (not listed on this page is the Application-only authentication). I ended up choosing the method that required the least work on the Feeds OAuth module, namely obtaining OAuth tokens from dev.twitter.com as a pre-processing step. To do this, I manually added an entry to the feeds_oauth_access_tokens table, with the tokens that were handed to me by Twitter on my application page. This way, Feeds OAuth would not have to ask me to login to Twitter in order to make the API call. Obviously, this is a temporary hack and I will work on enhancing the module support for different authentication options.

Twitter 1.1 API only returns JSON results. To parse JSON instead of RSS/Atom, I used Feeds JSONPath Parser. It does the job as advertised, but the only challenge here was to retrieve the tweet URL for each result. The Twitter search API itself does not return tweet URLs, for some unfathomable reason. My setup needs the tweet URL to pass it to oEmbed, which renders the tweet on the view. Tweet URLS are of the form https://twitter.com/<user_screen_name>/status/<tweet_id>.

To get the URL, I had to resort to coding. Here's how I did it:

  • First convince Feeds JSONPath Parser to retrieve the user's screen name. To do this, I had to map it to some field - I chose the node body, although a better solution would be to use a NULL target field, just to fool the parser into returning the value.

  • In a custom module, I created a new programmatic source field called "Tweet URL" that synthesizes the URL:

/**
 * Implements hook_feeds_parser_sources_alter().
 */ 
function demo_feeds_parser_sources_alter(&$sources, $content_type) {
  $sources['tweet_url'] = array(
    'name' => t('Tweet URL'),
    'description' => t('The URL of a tweet.'),
    'callback' => 'demo_feeds_tweet_url',
  );
}

/**
 * Populates the "tweet_url" field for each result.
 */
function demo_feeds_tweet_url(FeedsSource $source, FeedsParserResult $result, $key) {
  $item = $result->currentItem();
  // jsonpath_parser:2 corresponds to user screen name in my importer.
  // jsonpath_parser:0 corresponds to tweet ID in my importer.
  return 'https://twitter.com/' . $item['jsonpath_parser:2'] . '/status/' . $item['jsonpath_parser:0'];
}
  • Finally, I mapped this new source field to my target field, the URL of a Link that renders using oEmbed.

This little exercise took a good couple of hours - and that's just for a demo. API changes are always painful, but at least the Feeds OAuth module got some love and fixes in the process.

  • Enable the module twitter_feed_custom.
  • Copy the ping.php_.txt file to your Drupal root folder and rename it to ping.php. Also edit the file to point the DRUPAL_ROOT definition to your actual Drupal root folder.
  • Copy the Consumer key and Consumer secret strings of the Twitter app to the Fetcher > HTTPS OAuth Fetcher Settings > Consumer key and Consumer secret settings, respectively.
  • Create a new node of type Twitter feed with your query URL (e.g. #drupal). Make a note of this node's nid.
  • Edit the (badly-named) hashdrupal view included in this feature, such that the filter Feeds item: Owner feed nid refers to the nid noted above.
Author: 
Original Post: 

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web