Feeds cheatsheet for developers

In this article I would like to create some notes about architecture of the Feeds module. This is really great module and it is real “must know” when we are talking about regular import of the data from some sources.

There is great documentation available on http://drupal.org/documentation/modules/feeds. Here I try to create a cheatsheet for 7.x version of the module that let developers brief understanding about how things work.

Diagram is clickable

More detailed explanation

FeedsSource object that contains settings of the source.

FeedsFetcher fetch() method should retrieve content of the feed that has been fetched (for example long string received from URL) FeedsFetcherResult object is created from this content (it is located in raw property) and passed to FeedsParser.

FeedsParser parse() method parse the raw content of the FeedsFetcherResult object and creates array of the items. FeedsParserResult object is created from these items and passed further to FeedsProcessor. Also it is very important to define mapping sources in method getMappingSources so your items fields will be visible in mapping.

FeedsProcessor process() this is where target entities got created. The idea is that we iterate through all items passed to us in FeedsParserResult, check the hashes of each item (in order to not reimport data that hasn't changed), map the fields (using map() method) and save target entities (using newEntity() or entitySave()).

As you might notice we also pass FeedsSource to each of above mentioned methods. This is needed for tracking the progress of the batch operations.

One of my last tasks I had was to import some data from external source that has REST interface. Only things I needed to implement was to write FeedsFetcher plugin where in fetch() method I did REST calls to my application. And I needed to implement FeedsParser where I did some manipulations with data and defined mapping sources.

Hope this cheatsheet will be handful for new people trying to use feeds and developers who already used feeds to remind the internal architecture.

Diagram has been created with Dia v0.97.1