Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Feb 03 2017
Feb 03

In my previous blog post on managing microsites with Drupal 8 I promised to write something further and fuller about designing web APIs. This is less directly about Drupal 8, but I will comment on how to implement the recommendations here in Drupal 8.

These are the things that I take time to think about when building a web API.

As a developer, it’s all too easy, and too tempting, to just jump right into coding something. It’s certainly a weakness I suffer from and that I have to cope with.

Before putting the proverbial pen to paper, though, it’s really important to understand why we’re building an API in the first place. What are the problems we’re trying to solve? What do the users need or want?

With regard to building an API, that means thinking about the consumers of the data provided by your API. If you’re building a decoupled CMS, the main user is the frontend system. In other circumstances it may also mean other websites, embedded widgets, apps on mobile devices, and so on. Whatever it is, due consideration needs to be given to the needs of those consumers.

That means understanding your user’s needs, examining the patterns of behaviour of those users, and ultimately translating those into a design.

Sound like familiar language? Yes, that’s the language of visual designers and user experience specialists. In my books, I’d suggest that means you would do well to work closely with specialist design colleagues when designing and building an API.

Your web API needs to be designed: needs; behaviours; analysis; patterns; traits; design; feedback; improve.

Be an artisan with your API

Take time. Research. Think. Plan. Design.

Beware, Drupal

When you’re working with Drupal, it is too easy to jump over the design step. Drupal does so much out of the box that it’s too easy to start coding without thinking properly about what we’re coding.

The availability bias when you’re a specialist Drupal developer, having it as the go-to toolkit, is that we think about the solutions to the problems (if we’ve even got as far as articulating the problems) in a Drupally way. For instance, since Drupal has a menu system it’s easy to think about navigation in a decoupled CMS system in terms of the way Drupal handles the menu system, which prevents you from thinking about other ways of handling navigation.

The same is true with Drupal 8’s support for REST. Drupal 8 core includes REST resources for most entities in a Drupal installation. That’s very useful. But, it can also make you lazy, just using these core RESTful API endpoints for nodes or comments or whatever, with all the guff they include, without even thinking about whether they’re appropriate, whether all the guff they include is appropriate, whether it’s useful or formatted appropriately.

That goes also for REST exports from Views. They can be useful, giving you a quick way of creating a RESTful API endpoint. The problem is, thought, that also confines you to working with the way Views works and what it can produce. You may find that a problem if you want to support optionally requesting for additional objects to be embedded in the response, for instance (see below).

Resist the temptation! Instead, take the time to think from the other end first.

I’ll return to the question of designing your API below, but first we need to talk about documentation, since designing and documenting your API can be part of the same process.

Yeah, I know. Most devs find this just the dullest thing in the world to write. With a web API, though, it’s incredibly important. If you want people to actually be able to use your API, they need to know how to work with it. It’s horrible trying to work with an undocumented or under-documented API.

So, what should go into the documentation for a web API? Here’s some pointers.

The basics:

API reference

Yeah, this is probably what everyone thinks of when they think of documentation for a web API, but it is in fact only part of the documentation—maybe the most important part, but only part.

There a plenty of good blog posts and descriptions of what your API reference should include, so there’s no need for me to reiterate that here.

The most important thing to say, though, is that, beyond identifying resource paths, actions and parameters, your reference should describe in full both what the request should and the response will look like.

Mock server

It is incredibly helpful to include a mock server with your API documentation. Preferably, your mock server will handle the documented requests and responses of each resource.

This will help those building apps and tools that will consume your API to get up-and-running quickly.

For gold stars and a round of applause:

Tutorials, guides, cookbooks

If your API gets to be any substantial scale then the developers who use your API will find it incredibly useful to have some tutorials and guides included in your documentation.

These should cover common tasks, or how to work with specific sections of your API. A guide to ‘best practices’ with your API may be appropriate to help people make the most out of your API.

Check out the guides in MailChimp’s API documentation for a good example. Twitter’s API docs ‘best practice’ section are great as well.

Quick start

One invaluable guide is the ‘getting started’ or ‘quick start’ guide. This can often be just a single page, with a succinct summary of the things you need to do to get going.

The YouTube API ‘getting started’ page is a useful example.

Useful tools

There’s lots of useful tools out there to help you get started when you document your API design. Here’s some suggestions.

API Blueprint is an open-source high-level API design language that is very useful for writing your documentation. The language is similar to Markdown, so it’s easy to work with. There are a number of SaaS tools offering services based on API Blueprint. One that I really like is Apiary.io (though they’ve recently been bought by Oracle so who know where that’ll take them), but there are others, like Gelato.

You might also consider Read the Docs and daux.io amongst others. There’s also the Open API Initiative, which is ‘focused on creating, evolving and promoting a vendor neutral API Description Format,’ though the initiative is ‘based on the Swagger Specification.’ Open API is an initiative of Swagger.io, and they have a list of tools and frameworks using the specification. The OpenAPI specification is on GitHub.

Whatever you use, your documentation should (probably) end up in a public location so that other developers can use it. (An exception might be for an API used in a secure decoupled system.)

So, let’s return more directly to the question of designing your web API.

An important rule of thumb for me is to ‘keep it simple, stupid.’ There is no need to include anything more in the resources of your API than is necessary.

I say this as a long-time Drupal developer, knowing full well that we have a superpower in overcomplicating things, all those extra divs and classes all over the markup, all those huge arrays.

This is still true in the core REST resources of Drupal 8. For example, when GETting the core Content resource for node 10 /node/10?_format=json the response gives us …

{
"nid": [
{
"value": "10"
}
],
"uuid": [
{
"value": "6bfe02da-b1d7-4f9b-a77a-c346b23fd0b3"
}
],
"vid": [
{
"value": "11"
}
],

}

Each of those fields is an array that contains an array that contains the value name:value pair as the only entry. Whew! Exhausting. An array within an array, when there’s only one level-1 array ? Really? Maybe we could render that a little more simply as …

{
"nid": "10",
"uuid": "6bfe02da-b1d7-4f9b-a77a-c346b23fd0b3",
"vid": "11",

}

… which might help our API’s consuming applications to parse and use the JSON data more easily. Like I said above, I’d suggest that just using the core entity REST resources isn’t often the place to start.

The simplicity mantra should pervade your API design. Include only the data that is needed for the consuming apps. Pare it down, so it’s as easy to read as possible.

As a result, when you come to build that API in your Drupal 8 backend system, it will demand a good discipline on you of not just throwing out in the API resource responses what’s easiest but rather what’s best.

What’s in a name?

This is true in particular when it comes to your naming conventions and API resource paths.

Don’t just add root-level endpoints ad infinitum. Use well-structured paths for your resources where the depth of the path elements make sense together. The result should be that your resources are explorable via a browser address bar. E.g.

GET /articles/5/comments/19

… makes intuitive sense as a path: get comment 19 on article 5.

On the other hand, don’t just add depth to your resource paths unnecessarily. Separating things out with some logic will help make things intelligible for developers using your API. E.g.

GET /articles/comments

Umm? What’s that? The comments on articles — why would I want that? However …

GET /comments?contenttypes=articles

… is more obvious — a path to get comments, with a content types filter. Obvious. It also suggest we might be able to filter content types with a comma-separated list of types—nice!

Find a straightforward naming convention. Make the names of resource endpoints and data fields obvious and sensible at first glance.

Overall, make the way you name things simple, intuitive and consistent. If the title field of a data object in your resources is called ‘title’ in one place, ‘name’ in others and ‘label’ in still others, for instance, then it adds unnecessary complexity for writing reusable code.

When designing your web API, it needs to be simple to use and work with. Help users to get just what they want from your API.

Support limiting response fields

You’ll make developers smile if you provide a way of limiting the fields that are returned in a response. You don’t always want to get everything from a resource. Being able to choose exactly what you want can help speed up usage of an API.

For example, consider supporting a fields parameter, that could be used like this:

GET /articles/5?fields=id,title,created

Support auto-loading related resources

The opposite might also be important, being able to load extra resources in the same request. If a request can combine related resources then fewer requests will need to be made, which again will help speed up using an API.

Supporting an embed query parameter could give you this. For example:

GET /articles/5?embed=author.name,author.picture,author.created

… would enable users to load also the article author’s name, their picture and the date their account was created. Note the dot syntax, which might be useful.

Flexible formats

Another way of making it easy for users is to support flexibility in the format of the data in the response. JSON is usually what people want to handle, but some do still prefer to use XML.

There’s also the problem that JSON has no support for hyperlinks, the building blocks of the web, which is a curiosity as the W3C admit. There are JSON protocol variants that attempt to address this, like HAL and JSON-LD, but I refer you to a fuller discussion of JSON and hypermedia and some useful resources on hypermedia and APIs from Javier Cervantes at this point.

When designing your API, you should expect it to have a certain lifetime. In fact, it’s bound to last long enough to need changing and improving. But what do you do about rolling out those changes?

Your devs will need the flexibility to change things, especially if they find bugs, and they’ll get frustrated if they can’t adapt the API to make improvements.

Your users need reliability and stability, though, and they’ll get frustrated if the API keeps changing and their consumer app dies without warning.

So, from the start, include versioning.

A pretty sensible thing is use a path element to specify the version number. E.g.

GET /api/v1/articles/5

You could use a query parameter instead, of course, though since query parameters are optional that would mean that without the version parameter your API would return the latest. Consumers who’d inadvertently missed including the version in their requests would be vulnerable to changes making their app die, which might result in some flame support emails.

Make sure there’s a way for your users to let you know when they have problems, there find a bug, or whatever.

If its an internal API, like with a decoupled CMS and frontend, then that is probably your bug tracker.

If it’s a public API, then you’ll need some public way for people to contact you. If you host your repository on e.g. GitHub then there’s support for issues baked in.

Respond.

Giant lists of bugs that never get addressed are soul-crushing.

Authentication and security

You’ll probably want to include some authentication to your API. You shouldn’t rely on cookies or sessions for your API as it should be stateless. Instead, by using SSL (you’re using SSL, right? yes, you’re using SSL.), you can implement a token-based authentication approach.

However, where a token approach is inappropriate, OAuth 2 (with SSL) is probably the best way to go. Here’s some further discussion on API security and authentication, if you’d like to read in more depth.

Caching

HTTP has a caching mechanism built in — woot! Just add some response headers and do some validation on request headers and it’s there.

I’ll point you elsewhere to read more about the 2 key approaches, ETag and Last-Modified.

Use HTTP status codes

HTTP defines lots of meaningful status codes that can be returned in your API responses. By using them appropriately, your API consumers can respond accordingly.

Useful errors

If a request has an error, don’t just return an error code. Your API should provide a useful error message in a format with which the consumer can work. You should use fields in your error message in the same way that a valid response does.

In summary, when building an API it’s not healthy to just jump in and start writing the code for the API from a specification. Neither is it healthy to just rely on the default resources of CMS tools like Drupal 8. APIs always need to be tailor-made for the task.

APIs need to be designed.

If you can make your web API simple to understand and adopt, easy to work with, incorporating plenty of flexibility, if it’s stable and reliable and well-supported, then you’re well on your way to being the proud owner of a healthy API.

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web