Rob Zazueta | Director, Platform Strategy
June 02, 2015

The Ultimate Solution to Versioning REST APIs: Content Negotiation

 

Versioning your API is terrifying. If you push out a “breaking change” – basically any change that runs counter to what client developers have planned for, such as renaming or deleting a parameter or changing the format of the response – you run the risk of bringing down many, if not all, of your customers’ systems, leading to angry support calls or – worse – massive churn. For this reason, versioning is the number one concern among the development teams I work with when helping them design their APIs.

The traditional way of handling versioning in REST is to apply some version number to the URL – either in the domain (i.e. apiv1.example.com/resource) or in the resource path (api.example.com/v1/resource). Non-breaking changes are frequently pushed out without much fanfare aside from a changelog posted to a blog or an update to the documentation. Breaking changes, on the other hand, require excessive testing, customer hand-holding, voluminous communication and a high level of planning, code maintenance and creative routing as you bend over backwards to ensure your customers can continue to work smoothly while you delicately push out a new release. With all that, everything can still go wrong.

Most API producers handle these issues by placing tight restrictions on their development teams regarding when and how the API may be updated, leading to convoluted policies that often confuse client developers and confound innovation and iteration. For example, Facebook recently described their new versioning strategy for their Marketing API. In the old days, Facebook was rather cavalier about pushing out breaking changes. “Move fast and break things” worked fine for them, but annoyed the developers who relied on their API. Though they have apparently learned their lesson, their solution to versioning – which, to be fair, is common among RESTful API providers – prevents their API from taking advantage of continous releases and forces their client developers to assiduously watch for announcements about new releases.

There’s a better way.

REST is closely tied to the HTTP specification, which has long had a way to communicate the format of the data exchanged between client and server – content negotiation. In fact, the “Representational” part of “Representational State Transfer” (for which REST is named) refers to this directly. Roy Fielding calls this out specifically in his 2000 thesis that defined REST (and which anyone talking about REST is obligated to reference). A resource may express one or more representations of its data based on the state of the resource. You typically see this manifest in APIs that support both JSON and XML responses – the client uses either a file extension in the URI or the “Accept” header to request their desired format. But that’s just the tip of the iceberg.

Resources should support more than simply the top-level format of their data – they should specify exactly what data that response contains. According to the Media Type specifications, you can define your own media types using either the “vendor tree” (i.e. “application/vnd.example.resource+json”) or, to avoid registration requirements, the “Unregistered x. Tree” (i.e. “application/x.example.resource+json”). In either case, you’re clearly communicating to the client developer that the response they’re receiving has a specified format beyond simply XML or JSON. You will need to provide documentation to your developers to describe the data each media type contains, but you should be doing that already.

It seems odd to many to define new media types that effectively take advantage of existing media types. In this case, you can define the specific format of your response using the parameters afforded by the Media Type specification. For example, if I have a resource named “product”, the media type for its basic JSON repesentation could be “application/json; profile=vnd.example.product”. Defining the profile as a parameter indicates to the client that they should treat the JSON formatted data in a predetermined way. I have also seen – and advocated for – the looser format “application/json;vnd.example.product”, but RFC wonks may shout you down on that.

Regardless of how you present your media type, the version of your resource should also be reflected in it. For example, if you’ve made one or more breaking changes to the Product resource, you should add the version parameter to it (i.e. “application/x.example.product+json; version=2″, “application/json; profile=vnd.example.product version=2″ or “application/json;vnd.example.product+v2″). Non-breaking changes should also be reflected using sub-versions (i.e. “version=2.1″). Removing versioning from the URI in this way has a number of benefits:

  • Rather than versioning the entire API when a single resource has a breaking change, you can version directly at the resource level. This allows teams responsible for only a handful of resources to operate independently and enforces the constraint that a resource is solely responsible for its state.
  • Clients will only need to update their code when the resources they commonly use change rather than every time you upversion your entire API.
  • Your team will no longer need to maintain various routing rules for URIs to redirect them to different back-end code bases, allowing for a cleaner and easier to use and cache set of URIs.
  • Best of all, your API will no longer be tied to arcane release cycles separate from the rest of your application stack. So long as you maintain older versions of your representations for a reasonable period of time, you may release at will without breaking client code.
  • You should also use content negotiation to define the different “shapes” of your resource – i.e. “brief” vs. “full”, “mobile” vs. “web” and whatever else makes the most sense to provide for your customers. You can do this either through an additional custom media type or by adding a parameter (i.e. “application/x.example.product+json; version=2 shape=’full'”).

It also introduces a few challenges:

  • While URI routing is no longer an issue, routing of the correct response type gets moved up further in your API application stack. If you’ve stuck to an MVC pattern, your controller will be responsible for mapping the data from your model into the appropriate data structure while your views will be responsible for presenting it according to the requested media type.
  • Clients will need to know the media type for each resource and request the same one throughout their use of your API to ensure their code continues to function normally as you push out new changes.
  • You will still need to clearly communicate any changes to your API to allow your developers to determine how and when they will need to update their client code accordingly.

Most of these challenges can be easily overcome. If you’re not already taking advantage of the MVC pattern in your application stack, for example, you’d be wise to consider rearchitecting. Content negotiation doesn’t necessarily eliminate the need for multiple code bases, but it does move it into a more manageble part of your code – specifically, the controllers and the views. As far as clients knowing the media type, I’d recommend they either store the media type they’re using in a variable that gets passed in the Accept header or they do an “Accept: /” on the initial call and locally cache the media type that comes back for use with all subsequent calls. Content negotiation is not a common REST client consideration at this point, but that’s no excuse for poor API design.

Communicating API changes to developers has always been a challenge. You can tweet about them, update your blog, send an email and even call each developer directly and still have more than a few of them contacting your support team complaining about code broken during an update. At last year’s API Craft conference in Detroit, some attendees recommended adding a link to resource responses that have been upversioned as an additional means of communication. I’m particularly fond of this as it gives the client developer the power to decide how to handle such updates directly from within their code. I recommend implementing this now, whether you’re using content negotiation to handle your versioning or not. Essentially, if a developer is using an outdated version of the API, they should see an additional link in the response like this:

"_links": { "rel": "urn:deprecated", "href": "http://api.example.com/alerts/735432", "method": "GET" }

The link relation follows the custom URN format (since this is not an IANA approved relation) with a relation of “deprecated”. While this is not currently a standard, it should be. The link itself should go to some kind of message resource that contains the details of the deprecation – where you direct that is ultimately up to you. However, this resource should respond with at least two representations – one formatted appropriately for code (i.e. JSON or XML) and the other for human readability (i.e. an HTML page, blog post, changelog, etc.). The data found at this resource should show what the current version is and clearly explain the changes made to the resource that warranted a version change.

The client developer should include a routine in their code to look for this link in all responses. If found, the client code should do whatever the developer deems necessary to alert the team responsible for maintaining this code about the change. This may include an email to an internal group, creating a Jira ticket using the details obtained by the deprecation link or any other preferred method guaranteed to get the attention of the team.

Content negotiation is a clean, well-documented, standards-compliant way of handling a lot of the complexity found in managing and maintaining RESTful APIs. With everyone clamoring about hypermedia, the media type often gets overlooked. You’ve nothing to lose by beginning to implement it in your existing RESTful API, but so much to gain in added usability for your developer customers and flexibility and agility for you internal teams.

This blog post originally appeared on The RESTed NARWHL on March 11, 2015.