May 14, 2012

A Series: Practical API Development


When you first set out to build your API, the number of factors to consider may be overwhelming.  Which functions do you need to expose to your clients?  Which should remain hidden?  What if you change your mind in the future?  Can your servers handle all of that additional load?  The list of questions goes on and on, and some of the trickiest issues may not apparent until you are stuck in the middle of them.  Fortunately, there are several existing patterns you can follow to address some of the most common concerns.

In this series, we outline the practical considerations when building an API.

APIs are now an essential part of building business.  Through a network of APIs, companies can more rapidly develop new applications and products and bring them to market more quickly.

APIs or Web Services are to web applications what interfaces are to classes in object oriented programming.  Versioning is an important component in the API lifecycle.

Versioning Overview

As web services continue to proliferate as more and more companies begin to offer APIs to their services, the importance of proper versioning and continued compatibility for their early adopters becomes more and more important. A successful and easily maintainable versioning strategy can mean the difference between happy customers and productive developers or, if executed poorly, a technical disaster.

An API is a Contract

Once it is live, you can't change it. More specifically, once you have customers, any changes you make to your API will increase the development costs for your customers. Renaming that endpoint that's been bugging your developers internally might mean hours of changes for each of your customers and possible downtime if you don't have a system for notifying them, staging changes, and coordinating releases. If you have hundreds of clients, coordinating a release can be daunting if not impossible.

Plan for Change

Though you cannot change the API while it is live, you can count on the fact that APIs do change. Plan accordingly. As soon as you release your API, a new use case that you did not account for will pop up and your customers will want you to add support for it. Or, perhaps a competing service offers some must have new feature that you simply can't ignore and you will be forced to add support for it. A critical bug will be discovered that puts the security of your backend and risk and you'll be forced to release a patch to your API. No matter how well you plan, you will have to change your API at some point. If you plan for changes and have a process to implement them, you will have happier customers and ultimately provide a better product.

Backwards Compatibility

One of the most critical features for any API is backwards compatibility. Achieving this is a lot harder than it sounds. Good products are constantly improved and refactored to ensure they keep pace with the market. As developers we strive for perfection and can't resist rewriting a bit of ugly code or fixing that embarrassing spelling mistake that somehow slipped through QA. Oftentimes, the pressure to bring a product or feature to market quickly will force us to release unfinished or rough code that isn't as intuitive as it could be or just doesn't work as expected.

The best way to ensure your changes aren't going to break existing applications, is to implement a test client or sample app that you can use as a regression testing tool for your API developers.
This should fit right in with your TDD and agile methodologies. As an added benefit you can use the sample app for demonstration purposes for potential customers to learn your API.

When to Version

Once you've released an API, the functionally is essentially locked. You can make changes, but only changes that maintain backwards compatibility. Examples of acceptable changes are things like adding a new endpoint, adding a new property to a response document, or adding new parameters to an API call to filter or constrain results differently. There are times when developers have written clients that do not handle new document formats effectively, so only endpoint additions are truly safe in all cases.

When Not To Version

Here are the types of non-destructive changes you can make without versioning your API:
1) Renaming endpoints - This change can be non-destructive if you proxy the previous resource location to the new one. This will add overhead to your operations department when it comes time to deploy or scale your API, so be careful with this one as it is likely to break at some point.
2) Adding endpoints. You can add new endpoints without breaking anything provided that these endpoints are not required for prior API clients (for example, if you started requiring a login endpoint to be called before a data retrieval endpoint that already existed, this would break the functionality for existing clients). Be careful using this technique to add new versions of existing endpoints as this practice can become unwieldy and confusing to your customers.
3) Renaming or Adding parameters. You can safely add new parameters to your API as long as you aren't changing the existing behavior of your API. New parameters could provide new ways of filtering or sorting your data (sorting and filtering as convenience methods or provided when it would be impractical to handle on the client, i.e.: too much data, filters on proprietary information or dynamic data, etc). Renaming parameters is non-destructive as long as you continue to support the old parameter names. It should be simple to alias parameters in your code.

Additionally, it is safe to make document format changes if you are positive there are no existing clients that will break when you change the format of your document. Let's say your Customer object provides first name and last name as distinct properties, but you've started collecting nicknames and want to add this to your API. Adding new fields or properties to your response documents is generally a safe way to introduce new data into our API. Existing customers will simply ignore the new fields (assuming they've coded defensively) and can make use of them at their leisure. One caveat I would mention here is that your API should never contain formatting. Leave content formatting to the client application and only provide raw data through your API.

If you want to be extra safe when changing your document, it is a good idea to add metadata to the document itself indicating the version of the document. This can (and should) be different from the version of the API; document version indicates the version of the document schema, whereas API version indicates the version of the request methods (changes to endpoints, functionality, etc.). This allows your clients to pass in a request parameter indicating which version of the document they want if they wish to continue using your current API version but aren't able to make use of the new document format.

In the next installment we will dive into the specific strategies you can use to add versioning information to your API, as well as an example that shows how you can put these strategies into practice.
Shane Holland is a Managing Partner at Above Labs where he and his team help startups and enterprise to develop great products and platforms. Shane has been developing and working with APIs for 15 years.