July 30, 2012

A Matter of Scale: When it Comes to APIs, an Ounce of Advanced Planning TODAY Increases the Likelihood of Success TOMORROW


If you are just now planning to build the first version of your API, as a technology leader you may wonder why you should be concerned with application scale at this early date. After all, it’s possible that customer feedback will take the API in a completely different direction, or that the project takes a back seat to other priorities within the organization, or that the project is shelved altogether. Isn’t it best to get something to market quickly and then worry about scalability later?

The answer is both yes and no. You are looking at your API’s chances of success optimistically – after all, you wouldn’t be investing the time to build it if you didn’t have high hopes for it’s future. And if your API is a success, you will find precious little time to engage in a top-to-bottom rewrite of your software once real, paying customers are using it. On the other hand, if you spend the next two years building out a massive Twitter-scale platform to support your 150 customers, then you’re losing time to your competition (and annoying your users).

What To Do From Day One

The most important thing for you to do at this point is to have a plan for future scaling. You don’t need to build all or even most of it now, but you do need to have a roadmap. It’s helpful to put together some quick notes on how many requests you optimistically expect to be serving two months, six months, a year, and two years from now (for most organizations, anything beyond that is pure speculation). With these notes in hand you can then work with your technology team to ensure you will have adequate hardware and software infrastructure in place to deal with the coming tidal wave of developers at each of those milestones.

Next, make sure your plan adequately gets you to the right level of support by each of those milestones. Suppose you initially deploy your API on a single server but you know that you want it to be load balanced to two machines by the two-month milestone. If it takes your hardware vendor two weeks to procure new machines for you and your operations team needs an additional three weeks for setup and testing, you’d better put in the hardware order the day you launch your API.

Things to Think About

While the following is by no means exhaustive, here are some of the key points you will want to consider when assessing your scaling needs:

• What will the software and network architectures for my deployment look like?

• How can I set up a test environment to ensure the architecture really does support the number of users I will have to support when it is deployed?

• Where can I use off-the-shelf components to help me scale, and where do I need to create software custom to my business?

• What hardware will I require to support that?

• What staff will I need to support that?

• What budget will I need to support that? How many customers do we need to have in order for the budget to be justified?

• Will the executive team require analytics on the usage of the API? If so, when should I start gathering them and how will I report on them?

Be Prepared

Even if the first version of your API will be deployed on a single server, there are things you will probably want to do from the get go to ensure you don’t cause yourself unnecessary pain down the road. For example, almost all scaling solutions will require you to incorporate load balancing of your servers at some point. If you write your API layer such that it expects to live on one and only one machine, you may be faced with a nightmare trying to undo this once you have thousands of live customers using a single server that is ready to collapse under the strain. Spending a few extra hours up-front to ensure your application can run in a load-balanced environment is well worth the weeks or even months you’d eat down the road to undo this costly mistake.

Likewise, any smart software architect will tell you that you don’t need to build your whole solution right now – but you do want to build in hooks where you can easily insert the more complicated stuff later on. You might add functions at several points in your code like create_backup_data_records() or retry_data_pull_from_backup_computer(), and they might do absolutely nothing for now. But by putting those hooks in today, you ensure that some poor developer doesn’t need to dig through 100,000 lines of code three years from now searching for all the places that they need to insert those calls. Instead, they can just implement the logic in a single place to manage the creation and restoring of data backups. Because you are already making calls to those methods at the appropriate places, it’s just a matter of deploying the new code that actually makes those functions do something. No hunting and pecking required.

Setting up proper scaling is a balancing act. You need to make sure that your developers don’t go off the deep end writing code to account for massive scale that you may never encounter – your API isn’t being written as an ivory tower academic project, but for your real-world customers. At the same time, you must ensure that you have a plan in place along with minimal hooks to ease the transition at each of those future milestones. Finally, remember that your plan isn’t set in stone – take stock of your usage projections every few months and make sure that your plan is still appropriate to the conditions you are seeing as your customers start putting more demands on your infrastructure.