August 08, 2011

In Search of: API Metrics - Total API Call Volume Activity

In this final installment of our In Search Of: API Metrics series exploring the top three most commonly reported API metrics, we take a look at how companies measure and report Total API Call Volume Activity. So far, our series has focused on evaluating the performance of API platforms and programs by distinguishing between metrics that measure bigger versus better. We kicked off the series with different ways of reporting on Total number of Developers who consume your API. Next, we followed up by digging into the various ways that API platforms and programs choose to report on Total number of Applications, which should reflect adoption and activity in an API ecosystem.

The running theme of the series bears repeating a final time: companies may opt to report 'bigger' for various reasons, but it may be even more important in the long run to actively measure and monitor 'better'.

Total API Call Volume Activity
Measuring and sharing this third, widely reported metric is a good counterbalance to the common misconception -- often perpetuated by bloggers and the press -- that APIs are only for third party developers. John Musser's API Billionaires Club, last updated in May 2011, is a fantastic reference point of API activity involved in major mobile and web platforms, such as Twitter with 13 billion API Calls daily, Facebook and Google with 5 billion API calls daily each, eBay, Netflix, Amazon and others. (More recently, Klout announced last month it had joined the big leagues by serving up 2 billion API calls in the month of June 2011). Moreover, Netflix is one of the companies in the API Billionaires Club in process of redesigning their "chatty" API platform so that Netflix developers and partners, both internal and external to Netflix, can build applications that rely on their APIs using *fewer* calls. This API redesign reflects a growing recognition that bigger is not always better when it comes to API Call Volume. 

API platforms including eBay, Netflix and NPR go one step further and breakdown the internal and external usage of their public API Call Volume numbers. We recommend this approach because it more accurately captures the enormous product development value and competitive advantage that these major platforms are gaining by 'dogfooding' their own API. However, we further recommend API platforms go beyond a two-factor breakdown of API call volume by 1) internal and 2) external development, and instead measure a three-factor breakdown of 1) internal, 2) partner, and 3) open development by API call volume. This three-factor breakdown more accurately reflects that many vendors, agencies, consultancies, ISVs, plus manufacturing, channel and biz dev partners are the parties building "internal" house applications for a company. It is important to capture the value that these "external" developers building "internal" applications bring to your business.

  Bigger: Total API Call Volume over a given time period (daily/weekly/monthly/quarterly)
Better: Total API Call Volume over a given time period (daily/weekly/monthly/quarterly) broken down by internal, partner and open development usage

When your API is built to scale to meet internal, partner and open developer demand, Total API Call Volume is a great number to complement the other metrics we've already covered. API Call Volume communicates scale, stability, trust, and that your company supports your API platform as a core component to its own business model. When your company is a true platform and consumes your own API for internal and partner projects, this is an important signal to external developers (both partner and open developers!) that you will react quickly to bug fixes, maintenance requests, documentation and prioritization of new features because you consume the API you are encouraging them to use.

In conclusion, there are dozens of other API metrics and ratios tied to revenue, capital expenditure, time-to-market, device & channel distribution, innovation, developer adoption, geography and user demographics that Mashery leverages for strategy and developer outreach recommendations. These recommendations are tailored to industry verticals and business models. Mashery manages over 100+ APIs, so we have experience with what works well and what doesn't. Any API platform or program that measures the basic three metrics of Total number of DevelopersTotal number of Applications, and Total API Call Volume Activity we covered in this series -- even if they opt not to announce them publicly -- is starting on the path towards creating more value for their company and their API ecosystem. We encourage API providers to analyze data usage patterns in order to better manage your platform using these important quantitative metrics. Qualitative feedback that you get from developer voices and insights on discussion board threads, surveys, and twitter replies is also very valuable. But "quant" data usage patterns help you prioritize and focus on where the most value is being created -- instead of only reacting to the consumers of your API who shout the loudest.

Fundamentally, your API platform and program is an extension of your business. If you're not measuring the performance of your APIs, you are not treating your platform like a core part of your business. Measure your API platform and program better, and turn your API platform into a competitive advantage for your company and your API ecosystem.