This post is part of our ReadWriteCloud channel, which is dedicated to covering virtualization and cloud computing. The channel is sponsored by Intel and VMware. Read the case study about how Intel Xeon processors and VMware helped virtualize 12 business critical database applications.
Twitter was one of the first to see what happened when traffic to the site came more from the API than the Web.
It now has more than 65 million tweets per day, most coming from services that use the Twitter API.
Twitter has made numerous changes to fix its API. Those experiences have taught providers what mistakes not to make when launching a service.
But there is still a lot for providers to learn.
Considering this, we asked developers and service providers to help us prepare a list of 10 common mistakes made by API providers. Hopefully, the list will provide some basic insights into what mistakes should be avoided when developing an API.
Our group of commentators include Adam DuVander executive editor at Programmable Web; Mike Pearce, a developer out of the United Kingdom who writes a lot about scrum and Agile; Mashery's Clay Loveless and Sonoa Systems Sam Ramji.
"Databases fail, backend dependencies get slow, and/or someone somewhere along the line doesn't escape output properly. The result? A nasty stack trace at the top of your API payload, which in some cases may even include production database credentials.Solution: Before you launch, kick over some dependency and see what your API does. Developers would rather have a predictable (valid XML/JSON) error message they can code for than a surprise invalid payload."
Source: Clay Loveless
"Sometimes we see providers expecting the API alone to attract developers. You need a good service and must communicate to developers what it is you provide. Also, along similar lines, it's easy to ignore the developer community management, but that communication can make or break your platform. When you get developers' attention, keep it by being responsive and helpful. Treat them like partners."Source: Adam DuVander
"APIs are about bringing the scale of the Internet to bear on your business - but what if that scale actually happens? Will a manual review for user on-boarding scale if you surge in popularity? Will auditing and liability processes get in the way of enabling free access to a subset of your data? Governance is less important when the API is less used, but when transaction volume becomes significant, APIs become important to the finance and legal teams. Don't let success lead to failure due to non-scalable processes."Source: Sam Ramji
"Case in point: twitter.com is now going to api.twitter.com for the API. The result is that they can scale the website and the API independently of each other as needed. I've never heard of anyone regretting the decision to split out API traffic from main website traffic, and nearly everyone to doesn't do that at the beginning wishes they had _and_ endures a lengthy & painful process of migrating developers to the new API location."'Source: Clay Loveless
You can often tell the APIs that have had some real-world use before launching, and those that haven't. For example, an API that lists _some_ information about an object, yet needs a secondary call to get MORE information about that object. If your API requires two calls to find out everything there is to know, you and your devs will hate you for it eventually.Build a couple of reference apps that might do something useful, even if trivial. You'll quickly get a sense for how sane your API is when you run into those inefficient API call combinations. If you haven't built your website on top of your API, consider doing that -- or at least what it would be LIKE to do that. If your API couldn't reproduce your website, your API needs fixing.
Source: Clay Loveless
1)Tactical Bad Behavior
- Inefficient client code can call an API too frequently, causing an accidental DDoS attack
- APIs can be attacked through the JSON or XML data passed as parameters or attachments;
- Attack techniques can include SQL injection, XML bombs, and shell scripts, as well as the usual social engineering and spoofing attacks.
2) Bad behavior "at scale"
- When your service is small and you have a hundred users, you may have only two bad actors. These can be pruned manually using firewall rules based on IP address.
- When your service is scaling to hundreds of thousands of users, you may have thousands of bad actors. These are placing high levels of "fake" demand on the service and can no longer be pruned manually without adding more staff dedicated just to these issues.
- To mitigate this do you apply rate limits to everyone and reduce the value of the service to good users? Do you find a way to apply policies to naturally separate patterns of use?
The anti-pattern we've seen here is the failure to anticipate bad behavior against your service, and then to respond either with draconian limits on use, or with intermittent systems availability as a result of dealing with the bad demand.
Source: San Ramji
"Many APIs have no testing at all, and those that do frequently have automated unit tests, etc. Very few have end-to-end black box tests that can be run surrounding releases. Your unit tests may pass, but if your latest push also included a web server config tweak, you want to have an end-to-end test suite to validate those changes."Source: Clay Loveless
"The companies who succeed in the new web economy recognize that APIs are a core line of business and essential path to customers and partners. Successful media, retail and web companies often see over 50% of their traffic from their API - and companies must treat it as a product."Source: SamRamji
Like any new project, business and technology need to be lined up together including having understood and documented goals, metrics, and rewards. What higher level objectives are being met and how do these map into departmental goals for marketing, customer support, and engineering? How will these be reported to senior management and on what cadence will performance be reviewed?Source: SamRamji
"The specific mistakes you see regularly are error tunnelling (sending back a "200OK" response code, even if the body of the response describes an error). GET/POST tunnelling (sending all requests through GET or POST and ignoring the other verbs (PUT, DELETE, etc). And content-types (not allowing the consumer to specify the content type they will accept). Most of the big players get it mostly right. Amazon, for example, sets a good example, whereas Flickr does not. But most API providers miss the more esoteric stuff like hypermedia is the engine of application state and ignoring caching. However, you can use an API without that stuff quite adequately. It's the incorrect tunnelling that will hamstring you and your consumers."Source: Mike Pearce
Follow the latest industry discussion here. Click a topic in the tag cloud to focus on it.