In Batching REST APIs - Part 1, I talked about why offering batching APIs and batching REST clients is important for high traffic web services. Today I wanted to talk about some of the thoughts and design decisions that went into building our first REST clients.
Before I started, I wanted to nail down the tao of our official Segment.io REST clients. Here are the requirements I came up with. I call them the seven golden rules of friendly REST clients.
1. Batch requests.
You can’t afford to make blocking HTTP requests on every operation. Put them in a resource-capped queue, and async flush ‘em on another thread. If you’re not convinced, it’s back to part 1 for you.
You can also ask your users to build their own background queue to flush, but that’s significantly more work for them to implement.
2. Cap memory and network usage, especially under extreme load.
A few months ago we stumbled upon a memory leak while working on Segment.io’s ingestion process that was happening during periods of high load. After scouring our application code for an entire day, we still couldn’t figure out why. Finally we tried removing all of our dependencies one by one. After we removed our cloud logging provider’s REST library, the memory leak suddently disappeared. You can be sure we won’t be using them ever again.
High performance computing is about trade-offs. You can’t keep sending data to your API at an ever increasing rate. At some point, either the network bandwidth or the available memory will run out. And its important you take action (or inaction) before that happens. That way, when your customer’s application crashes, it won’t be your fault.
3. Use asynchronous HTTP flush whenever possible.
Node.js will give you this by default, but languages like Java and Python won’t. It’s worth investigating an asynchronous HTTP request option, so that your flushing thread doesn’t have to spin for valuable CPU cycles waiting for an expensive network operation to complete. There are good options for most languages, like these for java and python.
4. Be secure.
Use HTTPS.
5. Fail gracefully, and report.
It’s important to plan out your error cases before hand, and guard against you accidentially crashing your client’s application. That means you should simulate and expect errors like:
- Your API servers going down
- The API returning 302, 404, or 500, or some other non-200 status code
- The API returning an outdated or unexpected result format
- The API endpoint changing
6. Speak but don’t yell.
Logging is important, but like everything else, must be capped in volume. If every one of your requests is failing, you don’t want to crash the server by overfilling the disk with a huge log file or filling the entire screen with error messages.
A good pattern to provide is optional error callbacks. That means that the user can get errors when they are debugging, but also won’t get them when everything is going to hell.
client.on('error', function (err, req) {
winston.error(err);
});7. Provide statistics on usage.
When the user asks, provide a simple object that communicates how many successful and failing operations happened. Maybe provide how many bytes have passed through the API calls, how fast, and how often.Your users will appreciate the extra debugging information.
In the next post, I’ll discuss how we designed our Java rest client to achieve these goals!