This past week, I've been neck-deep in projects involving some aspect of the EC2 APIs. From work on Eucalyptus to OpenStack to enStratus to AWS, it's been an EC2 kind of week. Of course, I tend to use Twitter as a release valve for annoyances while doing development, and I commented on the suitability of the EC2 API as a de facto standard. These comments ignited yet another round of debates about cloud APIs.
The argument for EC2 as a de facto standard is, at some level, the same as it is for any de facto standard: through the EC2 API, you eliminate the need for others to learn some custom API and you can leverage the existing, sizable ecosystem.
First, a caveat. This article is neither a commentary on the EC2 API itself, on the tools that talk to it, or the people who have tried implementing compatible versions. EC2 was not designed as an industry-standard API. It was designed and has been maintained only with the needs of Amazon Web Services in mind and it has obviously served that end spectacularly well. I am critiquing an aspect of the API for which it specifically was not designed.
The true problem with EC2 as a de facto standard is that there's really no such thing as one EC2 API. All the clients out there are essentially written against different APIs with different authentication schemes. Being a cloud provider who supports EC2 as an API into your system therefore means supporting many divergent APIs.
EC2 Cheat Sheet
The EC2 API is divided into a SOAP API and a query API. So, if you want to be true to the docs, you have to support both. Though leveraging the SOAP API is not very common among the AWS ecosystem, it does tend to be more common among the in-house apps you might want to support.
The query API is further divided into query by GET and query by POST. So, for each operation that can be performed in your infrastructure, you have 3 choices when writing a client:
- HTTP GET
- HTTP POST
Each service then has its own specific protocol with (in some cases) many different versions. Several services have distinct authentication and request signing mechanisms. EC2 itself has three different versions of authentication. Most of the differences aren't documented well (if at all). After all, clients implementing the API should need understand only the current version.
There Is No EC2 API
Because of the way the API was designed and has evolved, there is no single API against which the ecosystem of AWS tools is written. In fact, it's highly probable that, given any two ecosystem tools, you will see two very different calls across the wire for the same exact logical query.
So, what API do you implement?
Do you implement the API (currently) documented by AWS? Most clients don't follow this documentation flawlessly. Some from historical reasons; others are just broken and AWS is forgiving. If you write a client to the documentation, you know it will work. If you write a service that exposes the documented API, you will serve no one.
Do you implement the API as used by the majority of ecosystem tools? I don't think there is such a thing. You might go for the heavyweights like ElasticFox, boto, jclouds, lib cloud, and Dasein Cloud. But even those tools are all using slightly different variants on EC2. Some are using the HTTP GET, some are using the HTTP post. Some use one timestamp format, others use different ones. Etc. etc.
Do you try to implement it all? If so, you've moved away from supporting a single API to supporting numerous APIs that are constantly evolving and out of your control.
I don't have a good answer for you. My approach was to write my own API.