Web Services: Avoiding APIs

I’ve spent some time thinking about the pros and cons of using APIs vs. using standardized wire formats, prompted by a whole bunch of references to this issue - by Don Box, Dave Orchard, and Sean McGrath. It’s becoming more and more clear to me what the advantages of standardizing on the wire formats are, so I thought I might as well share some of my ideas. Feedback, as usual, is very welcome.

So what’s the issue? I believe that when you take your first look at Web services, and have a strong background in Distributed Objects, or other, older RPC-based technologies, you get a strong feeling of déjà vu. After all, isn’t this all just CORBA or DCOM reinvented, with a different wire protocol? What could be the advantage of using a fat, text-based format instead of a nice, binary, efficient protocol like DCE RPC, IIOP, or plain RMI? Why would anyone in their right mind even care about what the bits you send over the wire look like?

You might be tempted to support Web services standards, most notably SOAP, just as another, additional transport. Create a common API, and use it to send messages via RMI, CORBA, or JMS - without having to change anything in your application code. Have custom transports, and plug them in as needed. Isn’t this the best possible strategy?

Unsurprisingly, I think the answer is: no - on the contrary, it’s the worst thing you could possibly do.

First of all, there is an architectural difference between the way applications should be designed for tightly coupled vs. loosely coupled interactions. This is mostly related to the granularity of your services, as opposed to the granularity of your components’ or objects’ interfaces. While I believe this argument to be strong, it’s not really related to the technology being used - you can just as easily build a loosely coupled system based on, say, JMS. In fact, I have seen customers do exactly that - independent components on a common bus, with the ability to add and remove components that listen and send to specific topics or queues, allowing for great flexibility in system evolution.

But there is another aspect, and a downside that seems to be less clear: Standardizing on the API doesn’t buy you very much in terms of interoperability.

Let’s say you standardize an API used to access services (or objects or whatever) that reside somewhere in your infrastructure. The effect is that you have a high interoperability between the partners that use the same API. You can change the API implementation, and everyone using that API will be interoperable again - in the cases where you use dynamic linking, this will happen instantaneously.

But what about others? What about third party products, or products developed in different departments of a large organization? Do they use the same API? Are they even developed in the same programming language, and if not, do you have an implementation of the API for that programming language as well? How sure can you be that even if you have one, it’s in sync with the other implementations?

With a protocol like IIOP (which is what CORBA and J2EE use for transport), there is simply no way you could standardize the message on the wire format level. There’s no easy way to describe it, and the only way to make sure everybody can interoperate is that you use the same CORBA version and 100% compatible implementations. Of course, CORBA interop has become a lot better in the last few years. But problems always occur when the underlying format needs to be changed - as is the case e.g. for transactions (with the need to propagate a transaction context) or security.

The beauty of SOAP - wow, that alone should have somebody flaming me - is that it actually makes it very easy to standardize on the format level. XML in general, and SOAP headers in particular, are very easily extensible. Basing your interop standards on a certain kind of SOAP message, including standards-based and non-standards-based headers, yields interoperability on the wire level. This in turn, enables a C++ app to talk to a Java or C# one, and if there’s anything e.g. in the SOAP message’s header that is specific to a certain type of application or interaction, implementations that don’t understand this header can simply ignore it. With the level of support from a standardization perspective — after all, other people are likely to experience the same problems that any given organization does, so it’s likely there’ll be a common standard at some point in time —, and with more and more applications that provide SOAP interfaces out of the box, integrating applications becomes an order of magnitude easier.