REST, RPC, mountains, molehills, and a retraction (sort of)

Jeff Bone jbone@jump.net
Wed, 08 Aug 2001 13:11:05 -0500


So...

Over the course of the FoRKBreak, I've had several good discussions
with various folks about this whole REST vs. RPC thing, in particular
a great and detailed exchange with Mark Baker.  (Thanks Mark!)
Surprisingly, and as a result, my position on the matter has shifted
somewhat;  previously, my position was basically that REST was much
ado about nothing, a matter of turning mountains into molehills, or at
worst a case of retroactively taking the incidentals of Web
development to date and casting them as architectural imperatives.
I've come to the conclusion that there's something of significant
value in REST, with serious consequences.  I thought I'd write up a
bit about my current thinking on this, FoRK it, and gather some
comments.  I'd like to offer a bit of a different perspective on
what's important about REST, why, and how to "sell" it.  To start off,
let's distinguish the REST and RPC design styles, and recap the
arguments that are usually used to support REST.  (This is also in
part to test my newer / deeper understanding of the REST argument.)


THE REST ARGUMENT
---------------------------

REST is (IMO) a design style which mandates that all objects referred
to by URIs (a) represent, in some sense, entities of interest rather
than behaviors, actions, etc., and (b) share a generic interface
defined completely and solely by the specified semantics of HTTP, i.e.
GET / PUT / POST, etc.  "Entities" is taken to mean nouns or
collections of nouns representing objects from a particular
application domain.  REST argues that RPCs, by having method names ---
in URIs, in the bodies of posts, etc. --- breaks one or both of these
characteristics.

The problem with the REST arguments that I've been exposed to at this
point are twofold.  First, they fail to strongly emphasize the
"virtues" of REST and why everybody can and should embrace REST and
pay the costs associated with doing so.  Second, they overemphasize
the "dire" consequences of breaking with REST, possibly because they
rest on faulty or narrow assumptions about how developers ---
particularly enterprise IT guys --- view the Web.  REST argues
vehemently that such breakage has dire consequences:

    (1)  It "breaks" the architecture of the Web, particularly caching

    (2)  It fails to scale
    (3)  It results in significantly higher coordination costs

Let's look at these.  The caching argument (1) is, IMO, the weakest of
the arguments.  The caching architecture of the Web isn't always a
good thing, or the right thing;  the (in practice) loose semantics of
POST has been a good way of dealing with this.  (2)  is not a
particularly good argument against RPC, either;  first, there are many
dimensions of scaling, and which dimensions are most important is an
application-specific consideration.  Not everybody is building Hotmail
or HotOrNot;  an integration pipeline between two corporate apps has
different scaling requirements, availability requirements, etc. than a
broad-based service like Hotmail, and the right approach to one isn't
necessarily the right approach to the other.  (3) is probably the most
compelling but under-elaborated argument;  a service offered in a REST
style will inherently be more "consumable" than some complex API, and
will have a lower learning curve for the consumer and lower support
overhead for the producer.


SUBTLE EXAMPLE
----------------------

Let's consider an example, drawn directly from my conversation with
Mark.  Consider a StockTicker service...  (Is it just my imagination,
or is this getting to be the "Hello, World" of the Web Services
world?)  We have a StockTicker class which exports a single method,
String getStockPrice(String strSymbol).  (Side note:  it's trivial to
write "glue" code that introspects a class definition and exports the
interface via HTTP-based RPCs of various flavors;  .NET provides a
facility for doing this, and various Pythonish things can do
similar...)  A non-REST or RPC-like interface for this thing,
foregoing SOAP for the moment and assuming URI + query string calling
conventions, might look like this:

   GET  http://localhost/StockTicker/getStockPrice?strSymbol=MSFT

The REST position is, essentially, that this an RPC, breaks the Web
and causes all sorts of nastiness, and is very RESTless.  The basic
problem is that the method --- getStockPrice --- doesn't represent a
resource;  it's not clear what what it means to GET, PUT, and POST to
something called "getStockPrice".  However, a simple change of
nomenclature from "getStockPrice" to "CurrentStockPrice" suffices!
I.e., the first example below is entirely valid from a REST design
perspective, and the second even better:

   GET  http://localhost/StockTicker/CurrentStockPrice?strSymbol=MSFT
   GET  http://localhost/StockTicker/MSFT/CurrentStockPrice

Obviously, the differences between "good REST design" and "bad RPC
design" can be quite subtle.  All of these are variations which can
easily be mapped / transformed to each other via simple manipulation
of URI format;  the difference is minimal and abstract.  Hence, my
conclusion is that, if you only consider the kinds of arguments that
have been presented so far, REST is merely a design style, not an
architectural style.  Furthermore, as a design style, it's one that
when pushed to its extreme *can* have significant costs for
developers;  rather than developing object models using traditional
OOP design techniques and then automagically exporting them to the
Web, developers must decompose their objects in a different fashion
and, perhaps, even go to a phenomenal amount of trouble to explicitly
provide the marshalling / unmarshalling / parsing of things to and
from the request and response streams.  StockTicker is much more work
to build as a servlet, say, than as a Web-naive class that is
automagically published to the Web via some RPC framework...


THE COORDINATION COST ARGUMENT
-------------------------------------------------

The coordination cost argument is the strongest of the arguments for
REST, but it hasn't really been fully elaborated.  The argument is
that generic interfaces require less coordination between producers
and consumers than elaborate object models where each entity has a
unique interface.  This is hard to deny;  one of the key strengths of
UNIX has always been its software tools philosophy, i.e. the ability
to string together lots of little special-purpose tools with generic
interfaces.

The immediate reaction by any verteran of the protocol wars will be to
argue as follows:  what happens when you need application semantics
that don't fit into the GET / PUT / POST generic interfaces and
representational state model?  Protocol vets will immediately assume
that the REST argument is one of the following:

   (1)  If it doesn't fit, it's not HTTP, build another protocol
   (2)  Extend HTTP by adding new HTTP methods

Neither of these is a particularly attractive, efficient, or practical
solution.  But wait....  it turns out that this isn't really the REST
perspective.


WHAT'S GREAT ABOUT REST?!
-------------------------------------

It turns out that the best argument for REST is this:  *there are no
applications you can think of which cannot be made to fit into the GET
/ PUT / POST / resources / representations model of the world!*  You
never need RPC, because these generic interfaces are sufficiently
general;  indeed, RPC is considered harmful because it increases the
costs of consuming a particular service.  It took some convincing, but
over the last week's experimentation and interaction with Mark I've
come to believe this.  REST is a coordination language, in the same
sense as Linda (specifically) or UNIX (in a general sense);  with a
little design thought up-front, it is complete enough and flexible
enough to subsume all the application semantics you might want.

One of the original design decisions in UNIX was "everything is a
file."  Files had a narrow and generic interface, and the idea was
that all resources in the system could be accessed through these
narrow interfaces.  The advent of networking broke with this design
philosophy, yielding a space of network IO that was disjoint and had
different interfaces than other system resources.  Plan 9 went back to
the drawing board and forced the issue;  everything *truly* is a file
in Plan 9, and the design of applications and services on Plan 9
centers around designing resource or service namespaces to represent
application domain abstractions.

This is *precisely* the design philosophy of REST;  rather than
(psuedo-)files, we have resources;  rather than a filesystem namespace
we have URI-space;  and the narrow generic interface is slightly
different.  But a REST design philosophy is the same kind of
philosophy;  it centers around a generic shared abstraction, rather
than around point-to-point interface coordination.

Coordination languages are elegant, simple, and powerful.  Viewing
HTTP as such a coordination language --- reconsidering your app in
terms of the provided interfaces and semantics rather than trying to
figure out how to subvert or extend HTTP to encompass what you think
your application semantics are --- is a useful and powerful
perspective.


ASIDE:  WHY RPCs AND DISTRIBUTED OBJECTS HAVE "FAILED"
-----------------------------------------------------------------------------

Mark and others have offered the hypothesis that RPCs and distributed
objects have in some sense "failed" --- at least in the sense that
they haven't achieved the ubiquity of today's Web --- because of the
high degree of complexity and coordination involved in having
application-specific object models and wide, specialized interfaces.
First, they haven't really "failed' to serve their purpose.  ONC and
DCE RPC are the basis of lots of enterprise software, even some
widely-deployed "horizontal" systems like NFS.  CORBA and DCOM are in
*lots* of industrial-strength enterprise software.  While it's
undeniable that specialized interfaces have much higher coordination
costs than generic ones, I believe that the "failure" to achieve
ubiquity by the various RPC-like things is somewhat less
fundamental...

All of the aforementioned mechanisms require *administrative*
coordination as well as developer coordination;  the costs to
"federate" and manage such systems, especially across adminstrative
domain boundaries, are significant.  In particular, there's a serious
difference in architectural understanding between the developers who
create such systems and the admins who must support it, and this leads
to all kinds of problems and costs.  Over the lifetime of a given
system, these costs can swamp the coordination costs in development.
The Web has *none* of these issues, and that --- the ability to
independently deploy systems coupled with the reasonable understanding
of Web architecture on the part of the admin community --- has been a
significant factor in the phenomenal adoption of the Web, particularly
as an integration framework.


WHAT'S WRONG WITH REST?
------------------------------------

REST has some "warts" that need to be dealt with.  For starters, there
are some fundamental architectural constraints to the Web's
interaction style which need to be dealth with.  The space of
"connectors" for the Web is insufficiently rich for all desirable
application communication patterns;  KnowNow addresses one particular
example of this, but there are others.  Rather than a world of unique
implementations of such things, it would be better if standard
interfaces for a variety of such connectors were defined.

Furthermore, there's a real impedance mis-match between the design
perspectives of the OOP /  IT developer communities and REST;
RPC-over-HTTP does not have this problem.  It's a *really attractive*
proposition to the OOP / IT guys to think of taking an existing object
model and, with no or minimal Web-specific "glue", simply exporting
those interfaces to the Web.  (The problems creep in down the road;
Adam Bosworth's frequent comments on the implicit state transitions in
using APIs are particularly pertinent.  His solution appears to be,
rather than use generic interfaces, extend interface descriptions to
encompass the state transition model of the API.  Ick.)

The REST alternative seems to be:  "re-think your design in terms of
generic interfaces, then build servlet-style implementations of each
resource / sub-URL, with the attendant possibility of having to
essentially unpack / pack Request and Response objects...  i.e., the
glue is up to you."  Now, doubtless, there's a lot of servlet and Web
development knowledge out there;  but most enterprise developers, data
modellers, etc. I know still live in UML- and OOP-land, and this is as
significant a change as the procedural->object transition of a decade
or so ago.

How can the gap between REST and OOP be bridged?  The differences
between a REST and RPC interface design can be subtle;  the example
above was ripped from a .NET example, and so clearly you *can*
automate the publication of appropriately designed, non-servlet object
models to the Web while conforming to REST.  This may not be true in
the general case, however;  and the only thing that can really serve
in the macro sense is broad education of what, exactly, the REST style
*is*, how to design to it, and why it's a good thing.  On the other
hand, the REST community should work with the Web Services community
to ensure that things like WSDL, UDDI, etc. are sufficiently rich to
be useful in the REST model of the world...  these are two different
design styles, informed by different needs and values, but they should
share a technology base to the largest extent possible.


I'm sold;  I just wish there was a larger body of literature about
REST out there already.  Would've saved some painful flaming with Roy
and lots of interaction with Mark on my part...  ;-)  (Where's Dave in
all this???)

$0.02,

jb