[FoRK] overhead of RESTful stuff

Stephen Williams sdw at lig.net
Fri Mar 9 11:39:31 PST 2012

The whole server-oriented solution, a la databases or web servers or stupid products like IBM MQ, is mostly broken from a 
scalability, management, power, flexibility point of view.
Some day soon perhaps I'll build a good open source solution.

The best systems allow easy definition / membership of servers, where a server can be a app server, router, database, client, or 
communication gateway.  The communication fabric needs to connect nodes that need to be connected, as directly as possible but 
also through communications concentrators to avoid too many connections.  The equivalent of ZeroMQ or RabbitMQ or similar should 
be built into each endpoint, where some endpoints are concentrators and have little or no processing steps.

I've seen it done and scale giant systems, on far less hardware and network than we have now.  And now the market is getting 
past broken app models that couldn't be fit into a nice architecture.

MPI is a little like this, but far too narrowly focused and perhaps fragile.  Various cloud / NoSQL / etc. new architectures 
sometimes have bits in the right direction.


On 3/9/12 11:20 AM, J. Andrew Rogers wrote:
> On Mar 9, 2012, at 9:12 AM, John D. Mitchell wrote:
>> On Mar 9, 2012, at 08:42 , Stephen Williams wrote:
>> [...]
>>> You're situation sounds perfect for message oriented communication, but simple REST is probably simpler to get working.  And if you're IO bound rather than latency or bandwidth-in-flow bound, it is probably fine.
>> Given the controlled trust boundary and multi-language gluing, you might want to check out 0MQ.
> Yeah, ZeroMQ is another option if the communication patterns are relatively simple (sounds like it), and very fast in its element. Not so good for complex topologies and behaviors but most people don't do that.

More information about the FoRK mailing list