[FoRK] Whither BEEP? Re: HTTP 0.2, 9p, and op

Stephen Williams sdw at lig.net
Thu Dec 24 15:02:41 PST 2009

Sean Conner wrote:
> It was thus said that the Great Dr. Ernie Prabhakar once stated:
>> Hi all,
>> On Dec 24, 2009, at 10:39 AM, Stephen Williams wrote:
>>> Anything sane in a general sense must be async, pipelined, channelized
>>> lightweight message oriented with end to end flow control and adaptive
>>> rate handling.
>   Sounds like SCTP [4].

Similar, but without assumptions on transport and with IETF application 
protocol like framing (ASCII), yet very lightweight and able to be 
optimized (pad numbers, etc.).

If SCTP were easily used both as the transport and over an arbitrary 
transport and if it included TLS, then it would give you most or all of 
what BEEP provides.
Since it is implemented natively on many systems, and it seems to be 
well thought out, perhaps the ideal solution is to use SCTP when it is 
available at both endpoints and can pass, or to use SCTP layered over 
one or more alternatives.  I happen to be designing yet another async 
messaging solution, so it's a convenient time to think about it.

I wonder if A) SCTP implementations all include TLS [1] and B) if SCTP 
includes rate-based [3] transmission and all of the improvements in UDT [2].

Another part of some problems is end-to-end flow control.  It would be 
convenient if you could map transactions, messages, and chunks through 
both each transport and over the whole message path using the same 
mechanism, with or without various kinds of nesting.

SCTP as native & library, with UDT/Rate based flow, TLS, full TCP/select 
semantics, and able to run over raw packets, TCP, UDP, UDT, HTTP, and 
libOpenSkype, with a coherent solution for link vs. end-to-end flow 
control would be the ultimate solution.  Where libOpenSkype == automatic 
secure communication linkage between any 2 or more endpoints based on a 
unique ID, secure authentication mechanism, and over any required 
combination of communication link types to get the most efficient 
communication path: local packets, TCP, UDP, UDT, routed directly in 
whichever direction is required or routed through a "nearby" system with 
some policy.  All intermediated by either a DNS-like neutral 
coordination server, or ISP-nearby, or private configured coordination 
server.  Perhaps also with distributed self-healing hash/cache like HTTP 
cache / memcached + BitTorrent, pub/sub propagation, etc.
>>> HTTP-like messages over BEEP or AMQP would be fine.  
>> Just out of curiosity -- does anyone know why BEEP doesn't seem to have
>> taken off? It seems more general and lighter than, say, XMPP, and on
>> alternate weeks I agree with you that it makes more sense than abusing
>> HTTP.
>   I took one look at the RFC for BEEP (RFC-3080) and saw:
> 	At BEEP's core is a framing mechanism that permits simultaneous and
> 	independent exchanges of messages between peers.  Messages are
> 	arbitrary MIME [1] content, but are usually textual (structured using
> 	XML [2]).
>   Okay, so now my application needs to link against a MIME parser and an XML
> parser (SAX, lighter weight but difficult to use, or DOM, easy to use but my
> God the memory requirements [3]).  So now my app becomes a bit more
> complicated.  Thanks, I'll stick to UDP [5].

Those are naive assumptions.  First, for a typical application, you 
would only implement the Mime types you are interested in, which 
basically means you write a fixed string and read a fixed string, puking 
on anything else.  A set of applications could even define private mime 
types as very short strings.  The point is that the messages are typed 
and allow Mime-compatible strings.  No library, no parsing.  Second, 
BEEP messages are opaque and can be anything you want, including Google 
Protocol Buffers, XML, SMTP, images, etc.  The RFC is just saying that 
XML messages work well with the text-based framing so you can debug 
easily, in keeping with SMTP, Imap, HTTP, etc.
>   But overall, it sounds like it's trying to do SCTP.

It was, for the application level.  Their stated goal was to create the 
base application framing for most non-low level IETF application protocols.

[1] http://www.ietf.org/rfc/rfc3436.txt
[2] http://udt.sourceforge.net/
[3] http://www.ee.duke.edu/~romit/pubs/xrtp.pdf
>   -spc (A greybearded C programmer)
> [1]	Not My Footnote
> [2]	Not My Footnote
> [3]	Yeah yeah yeah, I know, gigs of RAM these days, it doesn't matter,
> 	but I say Nay!  Nay!  It *does* matter!
> 	Okay, at least to me, it does.
> [4]	Stream Control Transmission Protocol---think reliability and
> 	congestion control of TCP but message based like UDP, and can even
> 	work of one, or both, of the endpoints change their IP address.
> 	And it's at the same level as TCP and UDP.
> [5]	Used UDP in my greylist daemon [6], which wasn't that difficult to
> 	use actually.
> [6]	http://www.x-grey.com/
> 	Implements an anti-spam technique that works with both sendmail and
> 	postfix (and should support other MTAs with a minimum of work).  It
> 	uses UDP as the underlying protocol.
Cool.  UDP is nice, except when you need efficient security 
(authentication, encryption), minimal message count (NOT one message per 
packet), large messages (multiple packets to make a message), or any 
serious flow control (OS can be better at dealing with start/stop, 
resend, etc.), or you want to maximize throughput / separation of tasks 
(multiple tasks sharing 1 TCP port). || fail

Have you used SCTP?  Has anyone?


More information about the FoRK mailing list