[FoRK] Whither BEEP? Re: HTTP 0.2, 9p, and op
sdw at lig.net
Thu Dec 24 15:02:41 PST 2009
Sean Conner wrote:
> It was thus said that the Great Dr. Ernie Prabhakar once stated:
>> Hi all,
>> On Dec 24, 2009, at 10:39 AM, Stephen Williams wrote:
>>> Anything sane in a general sense must be async, pipelined, channelized
>>> lightweight message oriented with end to end flow control and adaptive
>>> rate handling.
> Sounds like SCTP .
Similar, but without assumptions on transport and with IETF application
protocol like framing (ASCII), yet very lightweight and able to be
optimized (pad numbers, etc.).
If SCTP were easily used both as the transport and over an arbitrary
transport and if it included TLS, then it would give you most or all of
what BEEP provides.
Since it is implemented natively on many systems, and it seems to be
well thought out, perhaps the ideal solution is to use SCTP when it is
available at both endpoints and can pass, or to use SCTP layered over
one or more alternatives. I happen to be designing yet another async
messaging solution, so it's a convenient time to think about it.
I wonder if A) SCTP implementations all include TLS  and B) if SCTP
includes rate-based  transmission and all of the improvements in UDT .
Another part of some problems is end-to-end flow control. It would be
convenient if you could map transactions, messages, and chunks through
both each transport and over the whole message path using the same
mechanism, with or without various kinds of nesting.
SCTP as native & library, with UDT/Rate based flow, TLS, full TCP/select
semantics, and able to run over raw packets, TCP, UDP, UDT, HTTP, and
libOpenSkype, with a coherent solution for link vs. end-to-end flow
control would be the ultimate solution. Where libOpenSkype == automatic
secure communication linkage between any 2 or more endpoints based on a
unique ID, secure authentication mechanism, and over any required
combination of communication link types to get the most efficient
communication path: local packets, TCP, UDP, UDT, routed directly in
whichever direction is required or routed through a "nearby" system with
some policy. All intermediated by either a DNS-like neutral
coordination server, or ISP-nearby, or private configured coordination
server. Perhaps also with distributed self-healing hash/cache like HTTP
cache / memcached + BitTorrent, pub/sub propagation, etc.
>>> HTTP-like messages over BEEP or AMQP would be fine.
>> Just out of curiosity -- does anyone know why BEEP doesn't seem to have
>> taken off? It seems more general and lighter than, say, XMPP, and on
>> alternate weeks I agree with you that it makes more sense than abusing
> I took one look at the RFC for BEEP (RFC-3080) and saw:
> At BEEP's core is a framing mechanism that permits simultaneous and
> independent exchanges of messages between peers. Messages are
> arbitrary MIME  content, but are usually textual (structured using
> XML ).
> Okay, so now my application needs to link against a MIME parser and an XML
> parser (SAX, lighter weight but difficult to use, or DOM, easy to use but my
> God the memory requirements ). So now my app becomes a bit more
> complicated. Thanks, I'll stick to UDP .
Those are naive assumptions. First, for a typical application, you
would only implement the Mime types you are interested in, which
basically means you write a fixed string and read a fixed string, puking
on anything else. A set of applications could even define private mime
types as very short strings. The point is that the messages are typed
and allow Mime-compatible strings. No library, no parsing. Second,
BEEP messages are opaque and can be anything you want, including Google
Protocol Buffers, XML, SMTP, images, etc. The RFC is just saying that
XML messages work well with the text-based framing so you can debug
easily, in keeping with SMTP, Imap, HTTP, etc.
> But overall, it sounds like it's trying to do SCTP.
It was, for the application level. Their stated goal was to create the
base application framing for most non-low level IETF application protocols.
> -spc (A greybearded C programmer)
>  Not My Footnote
>  Not My Footnote
>  Yeah yeah yeah, I know, gigs of RAM these days, it doesn't matter,
> but I say Nay! Nay! It *does* matter!
> Okay, at least to me, it does.
>  Stream Control Transmission Protocol---think reliability and
> congestion control of TCP but message based like UDP, and can even
> work of one, or both, of the endpoints change their IP address.
> And it's at the same level as TCP and UDP.
>  Used UDP in my greylist daemon , which wasn't that difficult to
> use actually.
>  http://www.x-grey.com/
> Implements an anti-spam technique that works with both sendmail and
> postfix (and should support other MTAs with a minimum of work). It
> uses UDP as the underlying protocol.
Cool. UDP is nice, except when you need efficient security
(authentication, encryption), minimal message count (NOT one message per
packet), large messages (multiple packets to make a message), or any
serious flow control (OS can be better at dealing with start/stop,
resend, etc.), or you want to maximize throughput / separation of tasks
(multiple tasks sharing 1 TCP port). || fail
Have you used SCTP? Has anyone?
More information about the FoRK