[FoRK] Whither BEEP? Re: HTTP 0.2, 9p, and op
sdw at lig.net
Thu Dec 24 19:47:08 PST 2009
I remember Marshall Rose announcing BXXP in about 1999 or 2000, however
the RFC is dated March 2001. SCTP was in an RFC in Oct. 2000. It would
be interesting to know the relationship. Did Marshall announce BXXP and
cause others to work on SCTP? Did SCTP work inspire Marshall to create
a complementary (somewhat) solution at the "application protocol
building block" level? We ended up not using either very much for
various reasons. It is possible that both undermined the other.
We get by on HTTP by sheer bandwidth, CPU, memory, and lack of good
competition. That is even somewhat true in mobile now, although I think
they may be using a rate-based tunnel to a web proxy to avoid severe
problems from packet loss, slow start on multiple streams, etc. The
Palm Treo / Centro models used a web proxy that resized images,
filtered, etc. Seemed to slow things down a lot because, I think, the
proxy was either underpowered or it throttled. I'm glad the current
generation, Pre et al, are past that.
Sean Conner wrote:
> It was thus said that the Great Stephen Williams once stated:
>> Similar, but without assumptions on transport and with IETF application
>> protocol like framing (ASCII), yet very lightweight and able to be
>> optimized (pad numbers, etc.).
> Ah, silly me ... I thought that IP was the networking protocol that didn't
> make assumptions on transport, but then again, I never did like the OSI 7
> layer burrito.
>> If SCTP were easily used both as the transport and over an arbitrary
>> transport and if it included TLS, then it would give you most or all of
>> what BEEP provides.
> I seem to be missing something. SCTP rides on top of IP; what else would
> you need SCTP to ride on top of, if IP isn't available? (And yes, there are
> RFCs covering IP over SCSI, ATM and avaian carriers, so I'm having
> difficulty seeing where you would have SCTP and *not* IP)
TCP, UDP, UDT, Skype, XMPP, AMQP/RabbitMQ, HTTP, SCTP->app->SCTP->app...
If you can't do SCTP directly, or if your problem involves multiple hops
or nesting, then you need to layer. There is a lot more to the world
than a point to point link, even with full Internet routing.
>> Another part of some problems is end-to-end flow control. It would be
>> convenient if you could map transactions, messages, and chunks through
>> both each transport and over the whole message path using the same
>> mechanism, with or without various kinds of nesting.
> I'm haing trouble parsing that paragraph. What do you mean by it all?
The problems that networking protocols solve appear at multiple levels
of abstraction and also sometimes need to be nested. Let's say you are
one of the few who realize that doing everything as chains of
synchronous RPC calls is terribly inefficient, wasteful, and provides a
fraction of the possible throughput. Using asynchronous message queues,
you send messages through TCP/UDT/SCTP/* to the first server. That
server may do some processing and/or it may be a message router / broker
/ bus. It produces messages that flow on TCP/SCTP/... to the next
tier. That tier does the same, and so on. Now, in a simple application
scenario, the sender may be just farming out of a few application
requests to get some pipelining and basically expecting the results
quickly. Readers all along the pipeline are expected to always pretty
much keep up with writers. Describes some Wall Street apps.
Now, what happens when the writer has a lot to send and it is always
faster than one or more tiers downstream? The chokepoints could be CPU,
an Internet data link, writing to storage, etc. The result: packets
pile up in memory at the various queues because the individual
TCP/SCTP/UDT links are only doing flow control for their endpoints. So,
either your application is fragile and takes huge resources, or you
build additional flow control at the next layer up.
I've architected, designed, and built solutions for this a few times.
Here is a SIP example:
> In the following figure, a request starting at the caller traverses a
> UDP hop, an SCTP hop, and a TCP hop before reaching the callee.
Works fine for voice traffic, might work with enough bandwidth for
video, and would fall apart completely without a higher level flow
control for any serious bulk transfer.
>> SCTP as native & library, with UDT/Rate based flow, TLS, full TCP/select
>> semantics, and able to run over raw packets, TCP, UDP, UDT, HTTP, and
>> libOpenSkype, with a coherent solution for link vs. end-to-end flow
>> control would be the ultimate solution. Where libOpenSkype == automatic
>> secure communication linkage between any 2 or more endpoints based on a
>> unique ID, secure authentication mechanism, and over any required
>> combination of communication link types to get the most efficient
>> communication path: local packets, TCP, UDP, UDT, routed directly in
>> whichever direction is required or routed through a "nearby" system with
>> some policy. All intermediated by either a DNS-like neutral
>> coordination server, or ISP-nearby, or private configured coordination
>> server. Perhaps also with distributed self-healing hash/cache like HTTP
>> cache / memcached + BitTorrent, pub/sub propagation, etc.
> Um ... yeah. The only thing I have to say is ... I hope it doesn't break
Although I buzzworded that to the max, an open solution that fits that
description would benefit a lot of people. And a closed version already
exists that provides the core of that: Skype does all of that automatic
virtual secure network plumbing. It just isn't the solution in general
because they have to keep control to retain their value.
>>> I took one look at the RFC for BEEP (RFC-3080) and saw:
>>> At BEEP's core is a framing mechanism that permits simultaneous and
>>> independent exchanges of messages between peers. Messages are
>>> arbitrary MIME  content, but are usually textual (structured using
>>> XML ).
>>> Okay, so now my application needs to link against a MIME parser and an XML
>>> parser (SAX, lighter weight but difficult to use, or DOM, easy to use but my
>>> God the memory requirements ). So now my app becomes a bit more
>>> complicated. Thanks, I'll stick to UDP .
>> Those are naive assumptions. First, for a typical application, you
>> would only implement the Mime types you are interested in, which
>> basically means you write a fixed string and read a fixed string, puking
>> on anything else. A set of applications could even define private mime
>> types as very short strings. The point is that the messages are typed
>> and allow Mime-compatible strings. No library, no parsing.
> No library I'll grant you, but there's still parsing, even if it is
> minimal in nature.
2 strcmp's for a typical application. If minimal, fixed size "mime"
types were used, a quick compare of 4 bytes.
>> BEEP messages are opaque and can be anything you want, including Google
>> Protocol Buffers, XML, SMTP, images, etc. The RFC is just saying that
>> XML messages work well with the text-based framing so you can debug
>> easily, in keeping with SMTP, Imap, HTTP, etc.
> As I keep reading the BEEP specification, I keep asking myself "Self,
> what's the point?" BEEP over TCP includes a window size, which is
> duplicating what TCP already provides (but I can see why it's done, which is
> to get around the whole "TCP dropped packets and blocking" issue, which gets
> into the whole "leaky abstraction" thang and seems to introduce yet another
> layer that can break on a whim).
> Looking even further, I see SOAP over BEEP and XML-RPC over BEEP and well
> ... it just seems overly engineered and complicated (just bcecause it can)
> just to schlep bits from point A to point B and back.
Then why put SOAP over HTTP? Why do you need HTTP at all? Answer that,
and you will have taken your first step.
For just part of the answer, and one of the features that few other
protocols have: BEEP provides flow control on multiple simultaneous
channels, all running over a single TCP (or whatever) connection. This
means you could many separate IM/chat/presence channels with minimal
traffic (no need to keep identifying the destination), including bulk
data transfer at lower priority, all playing nicely with TCP and sharing
packets when possible. Instead of sending 5 small packets, plus acks,
for 5 separate IM / data transfer channels, you can send 1 packet with
the data for all of them. Huge win in some cases.
> It also seems that a bunch of these layers exist solely to bypass
> firewalls run by facist network admins that only allow ports 80 (which is
> invisibly proxied anyway: http://boston.conman.org/2009/11/17.1) and 443
> (and really, what's the port of allowing *any* open port if you can tunnel
> anything through it anyway? http://boston.conman.org/2009/11/18.1), which
> means that firewalls have to get more instrusive, complicated, and make
> network troubleshooting all that more difficult (true story: setting up a
> route for a client in their cabinet at a large data center. One hour later
> and we still can't get to it from outside because a) the IP is private, and
> b) the consultants we're working with for the client can't figure out how to
> forward the network traffice from the public side to the router).
Actually, none of this is to bypass firewalls. Layering everything over
HTTP is often done for that reason. BEEP, if it had become popular,
would have been its own border protocol with explicit firewalling just
like HTTP. Already, various protocols are supported directly for
security filtering by the Cisco ACE product, including XML messages,
HTTP, and others. It does all of the SSL/TLS wrapping and unwrappting,
determine what is allowed where, creates audit trails when necessary,
and is intended to be the corporate central switching point for message
queue traffice, web, email, SOA, database (I think), etc.
And Skype is already a master at getting through firewalls. Because
they kept tight control of the application, have great although opaque
security, and didn't go down the p2p file route (again, it's from the
guys who did Kazaa), it is ignored by many corporate and other
For more open networks, uPnP mostly solves the reachability problem, at
least for UDP. Not sure if SCTP would flow through or not. Good thing
>> Have you used SCTP? Has anyone?
> Well, _Unix Network Programming, 3rd Edition_ shows that SCTP is used for
> ISDN over IP, M2UA, M3UA (SS7 telephony signaling), H.248, H.323 (IP
> telephony) and SIP. So it is in use.
SS7 is a bit esoteric for most companies and people.
It looks like SIP is not actually available and used over SCTP yet:
> What are the challenges of building a large scale SIP-based signaling
> network using SCTP as the foundation?
> -spc (Having a hard time wrapping my brain around all these layers)
More information about the FoRK