[FoRK] scribd sucks dead dogs balls

Stephen D. Williams sdw at lig.net
Sat Nov 15 17:15:21 PST 2008

Dan Brickley wrote:
> damien morton wrote:
>> On Sun, Nov 16, 2008 at 9:58 AM, Luis Villa <luis at tieguy.org> wrote:
>> ...
>>  Merely putting this or that into the browser isn't sufficient if users
>> cannot directly connect to each other. I know how to open my 
>> firewalls to
>> bittorrent, but from what I can see by looking at the other clients when
>> downloading, the great majority of users can not or do not do so.
>> I can imagine putting it all together with http and range requests.
> Yup, Opera integrated BitTorrent a couple of years ago. 
> http://www.opera.com/pressreleases/en/2006/02/06/ but it seems 
> primarily targeted at consumers rather than publishers of data.
> Re firewalls/NAT, I wonder whether XMPP/Jabber might provide a 
> poor-users P2P. I've made a few experiments with it for publishing 
> query services - http://danbri.org/words/2008/02/11/278 - but not for 
> bulk data.
One concern is transferring the data directly rather than through a server.
Another is how the transfer is coordinated to begin with.  What is the 
presence, communication, and routing method?  Is there an external 
authentication method?  How is spam, spoofing, feeding malware, etc. 

You can layer any kind of communication over a sufficiently complete 
communication mechanism.  A good presence / messaging architecture like 
XMPP/Jabber is sufficient.  As is email.  The only real problem with 
Jabber is that ISPs have not quite had enough incentive to run their own 
servers, like they must for email.  This means that it doesn't really 
solve the bandwidth issue properly.

The point of adding P2P (similar to Bittorrent) to web publishing is 
that it allows any web server to gracefully handle any surge in traffic 
in a very inexpensive or free way.  Amazon S3 has it, which is cool.  It 
ought to be nearly automatic to get the same capability in Apache / 
Lightppd / various web services / etc.  It should be sufficient to 
simply publish the file.  If web browsers, when retrieving a file, were 
able to signal with a transfer encoding or something that they can 
handle "webtorrent", they would then go to the P2P mode to get the 
file.  If that fails, they then go back to the web server.  Then, as 
long as they have that browser session open, at least, they then serve 
the file back to the P2P network.  Additionally, any client should be 
able to be sure that they have received only the unmodified data.

While firewalls/NAT are a problem, there are standard methods that will 
get through most of those now, especially if the connection is 
coordinated via a server on the web.  This server acts a go between to 
get the connection started, however the data flows directly between 
endpoints.  It does this by having both ends open a connection to the IP 
address of the other end point at the same time.  The firewall/NAT, 
seeing outgoing streams, will allow corresponding incoming streams at 
both ends, allowing direct bidirectional transfer.  This is widely used, 
however not necessarily available as a good raw component.  Has anyone 
tracked down libraries for this lately?  I have a variety of interests 
on this, including the desire to do an open source Skype, but mainly 
having to do with apps that can transfer bulk data directly between 
multiple users, a la Groove, but better.

> cheers,
> Dan
> -- 
> http://danbri.org/

More information about the FoRK mailing list