[FoRK] scribd sucks dead dogs balls
Stephen D. Williams
sdw at lig.net
Sat Nov 15 17:15:21 PST 2008
Dan Brickley wrote:
> damien morton wrote:
>> On Sun, Nov 16, 2008 at 9:58 AM, Luis Villa <luis at tieguy.org> wrote:
>> Merely putting this or that into the browser isn't sufficient if users
>> cannot directly connect to each other. I know how to open my
>> firewalls to
>> bittorrent, but from what I can see by looking at the other clients when
>> downloading, the great majority of users can not or do not do so.
>> I can imagine putting it all together with http and range requests.
> Yup, Opera integrated BitTorrent a couple of years ago.
> http://www.opera.com/pressreleases/en/2006/02/06/ but it seems
> primarily targeted at consumers rather than publishers of data.
> Re firewalls/NAT, I wonder whether XMPP/Jabber might provide a
> poor-users P2P. I've made a few experiments with it for publishing
> query services - http://danbri.org/words/2008/02/11/278 - but not for
> bulk data.
One concern is transferring the data directly rather than through a server.
Another is how the transfer is coordinated to begin with. What is the
presence, communication, and routing method? Is there an external
authentication method? How is spam, spoofing, feeding malware, etc.
You can layer any kind of communication over a sufficiently complete
communication mechanism. A good presence / messaging architecture like
XMPP/Jabber is sufficient. As is email. The only real problem with
Jabber is that ISPs have not quite had enough incentive to run their own
servers, like they must for email. This means that it doesn't really
solve the bandwidth issue properly.
The point of adding P2P (similar to Bittorrent) to web publishing is
that it allows any web server to gracefully handle any surge in traffic
in a very inexpensive or free way. Amazon S3 has it, which is cool. It
ought to be nearly automatic to get the same capability in Apache /
Lightppd / various web services / etc. It should be sufficient to
simply publish the file. If web browsers, when retrieving a file, were
able to signal with a transfer encoding or something that they can
handle "webtorrent", they would then go to the P2P mode to get the
file. If that fails, they then go back to the web server. Then, as
long as they have that browser session open, at least, they then serve
the file back to the P2P network. Additionally, any client should be
able to be sure that they have received only the unmodified data.
While firewalls/NAT are a problem, there are standard methods that will
get through most of those now, especially if the connection is
coordinated via a server on the web. This server acts a go between to
get the connection started, however the data flows directly between
endpoints. It does this by having both ends open a connection to the IP
address of the other end point at the same time. The firewall/NAT,
seeing outgoing streams, will allow corresponding incoming streams at
both ends, allowing direct bidirectional transfer. This is widely used,
however not necessarily available as a good raw component. Has anyone
tracked down libraries for this lately? I have a variety of interests
on this, including the desire to do an open source Skype, but mainly
having to do with apps that can transfer bulk data directly between
multiple users, a la Groove, but better.
More information about the FoRK