[FoRK] Stand up and face the future
sdw at lig.net
Mon Dec 21 11:36:13 PST 2009
Jeff Bone wrote:
> Stephen says:
>> Buddylist @ AOL in 1995-1998 handled 3-8 million simultaneous users
> Mmm hmmm.... and *surely* its system architecture, including
> technical architecture (mapping of the software architecture onto
> hardware resources, including the underlying networking
> infrastructure) could easily and linearly scale by the requisite 3
> orders of magnitude necessary to handle every person on the planet.
> (It couldn't, of course. The networking infrastructure alone required
> for such a centralized service didn't exist at that time, and the
> solution is clearly subject to the "social supernode" problem eluded
> to in the linked article. And as I recall, such fails among other
> things inspired AOL's purchase of Mirabilis, which even then continued
> to have occasional outages as spikes ripped past the pace at which
> they could grow system capacity... Ding! circa the same timeframe
> tackled the first problem by being entirely decentralized /
> peer-to-peer, with uncoupled "servers" only serving to accomplish
> rendezvous and to host presence information when a user was offline,
> and otherwise leveraging the connectivity aspects of the social
> network graph along with peering in order to squeeze an order or two
> magnitude greater scalability (in theory) out of the system as a
> whole. But even that sort of architecture doesn't address the
> supernode problem, or supernode clustering, etc....)
A) Most architectures seem to do it wrong the first time or two. I saw
many people assume that everything had to go through a database server,
which is silly.
B) My response was tongue-in-cheek in the spirit of "That's the same
thing we were doing..."
C) Clearly, distributed systems with highly interconnected groups of
people can work just fine. See email, etc.
> Now add every cell phone. Now add in every building. Every room in
> every building. Every light switch. Every power outlet. Every
> appliance. Every manufactured good --- every book, every can of cola,
> every article of clothing. Every streetlight. Every car, bike,
> train, train car, boat, plane, helicopter, etc. Every webcam peering
> into every space, public or private. Every GPS or other localizer
> embedded in or on something. Every RFID or other smart tag. Every
> piece of paper currency. Every box being shipped... all moving
> around relative to each other. Push the network into everything,
> yammering at everything else. How many orders of magnitude? How many
> names / network addresses *per person on the planet?* How many
> simultaneous conversations? How much presence data at a time? How
> much historical presence and communication data generated merely by
> logging? How to mine all that data, much less sift through it and
> react to it in real time?
Mining, sifting, and real-time do not go together. At least not in the
> We can't even begin to understand the implications because, among
> other things, we only have a very tentative and nascent model for even
> *human* networks at present, and that only captures static structure
> rather than, particularly, dynamics. The communication patterns
> (static and dynamic) that will be common on the Internet of Things
> aren't particularly well-understood or even much-studied at present.
> (Robert Poor to the contrary...)
> It's truly difficult to envision applications of such higher-order
> scale vs. what you presently have at your disposal; about ten years
> ago one of the founders of Grande Communications, at the time of its
> inception, asked me what kinds of qualitatively different killer apps
> you could build if you had a gigabit / sec coming into your house...
> Sadly the only thing I could come up with at the time, despite much
> thought, was cloud-hosted infinite DVR and any-show any-time on-demand
> hi-def television. (And that was actually more of a storage problem
> than a bandwidth / delivery problem, which realization led me down the
> road toward Deepfile...) Maybe interactive hi-def games of a
> qualitatively different (but only evolutionarily so) variety. Ugh.
> Vision fail.
I found advertising distasteful and irritating, so I was blind to the
best revenue engine for too long... And I even wrote the code that
powered the very first popups on AOL and knew the revenue numbers for
quite a while. BTW, the very first popup was actually an FTC required
notice. I had to make sure everyone saw and acknowledged it. Which led
to: Gee, you know what that would be handy for... Vision fail, and even
with glaringly obvious information.
> Forget the apps. I'm sort of with silky on at least part of this, we
> don't even have any idea what might *require* a truly planetary-scale
> network. (Well, I have a few ideas...) To crib a phrase, in such a
> case *the network is the application.* But will it happen? Surely,
> if we don't hit some existential risk boundary first.
> Scale is everything. Two problems: figuring out what to do with it,
> and figuring out how to do it. Not the same. Both essential.
More information about the FoRK