[FoRK] Stand up and face the future

Stephen Williams sdw at lig.net
Mon Dec 21 11:36:13 PST 2009


Jeff Bone wrote:
>
> Stephen says:
>
>> Buddylist @ AOL in 1995-1998 handled 3-8 million simultaneous users
>
> Mmm hmmm....  and *surely* its system architecture, including 
> technical architecture (mapping of the software architecture onto 
> hardware resources, including the underlying networking 
> infrastructure) could easily and linearly scale by the requisite 3 
> orders of magnitude necessary to handle every person on the planet.  
> (It couldn't, of course.  The networking infrastructure alone required 
> for such a centralized service didn't exist at that time, and the 
> solution is clearly subject to the "social supernode" problem eluded 
> to in the linked article.  And as I recall, such fails among other 
> things inspired AOL's purchase of Mirabilis, which even then continued 
> to have occasional outages as spikes ripped past the pace at which 
> they could grow system capacity...  Ding! circa the same timeframe 
> tackled the first problem by being entirely decentralized / 
> peer-to-peer, with uncoupled "servers" only serving to accomplish 
> rendezvous and to host presence information when a user was offline, 
> and otherwise leveraging the connectivity aspects of the social 
> network graph along with peering in order to squeeze an order or two 
> magnitude greater scalability (in theory) out of the system as a 
> whole.  But even that sort of architecture doesn't address the 
> supernode problem, or supernode clustering, etc....)

A) Most architectures seem to do it wrong the first time or two.  I saw 
many people assume that everything had to go through a database server, 
which is silly.
B) My response was tongue-in-cheek in the spirit of "That's the same 
thing we were doing..."
C) Clearly, distributed systems with highly interconnected groups of 
people can work just fine.  See email, etc.
>
> Now add every cell phone.  Now add in every building.  Every room in 
> every building.  Every light switch.  Every power outlet.  Every 
> appliance.  Every manufactured good --- every book, every can of cola, 
> every article of clothing.  Every streetlight.  Every car, bike, 
> train, train car, boat, plane, helicopter, etc.  Every webcam peering 
> into every space, public or private.  Every GPS or other localizer 
> embedded in or on something.  Every RFID or other smart tag.  Every 
> piece of paper currency.  Every box being shipped...  all moving 
> around relative to each other.  Push the network into everything, 
> yammering at everything else.  How many orders of magnitude?  How many 
> names / network addresses *per person on the planet?*  How many 
> simultaneous conversations?  How much presence data at a time?  How 
> much historical presence and communication data generated merely by 
> logging?  How to mine all that data, much less sift through it and 
> react to it in real time?

Mining, sifting, and real-time do not go together.  At least not in the 
traditional forms.
>
> We can't even begin to understand the implications because, among 
> other things, we only have a very tentative and nascent model for even 
> *human*  networks at present, and that only captures static structure 
> rather than, particularly, dynamics.  The communication patterns 
> (static and dynamic) that will be common on the Internet of Things 
> aren't particularly well-understood or even much-studied at present.  
> (Robert Poor to the contrary...)
>
> It's truly difficult to envision applications of such higher-order 
> scale vs. what you presently have at your disposal;  about ten years 
> ago one of the founders of Grande Communications, at the time of its 
> inception, asked me what kinds of qualitatively different killer apps 
> you could build if you had a gigabit / sec coming into your house...  
> Sadly the only thing I could come up with at the time, despite much 
> thought, was cloud-hosted infinite DVR and any-show any-time on-demand 
> hi-def television.  (And that was actually more of a storage problem 
> than a bandwidth / delivery problem, which realization led me down the 
> road toward Deepfile...)  Maybe interactive hi-def games of a 
> qualitatively different (but only evolutionarily so) variety.  Ugh.  
> Vision fail.

I found advertising distasteful and irritating, so I was blind to the 
best revenue engine for too long...  And I even wrote the code that 
powered the very first popups on AOL and knew the revenue numbers for 
quite a while.  BTW, the very first popup was actually an FTC required 
notice.  I had to make sure everyone saw and acknowledged it.  Which led 
to: Gee, you know what that would be handy for...  Vision fail, and even 
with glaringly obvious information.
>
> Forget the apps.  I'm sort of with silky on at least part of this, we 
> don't even have any idea what might *require* a truly planetary-scale 
> network.  (Well, I have a few ideas...)  To crib a phrase, in such a 
> case *the network is the application.*  But will it happen?  Surely, 
> if we don't hit some existential risk boundary first.
>
> Scale is everything.  Two problems:  figuring out what to do with it, 
> and figuring out how to do it.  Not the same.  Both essential.
>
>
> jb

sdw



More information about the FoRK mailing list