[FoRK] Stand up and face the future

Stephen Williams sdw at lig.net
Tue Dec 22 21:11:19 PST 2009


Ken Ganshirt @ Yahoo wrote:
> --- On Tue, 12/22/09, Jeff Bone <jbone at place.org> wrote:
>
>   
>> Stephen says:
>>
>>     
>>> Mining, sifting, and real-time do not go together. At
>>>       
>> least not in the traditional forms.
>>
>> With the caveat that mining and sifting / reacting in
>> realtime are (today) different activities, I have to point
>> at the existence proof (what I do for a living these days)
>> as contrary to this point of view.  
>>
>>     
>
> Perhaps you can "sift"/react in near-realtime (I assume you mean you're filtering the data pretty much as it's created?). But you can't "mine" in realtime. Mining requires a pre-existing source of accumulated data. Or it's not mining. By definition. ... Or is that what you just said?? :-)
>
> And you can't even sift in near-realtime if the data being filtered isn't hitting the filter immediately it's created.
>
> So for "remote" data sources, how do you position your filters in order for the data to hit them in near-realtime? 
>
> Or is your definition of "realtime" sorta looser than mine (e.g. you don't count such things as latency, etc.)? 
>   

"Realtime" is as real as you wanna be.  I started out doing embedded 
machine control, factory machines, for GE, making light bulbs.  Low 
wattage arc lamps actually.  Realtime was in the millisecond range.  
"Realtime" for AOL was maybe 200ms. for the end-user, 1 ms. at the 
server.  "Realtime" for Geico was like 500-1000 ms. (horrifying!).
> (Notice I don't really like to use the term "realtime" because it never is. Comes from my days as a SCADA system developer. Users figured we really could do stuff in real time. And got pretty pissed when we didn't. So I quit using the term.)
>   

It's still a useful term, just has to have a parameter or two to be 
meaningful.  Soft/hard, max/avg. latency, throughput requirements and 
distribution, etc.

Stephen
>       ...ken...
>
>   



More information about the FoRK mailing list