[FoRK] any IM experts here

Stephen D. Williams <sdw at lig.net> on Thu Mar 29 09:24:13 PDT 2007

I knew about Nutch and some other components, but Carrot is cool.  I was 
planning to use Nutch for the crawling / indexing side, but I haven't 
found a truly scalable search engine.

sdw

Ken Meltsner wrote:
> Nutch + Carrot?  It's a nice combination since Nutch is a good
> straightforward search engine and Carrot gives you spiffy clustering
> capabilities.    Nutch is nearing its 1.0 release (0.9 is coming soon)
> so it's probably almost ready for prime time.
>
> http://lucene.apache.org/nutch/
>
> http://project.carrot2.org/
>
> If you don't like the indexing component (Lucene), you could probably
> reuse the other parts of Nutch: the distributed store and the
> map-reduce system (Hadoop), the crawler and filters, etc.
>
> Of course, it's Java, which may trigger certain allergies.
>
> [Experience is a year or two old, from another prototype effort/proof
> of concept which went nowhere, although I did like the combination of
> Davenport (a CIFS<->Web server) and Nutch.]
>
> Ken "I've abandoned or failed to follow through on more ideas than
> most people have had" Meltsner
>
>
> On 3/29/07, Stephen D. Williams <sdw at lig.net> wrote:
>> I assume that you mean something like Meebo, Kool-IM, etc.
>> http://wwwl.meebo.com/index-en.html
>> http://www.koolim.com
>>
>> I've built mainly servers, and very specialized client code.  A few
>> years ago, I did build a web client using a frame that processed
>> streaming javascript.  It was kind of cool in that it allowed on-the-fly
>> translation in each direction (using public translation engines).
>>
>> Given libraries that communicate to those services properly (is that war
>> finally over??), arrangements (if still needed) with services to allow
>> many connections and/or messages through a small number of IP addresses,
>> and a small set of features (not video, direct file transfer, etc.), it
>> shouldn't take more than a month of solid work by 1 or 2 developers to
>> get the main functionality done.  Another month or two would be needed
>> to get true scalability.  Polishing and extra features would follow.
>>
>> For various reasons, I've been looking at what it would take to create a
>> complete, scalable search engine.  Not Google exactly, but something
>> more than a toy.  There's a lot available for the crawling, indexing,
>> and basic search functionality.  The search engine seems pretty straight
>> forward to scale also.
>>
>> sdw
>>
>> Sat N wrote:
>> > any one who has built an IM client?
>> >
>> > How long does it take to build a scalable web based IM client that can
>> > talk to Yim, MSN, AIM and Gtalk?
>> > _______________________________________________
>> > FoRK mailing list
>> > http://xent.com/mailman/listinfo/fork
>>
>> _______________________________________________
>> FoRK mailing list
>> http://xent.com/mailman/listinfo/fork
>>
> _______________________________________________
> FoRK mailing list
> http://xent.com/mailman/listinfo/fork


More information about the FoRK mailing list