[FoRK] A Theory of Products: Magic, Alchemy, Science... and Beyond?

Jeff Bone jbone at place.org
Sat Feb 6 07:20:15 PST 2010

sdw says:

> Isn't this where the fun is? Even in the regimes below, you won't  
> typically hill climb out of any serious local minima without at  
> least some benefit from a Magician. Google gets this at least by  
> buying companies and giving people a lot of autonomy.

Not making any judgment calls here, other than to note that "magic  
isn't reliable."  (I.e., the results are not always reproducible, and  
what worked once doesn't necessarily work a second time, even when  
faithfully repeated.)  It may be "fun" --- but is that really the  
point?  (I'd say it is, at least with respect to creatives.  OTOH,  
there's no reason that creativity can't be taken up a meta-level, i.e.  
creativity becomes meta-creativity, the human creatives / magicians  
become the creators, curators, and custodians of the machine creative  
processes.  (Of course, they need a little bit of alchemy at least,  
but probably meta-applied post-science, to do *this* part well,  
too...  and so on, recursively and ever indirectly.

OTOH, I've now heard the "stuck on a local hilltop" argument in  
several contexts here with respect to optimization processes,  
particularly data- / machine learning-driven processes, evolutionary  
computing, etc.  It's a problem, that's true;  but it's one that's  
easily avoided.  The first step is being able to *determine* that  
you're not improving;  and that's trivial.  The second step is to  
relocate some distance / direction in the search space, and see what  
happens.  Generally this "jolt" is done randomly;  it can be the  
mutation step in a EC, or whatever.  That doesn't work, start  
exploring ever-more-distant parts of the space.  Standard operating  
procedure.  How well it works depends on your ability to assess and to  
move around the fitness landscape as well as the shape of the  
landscape itself.  But n-gradient detection isn't hard.

Also worth pointing out that for most interesting problems the fitness  
landscape isn't static, it's dynamic;  so even under most standard EC  
mechanisms, with even naive hilltop-avoidance, you get coevolution.

So I'm not convinced that optimization processes necessarily involve  
getting eventually stuck on a local hilltop that you can't get off  
of.  Indeed, the theory(-ies) of universal optimizers e.g. AIXI would  
have it otherwise.  To assume that getting stuck is essential to such  
mechanical processes is to assume something, ahem, "magical" about the  
human brain itself.



Aside, re: rvk:  nice post, thanks.

More information about the FoRK mailing list