[FoRK] What will the big political issues be in the future?

Eugen Leitl eugen at leitl.org
Sun Aug 8 03:21:29 PDT 2004

On Wed, Aug 04, 2004 at 03:16:20PM -0600, daniel grisinger wrote:

> Ok, here's my problem with the entire runaway AI idea.
> What this notion is doing is making a strong statement about
> the relation between how hard it is to advance from one level

I don't see how intelligence advances in discrete levels. There are no
dramatic differences in the makeup of a P. paniscus brain or H. sapiens.
The differences are all in the (micro)architecture and the size.

> of intelligence to the next.  Basically, it's saying that after
> some threshhold is reached intelligence will begin to accelerate

There is no threshold if you look at the fossil record. Co-evolution drives
better infoprocessing. Targeted genome modification could increment human
intelligence far quicker, but still progressing in human generation

The issue with AI is that it's not there as a self-enhancing chain of beings.
As such there is no AI at all, only human-encoded domain specific knowledge,
brittle as shit, and something with a low ceiling of complexity (because
humans can't deal with complexity explicitly, and the ceiling gets lower as
the team size goes up).

> at some rate, say I(x) = 2^x.  But there's an implicit assumption
> that that rate is faster than the rate at which the problem of
> becoming more intelligent is becoming hard.  If how hard it is

Why should the problem become more hard? Scaling up the human bauplan will
result in increased backpressure, clustering modular hardware has no such

> to become more intelligent is described by H(x) = 2^x^x, then the
> entire runaway becomes impossible.  Sure, you become 2^x smarter
> at each step, but if the next step is 2^x^x times harder to take
> you certainly aren't running away.

We *are* running away. Exponential processes have no threshold. It's just
it's getting quicker and quicker, and at some point the rate change increase
goes through the roof. But it's not thresholded, and it's not a mathematical
Singularity, because physics doesn't do infinities in the real world.
(Presence of infinities is an earmark of an immature theory).

Eugen* Leitl <a href="http://leitl.org">leitl</a>
ICBM: 48.07078, 11.61144            http://www.leitl.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
http://moleculardevices.org         http://nanomachines.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://lair.xent.com/pipermail/fork/attachments/20040808/93c109d2/attachment.pgp

More information about the FoRK mailing list