From: Eliezer S. Yudkowsky (email@example.com)
Date: Fri Aug 18 2000 - 11:09:06 PDT
Jeff Bone wrote:
> Eugene's not proposing to *do* anything to me with his uberbeing, like
> subjegate me to some loony set of seed Rules that he thinks are spiffy and
> fair. Yet, anyway. ;-) (Eugene?)
As far as I can tell, 'gene wants to turn you loose in an ecology of
competing superintelligences with no particular motivation to be nice to
I challenge your use of the term "subjugate". Name a specific example
of something that you can't do, and feel you should be able to. Not a
general instance like "do whatever I want" - a specific instance like
"upgrade myself to superintelligence" or "build a time machine".
If you can do whatever you want, in what sense have you been subjugated?
> > Put your eggs in enough baskets and I GUARANTEE that one of them will
> > break.
> Is it really worth risking breaking ALL the eggs in order to ensure that
> NONE of them break? If your answer is yes, well man, you've got some
> issues. At a minimum you're on the outside of consensus-space on that
> issue. That *should* give you some pause.
No; I mean that I expect that, with Ultimate Technology as with
nanotechnology and nuclear weapons, attack will win out over defense -
i.e., that a single rogue MiniSysop could lay waste to all of Human
Space. It's always easier to destroy than create. One broken basket
loses the entire game - at the very, very least, it would not be
possible to defend the fragile Mundanes on Old Old Earth. Even a
nongenocidal angry MiniSysop could wreak havoc - make the Universe a far
less pleasant place to be in and significantly decrease (multiple orders
of magnitude) processing efficiency due to the requirements of mil-spec
processing. In reality, I don't think there *is* any defense against a
supernova dumped in your backyard. That's why a singleton scenario is
-- firstname.lastname@example.org Eliezer S. Yudkowsky http://singinst.org/home.html
This archive was generated by hypermail 2b29 : Fri Aug 18 2000 - 11:27:18 PDT