[FoRK] Super-Intelligent Humans Are Coming

Gary Stock gstock at nexcerpt.com
Fri Oct 24 05:49:54 PDT 2014


The whole notion of context seems odd to introduce, since it appeared 
that the premise was "something from nothing."

Something akin to "context" may be:  Is, or is not, the development of 
super-intelligence foreordained by (perhaps unrecognized) principles?  
Is, or is not, the path to fully realized super-intelligencerelatively 
constrained or reproducible?

If the path is driven by principles / reproducible, then we might 
imagine where that leads.

However, if that path is ~not~ driven by principles / reproducible... 
hmmm...

If super-intelligence stumbles first, randomly, onto more ideas Jimmy 
Carter thought, then most problems on Earth will be solved.

If super-intelligence stumbles first, randomly, onto more ideas Ayn Rand 
thought, then most life on Earth will be eliminated.

An urge to begin "constructing the necessary context" seems to suggest 
hope of steering super-intelligence toward one or the other.

Watch more Star Trek.  Those models have all been exhausted ;-)

GS

On 10/24/14 3:48 AM, Stephen Williams wrote:
> On 10/23/14, 9:58 PM, J. Andrew Rogers wrote:
>>> On Oct 23, 2014, at 5:57 PM, Stephen D. Williams <sdw at lig.net> wrote:
>>> One super-intelligent human has a <50% of being moral (the Superman 
>>> Problem I'll call it) while a group (SuperFriends?) is very likely 
>>> to be moral due to fundamental and derived effects.  If the first 
>>> generation isn't, they'll likely kill each other off.  But some 
>>> generation will get it right.
>>
>> Define “moral”.
>>
>> Human morality is baskets of evolved heuristics that reflect biology 
>> and other local constraints. Islands of stability and game theoretic 
>> optimality are not even consistent across human populations. And a 
>> lot of popular human morality is predicated on near-parity of 
>> intelligence, which is being discarded as an assumption here. We 
>> would lack the ability to even understand the context in which a 
>> super-intelligent person is evaluating the morality of an action.
>>
>> See also: human morality as evaluated by mice in a mouse context
>>
>> Compounding this, humans can rationalize almost any action as “moral” 
>> if placed in the proper context. A super-intelligent human could be 
>> expected to be superb at constructing the necessary context. Normal 
>> humans are able to do this regularly.
>
> What example did you have in mind for "predicated on near-parity of 
> intelligence"?
> I would divide things people call "morals" into categories.  Some 
> categories are like common law and natural law. (Murder, theft, etc.)  
> Some are probabilistically correct in some context.  Some are correct 
> according to some mythical or fashionable story.  Some are new 
> theories yet to be proven. (Various PC)
>
> Each of those levels may have different applicability to widely 
> disparate entities and cultural boundaries, just as humans have those 
> divisions now.  But many of them will overlap.  Besides the mythical 
> and fashionable ones, those we can drop now.
>
> The ability of humans or any intelligence to falsely rationalize 
> rather than correctly applying known facts is either laziness, lack of 
> ability, or the presence of mendacity.  Your "constructing the 
> necessary context" really sounds like falsely rationalizing by a 
> method like straw man, false premises, false equivalence, correlation 
> used as causation, or selective memory.
>
> sdw
>
> _______________________________________________
> FoRK mailing list
> http://xent.com/mailman/listinfo/fork


More information about the FoRK mailing list