From test@demedici.ssec.wisc.edu Mon Jun 5 13:13:38 2006
Date: Mon, 5 Jun 2006 13:01:05 -0500 (CDT)
From: Bill Hibbard <test@demedici.ssec.wisc.edu>
Reply-To: sl4@sl4.org
To: sl4@sl4.org
Cc: wta-talk@transhumanism.org, extropy-chat@lists.extropy.org,
agi@v2.listbox.com
Subject: Re: Two draft papers: AI and existential risk; heuristics and
biases
Eliezer,
> These are drafts of my chapters for Nick Bostrom's forthcoming edited
> volume _Global Catastrophic Risks_. I may not have much time for
> further editing, but if anyone discovers any gross mistakes, then
> there's still time for me to submit changes.
>
> The chapters are:
> . . .
> _Artificial Intelligence and Global Risk_
> http://singinst.org/AIRisk.pdf
> The new standard introductory material on Friendly AI. Any links to
> _Creating Friendly AI_ should be redirected here.
In Section 6.2 you quote my ideas written in 2001 for
hard-wiring recognition of expressions of human happiness
as values for super-intelligent machines. I have three
problems with your critique:
1. Immediately after my quote you discuss problems with
neural network experiments by the US Army. But I never said
hard-wired learning of recognition of expressions of human
happiness should be done using neural networks like those
used by the army. You are conflating my idea with another,
and then explaining how the other failed.
2. In your section 6.2 you write:
If an AI "hard-wired" to such code possessed the power - and
[Hibbard, B. 2001. Super-intelligent machines. ACM SIGGRAPH
Computer Graphics, 35(1).] spoke of superintelligence - would
the galaxy end up tiled with tiny molecular pictures of
smiley-faces?
When it is feasible to build a super-intelligence, it will
be feasible to build hard-wired recognition of "human facial
expressions, human voices and human body language" (to use
the words of mine that you quote) that exceed the recognition
accuracy of current humans such as you and me, and will
certainly not be fooled by "tiny molecular pictures of
smiley-faces." You should not assume such a poor
implementation of my idea that it cannot make
discriminations that are trivial to current humans.
3. I have moved beyond my idea for hard-wired recognition of
expressions of human emotions, and you should critique my
recent ideas where they supercede my earlier ideas. In my
2004 paper:
Reinforcement Learning as a Context for Integrating AI Research,
Bill Hibbard, 2004 AAAI Fall Symposium on Achieving Human-Level
Intelligence through Integrated Systems and Research
http://www.ssec.wisc.edu/~billh/g/FS104HibbardB.pdf
I say:
Valuing human happiness requires abilities to recognize
humans and to recognize their happiness and unhappiness.
Static versions of these abilities could be created by
supervised learning. But given the changing nature of our
world, especially under the influence of machine
intelligence, it would be safer to make these abilities
dynamic. This suggests a design of interacting learning
processes. One set of processes would learn to recognize
humans and their happiness, reinforced by agreement from
the currently recognized set of humans. Another set of
processes would learn external behaviors, reinforced by
human happiness according to the recognition criteria
learned by the first set of processes. This is analogous
to humans, whose reinforcement values depend on
expressions of other humans, where the recognition of
those humans and their expressions is continuously
learned and updated.
And I further clarify and update my ideas in a 2005
on-line paper:
The Ethics and Politics of Super-Intelligent Machines
http://www.ssec.wisc.edu/~billh/g/SI_ethics_politics.doc
Please adjust your discussion of my ideas to:
1. Not conflate my ideas with others.
2. Not assume a poor implementation of my ideas.
3. Not critique my old ideas when they have been
replaced by newer ideas in my publications.
Thank you,
Bill