From billh@ssec.wisc.edu Sat Feb 7 11:08:22 2015
Date: Sat, 7 Feb 2015 11:08:05 -0600 (CST)
From: Bill Hibbard <billh@ssec.wisc.edu>
To: Benja Fallenstein <benja.fallenstein@gmail.com>
Cc: "researchassociates@intelligence.org" <researchassociates@intelligence.org>
Subject: Re: Weekly report 2015-01-19 through 2015-01-25
Hi Benja,
I enjoyed seeing you, Nate and Kaj at AAAI-15.
On Tue, 27 Jan 2015, Benja Fallenstein wrote:
> Also, I got to chat again with Bill Hibbard about his
> idea for -- translating into my own way of thinking
> about it -- using a bounded resource variant of AIXI
> for some types of logical uncertainty tasks. Roughly,
> the idea is illustrated by the following example:
> Suppose that every time you upgrade the computing
> hardware you run on, your average score increases
> significantly. A probabilistic environment model
> which is far too computationally constrained to
> simulate the real world (which contains the real you)
> might still be able to make probabilistic predictions
> about your score if you buy additional hardware that
> reflect this relationship. Bill had a version of this
> system on his poster, and thinking about that and
> talking to him about it helped me understand it
> somewhat better than I had before.
The self-modeling agent framework proposes
to address several problems. As you note,
one is predicting the value of increases to
agent resources.
Another is "inconsistency between the agent's
utility function and its definition." One case
of this problem is what Armstrong calls
"motivated value selection" and describes as
"a conflict between agents learning their
future values and following their current
values." In the self-modeling framework the
agent learns a function from interaction
history and proposed action to value. All
elements of the agent defintion can be included
in the function to be learned, including
evolution of the agent's environment model and
utility function as interaction history length
increases. Thus this framework can avoid any
"inconsistency between the agent's utility
function and its definition."
As defined in Section 8.4 of my book
(http://arxiv.org/abs/1411.1373), the self-
modeling framework proposes to address the
problem of being predicted by other agents in
the environment by learning when to choose
actions stochastically.
Best wishes,
Bill