Minsky

WHY PEOPLE THINK COMPUTERS CAN'T

Marvin Minsky, MIT

First published in AI Magazine, vol. 3 no. 4, Fall 1982. Reprinted in

Technology Review, Nov/Dec 1983, and in The Computer Culture,

(Donnelly, Ed.) Associated Univ. Presses, Cranbury NJ, 1985

Most people think computers will never be able to think. That is, really

think. Not now or ever. To be sure, most people also agree that computers

can do many things that a person would have to be thinking to do. Then

how could a machine seem to think but not actually think? Well, setting

aside the question of what thinking actually is, I think that most of us

would answer that by saying that in these cases, what the computer is

doing is merely a superficial imitation of human intelligence. It has been

designed to obey certain simple commands, and then it has been provided

with programs composed of those commands. Because of this, the

computer has to obey those commands, but without any idea of what's

happening.

Indeed, when computers first appeared, most of their designers intended

them for nothing only to do huge, mindless computations. That's why the

things were called "computers". Yet even then, a few pioneers --

especially Alan Turing -- envisioned what's now called "Artificial

Intelligence" - or "AI". They saw that computers might possibly go

beyond arithmetic, and maybe imitate the processes that go on inside

human brains.

Today, with robots everywhere in industry and movie films, most people

think Al has gone much further than it has. Yet still, "computer experts"

say machines will never really think. If so, how could they be so smart,

and yet so dumb?

CAN MACHINES BE CREATIVE?

We naturally admire our Einsteins and Beethovens, and wonder if

computers ever could create such wondrous theories or symphonies. Most

people think that creativity requires some special, magical "gift" that

simply cannot be explained. If so, then no computer could create - since

anything machines can do (most people think can be explained.

To see what's wrong with that, we must avoid one naive trap. We mustn't

only look at works our culture views as very great, until we first get good

ideas about how ordinary people do ordinary things. We can't expect to

guess, right off, how great composers write great symphonies. I don't

believe that there's much difference between ordinary thought and

highly creative thought. I don't blame anyone for not being able to do

everything the most creative people do. I don't blame them for not being

able to explain it, either. I do object to the idea that, just because we can't

explain it now, then no one ever could imagine how creativity works.

We shouldn't intimidate ourselves by our admiration of our Beethovens

and Einsteins. Instead, we ought to be annoyed by our ignorance of how

we get ideas - and not just our "creative" ones. Were so accustomed to the

marvels of the unusual that we forget how little we know about the

marvels of ordinary thinking. Perhaps our superstitions about creativity

serve some other needs, such as supplying us with heroes with such

special qualities that, somehow, our deficiencies seem more excusable.

Do outstanding minds differ from ordinary minds in any special way? I

don't believe that there is anything basically different in a genius, except

for having an unusual combination of abilities, none very special by

itself. There must be some intense concern with some subject, but that's

common enough. There also must be great proficiency in that subject;

this, too, is not so rare; we call it craftsmanship. There has to be enough

self-confidence to stand against the scorn of peers; alone, we call that

stubbornness. And certainly, there must be common sense. As I see it, any

ordinary person who can understand an ordinary conversation has

already in his head most of what our heroes have. So, why can't

"ordinary, common sense" - when better balanced and more fiercely

motivated - make anyone a genius,

So still we have to ask, why doesn't everyone acquire such a combination?

First, of course, it sometimes just the accident of finding a novel way to

look at things. But, then, there may be certain kinds of difference-in-

degree. One is in how such people learn to manage what they learn:

beneath the surface of their mastery, creative people must have

unconscious administrative skills that knit the many things they know

together. The other difference is in why some people learn so many more

and better skills. A good composer masters many skills of phrase and

theme - but so does anyone who talks coherently.

Why do some people learn so much so well? The simplest hypothesis is

that they've come across some better ways to learn! Perhaps such "gifts"

are little more than tricks of "higher-order" expertise. Just as one child

learns to re-arrange its building-blocks in clever ways, another child

might learn to play, inside its head, at Fe-arranging how it learns!

Our cultures don't encourage us to think much about learning. Instead

we regard it as something that just happens to us. But learning must itself

consist of sets of skills we grow ourselves; we start with only some of them

and and slowly grow the rest. Why don't more people keep on learning

more and better learning skills? Because it's not rewarded right away, its

payoff has a long delay. When children play with pails and sand, they're

usually concerned with goals like filling pails with sand. But once a child

concerns itself instead with how to better learn, then that might lead to

exponential learning growth! Each better way to learn to learn would lead

to better ways to learn - and this could magnify itself into an awesome,

qualitative change. Thus, first-rank "creativity" could be just the

consequence of little childhood accidents.

So why is genius so rare, if each has almost all it takes? Perhaps because

our evolution works with mindless disrespect for individuals. I'm sure no

culture could survive, where everyone finds different ways to think. If

so, how sad, for that means genes for genius would need, instead of

nurturing, a frequent weeding out.

PROBLEM SOLVING

We can hardly expect to be able to make machines do wonders before we

find how to make them do ordinary, sensible things. The earliest

computer programs were little more than simple lists and loops of

commands like "Do this. Do that. Do this and that and this again until that

happens". Most people still write programs in such languages (like BASIC

or FORTRAN) which force you to imagine everything your program will

do from one moment to the next. Let's call this "do now" programming.

Before long, Al researchers found new ways to make programs. In their

"General Problem Solver" system, built in the late 1950's- Allen Newell,

J.C.Shaw and Herbert A.Simon showed ways to describe processes in terms

of statements like "If the difference between what you have and what you

want is of kind D, then try to change that difference by using method M."

This and other ideas led to what we call "means-ends" and "do if needed"

programming methods. Such programs automatically apply rules

whenever they're needed, so the programmers don't have to anticipate

when that will happen. This started an era of programs that could solve

problems in ways their programmers could not anticipate, because the

programs could be told what sorts of things to try, without knowing in

advance which would work. Everyone knows that if you try enough

different things at random, eventually you can do anything. But when

that takes a million billion trillion years, like those monkeys hitting

random typewriter keys, it's not intelligence -- just Evolution. The new

systems didn't do things randomly, but used "advice" about what was

likely to work on each kind of problem. So, instead of wandering around

at random, such programs could sort of feel around, the way you'd climb a

hill in the dark by always moving up the slope. The only trouble was a

tendency to get stuck on smaller peaks, and never find the real mountain

tops.

Since then, much Al research has been aimed at finding more "global"

methods, to get past different ways of getting stuck, by making programs

take larger views and plan ahead. Still, no one has discovered a

"completely general" way to always find the best method -- and no one

expects to.

Instead, today, many Al researchers aim toward programs that will match

patterns in memory to decide what to do next. I like to think of this as "do

something sensible" programming. A few researchers -- too few, I think

-- experiment with programs that can learn and reason by analogy. These

programs will someday recognize which old experiences in memory are

most analogous to new situations, so that they can "remember" which

methods worked best on similar problems in the past.

CAN COMPUTERS UNDERSTAND?

Can we make computers understand what we tell them? In 1965, Daniel

Bobrow wrote one of the first Rule-Based Expert Systems. It was called

"STUDENT" and it was able to solve a variety of high-school algebra "word

problems"., like these:

The distance from New York to Los Angeles is 3000 miles. If the

average speed of a jet plane is 600 miles per hour, find the time it

takes to travel from New York to Los Angeles by jet.

Bill's father's uncle is twice as old as Bill's father. Two years from

now I Bill's father will be three times as old as Bill. The sum of their

ages is 92.

Find Bill's age.

Most students find these problems much harder than just solving the

formal equations of high school algebra. That's just cook-book stuff -- but

to solve the informal word problems, you have to figure out what

equations to solve and, to do that, you must understand what the words and

sentences mean. Did STUDENT understand? It used a lot of tricks. It was

programmed to guess that "is" usually means "equals". It didn't even try to

figure out what "Bill's fathers' uncle" means -- it only noticed that this

phrase resembles "Bill's father". It didn't know that "age" and "old" refer

to time, but it took them to represent numbers to be put in equations. With

a couple of hundred such word-trick-facts, STUDENT sometimes managed

to get the right answers.

Then dare we say that STUDENT "understands" those words? Why bother.

Why fall into the trap of feeling that we must define old words like

"mean" and "understand"? It's great when words help us get good ideas,

but not when they confuse us. The question should be: does STUDENT avoid

the "real meanings" by using tricks?

Or is it that what we call meanings really are just clever bags of tricks.

Let's take a classic thought-example, such as what a number means.

STUDENT obviously knows some arithmetic, in the sense that it can find

such sums as "5 plus 7 is 12". But does it understand numbers in any other

sense - say, what 5 "is" - or, for that matter, what are "plus" or "is"? What

would �say if I asked you, "What is Five"? Early in this century, the

philosophers Bertrand Russell and Alfred North Whitehead proposed a

new way to define numbers. "Five", they said, is "the set of all possible sets

with five members". This set includes each set of five ball-point pens, and

every litter of five kittens. Unhappily, it also includes such sets as "the

Five things you'd least expect" and "the five smallest numbers not

included in this set" -- and these lead to bizarre inconsistencies and

paradoxes. The basic goal was to find perfect definitions for ordinary

words and ideas. But even to make the idea work for Mathematics, getting

around these inconsistencies made the Russell-Whitehead theory too

complicated for practical, common sense, use. Educators once actually

tried to make children use this theory of sets, in the "New Mathematics"

movement of the 1960's; it only further set apart those who liked

mathematics from those who dreaded it. I think the trouble was, it tried to

get around a basic fact of mind: what something means to me depends to

some extent on many other things I know.

What if we built machines that weren't based on rigid definitions? Wont

they just drown in paradox, equivocation, inconsistency? Relax! Most of

what we people "know" already overflows with contradictions; still we

survive. The best we can do is be reasonably careful; let's just make our

machines that careful, too. If there remain some chances of mistake, well,

that's just life.

WEBS OF MEANING

If every meaning in a mind depends on other meanings in that mind,

does that make things too ill-defined to make a scientific project work?

No, even when thing go in circles, there still are scientific things to do!

Just make new kinds of theories - about those circles themselves! The

older theories only tried to hide the circularities. But that lost all the

richness of our wondrous human meaning-webs; the networks in our

human minds are probably more complex than any other structure

Science ever contemplated in the past. Accordingly, the detailed theories

of Artificial Intelligence will probably need, eventually, some very

complicated theories. But that's life, too.

Let's go back to what numbers mean. This time, to make things easier,

well think about Three. I'm arguing that Three, for us, has no one single,

basic definition, but is a web of different processes that each get meaning

from the others. Consider all the roles "Three" plays. One way we tell a

Three is to recite "One, Two, Three", while pointing to the different

things. To do it right, of course, you have to (i) touch each thing once and

(ii) not touch any twice. One way to count out loud while you pick up each

object and remove it. Children learn to do such things in their heads or,

when that's too hard, to use tricks like finger-pointing. Another way to

tell a Three is to use some Standard Set of Three things. Then bring �set of

things to the other set, and match them I one-to-one: if all are matched

and none are left, then there were Three. That "standard I Three" need

not be things, for words like "one, two, three" work just as well. For Five

we have a wider choice. One can think of it as groups of Two and Three, or

One and Four. Or, one can think of some familiar shapes -. a pentagon, an

X, a Vee, a cross, an aeroplane; they all make Fives.

o o o o o o o o

o o o o o o o o o o o

o o o o o o

Because each trick works in different situations, our power stems from

being able to shift from one trick to another. To ask which meaning is

correct - to count, or match, or group - is foolishness. Each has its uses

and its ways to support the others. None has much power by itself, but

together they make a versatile skill-system. Instead of flimsy links in

chain of definitions in the mind, each word we use can activate big webs

of different ways to deal of things, to use them, to remember them, to

compare them, and so forth. With multiply-connected knowledge-nets,

you can't get stuck. When any sense of meaning fails, you can switch to

another. The mathematician's way, once you get into the slightest trouble,

you're stuck for good!

Why, then, do mathematicians stick to slender chains, each thing

depending as few things as is possible? The answer is ironic:

mathematicians want to get stuck! When anything goes wrong, they want

to be the first to notice it. The best way to be sure of that is having

everything collapse at once! To them, fragility is not bad, because it helps

them find the perfect proof, lest any single thing they think be

inconsistent with any other one. That's fine for Mathematics; in fact,

that's what much of mathematics is. It's just not good Psychology. Let's

face it, our minds will always hold some beliefs that turn out wrong.

I think it's bad psychology, when teachers shape our children's

mathematics into long, thin, fragile, definition tower-chains, instead of

robust cross-connected webs. Those chains break at their weakest links,

those towers topple at the slightest shove. And that's what happens to a

child's mind in mathematics class, who only takes a moment just to watch

a pretty cloud go by. The purposes of ordinary people are not the same as

those of mathematicians and philosophers, who want to simplify by

having just as few connections as can be. In real life, the best ideas are

cross-connected as can be. Perhaps that's why our culture makes most

children so afraid of mathematics. We think we help them get things

right, by making things go wrong most times! Perhaps, instead, we ought

to help them build more robust networks in their heads.

CASTLES IN THE AIR

The secret of what something means lies in the ways that it connects to all

the other things we know. The more such links, the more a thing will

mean to us. The joke comes when someone looks for the "real" meaning of

anything. For, if something had just one meaning, that is, if it were only

connected to just one other thing, then it wold scarcely "mean" at all!

That's why I think we shouldn't program our machines that way, with

clear and simple logic definitions. A machine programmed that way

might never "really" understand anything -- any more than a person

would. Rich, multiply-connected networks provide enough different ways

to use knowledge that when one way doesn't work, you can try to figure

out why. When there are many meanings in a network, you can turn

things around in your mind and look at them from different perspectives;

when you get stuck, you can try another view. That's what we mean by

thinking!

That's why I dislike logic, and prefer to work with webs of circular

definitions. Each gives meaning to the rest. There's nothing wrong with

liking several different tunes, each one the more because it contrasts

with the others. There's nothing wrong with ropes - or knots, or woven

cloth - in which each strand helps hold the other strands together - or

apart! There's nothing very wrong, in this strange sense, with having all

one's mind a castle in the air!

To summarize: of course no computer could understand anything real --

or even what a number is - if forced to single ways to deal with them. But

neither could a child or philosopher. So such concerns are not about

computers at all, but about our foolish quest for meanings that stand by

themselves, outside any context. Our questions about thinking machines

should really be questions about our own minds.

ARE HUMANS SELF-AWARE?

Most people assume that computers can't be conscious, or self-aware; at

best they can only simulate the appearance of this. Of course, this

assumes that we, as humans, are self-aware. But are we? I think not. I

know that sounds ridiculous, so let me explain.

If by awareness we mean knowing what is in our minds, then, as every

clinical psychologist knows, people are only very slightly self-aware, and

most of what they think about themselves is guess-work. We seem to build

up networks of theories about what is in our minds, and we mistake these

apparent visions for what's really going on. To put it bluntly, most of

what our "consciousness" reveals to us is just "made up". Now, I don't

mean that we're not aware of sounds and sights, or even of some parts of

thoughts. I'm only saying that we're not aware of much of what goes on

inside our minds.

When people talk, the physics is quite clear: our voices shake the air; this

makes your ear-drums move -- and then computers in your head convert

those waves into constituents of words. These somehow then turn into

strings of symbols representing words, so now there's somewhere in your

head that "represents" a sentence. What happens next?

When light excites your retinas, this causes events in your brain that

correspond to texture, edges, color patches, and the like. Then these, in

turn, are somehow fused to "represent" a shape or outline of a thing.

What happens then?

We all comprehend these simple ideas. But there remains a hard problem,

still. What entity or mechanism carries on from there? We're used to

saying simply, that's the "self". What's wrong with that idea? Our standard

concept of the self is that deep inside each mind resides a special, central

"self" that does the real mental work for us, a little person deep down

there to hear and see and understand what's going on. Call this the

"Single Agent" theory. It isn't hard to see why every culture gets attached

to this idea. No matter how ridiculous it may seem, scientifically, it

underlies all principles of law, work, and morality. Without it, all our

canons of responsibility would fall, of blame or virtue, right or wrong.

What use would solving problems be, without that myth; how could we

have societies at all?

The trouble is, we cannot build good theories of the mind that way. In

every field, as Scientists we're always forced to recognize that what we

see as single things - like rocks or clouds, or even minds - must sometimes

be described as made of other kinds of things. We'll have to understand

that Self, itself, is not a single thing.

NEW THEORIES ABOUT MINDS AND MACHINES

It is too easy to say things like, "Computer can't do (xxx), because they

have no feelings, or thoughts". But here's a way to turn such sayings into

foolishness. Change them to read like this. "Computer can't do (xxx),

because all they can do is execute incredibly intricate processes, perhaps

millions at a time". Now, such objections seem less convincing -- yet all

we did was face one simple, complicated fact: we really don't yet know

what the limits of computers are. Now let's face the other simple fact: our

notions of the human mind are just as primitive.

Why are we so reluctant to admit how little is known about how the mind

works? It must come partly from our normal tendency to repress

problems that seem discouraging. But there are deeper reasons, too, for

wanting to believe in the uniqueness and inexplicability of Self. Perhaps

we fear that too much questioning might tear the veils that clothe our

mental lives.

To me there is a special irony when people say machines cannot have

minds, because I feel we're only now beginning to see how minds

possibly could work -- using insights that came directly from attempts to

see what complicated machines can do. Of course we're nowhere near a

clear and complete theory - yet. But in retrospect, it now seems strange

that anyone could ever hope to understand such things before they knew

much more about machines. Except, of course, if they believed that minds

are not complex at all.

Now, you might ask, if the ordinary concept of Self is so wrong, what

would I recommend in its place? To begin with, for social purposes, I don't

recommend changing anything - it's too risky. But for the technical

enterprise of making intelligent machines, we need better theories of

how to "represent", inside computers, the kinds of webs of knowledge and

knowhow that figure in everyone's common-sense knowledge systems.

We must develop programs that know, say, what numbers mean, instead of

just being able to add and subtract them. We must experiment with all

sorts of common sense knowledge, and knowledge about that as well.

Such is the focus of some present-day Al research. True, most of the world

of "Computer Science" is involved with building large, useful, but shallow

practical systems, a few courageous students are trying to make

computers use other kinds of thinking, representing different kinds of

knowledge, sometimes, in several different ways, so that their programs

won't get stuck at fixed ideas. Most important of all, perhaps, is making

such machines learn from their own experience. Once we know more

about such things, we can start to study ways to weave these different

schemes together. Finally, we'll get machines that think about themselves

and make up theories, good or bad, of how they, themselves might work.

Perhaps, when our machines get to that stage, we'll find it very easy to

tell it has happened. For, at that point, they'll probably object to being

called machines. To accept that will be will be difficult, but only by this

sacrifice will machines free us from our false mottos.

KNOWLEDGE AND COMMON SENSE

We've all enjoyed those jokes about the stupid and literal behavior of

computers. They send us silly checks and bills for $0.00. They can't tell

when we mean "hyphen" from when we mean minus They don't mind

being caught in endless loops, doing the same thing over again a billion

times. This total lack of common sense is one more reason people think

that no machine could have a mind. It's not just that they do only what

they're told, it's also that they're so dumb it's almost impossible to tell

them how to do things right.

Isn't it odd, when you think about it, how even the earliest Al programs

excelled at "advanced" subjects, yet had no common sense? A 1961

program written by James Slagle could solve calculus problems at the

level of college students; it even got an A on an MIT exam. But it wasn't till

around 1970 that we managed to construct a robot programs that could see

and move well enough to handle ordinary things like children's building

blocks and do things like stack them up, take them down, rearrange them,

and put them in boxes.

Why could we make programs do those grown-up things before we could

make them do those childish things? The answer is a somewhat

unexpected paradox: much "expert" adult thinking is basically much

simpler than what happens in a child's ordinary play! It can be harder to

be a novice than to be an expert! This is because, sometimes, what an

expert needs to know and do can be quite simple -- only, it may be very

hard to discover, or learn, in the first place. Thus, Galileo had to be smart

indeed, to see the need for calculus. He didn't manage to invent it. Yet any

good student can learn it today.

The surprising thing, thus, was that when it was finished, Slagle's

program needed only about a hundred "facts" to solve its college-level

calculus problems. Most of them were simple rules about algebra. But

others were about how to guess which of two problems is likely to be

easier; that that kind of knowledge is especially important, because it

helps the program make good judgments about what to do next. Without

this such programs only thrash about; with it they seem much more

purposeful. Why do human students take so long to learn such rules? We

do not know.

Today we know much more about making such "expert" programs -- but

we still don't know much more about making programs with more

"common sense". Consider all the different things that children do, when

they play with their blocks. To build a little house one has to mix and

match many different kinds of knowledge: about shapes and colors, space

and time, support and balance, stress and strain, speed, cost, and keeping

track. An expert sometimes can get by with deep but narrow bodies of

knowledge - but common sense is, technically, a lot more complicated.

Most ordinary computer programs do just the things they're programmed

for. Some Al programs are more flexible; when anything goes wrong,

they can back up to some previous decision and try something else. But

even that is much too crude a base for much intelligence. To make them

really smart, we'll have to make them more reflective. A person tries,

when things go wrong, to understand what's going wrong, instead of just

attempting something else. We look for causal explanations, or excuses,

and, when we find them, add them to our networks of belief and

understanding. We do intelligent learning. Some day programs, too, could

do such things -- but first we'd need a lot more research to find out how.

UNCONSCIOUS FEARS AND PHOBIAS

I'll bet that when we try to make machines more sensible, we'll find that

learning what is wrong turns out to be as important as learning what's

correct. In order to succeed, it helps to know the likely ways to fail. Freud

talked about censors in our minds, that keep us from forbidden acts or

thoughts. And, though those censors were proposed to regulate our social

activity, I think we use such censors, too, for ordinary problem solving --

to know what not to do. Perhaps we learn a new one each time anything

goes wrong, by constructing a process to recognize similar

circumstances, in some "subconscious memory".

This idea is not popular in contemporary psychology, perhaps because

censors only suppress behavior, so their activity is invisible on the

surface. When a person makes a good decision, we tend to ask what "line

of thought" lies behind it. But we don't so often ask what thousand

prohibitions might have warded off a thousand bad alternatives. If

censors work inside our minds, to keep us from mistakes and absurdities,

why can't we feel that happening? Because, I suppose, so many thousands

of them work at once that, if you had to think about them, you'd never get

much done. They have to ward off bad ideas before you "get" those bad

ideas.

Perhaps this is one reason why so much of human thought is

"unconscious". Each idea that we have time to contemplate must be a

product of many events that happen deeper and earlier in the mind. Each

conscious thought must be the end of processes in which it must compete

with other proto-thoughts, perhaps by pleading little briefs in little

courts. But all that we do sense of that are just the final sentences.

And how, indeed, could it be otherwise? There's no way any part of the

mind could know everything that happens in the rest. Our conscious

minds must be like high executives, who can't be burdened with the small

details. There's only time for summaries from other, smaller parts of

mind, that know much more about much less; the ones that do the real

work.

SELF-CONSCIOUS COMPUTERS

Then, is it possible to program a computer to be self-conscious? People

usually expect the answer to be "no". What if we answered that machines

are capable, in principle, of even more and better consciousness than

people have?

I think this could be done by providing machines with ways to examine

their own mechanisms while they are working. In principle, at least, this

seem possible; we already have some simple Al programs that can

understand a little about how some simpler programs work. (There is a

technical problem about the program being fast enough, to keep up with

itself, but that can be solved by keeping records.) The trouble is, we still

know far too little, yet, to make programs with enough common sense to

understand even how today's simple Al problem-solving programs work.

But once we learn to make machines that are smart enough to understand

such things, I see no special problem in giving them the "self-insight"

they would need to understand, change, and improve themselves.

This might not be so wise to do. But what if it turns out that the only way

to make computers much smarter is to make them more self-conscious?

For example, it might turn out to be too risky to assign a robot to

undertake some important, long-range task, without some "insight" about

it's own abilities. If we don't want it to start projects it can't finish, we'd

better have it know what it can do. If we want it versatile enough to solve

new kinds of problems, it may need to be able to understand how it

already solves easier problems. In other words, it may turn out that any

really robust problem solver will to understand itself enough to change

itself. Then, if that goes on long enough, why can't those artificial

creatures reach for richer mental lives than people have. Our own

evolution must have constrained the wiring of our brains in many ways.

But here we have more options now, since we can wire machines in any

way we wish.

It will be a long time before we learn enough about common sense

reasoning to make machines as smart as people are. Today, we already

know quite a lot about making useful, specialized, "expert" systems. We

still don't know how to make them able to improve themselves in

interesting ways. But when we answer such questions, then we'll have to

face one, even stranger, one. When we learn how, then should we build

machines that might be somehow "better" than ourselves? We're lucky

that we have to leave that choice to future generations. I'm sure they

won't want to build the things that well unless they find good reasons to.

Just as Evolution changed man's view of Life, Al will change mind's view

of Mind. As we find more ways to make machines behave more sensibly,

we'll also learn more about our mental processes. In its course, we will

find new ways to think about "thinking" and about "feeling". Our view of

them will change from opaque mysteries to complex yet still

comprehensible webs of ways to represent and use ideas. Then those

ideas, in turn, will lead to new machines, and those, in turn, will give us

new ideas. No one can tell where that will lead and only one thing's sure

right now: there's something wrong with any claim to know, today, of

any basic differences between the minds of men and those of possible

machines.