Mustafa Suleyman transcript

Intro

are you uncomfortable talking about this

yeah I mean it's pretty wild right

Mustafa Suleiman the billionaire founder

of Google's AI technology he's played a

key role in the development of AI from

its first critical steps in 2020 I moved

to work on Google's chatbot it was the

ultimate technology we can use them to

turbocharge our knowledge unlike

anything else why didn't they release it

we were nervous we were nervous every

organization is going to race to get

their hands on intelligence and that's

going to be incredibly destructive this

technology can be used to identify

cancerous tumors as it can to identify a

Target on the battlefield a tiny group

of people who wish to cause harm are

gonna have access to tools that can

instantly destabilize our world that's

the challenge how to stop something that

can cause harm or potentially kill

that's where we need containment do you

think that it is containable it has to

be possible why it must be possible why

must it be because otherwise it contains

us yet you chose to build a company in

this space why did you do that because I

want to design an AI That's on your side

I honestly think that if we succeed

everything is a lot cheaper it's going

to power New forms of transportation

reduce the cost of healthcare but what

if we fail the really painful answer to

that question is that

do you ever get sad about it

yeah it's intense

I think this is fascinating I looked at

the back end of our YouTube channel and

it says that since this channel started

69.9 of you that watch it frequently

haven't yet hit the Subscribe button so

I have a favor to ask you if you've ever

watched this Channel and enjoyed the

content if you're enjoying this episode

right now please could I ask a small

favor please hit the Subscribe button

helps this channel more than I can

explain and I promise if you do that to

return the favor we will make this show

better and better and better and better

and better that's the promise I'm

willing to make you if you hit the

Subscribe button do we have a deal

[Music]

everything that's going on with

How do you feel emotionally about what's going on with AI?

artificial intelligence now and

um this new wave and all these times

like AGI and I saw another time in your

your book called ACI it's the first time

I'd heard that term how do you feel

about it emotionally if you had to

encapsulate how you feel emotionally

about what's going on in this moment how

would you do what words would you use I

would say

in the past

it would have been petrified

and I think that over time

as you really think through

the consequences and the pros and cons

and the trajectory that we're on

you adapt

and you understand that

actually there is something

incredibly inevitable about this

trajectory and that we have to wrap our

arms around it and guide it and control

it as a collective species as a as

Humanity

and I think the more you realize

how much influence we collectively can

have over this outcome the more

empowering it is because on the face of

it this is really going to be the tool

that helps us tackle all the challenges

that we're facing as a species right we

need to

fix water desalination we need to grow

food 100x cheaper than we currently do

we need renewable energy to be you know

ubiquitous and everywhere in our lives

we need to adapt to climate change

everywhere you look in the next 50 years

we have to do more with less

and there are very very few

proposals let alone practical solutions

for how we get there

training machines to help us as AIDS

scientific research Partners inventors

creators is absolutely essential and so

the upside is phenomenal it's enormous

but AI isn't just a thing it's not an

inevitable whole its form isn't

inevitable right it's form the exact way

that it manifests and appears in our

everyday lives and the way that it's

governed and who it's owned by and how

it's trained

that is a question

that is up to us collectively as a

species to figure out over the next

decade because if we don't Embrace that

challenge then it happens to us

and that's really what I'm I have been

wrestling with for 15 years of my career

is how to intervene in a way that this

really does benefit everybody and those

benefits far far outweigh the potential

risks at what stage were you petrified

so I found a deep mind in 2010

and

no over the course of the first few

years

our progress was fairly modest

but

quite quickly in sort of 2013 as the

Deep learning Revolution began to take

off

I could see glimmers

of very early versions of AIS learning

to do really clever things so for

example one of our big initial

achievements was to teach an AI to play

the Atari games so remember Space

Invaders and and pong where you bat a

ball from left to right

and we trained this initial AI to purely

look at the raw pixels screen by screen

flickering or moving in front of the AI

and then control the actions up down

left right shoot or not

and it got so good at learning to play

this simple game simply through

attaching a value between the reward

like it was it was getting score and

taking an action that it learned some

really clever strategies uh to play the

game really well that Us games players

and humans hadn't really even noticed or

at least people in the office haven't

noticed it some professionals did

um and that was amazing to me because I

was like wow this simple system that

learns through a set of stimuli plus a

reward to take some actions

can actually discover many strategies

clever tricks to play the game well that

us humans

haven't occurred to us right and that to

me is both thrilling

because it presents the opportunity to

invent new knowledge and Advance our

civilization

but of course in the same measure is

also

petrifying

was there a particular moment when you

were you were at deepmind where you go

whether you had that kind of Eureka

Moment Like a day when something

happened and it caused that

that Epiphany I guess was it yeah it it

was actually a moment even before 2013

where I remember standing in the office

and watching a very early prototype of

one of these image recognition image

generation models

that was trained to generate new

handwritten black and white digits so

imagine zero to one two three four five

six seven eight nine

all in different style of handwriting on

a tiny grid of like 300 pixels by 300

pixels in black and white

and we were trying to train the AI to

generate a new version of one of those

digits a number seven in a new

handwriting sounds so simplistic today

given the incredible photorealistic

images that are being generated right

um and I just remember so clearly it it

took sort of 10 or 15 seconds and it

just resolved it that the number

appeared it went from complete Black to

like slowly gray and then suddenly these

were like white pixels appeared out of

the the black darkness and it revealed a

number seven

and that sounds so simplistic in

hindsight but it was amazing I was like

wow

the model

kinda understands the representation of

a seven well enough to generate a new

example of a number seven

an image of a number seven

you know and you roll forward 10 years

and our predictions were correct in fact

it was quite predictable

in hindsight the trajectory that we were

on more compute plus vast amounts of

data has enabled us within a decade to

go from predicting black and white

digits

generating new versions of those images

to now generating unbelievable

photorealistic not just images but

videos novel videos

with a simple natural language

instruction or a prompt

what has surprised you you said you

What's surprised you most about the last decade?

referred to that as predictable but what

has surprised you about what's happened

over the last decade

so

I think what was predictable to me back

then was the generation of images and of

audio

um because the structure of an image is

locally contained so pixels that are

near one another create straight lines

and edges and corners and then

eventually they create eyebrows and

noses and eyes and faces and entire

scenes and I could just intuitively in a

very simplistic way I could get my head

around the fact that okay well we're

predicting these number sevens you can

imagine how you then can expand that out

to entire images maybe even to videos

maybe you know to audio too you know

what I said you know a couple seconds

ago is connected in phoneme space in the

spectrogram

but what was much more surprising to me

was that those same methods for

Generation applied in the space of

language

you know language seems like such a

different abstract space of ideas when I

say like the cat sat on the

most people would probably predict Matt

right but it could be table car chair

tree it could be Mountain Cloud I mean

there's a gazillion possible

next word predictions

and so the space is so much larger the

ideas are so much more abstract I just

couldn't wrap my intuition around the

idea that we would be able to create the

incredible large language models that

you see today

you check gpts

chat GPT

the Google's Bard inflection my new

company has an AI called Pi a pi dot AI

which stands for personal intelligence

and it's as good as chat GPT but much

more emotional empathetic and kind

so it's just super surprising to me that

just growing the size of these large

language models as we have done by 10x

every single year for the last 10 years

we've been able to produce this and that

that's just an amazingly large number if

you just kind of pause for a moment to

Grapple with the numbers here

in 2013 when we trained the Atari AI

that I mentioned to you at deepmind

that used two Peta flops of computation

so peta peta stands for a million

billion calculations a flop is a

calculation so two million billion

right which is already an insane number

of calculations it's totally crazy yeah

just two of these units that are already

really large

and every year since then we've 10xed

the number of calculations that can be

done such that today

the biggest language model that we train

at inflection uses 10 billion petaflops

so 10 billion million billion

calculations I mean it's just

unfathomably large number

and what we've really observed is that

scaling these models by 10x every single

year

produces this magical experience of

talking to an AI that feels like you're

talking to a human that is super

knowledgeable and super smart

there's so much that's happened in

I'm scared of this coming wave.

public conversation around AI um and

there's so many questions that I have

I've I've been speaking to a few people

about artificial intelligence to try and

understand it and

I'm I think where I am right now is I

feel quite scared

um but when I get scared I don't get

it's not the type of scared that makes

me anxious it's not like an emotional

scared it's a very logical scared it's

my very logical brain hasn't been able

to figure out how the inevitable outcome

that I've arrived at which is that

humans become the less dominant species

on this planet

um how that is to be avoided in any way

the first chapter of your book The

Coming wave is a is is a is

titled

appropriately to how I feel containment

is not possible

you say in that chapter the widespread

emotional reaction I I was observing is

something I've come to call the

pessimism aversion trap correct what is

the pessimism aversion trap

well

so all of us be included

feel what you just described when you

first get to grips with the idea of this

new coming wave it's scary it's

petrifying it's threatening is it going

to take my job is my daughter or son

gonna fall in love with it you know what

does this mean what does it mean to be

human in a world where there's these

other human-like things that aren't

human how do I make sense of that it's

super scary

and a lot of people over the last few

years

I think things have changed in the last

six months I have to say but over the

last few years

I would say the default reaction has

been to avoid the pessimism and the fear

right to just kind of recoil from it and

pretend that it's like either not

happening or that it's all going to work

out to be Rosy it's going to be fine we

don't have to worry about it people

often say well we've always created new

jobs we've never permanently displaced

jobs we've only ever seen new jobs be

created unemployment is an all-time low

right so there's this default optimism

bias that we have and I think it's less

about a need for optimism and more about

a fear of pessimism

and so that trap

particularly in Elite circles means that

often we aren't having the tough

conversations that we need to have in

order to respond to the coming wave

are you scared in part about having

those tough conversations because of how

it might be received

um not so much anymore so I've spent

most of my career

trying to put those tough questions on

the policy table right I've been raising

these questions the ethics of AI safety

and questions of containment for as long

as I can remember with governments and

civil societies and all the rest of it

and so I've become used to talking about

that and

you know I think

it's

essential that we have the honest

conversation because we can't let it

happen to us we have to openly talk

about it

is

Is containment possible?

I mean this is a this is a big a big

question but

as you sit here now do you think that it

is containable because I

I can't see how

I can't see how it can be contained

chapter three is the containment problem

are you giving give the example of how

Technologies are often invented for good

reasons and for certain use cases like

the hammer you know which is used you

know maybe to build something but it

also can be used to kill people

um

and you say in history we haven't been

able to ban a technology ever

really it has always found a way into

society

um because of other societies have an

incentive to have it even if we don't

and then we need we need it like the

nuclear bomb because if they have it

then we don't then we're at disadvantage

so are you optimistic

honestly

I don't think

an optimism or a pessimism frame

is the right one because the E both are

equally biased in ways that

I think distract us

as I say in the book on the face of it

it does look like containment isn't

possible we haven't contained or

permanently banned a technology of this

type in the past there are some that we

have done right so we banned CFCs for

example because they were producing a

hole in the ozone layer we've banned

certain weapons chemical and biological

weapons for example or blinding lasers

Believe It or Not There are such things

as lasers that were instantly blind you

you know so we have stepped back from

the frontier

in some cases but that's largely where

there's either cheaper or you know

equally effective Alternatives that are

quickly adopted in this case these

Technologies are Omni use so the same

core technology can be used to identify

you know cancerous tumors in chest

x-rays as it can to identify a Target on

the battlefield for an aerial strike so

that mixed use or Omni use

is going to drive the proliferation

because there's huge commercial

incentives because it's going to deliver

a huge benefit and do a lot of good

and that's the challenge that we have to

figure out is how to stop something

which on the face of it is so good but

at the same time can be used in really

bad ways too

do you think we will

I do think we will so I think that

nation states Remain the backbone of our

civilization we have

chosen to concentrate power

in a single Authority the nation-state

and we pay our taxes and we've given the

nation-state a monopoly over the use of

violence

and now the nation-state is going to

have to

update itself quickly to be able to

contain this technology because without

that kind of

essentially oversight both of those of

us who are making it but also crucially

of the open source

then it will proliferate and it will

spread but regulation is still a real

tool and we can use it and we must what

does what does the world look like in

um let's say 30 years if that doesn't

happen in your view

people because people the average person

can't really grapple their head around

artificial intelligence when they think

of it they think of like these large

language models that you can chat to and

ask it about your homework

that's like the average person's

understanding of artificial intelligence

because that's all they've ever been

exposed to of it you have a different

View

because of the work you've spent the

last decade doing so to try and give

Dave

who's I don't know an Uber driver in

Birmingham an idea who's listening to

this right now what artificial

intelligence intelligence is and its

potential capabilities if

you know there's no there's no

containment what does it what does the

world look like in 30 years

so I think it's going to feel largely

like another human

so think about the things that you can

do

not again in the physical world but in

the digital world 2050 I'm thinking of

I'm in 2050.

2050 we will have robots

2050 we will definitely have robots I

mean more than that 2050 we will have

new biological beings as well

because the same trajectory that we've

been on with hardware and software is

also going to apply to the platform of

biology are you uncomfortable talking

about this yeah I mean it's pretty wild

right it donates you cross your arms

no I always I always look I always I

always use that as a key for someone

when a subject matter is uncomfortable

and it's interesting because I know you

know so much more than me

and about this and I know you've spent

way more hours thinking off into the

future about the consequences of this I

mean you've written a book about it so

yeah like you spent 10 years at the very

deep mine is one of the the Pinnacle

companies the Pioneers in this whole

Space

so you know you know some stuff and it's

funny because when I was I watched an

interview with Elon Musk and he was

asked a question similar to this I know

he speaks in certain

sentence had a voice but he said that he

he's almost he's gotten to the point

where he thinks he's living in suspended

disbelief

where he thinks that if he spent too

long thinking about it he wouldn't

understand the purpose of what he's

doing right now and he he says that it's

more dangerous than nuclear weapons um

and that it's too late too late to stop

it this one interview that's chilling

and I was filming Dragons Den the other

day and I showed the dragons the clip I

was like look what Elon Musk said when

he was asked about what his child what

advice he should give to his children in

a world of in an inevitable world of

artificial intelligence it's the first

time I've seen Elon Musk stop for like

20 seconds and not know what to say

stumble stumble stumble stumble stumble

and then conclude that he's living in

suspended disbelief yeah I mean I think

it's a great phrase

that is the moment we're in we have to

it's what I said to you about the

pessimism's aversion trap we have to

confront

the probability of seriously dark

outcomes and we have to spend time

really thinking about those consequences

because the competitive nature of

companies and of nation states is going

to mean that every organization is going

to race to get their hands on

intelligence intelligence is going to be

a new form of of capital right just as

there was a grab for land or there's a

grab for oil there's a grab for anything

that enables you to do more with less

faster better smarter right and we can

clearly see the predictable trajectory

of the exponential improvements in these

Technologies and so we should expect

that wherever there is power there's now

a new tool to amplify that power

accelerate that power Turbo Charge it

right and you know in 2050 if you asked

me to look out there

I mean of course it makes me Grimace

that's why I was like oh my God it's

it really does feel like a new species

and and that has to be brought under

control we cannot allow ourselves to be

dislodged from our position as the

dominant species on this planet we

cannot allow that

you mentioned robots so these are sort

What will these AI biological beings look like?

of adjacent technologies that are rising

with artificial intelligence robots you

mentioned

um biological new biological species

give me some light on what you mean by

that

well so so far the dream of Robotics

hasn't really come to fruition right I

mean we're we still have the most we

have now are sort of drones and a little

bit of self-driving cars

but that is broadly on the same

trajectory as these other Technologies

and I think that over the next 30 Years

you know we are going to have humanoid

robotics we're going to have

um you know physical

tools within our everyday system that

we can rely on that will be pretty good

that'll be pretty good to do many of the

physical tasks and that's a little bit

further out because I think it you know

there's a lot of tough problems there

but it's still coming in the same way

and likewise with Biology you know we

can now sequence a genome for a

millionth of the cost

of the first genome which took place in

2000 so 20-ish years ago

the cost has come down by a million

times and we can now increasingly

synthesize that is create or manufacture

new bits of DNA which obviously

give rise to life in every possible form

and we're starting to engineer that DNA

to either remove traits or capabilities

that we don't like or indeed to add new

things that we want it to do we want you

know fruit to last longer or we want

meat to have higher protein etc etc

synthetic meat to have higher protein

levels

and what's the implications of that

well potential implications

I think that the darkest scenario there

is that people will experiment with

pathogens

engineered

you know synthetic pathogens that might

end up accidentally or intentionally

being more transmissible

I.E they they can spread faster

or more lethal I.E you know they cause

more harm or potentially kill like a

pandemic like a pandemic

um and

that's where we need containment right

we have to limit access to the tools and

the know-how to carry out that kind of

experimentation so one framework of

thinking about this with respect to

making containment possible

is that

we really are experimenting with

dangerous materials and Anthrax is not

something that can be bought over the

Internet that can be freely experimented

with and likewise the very best of these

tools in a few years time are going to

be capable of creating

you know new synthetic

pandemic pathogens and so we have to

restrict access to those things that

means restricting access to the compute

Would we be able to regulate AI?

it means restricting access to the

software that runs the models to the

cloud environments that provide apis

provide you access to experiment with

those things and of course on the

biology side it means restricting access

to some of the substances and people

aren't going to like this

people are not going to like

that claim because it means that those

who want to do good with those tools

those who want to create a startup the

small guy the little developer that

struggles to comply with all the

regulations they're going to be pissed

off understandably right but that is the

age we're in

deal with it like we have to confront

that reality that means that we have to

approach this with the precautionary

principle right never before in the

invention of a technology or in the

creation of a regulation have we

proactively said we need to go slowly we

need to make sure that this first does

no harm the precautionary principle and

that is just an unprecedented moment no

other Technology's done that right

because I think

we collectively in the industry those of

us who are closest to the work can see a

place in five years or ten years

where it could get out of control and we

have to get on top of it now and it's

better to forgo like that is give up

some of those potential upsides or

benefits until we can be more sure

that it can be contained that it can be

controlled that it always serves our

Collective interests

think about that so I think about what

you've just said there about being able

to create these pathogens these diseases

and viruses Etc that you know could

become weapons or whatever else but with

artificial intelligence and the power of

that intelligence with these um

pathogens

you could theoretically ask one of these

systems to create a virus

that

a very deadly virus

um you could ask the artificial

intelligence to create a very deadly

virus that has certain properties

um

maybe even that mutates over time in a

certain way so it only kills a certain

amount of people kind of like a nuclear

bomb of viruses that you could just pop

hit an enemy with now if I'm if I hear

that and I go okay that's powerful

I would like one of those you know there

might be an adversary out there that

goes I would like one of those just in

case America get out of hand in America

is thinking you know I want one of those

in case Russia gets out of hand

and so okay you might take a

precautionary approach in the United

States but that's only going to put you

on the back foot when China or Russia or

one of your adversaries accelerates

forward in that in that path and it's

the same with the the nuclear bomb and

you know you nailed it I mean that is

the race condition we refer to that as

the race condition the idea that if I

don't do it the other party is gonna do

it and therefore I must do it but the

problem with that is that it creates a

self-fulfilling prophecy so the default

there is that we all end up doing it and

that can't be right because there is a

opportunity for massive cooperation here

there's a shared that is between us and

China and every other quote unquote them

or they or enemy that we want to create

we've all got a shared interest

in advancing the collective health and

well-being of humans and Humanity how

well have we done it promoting shared

interest well in the development of

Technologies over the years even at like

a corporate level even you know

you know the nuclear non-proliferation

treaty has been reasonably successful

there's only nine nuclear states in the

world today we've stopped many like

three countries actually gave up nuclear

weapons because we incentivized them

with sanctions and threats and economic

rewards

um small groups have tried to get access

to nuclear weapons and so far have

largely failed it's expensive though

right and hard to like uranium as a as a

chemical to keep it stable and to to buy

it and to house it I mean I can just put

it in the shed you certainly couldn't

put it in a shed you can't download

uranium 235 off the Internet it's not

available open source that is totally

true so it's got different

characteristics for sure but a kid in

Russia could you know in his bedroom

could download something onto his

computer that's

incredibly harmful in the in artificial

intelligence Department right

I think that that will be possible at

some point in the next five years it's

true because there's a weird Trend

that's going on here on the one hand

You've Got The Cutting Edge

AI models that are built by Google and

open Ai and my company inflection and

they cost hundreds of millions of

dollars and there's only a few of them

but on the other hand the what was

cutting edge a few years ago is now open

source today so gpt3 which came out in

the summer of 2020

is now

reproduced as an open source model so

the code and the weights of the model

the design of the model and the actual

implementation code is completely freely

available on the web and it's tiny it's

like 60 times or 60 70 times smaller

than the original model which means that

it's cheaper to use and cheaper to run

and that's as you know we've said

earlier like that's the natural

trajectory of technologies that become

useful they get more efficient they get

cheaper and they spread further and so

that's the containment challenge that's

really the essence of

well I'm sort of trying to raise in my

book is to frame the challenge of the

next 30 to 50 years as a round

containment and around confronting

proliferation

do you believe because we're both going

In 30 years' time, do you think we would have contained AI?

to be alive unless there's you know some

robot kills us but we're both going to

be 11 30 years time

I hope so maybe the podcast will still

be going unless AI is now taking my job

it's very possible so I'm going to sit

you here and you know when you're I mean

you'll be in what 60 68 years old I'll

be 60.

um

and I'll say

at that point when we have that

conversation do you think we would have

been successful in containment on a

global level

I think we have to be I can't even think

that we're not why

because I'm fundamentally a humanist

and I think that we have to make a

choice

to put our species first

and

I think that that's what we have to be

defending for the next 50 years that's

what we have to defend because look it

it's it's certainly possible that we

invent these agis in such a way that

they are always going to be provably

um subservient uh to humans and take

instructions you know from their human

controller every single time

but enough of us think that we can't be

sure about that

that I don't think we should take the

gamble basically so

that's why I think that we should focus

on containment and non-proliferation

because some people if they do have

access to the technology will want to

take those risks and they will just want

to see like what's on the other side of

the door you know and they might end up

opening Pandora's Box and that's a

decision that affects all of us

and that's the challenge of the

networked age you know we live in this

globalized world and we use these words

like globalization and we you sort of

forget what globalization means this is

what globalization is this is what a

networked world is it means that someone

taking one small action can suddenly

spread everywhere instantly regardless

of their intentions when they took the

action it may be

you know unintentional like you say it

may be that they're never they weren't

ever meaning to do harm

when I think I asked you when I said it

Why would such a being want to interact with us?

in 30 years time you said that there

will be like human

level intelligence you'll be interacting

with

you know this new species but the

species

for me to think the species were one to

interact with me is

feels like wishful thinking because what

will I be to them you know like I've got

a French Bulldog Pablo and I can't

imagine our IQ is that far apart like

you know in relatives terms my the IQ

between me and my dog Pablo I can't

imagine that's that far apart even when

I think about is it like the orangutan

where we only have like one percent

difference in DNA or something crazy and

yet they throw their poop around and I'm

sat here broadcasting around the world

there's quite a difference in that one

percent you know

and then I think about this new species

where as you write in your book in

chapter four

there seems to be no upper limit to ai's

potential intelligence

why would such an intelligence want to

interact with me

well it depends how you design it so

I think that our goal one of the

challenges of containment is to design

AIS that we want to interact with that

want to interact with us right if you

set an objective function for an AI a

goal for an AI by its design

which you know inherently disregards or

disrespects you as a human and your

goals then it's going to wander off and

do a lot of strange things whatever has

kids and the kids

you know what I mean what if it

replicates in a way where because

because I've I've heard this this

conversation around like it depends how

we design it but you know

I think about

it's kind of like if I have a kid and

the kid grows up to be a thousand times

more intelligent than me

to think that I can have any influence

on it on it when it's a thinking

sentient

developing

species again feels like

I'm overestimating my version of

intelligence and importance and

significance in the face of something

that is incomprehensibly like even a

hundred times more intelligent than me

and the speed of its computation is a

thousand times what my

the meat in my skull can do yeah like

how how is it gonna how how do I know

it's gonna respect me or care about me

or understand you know that I may you

know

I think that comes back down to the

containment challenge I think that if we

can't be confident

that it's going to respect you and

understand you and work for you and us

as a species overall

then that's where we have to adopt the

precautionary principle I don't think we

should be taking those kinds of risks in

experimentation and design and now I'm

not saying it's possible to design an AI

that doesn't have those self-improvement

capabilities in the limit in like 30 or

50 years I think it you know that's kind

of what I was saying is like it seems

likely that if you have one like that

it's going to take advantage of infinite

amounts of data and Infamous amounts of

computation and it's going to kind of

outstrip our ability

to act and so I think we have to step

back from that precipice that's what

the containment problem is is that it's

it's actually saying no sometimes it's

saying no and

that's a different sort of muscle that

we've never really exercised as a

civilization and that's obviously why

containment appears not to be possible

because we've never done it before we've

never done it before and every inch of

our you know Commerce and politics and

our war and our all of our instincts are

just like Clash complete Clash compete

profit profit grow beat exactly dominate

you know fear them be paranoid like now

all this nonsense about like China being

this new evil like it how does that slip

into our culture how are we suddenly all

shifted from thinking it's the the

Muslim terrorists about to blow us all

up to now it's the Chinese who are about

to you know blow up Kansas it's just

like what

about like we really have to pair back

the paranoia and the fear and the

othering

um because those are the incentive

dynamics that are going to drive us to

you know cause self-harm to humanity

thinking the worst of each other there's

a couple of key moments when in my

understanding of artificial intelligence

that have been kind of Paradigm Paradigm

shifts for me because I think like many

people I thought of artificial

intelligence as

you know like a like a child I was

Raising and I would program I would code

it to do certain things so I'd code it

to play chess and I would tell it the

moves that are conducive with being

successful in chess and then I remember

watching that like alphago documentary

right which I think was deep deep mind

that was us yeah you guys so you

programmed this this um artificial

intelligence to play the game go which

is kind of like just think of it kind of

like a chess or a black cam or whatever

and it eventually just beats the best

player in the world of all time and it

and the way it learned how to beat the

best player in the world of all time the

world champion he was by the way

depressed when he got beat

um was just by playing itself right

and then there's this moment I think is

it game four or something right right it

does this move that no one could have

predicted right it's a move that

seemingly makes absolutely no sense

right

in those moments where no one trained it

to do that and it did something

unexpected Beyond where humans are

trying to figure it out in hindsight

this is where I go how do you how do you

train it if it's doing things we didn't

anticipate right

like how do you control it when it's

doing things that humans couldn't

anticipate it doing well we're looking

at that move it's called like move 37

correct yeah is it move 37 it is nice

intelligence nice work yeah a bit longer

than I thought it's like 30. so at least

another decade

um move 37 does this crazy thing and you

see everybody like lean in and go why

has it done that and it turns out to be

brilliant that humans couldn't couldn't

afford the commentator actually thought

it was a mistake yeah he was a pro and

he was like this is definitely a mistake

you know it's the alphagos lost the game

but it was so far ahead of us that I

knew something we didn't right right

that's when that's when I lost hope in

this whole idea of like oh train it to

do what we want like a dog like sit pour

roll over right

well the real challenge is that we

actually wanted to do those things like

when it discovers a new strategy or it

invents a new idea or it helps us find

like you know a cure for some disease

right that's why we're building it right

because we're reaching the limits

of what we as you know humans can invent

and solve right especially with what

we're facing of you know in terms of

population growth over the next 30 Years

and how climate change is going to

affect that and so on like

we really want these tools to

turbocharge us right and yet like it's

that creativity and that invention which

obviously makes us also feel

well

maybe it it is really going to do things

that we don't like for sure right

how do you contend with all of this how

do you contend with the

the clear-up side and then you must like

Elon must be completely aware of the

the horrifying existential risk at the

same time and you're building a big

company in this space which I think is

valued at 4 billion now inflection AI

which has got this its own

model called pi

so you're building in this space you

understand the incentives that both the

nation-state level and a corporate level

that we're going to keep playing forward

even if the US stops there's going to be

some other country that sees that as a

huge Advantage their economy will swell

because they did

if this company stops then this one's

gonna get a get a huge advantage and

then shareholders are you know

everyone's investing in AI full steam

ahead

but you feel you can see this huge

existential risk is it suspended is that

the password suspended disbelief

I mean just kind of like just know that

it's I feel like I know that's going to

happen

no one's been able to tell me otherwise

but just don't think too much about it

and you'll be okay

I think you can't give up

right I think that in some ways

your realization exactly what you've

just described like weighing up

two conflicting and horrible truths

about what is likely to happen those

contradictions

that is a kind of honesty and a wisdom I

think that we need all collectively to

realize because the only path through

this is to

be straight up and embrace you know the

risks and embrace the default trajectory

of all these competing incentives

driving forward to kind of make this

feel like inevitable and if you put the

blinkers on and you kind of just ignore

it or if you just be super Rosy and it's

all going to be all right and if you say

that we've always figured out anyway

then we're not going to get the energy

and the dynamism and engagement from

everybody to try to figure this out and

that's what gives me like reason to be

hopeful because I think that we make

progress by getting everybody paying

attention to this it isn't going to be

about those who are currently the AI

scientists or those who are the

technologists or you know like me or the

Venture capitalists or just the

politicians like all of those people no

one's got answers so that's what we have

to confront there are no obvious answers

to this profound question

and I've basically written the book to

say

prove that I'm wrong

you know containment must be possible

and I it must be it must be possible it

has to be possible it has to be you want

it to be I I desperately want it to be

yeah why must it be

because

otherwise I think you're in the camp of

believing that this is

the inevitable evolution of humans

the transhuman

kind of view

you know some people would argue like

what is okay let's part let's let's

stretch the timelines out okay so let's

not talk about 30 years

let's talk about 200 years

like what is this gonna look like in

Quantum computers & their potential

2200

you tell me you're smarter than me

I mean it's mind-blowing it's

mind-blowing we'll have quantum

computers by then what's a quantum

computer

a quantum computer is a completely

different type of computing architecture

which in simple terms

basically allows you to prove those

those calculations that I described at

the beginning billions and billions of

flops those billions of flops can be

done in a single computation

so everything that you see in the

digital world today relies on computers

processing information

and and the speed of that processing is

a friction it kind of slows things down

right you remember back in the day old

school modems 56k modem the dial-up

sound and the image pixel loading like

pixel by pixel that was because the

computers were slow and we're getting to

a point now where the computers are

getting faster and faster and faster and

Quantum Computing is like a whole new

leap like way way way Beyond where we

where we currently are and so by analogy

how would I understand that so like if

my I've got my dial-up modem over here

and then Quantum Computing over here

right

what's the how do I what's the

difference well I don't know what it's

really difficult to explain billion

times faster oh it's it's like it's like

billions of billions times faster it's

it's it's much more than that I mean one

way of thinking about it is like

a floppy disk which I guess most people

remember 1.4 megabytes a physical thing

back in the day

in 1960 or so

that was basically an entire pallet's

worth of computer that was moved around

by a forklift truck

right which is insane today you know you

have billions and billions of times that

floppy disk

in your smartphone in your pocket

tomorrow you're going to have

billions and billions of smartphones in

minuscule wearable devices there'll be

cheap fridge magnets that you know are

constantly on everywhere sensing all the

time

monitoring processing analyzing

improving optimizing

you know and they'll be super cheap

so

it's super unclear what do you do with

all of that knowledge and information I

mean it's ultimately knowledge

creates value when you know the

relationship between things you can

improve them you know make it more

efficient

and so more data is what has enabled us

to build all the value of you know

online in the last 25 years

and so what does that look like in 150

years I can't really even imagine to be

honest with you it's very hard to say I

don't think everybody is going to be

working

why would we yeah well

we wouldn't be working in that kind of

environment I mean look the other

trajectory to add to this

is the cost of energy production

you know AI if it really helps us solve

battery storage

which is the missing piece I think to

really tackle climate change

then we will be able to source

basically source and store

Infinite Energy

from the Sun

and I think in 20 or so years time 20 30

years time

that is going to be a cheap and widely

available if not completely freely

available resource and if you think

about it everything in life

has the cost of energy built into its

production value

and so if you strip that out

everything is likely to get a lot

cheaper we'll be able to desalinate

water we'll be able to grow crops much

much cheaper we were able to grow much

higher quality food right it's going to

power New forms of transportation it's

going to reduce the cost of drug

production and health care right so all

of those gains obviously there'll be a

huge commercial incentive to drive the

production of those gains but the cost

of producing them is going to go through

the floor I think that's one key thing

that a lot of people don't realize that

is a reason to be hugely hopeful and

optimistic about the future everything

is going to get radically cheaper in 30

to 50 years

hmm

so 200 years time we have no idea what

the world looks like it's uh this goes

back to the point about being is it did

you say transhumanist right what does

that mean

transhumanism

I mean it's a group of people who

basically believe that

you the the humans and our soul and our

being will one day transcend or move

beyond our biological substrate okay so

our physical body our brain our biology

is just an enabler for your intelligence

and who you are as a person and

there's a group of kind of crack Bots

basically I think who think that we're

going to be able to upload ourselves

to a silicon substrate right a computer

that can hold the essence of what it

means to be Stephen so you in 220 uh in

in 2200

well could well still be You by their

reasoning

but you'll live on a server somewhere

why are they wrong I think about all

these adjacent Technologies like

biological

um biological advancements did you call

it like biosynthesis or something yeah

synthetic biology synthetic biology

um I think about the nanotechnology

development right about Quantum

computing

the the progress in artificial

intelligence that everything becoming

cheap and I think why why are they wrong

it's hard to say precisely

but broadly speaking I haven't seen any

evidence yet that we're able to extract

the essence of a being from a brain

right it's that that is that that kind

of dualism that you know there is a mind

and a body and a spirit that is a I I

don't think I don't see much evidence

for that even in Neuroscience

um that actually it's much more one and

the same so I don't think you know

you're going to be able to emulate the

entire brain so their thesis is that or

some of them cryogenically store their

brain after death Jesus so they they

have it they've they wear these like you

know how you have like an organ donor

tag or whatever so they have a

cryogenically freeze me when I die tag

and so they there's like a special

ambulance services that will come pick

you up because obviously you need to do

it really quickly the moment you die you

need to get put into a cryogenic freezer

to preserve your you know brain forever

I personally think this is this is nuts

but you know their belief is that you'll

then be able to reboot that biological

brain and then transfer you over

um

it doesn't seem plausible to me when you

said at the start of this this little

topic here that you it must be possible

to contain it so it must be possible

um the reason why I I struggle with that

is because in chapter seven you say a

line in your book that AI is more

autonomous than any other technology in

history

for centuries the idea that

technologists is somehow running out of

control are self-directed and

self-propelling force beyond the Realms

of human agency remained a fiction not

any more

and this idea of autonomous technology

that is

acting un instructed

um and is intelligent and then you say

we must be able to contain it it's kind

of like a massive dog like a big

rottweiler yeah that is you know a

thousand times bigger than me and me

looking up at it and going I'm gonna

give take you for a walk yep yeah and

then it just looking down at me and just

stepping over me or stepping on me well

that's actually a good example because

we have actually contained Rottweilers

before we've contained gorillas and you

know tigers and crocodiles and pandemic

pathogens and nuclear weapons and so you

know it's easy to be you know a hater on

what we've achieved

but this is the most peaceful moment in

the history of our species this is a

moment when our biggest problem is that

people eat too much

think about that we've spent our entire

evolutionary period

running around looking for food and

trying to stop you know our enemies

throwing rocks at us

and we've had this incredible period of

500 years where you know each year

things have broadly were maybe each

each Century let's say there's been a

few ups and downs but things have

broadly got better and we're on a

trajectory for you know life spans to

increase and quality of life to increase

and health and well-being to improve and

I think that's because in many ways we

have succeeded in containing forces that

appear to be more powerful than

ourselves it just requires unbelievable

creativity and adaptation it requires

compromise and it requires a new

tone

right a much more humble tone to

governance and politics and and how we

run our world not this kind of like

hyper aggressive adversarial paranoia

tone that we talked about previously but

one that is like

much more wise than that much more

accepting that we are unleashing this

force that does have that potential to

be the Rottweiler that you described but

that we must

contain that as our number one priority

that has to be the thing that we focus

Cybersecurity

on because otherwise it contains us

I've been thinking a lot recently about

cyber security as well just broadly on

an individual level

in a world where there are these kinds

of tools which seems to be quite close

large language models it brings up this

whole new question about cyber security

and cyber safety and you know in a world

where there's um these ability to

generate audio and language and videos

that seem to be real

um

what can we trust and you know I was

watching a video of a

of a of a young girl whose grandmother

was called up by

a voice that was made to sound like her

son saying he'd been in a car accident

and asking for money

and her nearly sending the money or this

whole what you know because this really

brings into Focus that we our life is a

build on built on trust trusting the

things we see here and

watch and in in and now where it feels

like a

a moment where we're no longer going to

be able to trust what we see on the

internet on the phone

[Music]

what what advice do you do we you have

for people who are worried about this

um

so

skepticism I think is

healthy and necessary and I think that

we're going to need it

um even more than than we ever did right

and so if you think about how we've

adapted to the first wave of this which

was spammy email scams

um everybody got them and over time

people learn to identify them and be

skeptical of them and reject them

likewise you know I'm sure many of us

get like text messages I'd certainly get

loads of text messages trying to fish me

and ask me to meet up or do this that

and the other

and we've adapted right now I think

we should all know and expect that

criminals will use these tools to

manipulate us just as you described I

mean you know the voice is going to be

human-like the Deep fake is going to be

super convincing and there are actually

ways around those things so for example

the reason why the banks invented

OTP um one-time passwords where they

send you a text message with a special

code

um is precisely for this reason so that

you have a 2fa a two-factor

authentication increasingly we will have

a three or four Factor authentication

where you have to triangulate between

multiple separate independent sources

and it won't just be like call your bank

manager and release the funds right so

this is where we need the creativity and

energy and attention of everybody

because

defense the kind of defensive measures

have to evolve as quickly as the

potential offensive measures the attacks

that are coming

I heard you say this that you think um

some people are

for many of these problems we're going

to need to develop AIS to

Defenders from the AIS

right we kind of already have that right

so we have automated ways of detecting

spam online these days you know most of

the time there are machine Learning

Systems which are trying to identify

when your credit card is used in a

fraudulent way that's not a human

sitting there looking at patterns of

spending traffic in real time that's an

AI that is like flagging that something

looks off

um likewise with data centers or

security cameras a lot of those security

cameras these days are you know have

tracking algorithms that look for you

know surprising sounds or like if a if a

glass window is is smashed that will be

detected by an AI often that is you know

listening on the security camera

so you know that's kind of what I mean

by that is that increasingly those AIS

will get more capable and will want to

use them for defensive purposes and

that's exactly what it looks like to

have good healthy well-functioning

controlled AIS that serve us I went on

one of these large language models and

said to me give I said to the large

language model give me an example where

an artificial intelligence takes over

the world or whatever and just ends

results in the destruction of humanity

and then tell me what we'd need to do to

prevent it

and it said it gave me this wonderful

example of this AI called Cynthia that

threatens to destroy the world and it

says

the way to defend that would be a

different AI which had a different name

and it said that this one would be

acting in human interests and would

basically be fighting one AI with

another AI

and of course

of course of course at that level if

Cynthia started to wreak habit havoc on

the world and take control of the

nuclear weapons and infrastructure and

all that we would need an equally

intelligent

weapon to fight it

although one of the interesting things

that we found

um over the last few decades is that it

so far tended to be the AI plus the

human that has that is still dominating

that's the case in chess uh in go in

other games

um that in ghosts or yeah so there was a

paper that came out a few months ago two

months ago that showed that a human was

actually able to beat The Cutting Edge

go program

um even one that was better than alphago

with a new strategy that they had

discovered

um you know so obviously it's not just a

sort of game over environment where the

AI just arrives and it gets better like

humans also adapt they get super smart

they like I say get more cynical ask get

more skeptical ask you know good

questions invent their own things use

their own AIS to adapt and that's the

evolutionary nature of what it means to

have a technology right I mean

everything is a technology like your

pair of glasses

made you smarter in a way like before

there were glasses and people got bad

eyesight they weren't able to read you

know suddenly those who did adopt those

Technologies were able to read for you

know longer in their lives or under low

light conditions and they were able to

consume more information and got smarter

and so that is the trajectory of

Technology it's this iterative interplay

between you know human and machine that

makes us better over time

you know the potential um consequences

Why did you build a company in this space knowing the problems?

if if we don't reach a point of

containment yet you chose to build a

company in this space yeah

why why that

why did you do that because I believe

that

the best way to

uh demonstrate how to build safe and and

contained AI is to actually experiment

with it in practice

and I think that if we are just Skeptics

or critics and we stand back from The

Cutting Edge then we give up that

opportunity to shape outcomes to you

know all of those other actors that we

referred to whether it's like China or

in the US going at each other's throats

uh you know or other big companies that

are purely pursuing profit at all costs

and so it doesn't solve all the problems

of course it's super hard and again it's

full of contradictions but I honestly

think it's the right way for everybody

to proceed you know if you're at the

front yeah if you're afraid Russia Putin

understand right what reduces fear is

deep understanding spend time playing

with these models look at their

weaknesses they're not superhumans yet

they make tons of mistakes they're

crappy in lots of ways they're actually

not that hard to make the more you've

experimented has it has that correlated

with a reduction in fear

[Laughter]

cheeky question I just said no you're

totally right yes it has in the sense

that you know the problem is the more

you learn the more you realize

I was fine before I started talking

about AI

[Laughter]

it's true it's true it's it's sort of

pulling on a thread which

is crazy spiral

um yeah I mean like I think in the short

term It's Made Me way less afraid

because I I don't see that kind of

existential harm that we've been talking

about in the next decade or two but

longer term that's that's where I

struggle to wrap my head around how

things play out in 30 years

some people say

Will governments help us regulate it?

government regulation was sorted out

you discussed this in Chapter 13 of your

book where you which is titled

containment must be possible I love how

you didn't say is yeah containment must

be

containment must be possible

um what do you say to people that say

government regulation was sorted out I

had Rishi sunak did some announcement

and he's got a cobra committee coming

together

they'll handle it

that's right and the EU have a huge

piece of regulation called the euai ACT

um you know Joe but President Joe Biden

has you know gotten his own you know set

of proposals and

um you know we've been working with with

both you know Rishi sunak and and Biden

and you know trying to contribute and

shape it in the best way that we can

look it isn't going to happen without

regulation

so regulation is essential it's critical

um again going back to the precautionary

principle but at the same time

regulation isn't enough you know I often

hear people say well we'll just regulate

it we'll just stop

we'll just stop we'll just stop we'll

slow down

um and the problem with that is that it

kind of

ignores the fact that the people who are

putting together the regulation don't

really understand enough about the

detail today you know in their defense

they're rapidly trying to wrap their

head around it especially in the last

six months and that's a great relief to

me because I feel the burden is now

increasingly shared and you know just

from a personal perspective I'm like I

feel like I've been saying this for

about a decade and just in the last six

months now everyone's coming at me and

saying like you know what's going on I'm

like great this is the conversation we

need to be having because everybody can

start to see the glimmers of the future

like what will happen if a chat GPT like

product or a pie like product really

does improve over the next 10 years and

so when I say you know regulation is not

enough what I mean is it needs movements

it needs culture it needs people who are

actually building and making you know in

like modern creative critical ways not

just like giving it up to you know

companies or small groups of people

right we need lots of different people

experimenting with strategies for

containment isn't it predictable this

industry is a 15 trillion dollar

industry or something like that yeah

I've heard that it is it's a lot so if

I'm Rishi and I know that I'm going to

be chucked out of office

Rishi is the prime minister of the UK If

I'm going to be trucked out of office in

two years unless this economy gets good

I don't want to do anything to slow down

that 15 trillion dollar bag that I could

be on the receiving end of I would I

would definitely not want to slow that

15 billion trillion dollar bag and give

it to like America or Canada or some

other country I'd want that 15 trillion

dollar

windfall to be on my country

right so I have I have no

other than the long-term you know

health and success of humanity in my

four-year election window I've got to do

everything I can to boost these numbers

right and get us looking good

so I could give you lip service

but but listen I'm not going to be here

unless these numbers look good

right

exactly

that's another one of the problems

short-termism is everywhere

who is responsible for thinking about

the 20-year future

who is it I mean that's a deep question

right I mean we we the world is

happening to us on a decade by decade

time scale it's also happening hour by

hour so change is just ripping through

us and this arbitrary window of

governance of like a four-year election

cycle where actually it's not even four

years because by the time you've got in

you do some stuff for six months and

then by month you know 12 or 18 you're

starting to think about the next cycle

and are you going to pull you know this

is like the short-termism is killing us

right and we don't have an Institutional

body

whose responsibility is stability

you could think of it as like a you know

like a global technology stability

function

what is the global strategy for

containment that has the ability to

introduce friction when necessary to

implement the precautionary principle

and to basically keep the peace

that I think is the missing governance

piece which we have to invent in the

next 20 years and it's insane because

I'm basically describing

the UN Security Council bust the World

Trade Organization all these huge

you know Global institutions which

formed after you know the horrors of the

second World War

have actually been incredible they've

created interdependence and alignment

and stability right obviously there's

been a lot of bumps along the way in the

last 70 years but broadly speaking it's

an unprecedented period of peace and

when there's peace we can create

prosperity and that's actually what

we're lacking at the moment is that we

don't have an international mechanism

for coordinating among competing Nations

competing corporations

um to drive the peace in fact we're

actually going kind of in the opposite

direction we're resorting to the old

school language of a clash of

civilizations with like China is the new

enemy they're going to come to dominate

us we have to dominate them it's a it's

a battle between two polls China's

taking over Africa China's taking over

the Middle East we have to count I mean

it's just like that can only lead to

conflict that just assumes that conflict

is inevitable and so when I say it

regulation is not enough no amount of

good regulation in the UK or in Europe

or in the US is going to deal with that

Clash of civilization's language which

we seem to have been become addicted to

if we need that Global collaboration to

be successful here

are you optimistic now that we'll we'll

get it because the same incentives are

at play with climate change and AI you

know why would I want to reduce my

carbon emissions when it's making me

loads of money or well you know

why would I want to reduce my AI

development when it's going to make us

15 trillion yep

so the the really painful answer to that

question is that we've only

really ever driven extreme compromise

and consensus in two scenarios one off

the back of

unimaginable catastrophe and suffering

you know Hiroshima Nagasaki and the

Holocaust and World War II which drove

10 years of consensus and new political

structures right

and then the second is

um we did fire the bullet though didn't

we we find a couple of those nuclear

bombs exactly and that that's why I'm

saying the brutal truth of that is that

it takes a catastrophe to trigger

the need for alignment right so that

that's one the second is where there is

an obvious mutually assured destruction

um you know

Dynamic where both parties are afraid

that this would trigger nuclear meltdown

right and that means suicide and when

there was few parties

exactly

when there was just nine people exactly

could get all nine but in in when we're

talking about artificial technology

there's going to be more than nine

people right

that have power access to the full sort

of power of that technology for

nefarious reasons I don't think it has

to be like that I think that's the

challenge of containment is to reduce

the number of actors that have access to

the existential threat Technologies to

an absolute minimum and then use the

existing military and economic

incentives which have driven World Order

and peace so far

um to prevent the proliferation of

access to these super intelligences or

these agis a quick word on huel as you

know they're a sponsor of this podcast

and I'm an investor in the company and I

have to say it's moments like this in my

life where I'm extremely busy and I'm

flying all over the place and I'm

recording TV shows and I'm recording

shows in America and here in the UK that

huel is a necessity in my life I'm

someone that regardless of external

circumstances or professional demands

wants to stay healthy and nutritionally

complete and that's exactly where heel

fits in my life it's enabled me to get

all of the vitamins and minerals and

nutrients that I need in my diet to be

aligned with my health goals while also

not dropping the ball on my profession

goals because it's convenient and

because I can get it online in Tesco in

supermarkets all over the country if

you're one of those people that hasn't

yet tried Hill or you have before but

for whatever reason you're not a kill

consumer right now I would highly

recommend giving huel a go and Tesco

have now increased their listings with

huel so you can now get the RTD ready to

drink in Tesco expresses all across the

UK

10 areas of focus for containment you're

the first person I've met that's really

hazarded a laid out a blueprint for the

things that need to be done

um cohesively to try and reach this

point of containment so I'm super

What do we need to do to contain it?

excited to talk to you about these the

first one is about safety and you

mentioned that that's kind of what we

talked about a little bit about there

being AIS that are currently being

developed to help contain other AIS

Le tune audits

um which is being able to from what I

understand being able to audit what's

being built in the these open source

models

three choke points what's that

yeah so choke points refers to points in

the supply chain where you can throttle

who has access to what okay so on the

internet today everyone thinks of the

internet as an idea this kind of

abstract Cloud thing that hovers around

above our heads but really the internet

is a bunch of cables those cables you

know are physical things that transmit

information you know under the sea and

you know those points the end points can

be stopped and you can monitor traffic

you can control basically what traffic

moves back and forth and then the second

choke point is access to chips so the

gpus graphics processing units which are

used to train these super large clusters

I mean we now have the second largest

supercomputer in the world today uh at

least we you know just for this next six

months we will other people will catch

up soon but we're ahead of the curve

we're very lucky cost a billion dollars

and those chips are really the raw

commodity that we use to build these

large language models and access to

those chips is something that

governments can should and are you know

restricting that's a choke point you

spent a billion dollars on a computer we

did yeah it's a bit more than that

actually about 1.3

a couple of years time that'll be the

price of an iPhone

that's the problem everyone's gonna have

it

number six is quite curious you say that

um the need for governments to put

increased taxation on AI companies to be

able to find fund the massive changes in

society such as paying for reskilling

and education yeah

um

you put massive tax on it over here I'm

going to go over here

if you tax it if I'm an AI company and

you're taxing me heavily over here I'm

going to Dubai yep

or Portugal yep

so if it's that much of a competitive

disadvantage

I will not build my company where the

taxation is high right

so the way to think about this is what

are the strategies for containment if

we're agreed that long term we want to

contain that is closed down slow down

control

both the proliferation of these

Technologies and the way the really big

AIS are used

then the way to do that is to tax things

tax things taxing things slows them down

and that's what you're looking for

provided you can coordinate

internationally so you're totally right

that you know some people will move to

Singapore or to Abu Dhabi or Dubai or

whatever

the reality is that at least for the

next you know sort of period I would say

10 years or so the concentrations of

intellectual

you know horsepower will remain the big

mega cities right you know I I moved

from London in 2020 to go to Silicon

Valley and I started my new company in

Silicon Valley because the concentration

of talent there is overwhelming all the

very best people are there on in in Ai

and software engineering

so I think it's quite likely that that's

going to remain the case for the

foreseeable future but in the long term

you're totally right how do you it's

another coordination problem how do we

get nation states to collectively agree

that we want to try and contain that we

want to slow down because as we've

discussed with the proliferation of

dangerous materials or on the military

side there's no use one person doing it

or one country doing it if others race

ahead and that's the conundrum that we

face

I am I don't consider myself to be a

pessimist in my life I consider myself

to be an optimist generally I think and

I always I think that as you've said I

think we have no choice but to be

optimistic and I have faith in humanity

we've done so much so many incredible

things and overcome so many things and I

also think I'm really logical as in I'm

the type of person that needs evidence

to change my beliefs either way

um when I look at all of the whole

picture having spoken to you and several

others on this subject matter I see more

reasons why we won't be able to contain

the reasons why we will especially when

I dig into those incentives

um you talk about incentives at length

in your book

um at different different points and

it's clear that all the incentives are

pushing towards a lack of containment

especially in the short midterm which

tends to happen with new technologies in

the short and Midterm it's like a land

grab

the gold is in the Stream we all rush to

get the shovels and the you know the

sieves and stuff and then we realize the

unintended consequences of that

hopefully not before it's too late

in chapter 8 you talk about Unstoppable

incentives at play here

the coming wave represents the greatest

economic prize in history

and scientists and technologists are all

too human they crave status success and

Legacy

and they want to be recognized as the

first and the best they're competitive

and clever with a carefully nurtured

sense of their place in the world and in

history

right

I look at you I look at people like Sam

um from open AI Elon

you're all humans

with the same understanding of your

place in history and status and success

you all want that right right

there's a lot of people that maybe

aren't as don't have as good a track

record of you at doing the right thing

which you certainly have that will just

want the status and the success and the

money

incredibly strong incentives I always

think about incentives as being the

thing that you look at exactly you want

to understand how people will behave all

of the incentives on a geopolitical like

on a Global level suggests that

containment won't happen hmm am I right

in that assumption

that all the incentives suggests

containment won't happen in the short or

midterm until there is a cap a tragic

event that makes us forces us towards

that

idea of containment or if there is a

threat

of mutually assured destruction

right so that and that's the case that

I'm trying to make is that let's not

wait

for something catastrophic to happen so

it's self-evident that we all have to

work towards containment right

I mean you you would have thought

that the Potential Threat the potential

idea

that covid-19

was

a side effect let's call it of a

laboratory in Wuhan that was exploring

gain of function research where it was

deliberately trying to

basically make the pathogen more

transmissible you would have thought

that warning to all of us let's let's

not even debate whether it was or wasn't

but just the fact that it's conceivable

that it could be

that should really in my opinion have

forced all of us to instantly agree

that this kind of research should just

be shut down because well we should just

not be doing gain a function research on

what planet could we possibly persuade

ourselves that we can overcome the

containment problem in biology because

we've proven that we can't because it

could have potentially got out and

there's a number of other examples of

where it did get out of other diseases

like foot and mouth disease

back in the 90s in the UK

so but that didn't change our Behavior

right well foot and mouth disease

clearly didn't cause enough harm because

it only killed a bunch of cattle right

um and the pandemic we can't seem to you

know covid-19 pandemic we can't seem to

agree you know that it really was

from a lab and not from Venture bats

right and so

that's where I struggle where you know

now you catch me in a moment where I

feel angry and sad and pessimistic

because to me that's like a

straightforwardly obvious conclusion

that you know

this is a type of research that we

should be closing down and I think

we should be using these moments to give

us insight and wisdom about how we

handle other technology trajectories

in the next few decades we should we

should that's what I'm advocating for

must that's the best I can do I want to

know will will I think the odds are low

I can only do my best I'm doing my best

to advocate for it I mean you know like

I'll give you an example like I think

autonomy is a type of AI capability that

we should not be pursuing really like

autonomous cars and stuff well I I

autonomous cars I think are slightly

different because autonomous cars

operate within a much more constrained

physical domain right like you know you

you really can the containment

strategies for autonomous cars are

actually quite reassuring okay

they have you know GPS control you know

we know exactly all the Telemetry and

how exactly all of those you know

components on board a car operate and we

can observe repeatedly that it behaves

exactly as intended right whereas I

think with with other forms of autonomy

that people might be pursuing like

online okay you know where you have an

AI that is like designed to self-improve

without any human oversight or a

battlefield weapon which you know like

unlike a car hasn't been you know over

that particular moment in the

battlefield millions of times but is

actually facing a new enemy every time

you know every single time and we're

just going to go and you know allow

these autonomous weapons to have you

know these autonomous military robots to

have lethal Force

something that we should really resist I

don't think we want to have autonomous

robots that have lethal Force you're a

super smart guy and I I struggle to

believe that you're you you because you

you demonstrate such a clear

understanding of the incentives in your

book that I struggle to believe that you

don't think the incentives will win out

especially in the short near term and

then the problem is in the short and

near term as it is the case with most of

these waves is we

we wake up in 10 years time ago how the

hell did we get here right and why like

and we and as you say this precautionary

approach of we should have rang the bell

earlier we should have sounded the alarm

earlier but we waltzed in with optimism

right and with that kind of aversion to

confronting the realities of it and that

then we woke up in 30 years and we're on

a leash right and there's a big

rottweiler and we're we've lost control

we've lost you know

I I

I would love to know

[Music]

someone as smart as you I don't I don't

believe can be can believe that

containment is

possible

and that's me just being completely

honest I'm not saying you're lying to me

but I just can't see how someone Smiles

you and in the know is you can believe

that containment is going to happen well

I didn't say it is possible I said it

must be right which is this is what we

keep discussing like that's an important

distinction you know on the face of it

look what I I care about

I care about science I care about facts

I care about describing the world as I

see it and what I've set out to do in

the book is describe a set of

interlocking incentives which drive a

technology production process which

produces potentially really dangerous

outcomes and what I'm trying to do is

frame those outcomes in the context of

the containment problem and say this is

the big challenge of the 21st century

containment is the challenge and if it

isn't possible then we have serious

issues and on the face of it like I've

said in the book I mean the first

chapter is called containment is not

possible right the last chapter is

called containment must be possible for

all our sakes it must be possible but

that but I agree with you that I'm not

I'm not saying it is I'm saying

this is what we have to be working on we

have no choice we have no choice but to

work on this problem

this is a critical problem

how much of your time are you focusing

on this problem

basically all my time I mean building

and creating is about understanding how

these models work what their limitations

are how to build it safely and ethically

I mean we have designed the structure of

the company to focus on the safety and

ethics aspects so for example we are a

public benefit Corporation right which

is a new type of corporation which gives

us a legal obligation to balance profit

making with the consequences of our

actions as a company on the rest of the

world the way that we affect the

environment you know the way that we

affect people the way that we affect

users that people who aren't users of

our products

and that's a really interesting I think

an important New Direction it's a new

Evolution in corporate structure because

it says we have a responsibility to

proactively do our best to do the right

thing right and I think that if if you

were a tobacco company back in the day

or an oil company back in the day and

your legal Charter said that your

directors are liable if they don't meet

the criteria of stewarding your work in

a way that doesn't just optimize profit

which is what all companies are

incentivized to do at the moment talking

about incentives but actually in equal

measure attend to the importance of

doing good in the world to me that's a

incremental but important innovation in

how we organize society and how we

incentivize our work so it doesn't solve

everything is it's not a Panacea but

that's my effort to try and take a small

step in the right direction do you ever

get sad about it about what's happening

yeah

for sure

for sure it's intense

it's intense it's a lot to take in

Do you feel sad about all of this?

this is

it's a it's a very real

ity

does that weigh on you

yeah it does I mean every day every day

I mean I've been working on this for

many years now and it's uh

you know it's it's emotionally

a lot to take in it's it's it's hard to

think about

the far out future

and how your actions today our actions

collectively our weaknesses our failures

that you know that irritation that I

have that we can't

learn the lessons from the pandemic

right like all of those moments where

you feel the frustration of government's

not working properly

or corporations not listening or some of

the obsessions that we have in culture

where we're debating like small things

you know

and you're just like whoa

we need to focus on the big picture here

you must feel a certain sense of

responsibility as well that most people

won't carry because you've spent so much

of your life at the very cutting edge of

this technology and you understand it

better than most you can speak to it

better than most so you have a greater

chance than many at steering

that's a responsibility

yeah I embraced that I try to treat that

as a privilege

I feel lucky to have the

opportunity to try and do that there's

this wonderful thing in my favorite

theatrical play called Hamilton where he

says history has its eyes on you

do you feel that

yeah I feel that I feel that I feel that

it's a good way of putting it

I do feel that

you're happy right

well what is happiness

[Laughter]

um what's the range of emotions that you

you contend with on a

on a frequent basis if you're being

honest

I think

is kind of exhausting and exhilarating

in equal measure because

for me it is beautiful to see people

interact with AIS and get huge benefit

out of it I mean you know every day now

millions of people have a super smart

tool in their pocket that is making them

wiser and healthier and happier

providing emotional support answering

questions of every type making you more

intelligent and so on the face of it in

the short term that feels incredible

it's amazing what we're all building

but

in the longer term it is exhausting to

keep making this argument and you know

have been doing it for a long time

and in a weird way I feel a bit of a

sense of relief in the last six months

because after chat gbt and you know this

this wave feels like it started to

arrive and everybody gets it so I feel

like it's a shared problem now

and uh

that feels nice and it's not just

bouncing around in your head a little

bit it's not just in my head and a few

other people that deepmind and open Ai

and other places that have been talking

about it for a long time

ultimately human beings May no longer be

the primary planetary drivers as we have

become accustomed to being we are going

to live in an Epoch where the majority

of our daily interactions are not with

other people but with AIS

page

284 of your book

the last page yeah

Well slowly move more toward AI interactions over human ones.

think about how much of your day you

spend looking at a screen

12 hours pretty much right whether it's

a phone or an iPad or a desktop

versus how much time you spend looking

into the eyes of your friends and your

loved ones

and so to me it's like

we're already there in a way you know

what I meant by that was you know

this is a world that we're kind of

already in

you know the last three years people

have been talking about metaverse

metaverse metaverse

and the mischaracterization of the

metaverse was that it's over there

it was this like virtual world that we

would all Bop around in and talk to each

other as these little characters and but

that was totally wrong that was a

complete mis-framing

the metaverse is already here

it's the digital space that exists in

parallel time to our everyday life it's

the conversation that you will have on

Twitter or you know the video that

you'll post on YouTube or this podcast

that will go out and connect with other

people it's that meta space of

interaction you know and I use meta to

mean Beyond this space Not Just

that weird other over their space that

people seem to point to and that's

really what is emerging here it's this

parallel digital space that is going to

live alongside with and in relation to

our physical world your kids come to you

got kids no I don't have kids your

future kids if you ever have kids a

young child walks up to you and says

asks that question that Elon was asked

what should I do about with my future

what should I pursue in the light of

everything you know about how our

artificial intelligence is going to

What should young people be dedicating their lives to?

change the world and computational power

and all of these things what should I

dedicate my life to what'd you say

I would say knowledge is power embrace

understand

grapple with the consequences don't look

the other way when it feels scary

and do everything you can to understand

and participate and shape

because it is coming

and if someone's listening to this and

they want to do something to help this

battle for which I think you present as

a solution containment

what can the individual do

read listen use the tools try to make

the tools

understand the current state of

Regulation

see which organizations are organizing

around it like you know campaign groups

activism groups you know find solidarity

connect with other people

spend time online ask these questions

mention it at the pub you know

ask your parents ask your mom how she's

reacting to you know talking to Alexa or

whatever it is that she might do

pay attention

I think that's already enough and

there's no need to be more prescriptive

than that because I think people are

creative and independent and will

it will it will be obvious to you what

you as an individual feel you need to

contribute In This Moment provided

you're paying attention

last question

what if we fail and what if we succeed

what if we fail in containment and what

if we succeed in containment of

artificial intelligence

I honestly think that if we succeed

this is going to be the most productive

What happens if we fail in containment, and what happens if we succeed?

and the most meritocratic moment in the

history of our species we are about to

make intelligence widely available to

hundreds of millions if not billions of

people and that is all going to make us

smarter and much more creative and much

more productive and I think over the

next few decades we will solve many of

our biggest Social Challenges I really

believe that I really believe we're

going to reduce the cost of energy

production storage and distribution to

zero marginal cost we're going to reduce

the cost of producing healthy food and

make that widely available to everybody

and I think

the same trajectory with Healthcare with

Transportation with education

I think that ends up producing radical

abundance over a 30-year period and in a

world of radical abundance what do I do

with my day

I think that's another profound question

and believe me that is a good problem to

have if we can absolutely if we not need

meaning and purpose and oh man that is a

better problem to have than what we've

just been talking about for the last

like 90 minutes yeah and I think that's

wonderful

isn't that amazing I don't know I don't

know the reason I I I'm unsure is

because

everything that seems wonderful has a

has a unintended consequence I'm sure it

does we live in a world of food

abundance in the west and our biggest

problem is obesity right so I'll take

that problem in the grand scheme of

everything humans not need struggle

do we not need that kind of meaningful

voluntary struggle I think we'll create

new times other you know opportunities

to Quest okay you know I I think that's

an easier problem to solve and I think

it's an amazing problem like many people

really don't want to work right they

want to pursue their passion and their

Hobby and you know all the things that

you talk about and so on and absolutely

like we're now I think going to be

heading towards a world where we can

liberate people from the the shackles of

work unless you really want to Universal

basic income I've long been an advocate

of Ubi very long time I think it's a

check every month I don't think he's

going to quite take that form I actually

think it's going to be that we basically

reduce the cost of producing basic Goods

so that you're not as dependent on

income like imagine if you did have

basically free energy and food and you

you could use that free energy to grow

your own food you could grow it in a

desert because you would have adapted

seeds and so on you would have you know

desalination and so on that really

changes the structure of cities it

changes the structure of Nations it

means that you really can live in quite

different ways for very extended periods

without contact with the kind of Center

I mean I'm actually not a huge advocate

of that kind of libertarian you know wet

dream but like I think if you think

about it in theory it's kind of a really

interesting Dynamic that's what

proliferation of power means power isn't

just about access to intelligence it's

about access to these tools which allow

you to take control of your own destiny

and your life and create meaning and

purpose in the way that you you know

might Envision and that's incredibly

creative incredibly creative time that's

what success looks like to me and

well in some ways the downside of that I

think the failure is not achieving a

world of radical abundance in my opinion

and and more importantly failure is a

failure to contain right what does that

lead to

I think it leads to a mass proliferation

of power and people who have really bad

you know intentions what does that lead

to well potentially use that power to

cause harm to others this is part of the

challenge right a small in this

networked globalized World a tiny group

of people

who wish to deliberately cause harm

are going to have access to tools that

can instantly quickly have large-scale

impact on many many other people

and that's the challenge of

proliferation is preventing those Bad

actors from getting access to the means

to completely destabilize

um our world that's what containment is

about

we have a closing tradition on this

podcast where the last guest leaves a

question for the next guest not knowing

who they're leaving the question for

the question left for you is

what is a space or place that you

consider the most

sacred

The last guest's question

um

well I think one of the most beautiful

places I remember going to as a child

was

um Windermere Lake in the Lake District

um and

I was pretty young and on a

on a dinghy with uh some family members

and I just remember it being incredibly

Serene and beautiful and and calm I

actually haven't been back there since

but

that was a pretty beautiful place

seems like the antithesis of the world

we live in right maybe I should go back

there and chill out

maybe thank you so much for writing such

a great book it's wonderful to to read a

book on this subject matter that does

present Solutions because not many of

them do and it presents them in a

balanced way that appreciates both sides

of the argument doesn't isn't tempted to

just play to either what do they call it

playing to like the crowd no they call

like playing to the orchestra can't

remember right but just it doesn't

attempt to play to either side or Ponder

to either side in order to score points

it seems to be entirely nuanced

incredibly smart and Incredibly

necessary because of the stakes that the

book confronts

um that are at play in the world at the

moment and and that's really important

it's very very very important and it's

important that I think everybody reads

this book it's incredibly accessible as

well and I said to Jack who's the

director of this podcast before we

started recording that there's so many

term there's so many terms like

nanotechnology and um or the stuff about

like biotechnologies and Quantum

Computing that reading through the book

suddenly I understood what they meant

and these had been kind of exclusive

terms and Technologies and I also had

never understood the relationship that

all of these Technologies now have with

each other and how like robotics

emerging with artificial intelligence is

going to cause this whole new range of

possibilities that again

have a good side and a potential

downside

um It's a Wonderful book and it's

perfectly timed it's perfectly timed

wonderfully written perfectly timed I'm

so thankful that I got to read it and I

highly recommend that anybody that's

curious on this subject matter goes and

gets the book so thank you Mustafa

really really appreciate your time and

hopefully it wasn't too uncomfortable

for you thank you this was awesome I

loved it it was really fun and uh thanks

for such a amazing wide-ranging

conversation

you

if you've been listening to this podcast

over the last few months you'll know

that we're sponsored and supported by

Airbnb but it amazes me how many people

don't realize they could actually be

sitting on their very own Airbnb for me

as someone who works away a lot it just

makes sense to Airbnb my place at home

whilst I'm away if your job requires you

to be away from home for extended

periods of time why leave your home

empty you can so easily turn your home

into an Airbnb and let it generate

income for you whilst you're on the road

whether you could use a little extra

money to cover some bills or for

something a little bit more fun your

home might just be worth more than you

think and you can find out how much it's

worth at airbnb.com

host that's airbnb.co.uk host

[Music]

ha

ha ha

ha ha

Engels (automatisch gegenereerd)

Alle

Van The Diary Of A CEO

Gerelateerd

Bekeken