AI mystery

AI Mystery.

No one yet knows how ChatGPT and its artificial intelligence cousin will transform the world.

One reason is that no one really knows what goes on inside them.

Some of these system’s abilities go far beyond what they were trained to do.

Even the inventors are baffled as to why.

A growing number of tests suggests these AI systems develop internal models, 

of the real world, much as our brain does, although the machines technique is different.

Everything we want to do with them in order to make them better are safer, 

seems to be ridiculous, if we don’t understand how they work.

At one level scientists understand GPT, or Generator Pre-trained Transformer, 

and other Large Language Models or LLMs, perfectly well.

The models rely on a machine learning system called neural network.

Such networks have a structure modelled loosely, 

after the connected neurons of the human brain.

The code for these programs is relatively simple and fills just a few screens.

It sets up an auto correction algorithm, which chooses the most likely word, 

to complete a passage based on hundreds of gigabytes of internet text.

Additional training ensures the system will present its results in the form of dialogue.

In this sense, all it does is regurgitate what it learned.

It is a stochastic parrot.

But LLMs have also managed the bar exam, write a sonnet about the Higgs boson, 

and make an attempt to breakup their users’ marriage.

Few had expected a fairly straight forward autocorrection algorithm, 

to acquire such broad abilities.

A GBT and other AI systems performed tasks, they were not trained to do,

giving them emergent abilities, which has surprised scientists, 

who have been generally skeptical about the hype over LLMs.

Some scientists believe that it is certainly more than a stochastic parrot, 

and it certainly builds some representation of the world.

However, they do not think that it is quite like how humans build an internal world model.

The models had already demonstrated the ability to write computer code,

which is impressive but not too surprising, 

because there is so much code on the internet to mimic.

One scientist went a step further and showed that GPT can execute code also.

He typed in a program to calculate the 83rd number in the Fibonacci sequence.

It was a multistep reasoning of a very high degree.

And the bot nailed it.

When the scientists asked directly for the 83rd fibonacci number, however, GPT got it wrong.

This suggests the system wasn’t just parroting the internet.

Rather it was performing its own calculations to reach the correct answer.

Although LLMs run on a computer, it is still not a computer.

It lacks essential computational elements, such as the working memory.

In a tacit acknowledgment that GPT on its own should not be able to run codes, 

its inventor, tech company openAI, 

a specialised plugin - a tool that chatGPT can use when answering a query - 

that allows it to do so.

But the plugin was not used in the demonstration.

Scientists hypothesise that the machine improvised a memory by harnessing, 

its mechanisms for interpreting words according to their context. 

This is similar to how nature repurposes existing capacities for new functions.

This impromptu abilities demonstrates that LLMs develop a internal complexity, 

that goes well beyond a shallow statistical analysis.

Researches are finding that these systems seem to achieve genuine understanding, 

of what they have learnt.

In another experiment scientists made their own smaller copy of the GPT neural network,

so that they could study its inner workings.

They trained it on millions of matches of the board game Othello, 

by feeding in long sequences of moves in text form.

Their model became a really perfect player.

To study how the neural network encoded information, 

they created a miniature ‘probe’ network to analysis the main network, layer by layer.

This was similar to putting an electrical probe into the human brain.

In the case of AI the probe showed that its ‘neural activity’ matched the representation, 

of an Othello game board, Albeit in a convoluted form.

To confirm this, scientists ran the probe in reverse to implant information into the network.

For instance flipping one of the games black marker pieces to a white one.

It was like hacking into the brain of these language models.

The network adjusted its moves accordingly.

Scientists concluded that it was plain Othello roughly like a human, 

- by keeping the game board in its ‘mind’s eye’ and using this model to evaluate moves.

Scientists think that the system learns this skill, 

because it is the most parsimonious description of its training data.

Scientists say, given a whole lot of game scripts, trying to figure out the rule behind it, 

is the best way to compress.

The ability to infer the structure of the outside world is not limited, 

to simple game playing moves.

Scientists fed in sentences such as ‘the key is in the treasure chest’, 

followed by ‘you take the key’.

Using a probe, they found that networks encoded within themselves, 

variables corresponding to ‘chest’ and ‘you’, each with the property, 

of possessing a key or not, and updated these variables sentence by sentence.

The system had no independent way of knowing what a box or key is, 

yet it picked up the concept it needed for the task. 

There seems to be some representation of the state hidden inside the model.

Scientists marvel at how much LLMs are able to learn from text.

For example these networks absorb colour descriptions from internet text,

and construct internal representations of colour.

When they see the word ‘red’, they processes it not just as an abstract symbol, 

but as an concept that has certain relations to maroon, crimson, fuchsia, rust and so on.

Demonstrating this was some what tricky.

Instead of inserting a probe into the network, the scientists studied its response, 

to a series of text prompts.

To check whether it was merely echoing colour relations from online references,

they tried misdirecting the system by telling it that red is in fact green.

This is like the old philosophical thought experiment, 

in which one person’s red is another person’s green.

Rather than parroting back an incorrect answer, 

this system’s colour evaluations changed appropriately to maintain the correct relations.

Picking up on the idea that to perform its auto correct function, 

the system seeks the underlying logic of its training data.

Scientists suggest that the wider the range of data, 

the more general rules the system will discover.

They say, maybe we are seeing such a huge jump, 

because we have reached a diversity of data, 

which is large enough that the only underlying principle to all of it, 

is that intelligent beings produced them.

The only way for the model to explain all the data, is to become intelligent.

In addition to extracting the underlying meaning of language, LLMs can learn on the fly.

In the AI field, the term ‘learning’ is usually reserved, 

for the computationally intensive process, 

in which developers expose the neural network to gigabytes of data, 

and tweak its internal connections.

By the time you type a query into ChatGPT, the network should be fixed;

unlike humans, it should not continue to learn.

It came as a surprise that LLMs do learn from their users prompts - 

an ability known as in-context learning.

Scientists say that it is a different kind of learning, 

that wasn’t really understood to exist before.

One example of how an LLM learns comes from the way humans interact with chatbots,

such as ChatGPT.

You can give the system the examples of how you wanted to respond, and it will obey.

Its outputs are determined by the last several thousand words it has seen.

What it does, given these words, is prescribed by its fixed internal connections - 

but the word sequence nonetheless offers some adaptability.

Entire websites are devoted to ‘jailbreak’ prompts, 

that over come the system’s ‘guardrails’.

These are restrictions that stop the system from telling users how to make a pipe bomb,

for example - typically by directing the model to pretend to be a system without guardrails.

Some people use jailbreak for sketchy purposes.

Others deploy it to elicit more creative answers.

Another type of in context learning happens ‘via chains of thought’ prompting, 

which means asking the network to spell out each step of its reasoning.

This tactic makes it better at logic or arithmetic problems requiring multiple steps.

Scientists have shown that in-context learning, 

follows the same basic computational procedure, 

as standard learning, known as gradient descent.

This procedure was not programmed; the system discovered it without help.

Scientists say that it would need to be a learnt skill.

In fact they think LLMs may have other latent abilities that no one has discovered yet.

Every time they test for a new ability, that can be quantified, they find it.

Although LLMs have enough blind spots not to qualify as Artificial General Intelligence,

or AGI - the term for a machine that attains the resourcefulness of animal brain.

These emergent abilities suggest that tech companies are closer to AGI, 

than even optimists had guessed.

There is indirect evidence that we are probably not far from AGI.

Open AI’s plug-ins have given ChatGPT a modular architecture, 

a little like that of the human brain.

Combining GPT-4, the latest version of the LLM that powers ChatGPT, with various plug-ins,

might be a route towards a human like specialisation of function.

At the same time scientist worry that the window, 

may be closing on their ability to study these systems.

Open AI has not divulged of how it designed and trained GPT-4.

This lack of transparency does not harm researchers.

It also hinders the efforts to understand the social impacts, 

of the rush to adapt the AI technology.

Transparency about these models is very important to ensure safety.