Quantum Randomness

AT ITS deepest level, nature is random and unpredictable. That, most physicists would say, is the unavoidable lesson of quantum theory. Try to track the location of an electron and you'll find only a probability that it is here or there. Measure the spin of an atom and all you get is a 50:50 chance that it is up or down. Watch a photon hit a glass plate and it will either pass through or be reflected, but it's impossible to know which without measuring it.

Where does this randomness come from? Before quantum theory, physicists could believe in determinism, the idea of a world unfolding with precise mathematical certainty. Since then, however, the weird probabilistic behaviour of the quantum world has rudely intruded, and the mainstream view is that this uncertainty is a fundamental feature of everything from alpha particles to Z bosons. Indeed, most quantum researchers celebrate the notion that pure chance lies at the foundations of the universe.

However, a sizeable minority of physicists have long been pushing entirely the opposite view. They remain unconvinced that quantum theory depends on pure chance, and they shun the philosophical contortions of quantum weirdness. The world is not inherently random, they say, it only appears that way. Their response has been to develop quantum models that are deterministic, and that describe a world that has "objective" properties, whether or not we measure them. The problem is that such models have had flaws that many physicists consider fatal, such as inconsistencies with established theories.

Until now, that is. A series of recent papers show that the idea of a deterministic and objective universe is alive and kicking. At the very least, the notion that quantum theory put the nail in the coffin of determinism has been wildly overstated, says physicist Sheldon Goldstein of Rutgers University in New Jersey. He and a cadre of like-minded physicists have been pursuing an alternative quantum theory known as Bohmian mechanics, in which particles follow precise trajectories or paths through space and time, and the future is perfectly predictable from the past. "It's a reformulation of quantum theory that is not at all congenial to supposedly deep quantum philosophy," says Goldstein. "It's precise and objective - and deterministic."

If these researchers can convince their peers, most of whom remain sceptical, it would be a big step towards rebuilding the universe as Einstein wanted, one in which "God does not play dice". It could also trigger a search for evidence of physics beyond quantum theory, paving the way for a better and more intuitive theory of how the universe works. Nearly a century after the discovery of quantum weirdness, it seems determinism may be back.

The debate over quantum theory and determinism started in the 1920s, when physicists Niels Bohr and Werner Heisenberg suggested that the unpredictability of quantum phenomena reflected an inherent fuzziness in nature. Einstein and others countered that the unpredictability might instead reflect nothing more than a lack of adequate knowledge. In principle you could predict the outcome of a coin flip, they argued, if you had perfect knowledge of the coin's initial state and surroundings.

At the historic 1927 Solvay meeting in Brussels, physicist Louis de Broglie tried to further this idea, showing how quantum randomness might arise in a non-mysterious way. He suggested that quantum particles show wave-like phenomena because they are accompanied by "pilot waves" that influence their motion in just the right way as to make them obey the Schrödinger wave equation, a cornerstone of quantum theory. However, most dismissed de Broglie's ideas, citing in particular shortcomings pointed out by the physicist Wolfgang Pauli.

Yet de Broglie's ideas would not go quietly. In the early 1950s, physicist David Bohm developed a more consistent version of the pilot-wave model, one based on the same equations as ordinary quantum theory but offering a different interpretation of them. Bohm found buried within those equations a close link to the mathematics of classical physics, which is based on Newton's laws of motion. Bohmian mechanics asserts that the outcome of an experiment isn't truly random, but is determined by the values of certain "hidden variables". For instance, in quantum theory two electrons may be "entangled" such that their states appear to have a kind of spooky link; measuring the spin of one determines the spin of the other, say. Bohm's theory suggests that they share a hidden variable governing spin. The theory also shows how probabilistic quantum measurements can always arise from specific particle trajectories.

Take a key puzzle in quantum theory: explaining how a beam of particles passing through two slits in a screen will create a wave-like interference pattern, even if the particles are sent one at a time. While mainstream quantum theory insists that you can't give any account of exactly how a given particle moves, Bohmian mechanics can. It suggests that a quantum wave associated with each particle goes through both slits and sets up a pattern of constructive and destructive interference - just like the bright and dark interference bands produced with light. This wave pattern then acts on the particles, driving them towards the "bright" bands of constructive interference(see Diagram).

In the Bohmian view, the statistical interference pattern arises from individual particles following distinct trajectories. This does away with any inherent quantum fuzziness, and shows that it's still possible to believe not only in determinism but also in the intuitive notion that particles really act like particles, having definite positions at all times. "The wave function choreographs the motion of the particles," says physicist Detlef Dürr of Ludwig Maximilian University in Munich, Germany. "As a result, while everything is deterministic, the universe evolves in such way that the appearance of randomness emerges, and precisely as described by the quantum formalism."

Still, the majority of physicists didn't take Bohmian mechanics very seriously, often suggesting that it was contrived and afflicted with technical problems. One of the biggest of these was that Bohm's original model cannot cope with the physics of Einstein's special relativity. It worked only for particles with low energies and speeds, where the number of particles in any process remains fixed. At higher energies, relativistic processes routinely create and destroy particles, as when an electron and positron annihilate one another, turning their energy into light. The simplest version of Bohm's theory could not handle such processes.

So Goldstein and others have tried to develop modified versions of the theory that can. Their work began in the 1980s and 90s as part of an effort to develop Bohmian models that describe not only quantum particles but quantum fields as well, which provide the basic framework of all modern physics. In these models, the universe consists both of particles following precise trajectories and of continuous fields that, like classical magnetic or electric fields, also evolve in a deterministic way. Over the past decade, Goldstein, working with Dürr and physicist Nino Zanghi of the University of Genoa in Italy, has shown that this picture gives a consistent view of relativistic particle processes, while reproducing the accurate predictions of quantum field theory (Physical Review Letters, vol 93, p 090402).

The most promising result to come out of this framework was published last year by Ward Struyve and Hans Westman, both at the Perimeter Institute in Waterloo, Ontario, Canada. They developed a Bohmian model that matches one of the most accurate theories in the history of science - quantum electrodynamics, the theory of light and its interactions with charged particles. In fact, Struyve and Westman found that a number of Bohmian models can easily account for all such phenomena, while remaining fully deterministic (Proceedings of the Royal Society A, vol 463, p 3115).

The researchers have not yet ironed out all the wrinkles. In particular, critics contend that Bohmian models still don't satisfy the fundamental principle of relativity, that all frames of reference are on an equal footing. Nevertheless the models appear to have no serious difficulty in coping with particles being created or destroyed, as many physicists had thought they would. "This is real progress that's happened over the past decade," says Tim Maudlin, a philosopher of physics at Rutgers. "The main objections to the theory have now either been addressed, turned out not to be serious or represent issues for the standard theory as much as the Bohm theory."

Goldstein and others have also solved another nagging problem for Bohmian models: elucidating how a deterministic theory can give rise to the fuzziness observed in quantum experiments in the first place. The uncertainty principle of quantum mechanics states that measuring the position of a quantum particle limits your knowledge of its momentum, and vice versa. The standard explanation is that the particle's state is undetermined until you measure it, but in Bohmian mechanics the state is always well defined. The trick, Goldstein says, is that measuring one variable stirs up uncertainty in the other due to interactions between the measuring device and the particle, in a way that matches the uncertainty principle.

Even so, most physicists are not yet ready to embrace the new models, because one crucial problem remains: Bohmian theory, critics point out, doesn't make any predictions that differ from those of ordinary quantum mechanics. "The theory is successful only because it keeps standard wave mechanics unchanged," says Dieter Zeh of the University of Heidelberg in Germany. He adds that the rest of the theory is biased towards the ideas of classical physics and is "observationally meaningless".

That objection isn't really fair, say Bohmian supporters. After all, one might equally argue that the standard theory doesn't go beyond Bohm's theory. "If some historical circumstances had been only slightly different," says physicist Hrvoje Nikolic of the Rudjer Boskovic Institute in Zagreb, Croatia, "then it would have been very likely that Bohm's deterministic interpretation would have been proposed and accepted first, and would be dominating today."

Philosopher of science Arthur Fine of the University of Washington in Seattle says that ideological objections, not technical ones, have been the main factor in the reluctance to accept Bohmian models. "There are some real criticisms one could raise," he says, "but these aren't the ones you find in the physics literature over the past 80 years." That's why he doubts further theoretical development of Bohmian mechanics will change physicists' minds. "Only new experimental results will do that," he says.

Such experiments might seem impossible, as Bohmian models are supposed to make all the same predictions as ordinary quantum theory. Yet some physicists, such as Antony Valentini of the Perimeter Institute, are now suggesting that the form of Bohmian models naturally points to ways in which the universe might depart from the standard predictions - with experimentally observable consequences.

In the early 1990s, Goldstein, Dürr and Zanghi were able to show that the statistical observations of quantum theory could reflect an "equilibrium" of underlying hidden variables. They found that in many circumstances it is natural to expect those hidden variables to evolve so as to produce that equilibrium, and thus quantum theory as we know it. But their work also suggested that in some situations the hidden variables might be out of equilibrium, in which case Bohmian predictions would differ from those of conventional quantum theory.

Valentini has considered what those circumstances might be, and suggests that they would need to involve extreme conditions, with the most likely setting being the early universe. As he points out, the atoms and particles involved in today's laboratory experiments have all experienced violent astrophysical pasts. "They have histories stretching back to the formation of stars or even earlier," he says, "and have experienced complex interactions with other systems."

All these interactions would have provided ample time and opportunity for the hidden variables governing these particles to reach equilibrium, making their behaviour today follow ordinary quantum theory. In the early universe, however, some particles may have been out of equilibrium, and so followed different patterns. Valentini has proposed several astrophysical experiments that might be able to detect traces of such behaviour (Journal of Physics A: Mathematical and Theoretical, vol 40, p 3285).

For example, standard theories of cosmology suggest that spatial variations in the cosmic microwave background - the radiation left over from the big bang - can be linked to quantum fluctuations in fields of the very early universe. Valentini has shown that the character of these fluctuations would be affected by disturbances in the variables, and could show up as deviations from the predictions of standard cosmology about the spatial structure of the cosmic background. Sensitive maps of this background, from experiments like the Planck satellite scheduled to launch later this year, could detect such variations.

Valentini suggests that other traces of quantum non-equilibrium might turn up in measurements of gravitational waves or the dynamics of black holes. Positive results in any such experiment would not only bolster the Bohmian perspective, but could also provide evidence for completely new physics beyond quantum theory. "My ultimate aim," says Valentini, "is to find an empirical window that could help us determine what the true underlying theory actually is."

The debate over whether the universe is random or deterministic is not likely to end before such experiments become possible. That won't stop physicists and philosophers from continuing to examine whether or not the logical structure of quantum theory demands randomness, or might instead rest on some deeper deterministic layer. In the latter case, predicting the future would be as simple as knowing the fundamental quantum rules and current conditions in enough detail. But even if we could do that, would we really want to? A deterministic universe might just be a little too boring.

Quantum World - Learn more about a weird world in our comprehensivespecial report.

Quantum icons

The de Broglie-Bohm interpretation of quantum mechanics, pioneered by Louis de Broglie (1927) and David Bohm (1952), offers a different take on several popular scenarios from quantum theory. Unlike the conventional view known as the "Copenhagen interpretation", in which quantum particles have indeterminate states until they are measured, in Bohmian mechanics quantum particles have definite positions and states at all times.

Schrödinger's cat

Conventional quantum theory says that particles can be in a superposition of two states at once. This leads to the thought experiment of a cat being both alive and dead inside a box, depending on the state of a toxic subatomic particle. Only when you open the box or make a measurement is the animal's fate determined. Bohmian mechanics says this is nonsense: at any given time the cat is either alive or dead, because the particle's state is fixed, even if we don't know what that state is.

Spooky action at a distance

Einstein decried the idea of entanglement - that one particle could instantaneously affect another's spin, say, through a weird quantum link. This phenomenon, also known as non-locality, has since been demonstrated and is a key principle behind quantum computers and communications. Bohmian mechanics accepts non-locality as one of its basic tenets, postulating that particles can be linked by "hidden variables" that control properties like spin.

Objective reality

Does the moon exist if nobody is looking at it? Conventional quantum theory says there is no reality beyond what we observe, so in principle things don't exist unless they are being measured. Bohmian mechanics, on the other hand, supports the idea of an objective reality, including hidden variables that are beyond the scope of our measurements.

Uncertainty principle

If you measure the position of a quantum particle, you can't know its momentum precisely, and vice versa. The conventional explanation is that there is randomness inherent in the quantum universe. According to the Bohm theory, however, this uncertainty arises from interactions between the measuring device and the particle: it is not inherent in the universe.

Gregory T. Huang

Mark Buchanan's latest book is The Social Atom, published by Bloomsbury in the US and Cyan-Marshall Cavendish in the UK

[This Article] NewScientist Magazine (Mark Buchanan), 3/22/2008, Quantum randomness may not be random, NewScientist Magazine, 3/22/2008 issue 2648, (http://www.newscientist.com/article/mg19726485.700-quantum-randomness-may-not-be-random.html?full=true)