The Unification of Physics

(This essay was originally part of Chapter 4 of my new book, The Systems View of Life, coauthored with Pier Luigi Luisi. It had to be cut due to space restrictions, and I am posting it here because I am often asked about the latest developments in particle physics.)

The two basic theories of twentieth-century physics, quantum theory and relativity theory, transcended the principal aspects of the Cartesian worldview and of Newtonian physics. Quantum theory showed that subatomic particles are not isolated grains of matter but are probability patterns, interconnections in an inseparable cosmic web that includes the human observer and his or her consciousness. Relativity theory revealed the intrinsically dynamic character of this cosmic web by showing that its activity is the very essence of its being.

Current research in physics aims at unifying quantum theory and relativity theory into a complete theory of subatomic matter. Such a theory would need to give a full account of the four fundamental forces that operate at the subatomic level: electromagnetism (which binds electrons to the nucleus and controls all chemical processes), gravity, the strong nuclear force (which holds atomic nuclei together), and the weak nuclear force (which is responsible for radioactive decay). Physicists have not yet been able to formulate such a complete theory, but we now have several partial theories that describe some of the four fundamental forces and the phenomena associated with them very well.

The first successful “quantum-relativistic” theory was developed in the 1940s and is known as quantum electrodynamics, or QED. It involved the integration of the principles of quantum mechanics with Einstein’s relativistic theory of electromagnetism (the special theory of relativity). This led to the novel concept of the quantum field, a fundamental entity that can exist in continuous form, as a field, and in discontinuous form, as particles, different kinds of particles being associated with different fields. For example, the photon is the particle version of the electromagnetic field. Because of the central role of quantum fields, QED is known as a quantum field theory. It was developed independently by Sin-Itiro Tomonaga in Japan and by Richard Feynman and Julian Schwinger in the United States.

After the completion of QED, physicists concentrated their efforts on extending the formalism of quantum field theory to the strong and weak nuclear forces, which would take another quarter century. QED owes its success to the fact that the electromagnetic interactions are relatively weak and thus make it possible, to a large extent, to maintain the classical distinction between particles and the forces acting between them. It was soon noticed that this is also true for the weak interactions, but the formulation of a corresponding field theory was anything but easy.

The crucial breakthrough occurred in the 1960s with the discovery that a certain mathematical symmetry, known as “gauge symmetry,” is shared by the electromagnetic and weak interactions. Mathematical symmetries are used very widely in modern physics as fundamental principles that provide structure and coherence to the laws of nature. To a mathematician the symmetry of an object is a transformation that leaves the object looking exactly the same. For example, we can rotate a square about its center through one or more right angles, and we will always end up with an identical square. In the case of gauge symmetries, the transformation acts on the quantum fields without affecting the value of any measurable physical quantity.

The discovery of a common gauge symmetry of electromagnetic and weak interactions led to the development of a new type of quantum field theories, called gauge theories, which made it possible to unify the two interactions. In the resulting unified field theory — known as the Weinberg-Salam theory after its two main architects, Steven Weinberg and Abdus Salam — the two interactions remain distinct but become mathematically intertwined and are referred to collectively as “electroweak” interactions.

The extension of the gauge-theory approach to the strong interactions remained highly problematic for many years, since the forces between the so-called hadrons (protons, neutrons, and other strongly interacting particles) are so strong that the distinction between particles and forces becomes blurred. However, during the 1960s an unexpected solution to the problem emerged with the discovery that hadrons were not elementary after all. Physicists Murray Gell-Mann and Georg Zweig independently discovered that hadrons are made of smaller elementary units, called quarks. The proton and neutron each contain three quarks, while other hadrons, called mesons, are composed of two (a quark and an anti-quark, to be precise). Quarks, however, are not particles in the conventional sense. None of them has ever been observed in isolation, and yet the strong interactions between hadrons exhibit striking regularities that can be explained by the quark model. Over the years, physicists have come to accept this strange fact and to think of quarks as being permanently confined within hadrons, bound together by so-called gluons, the carriers of the strong nuclear force.

The discovery of the quark structure was an essential step toward extending the gauge-theory approach to hadrons, because now the strong interactions could be described in terms of the interactions between quarks and gluons, which are much simpler than those among hadrons. The result was a field theory called quantum chromodynamics (QCD), in which the fields are associated with quarks and gluons and “chromo” refers to three types of gluons, arbitrarily labeled “red,” “green,” and “blue.”

The development of QCD in the 1970s completed the representation of three of the four fundamental forces within the single theoretical framework of gauge theory. Like the Weinberg-Salam quantum field theory, QCD is modeled after quantum electrodynamics. In all three theories the forces are transmitted by so-called gauge fields: the electromagnetic field (carried by the photon) in QED, two gauge fields (carried by particles labeled W and Z) in the Weinberg-Salam theory, and three gauge fields (carried by “colored” gluons) in QCD. Both the Weinberg-Salam theory of electroweak interactions and QCD, the theory of strong interactions, have been confirmed by rigorous experimental tests over more than three decades. Together these theories are known today as the standard model of particle physics. The most spectacular confirmation came with the discovery of the W and Z gauge particles, exactly as predicted by the standard model, in the early 1980s by a team of experimenters at CERN, the European center of particle physics, under the leadership of Carlo Rubbia.

In the initial formulations of the standard model, the two gauge particles W and Z needed to be massless, because an exact gauge symmetry requires that the gauge fields correspond to massless particles. This is correct for the photon, the original gauge field, but not for the W and Z, which are very massive particles. In fact, in the original formulation of the standard model all particles were massless. This was an obvious flaw, and physicists immediately set out to search for modifications of the model that would contain the massive particles observed in nature.

The challenge was to modify the standard model in such a way that the gauge symmetry would be maintained in the model’s basic equations, but would be broken in their solutions (the various fields and their associated particles) in such a way that the particles could acquire their observed masses. In 1964, three teams of physicists independently proposed such a mechanism of “spontaneous symmetry breaking,” which became known as the Higgs mechanism after one of the authors, the British physicist Peter Higgs, who presented the most complete formulation of the underlying mathematics.

The Higgs mechanism required an additional field, known as the Higgs field, which permeates space and interacts with all particles in such a way that their gauge symmetries are broken spontaneously and they acquire their masses. This explained not only how particles acquire mass; it also predicted the mass ratio between the W and Z masses, as well as their couplings with all particles in the standard model. These predictions were subsequently confirmed by precise measurements in large particle accelerators, which dramatically increased the physicists’ confidence in the Higgs mechanism.

However, the particle corresponding to the Higgs field, the so-called Higgs boson, eluded detection for half a century, until the Large Hadron Collider, a gigantic particle accelerator at the European research center CERN with a circumference of 27 km, made it possible to analyze trillions of particle collisions at extremely high energies. In March 2012, after five decades of searching, scientists reported that the hunt for the elusive Higgs boson might finally be rewarded; and in July 2012 they announced that a “Higgs-like” particle had been discovered. If this particle indeed turns out to have all the required properties of the Higgs boson, this would represent a triumphant completion of the standard model.

The representation of all three forces as expressions of a single unifying principle — the gauge principle — must be ranked as one of the greatest achievements in particle physics. However, a grand unified theory (GUT), in which all subatomic particles and the forces between them are seen as different manifestations of just one kind of particle and one gauge field, still remains an elusive dream. Many physicists believe that this dream will not be realized until gravity, the fourth fundamental force, is integrated with the other three forces in a grand unification.

Integrating the force of gravity with the other three fundamental forces requires the unification of quantum theory with Einstein’s general theory of relativity. In spite of more than half a century of strenuous efforts, such a unification — known as quantum gravity — has not been achieved so far. The most popular and most impressive approach, which is still evolving, has been that of string theory, in which all subatomic particles are represented as different states of vibrating mathematical “strings” in an abstract 9-dimensional space.

By picturing all particles, including the gauge particles, as different vibrations of the same fundamental object, string theory presents a unified picture of particles and forces, and by naturally including gravitons as vibrating closed strings it unifies quantum theory and general relativity. The mathematical elegance of the theory, as it has been developed so far, is utterly compelling. Indeed, one of the best non-technical introductions to string theory, the book by physicist Brian Greene, is titled The Elegant Universe (Norton, New York, 1999).

In spite of its conceptual elegance, however, string theory has serious deficiencies. To begin with, there are several versions with different numbers of spatial dimensions, and the process of reducing these dimensions to the 4 dimensions of our actual space-time is not unambiguous. Even more serious is the problem that the theory has not been verified experimentally. There is a large number of different string theories, none of them capable of uniquely explaining the values of the basic parameters of the standard model.

The great potential of string theory, as well as its serious problems, are analyzed in fascinating detail by physicist Lee Smolin in his provocatively titled book The Trouble With Physics (Houghton Mifflin, New York, 2006). Smolin argues that the principal weakness of string theory as a theory of quantum gravity lies in the fact that it is formulated in terms of vibrating strings moving against a fixed background of geometries of space that do not evolve in time. This is inconsistent with general relativity, which shows that the geometry of space and time is not fixed but changes as matter moves about. Hence, any theory consistent with general relativity needs to be formulated in such a way that the structure of space-time emerges from it rather than being assumed as the arena in which the physical phenomena take place. According to Smolin, the fact that string theory is not formulated in such a “background-independent” way is its most serious shortcoming.

Moreover, Smolin argues that a major obstacle standing in the way of a unifying quantum theory with general relativity may be that both are formulated in terms of continuous space-time, but that still unresolved problems in the interpretation of quantum theory (associated with the processes of observation and measurement) point to the possibility that there may be a deeper level of reality which exhibits some kind of fundamental quantum structure out of which continuous space-time emerges as a result of a unified theory of quantum gravity. In Smolin’s view, the complete unification of physics will not be possible until the foundational problems of quantum theory are resolved.

I would like to end this essay on a personal note. In the last two chapters of my book The Tao of Physics, I discussed a theory known as “bootstrap theory,” which was very popular in the 1970s, and on which I worked myself during my ten years at the Lawrence Berkeley Laboratory. This theory, proposed by Geoffrey Chew, is based on the idea that nature cannot be reduced to fundamental entities, like fundamental constituents of matter, but has to be understood entirely through self-consistency. All of physics has to follow uniquely from the requirement that its components be consistent with one another and with themselves.

This idea constitutes a radical departure from the traditional spirit of basic research in physics, which has always concentrated on finding the fundamental constituents of matter. At the same time, it can be seen as the culmination of the conception of particles as interconnections in an inseparable cosmic web, which arose in quantum theory and acquired an intrinsically dynamic nature in relativity theory.

The bootstrap philosophy abandons not only the idea of fundamental constituents of matter but accepts no fundamental entities whatsoever — no fundamental laws or equations, and not even a fundamental structure of space-time. The universe is seen as a dynamic web of interrelated events. None of the properties of any part of this web are fundamental; they all follow from the properties of the other parts, and the overall consistency of their mutual interrelations determines the structure of the entire web.

During the 1980s and 1990s, the bootstrap theory was eclipsed by the success of the standard model, which is very different, as it postulates the existence of fundamental fields and their corresponding particles. And today, bootstrap physics has virtually disappeared from the scene. However, if a theory of quantum gravity continues to remain elusive, and if the a priori assumption of the structure of space-time is broadly recognized as the essential flaw of string theory, the bootstrap idea may well will be revived someday, in some mathematical formulation or other.