In 1905 Albert Einstein published four papers which transformed the
understanding of space, time, mass, and energy; provided physical evidence for
the quantisation of energy; and observational confirmation of the
existence of atoms. These publications are collectively called the
Annus Mirabilis papers,
and vaulted the largely unknown Einstein to the top rank of theoretical
physicists. When Einstein was awarded the Nobel Prize in Physics in
1921, it was for one of these 1905 papers which explained the
photoelectric effect. Einstein's 1905 papers are masterpieces of
intuitive reasoning and clear exposition, and demonstrated
Einstein's technique of constructing thought experiments based
upon physical observations, then deriving testable mathematical
models from them. Unlike so many present-day scientific publications,
Einstein's papers on
special relativity
and the
equivalence of mass and energy
were accessible to anybody with a college-level understanding
of mechanics and electrodynamics and used no special jargon or
advanced mathematics. Being based on well-understood concepts,
neither cited any other scientific paper.
While special relativity revolutionised our understanding of space
and time, and has withstood every experimental test to which it
has been subjected in the more than a century since it was
formulated, it was known from inception that the theory was
incomplete. It's called special relativity because
it only describes the behaviour of bodies under the special
case of uniform unaccelerated motion in the absence of
gravity. To handle acceleration and gravitation would require
extending the special theory into a general theory of
relativity, and it is upon this quest that Einstein next
embarked.
As before, Einstein began with a simple thought experiment. Just as in
special relativity, where there is no experiment which can be done in
a laboratory without the ability to observe the outside world that
can determine its speed or direction of uniform (unaccelerated) motion,
Einstein argued that there should be no experiment an observer could
perform in a sufficiently small closed laboratory which could distinguish
uniform acceleration from the effect of gravity. If one observed objects to
fall with an acceleration equal to that on the surface of the Earth,
the laboratory might be stationary on the Earth or in a space ship
accelerating with a constant acceleration of one gravity, and
no experiment could distinguish the two situations. (The reason for
the “sufficiently small” qualification is that since
gravity is produced by massive objects, the direction a test particle
will fall depends upon its position with respect to the centre of
gravity of the body. In a very large laboratory, objects dropped
far apart would fall in different directions. This is what causes
tides.)
Einstein called this observation the
“equivalence principle”:
that the effects of acceleration and gravity are indistinguishable,
and that hence a theory which extended special relativity to
incorporate accelerated motion would necessarily also be a
theory of gravity. Einstein had originally hoped it would be
straightforward to reconcile special relativity with acceleration
and gravity, but the deeper he got into the problem, the more he
appreciated how difficult a task he had undertaken. Thanks to the
Einstein Papers
Project, which is curating and publishing all of Einstein's extant
work, including notebooks, letters, and other documents, the author
(a participant in the project) has been able to reconstruct Einstein's
ten-year search for a viable theory of general relativity.
Einstein pursued a two-track approach. The bottom up path started with
Newtonian gravity and attempted to generalise it to make it compatible
with special relativity. In this attempt, Einstein was guided by the
correspondence
principle, which requires that any new theory which explains
behaviour under previously untested conditions must reproduce the
tested results of existing theory under known conditions. For example,
the equations of motion in special relativity reduce to those of
Newtonian mechanics when velocities are small compared to the speed of
light. Similarly, for gravity, any candidate theory must yield results
identical to Newtonian gravitation when field strength is weak and
velocities are low.
From the top down, Einstein concluded that any theory compatible with
the principle of equivalence between acceleration and gravity must
exhibit
general covariance,
which can be thought of as being equally valid regardless of the choice
of co-ordinates (as long as they are varied without discontinuities).
There are very few mathematical structures which have this property,
and Einstein was drawn to
Riemann's
tensor geometry. Over years of
work, Einstein pursued both paths, producing a bottom-up theory which
was not generally covariant which he eventually rejected as in conflict
with experiment. By November 1915 he had returned to the top-down
mathematical approach and in four papers expounded a generally covariant
theory which agreed with experiment. General relativity had arrived.
Einstein's 1915 theory correctly predicted the
anomalous
perihelion precession of Mercury and also predicted that
starlight passing near the limb of the Sun would be
deflected
by twice the angle expected based on Newtonian gravitation. This
was confirmed (within a rather large margin of error) in an
eclipse expedition in 1919, which made Einstein's general relativity
front page news around the world. Since then precision
tests of general
relativity have tested a variety of predictions of the theory
with ever-increasing precision, with no experiment to date yielding
results inconsistent with the theory.
Thus, by 1915, Einstein had produced theories of mechanics, electrodynamics,
the equivalence of mass and energy, and the mechanics of bodies under
acceleration and the influence of gravitational fields, and changed
space and time from a fixed background in which physics occurs to
a dynamical arena: “Matter and energy tell spacetime how to
curve. Spacetime tells matter how to move.” What do you do,
at age 36, having figured out, largely on your own, how a large part
of the universe works?
Much of Einstein's work so far had consisted of unification. Special
relativity unified space and time, matter and energy. General
relativity unified acceleration and gravitation, gravitation
and geometry. But much remained to be unified. In
general relativity and classical electrodynamics there were
two field theories, both defined on the continuum, both with
unlimited range and an inverse square law, both exhibiting static
and dynamic effects (although the details of
gravitomagnetism
would not be worked out until later). And yet the theories seemed
entirely distinct: gravity was always attractive and worked by
the bending of spacetime by matter-energy, while electromagnetism
could be either attractive or repulsive, and seemed to be propagated
by fields emitted by point charges—how messy.
Further, quantum theory, which Einstein's 1905 paper on the
photoelectric effect had helped launch, seemed to point in a very
different direction than the classical field theories in which
Einstein had worked. Quantum mechanics, especially as elaborated
in the “new” quantum theory of the 1920s, seemed to
indicate that aspects of the universe such as electric charge
were discrete, not continuous, and that physics could, even in
principle, only predict the probability of the outcome of experiments,
not calculate them definitively from known initial conditions.
Einstein never disputed the successes of quantum theory in
explaining experimental results, but suspected it was a theory
based upon phenomena which did not explain what was going on at
a deeper level. (For example, the physical theory of elasticity
explains experimental results and makes predictions within its
domain of applicability, but it is not fundamental. All
of the effects of elasticity are ultimately due to electromagnetic
forces between atoms in materials. But that doesn't mean that the
theory of elasticity isn't useful to engineers, or that they should
do their spring calculations at the molecular level.)
Einstein undertook the search for a
unified field theory,
which would unify gravity and electromagnetism, just as Maxwell had
unified electrostatics and magnetism into a single theory. In
addition, Einstein believed that a unified field theory would be
antecedent to quantum theory, and that the probabilistic results of
quantum theory could be deduced from the more fundamental theory, which
would remain entirely deterministic. From 1915 until his death in 1955
Einstein's work concentrated mostly on the quest for a unified field
theory. He was aided by numerous talented assistants, many of whom
went on to do important work in their own right. He explored
a variety of paths to such a theory, but ultimately rejected each
one, in turn, as either inconsistent with experiment or unable
to explain phenomena such as point particles or quantisation of
charge.
As the author documents, Einstein's approach to doing physics changed in
the years after 1915. While before he was guided both by physics and
mathematics, in retrospect he recalled and described his search of
the field equations of general relativity as having followed the path
of discovering the simplest and most elegant mathematical structure which
could explain the observed phenomena. He thus came, like Dirac, to argue
that mathematical beauty was the best guide to correct physical theories.
In the last forty years of his life, Einstein made no progress whatsoever
toward a unified field theory, apart from discarding numerous paths
which did not work. He explored a variety of approaches:
“semivectors” (which turned out just to be a reformulation
of spinors),
five-dimensional models including a cylindrically
compactified dimension based on
Kaluza-Klein theory,
and attempts to deduce the properties of particles and their
quantum behaviour from nonlinear continuum field theories.
In seeking to unify electromagnetism and gravity,
he ignored the strong and weak nuclear forces which had been discovered
over the years and merited being included in any grand scheme of
unification. In the years after World War II, many physicists ceased
to worry about the meaning of quantum mechanics and the seemingly
inherent randomness in its predictions which so distressed Einstein, and
adopted a “shut up and calculate” approach as their
computations were confirmed to ever greater precision by experiments.
So great was the respect for Einstein's achievements that only rarely
was a disparaging word said about his work on unified field theories,
but toward the end of his life it was outside the mainstream of
theoretical physics, which had moved on to elaboration of quantum
theory and making quantum theory compatible with special relativity.
It would be a decade after Einstein's death before astronomical
discoveries would make general relativity once again a frontier in
physics.
What can we learn from the latter half of Einstein's life and his
pursuit of unification? The frontier of physics today remains
unification among the forces and particles we have discovered. Now we
have three forces to unify (counting electromagnetism and the weak
nuclear force as already unified in the electroweak force), plus two
seemingly incompatible kinds of particles: bosons (carriers of force)
and fermions (what stuff is made of). Six decades (to the day) after
the death of Einstein, unification of gravity and the other forces
remains as elusive as when he first attempted it.
It is a noble task to try to unify disparate facts and theories into a
common whole. Much of our progress in the age of science has come from
such unification. Einstein unified space and time; matter and energy;
acceleration and gravity; geometry and motion. We all benefit every
day from technologies dependent upon these fundamental discoveries.
He spent the last forty years of his life seeking the next grand
unification. He never found it. For this effort we should applaud him.
I must remark upon how absurd the price of this book is. At Amazon as of this writing,
the hardcover is US$ 102.91 and the
Kindle edition is US$ 88. Eighty-eight Yankee dollars
for a 224 page book which is ranked #739,058 in the Kindle store?