The Man Who Changed Everything

James had always been a prolific letter writer. Now that Glenlair was his professional as well as his private address, consignments of journals, manuscripts and proofs started to come in, adding to a growing daily bundle of personal and business letters. To ease the postman’s burden, James had a post box set into the rough stone wall by the side of the road, about half a mile from the house. It was a good system. Every day he walked down the drive to take outgoing letters and parcels to the postbox and pick up the incoming mail, at the same time giving the dogs a run.

Among the correspondents were his old school friend P. G. Tait, who had beaten him to the professorship of natural philosophy at Edinburgh University, and William Thomson, who had long held the corresponding post at Glasgow University and was now getting rich from patents and consultancy work on the Atlantic telegraph. The three great Scottish physicists had for years written to each other sharing ideas, comments and gossip. Now this burgeoned into an exuberant, quick-fire three-way exchange. They often used postcards, for speed and convenience, and developed a jokey kind of code language so as to get as much as possible on a card. James and Tait had done this sort of thing as schoolboys and drew just as much delight from it now as then. Thomson probably looked on it all with benign tolerance, but he played along.

Names were the first things to be abbreviated: Thomson was T, Tait was T′. James became dp/dt, from an equation in Tait’s book on thermodynamics, dp/dt = JCM. Hermann Helmholtz, whom they admired, was H². John Tyndall, who was not held in such high regard, was T′′. Tait, whose benevolence to all men was rather less than James’, said privately that this was because Tyndall was a second order quantity. Alexander Macmillan, the publisher, was #, because he was a sharp character. Greek letters were useful in quasi-phonetic abbreviations: Σφαρξ stood for spherical harmonics and θΔics for thermodynamics.

Their feeling of common regard was so secure that it could withstand the most robust ribbing. Here, in a report to the Royal Society of Edinburgh, James pokes fun at Tait’s increasing fondness for brevity in his mathematical writing, which made some of the steps hard to follow:

I beg leave to report that I consider the first two pages of Professor Tait’s Paper on Orthogonal Isothermal Surfaces as deserving and requiring to be printed in the Transactions of the R.S.E. as a rare and valuable example of the manner of that Master in his middle or Transition period, previous to that remarkable condensation of his style, which has rendered it impenetrable to all but the piercing intellect of the author in his best moments.

Thomson and Tait were collaborating on their Treatise on Natural Philosophy, which attempted to map out the state of all aspects of physics, a huge undertaking. They asked James to check drafts of some of the chapters and got exactly the sort of constructive criticism a good author welcomes: as we have seen, he picked them up on the definition of mass. James was himself starting to compile his Treatise on Electricity and Magnetism, encompassing with rigour everything that was known on the topic. This was a monumental task, which was to take him 7 years.

The 6 years James spent at Glenlair were not in any sense a time of retirement. On the contrary, this was, by any standards, an outstandingly prolific period. Apart from preparing the Treatise, he published a book, The Theory of Heat, and 16 papers on an amazingly diverse range of topics, all with something profoundly original to say. We shall come to these later in the chapter.

He and Katherine did, however, find time for a touring holiday in Italy in the spring and early summer of 1867 while the house was being extended. It was usual in those days for anyone with enough money and leisure to make ‘the grand tour’ of the cultural and historical high spots. The Maxwells’ life style was far removed from that of the fashionable set and the holiday was one of their few extravagances. The first adventure was not one they had sought; their ship was put under quarantine at Marseilles. James’ fortitude came to the fore. Harking back to his childhood at Glenlair, he became the general water-carrier, and in other ways did everything possible to ease the discomfort of fellow passengers.

In Florence they happened by chance to bump into Lewis Campbell and his wife. Campbell later recalled how his friend’s enthusiasm for Italian architecture and music brought to mind reports he had read of ‘the joy of Michelangelo in etherealising the work of Brunelleschi’. Not that James was ever reverential; in his account the Vatican orchestra became ‘the Pope’s band’. He and Katherine took lessons in Italian and he quickly became fluent enough to discuss scientific matters with an Italian colleague in Pisa. He also took every chance of improving his French and German by talking to fellow tourists, but found himself at a loss with Dutch.

There were other breaks from home. Each spring James and Katherine stayed in London for several weeks and James travelled to British Association meetings in various parts of the country, sometimes acting as president of the Mathematics and Physics Section. He also made annual visits to Cambridge. The University had asked him to act as moderator, then examiner, for the Mathematical Tripos. This was an inspired move, possibly prompted by William Thomson, who had been appointed a public examiner. Cambridge had pulled itself out of a mathematical rut in the early 1800s, thanks to the efforts of Charles Babbage and some of his colleagues, but was now settling into a broader scientific one. The Mathematical Tripos exam was partly to blame. Its questions were still like those in James’ student days. A contemporary described them as ‘mathematical trifles and problems, so called, barren alike of practical results and scientific interest’. James set about making the examinations more interesting and more relevant to everyday experience, as he had done at King’s College. It was the start of a magnificent revival of Cambridge’s scientific tradition, in which James was yet to play the main part.

Ever since his brilliant but flawed paper of 1860 on gas theory, James had been mulling over new ideas on the topic. In 1866 he brought them to fruition in a paper, On the Dynamical Theory of Gases. His earlier paper had given the world its first statistical law of physics—the Maxwell distribution of molecular velocities —and had predicted that the viscosity of a gas was independent of its pressure, a remarkable result which James and Katherine had verified by experiment in their Kensington attic. In other parts of the paper he had made errors in calculation, which were embarrassing but easily corrected.

His results had greatly advanced and strengthened the theory that gases consist of myriads of jostling molecules, but two serious problems remained. One was to do with the ratio of the specific heat of air at constant pressure to that at constant volume; the theory predicted a different value from that observed in practice. As we shall see, the answer to this was to prove beyond the reach of nineteenth century scientists. The other problem was more amenable: James’ and Katherine’s own experiments had shown that the viscosity of a gas did not vary with the square root of its absolute temperature, although the theory predicted that it should. The fault seemed to lie in James’ original assumption that when molecules collided they behaved like billiard balls, in other words that they were perfectly elastic spheres. He now tried the alternative assumption that they did not actually come into contact at all but repelled one another with a force that varied inversely with the nth power of the separation distance: if n was, say, 4 or larger, the repulsive force would be great when two molecules came close together but negligible when they were far apart. Some fiendishly complicated mathematics followed because the molecules no longer travelled in straight lines but followed complex curves.

He found two ways to simplify the calculations. One was to introduce the notion of relaxation time, the time a system takes to return to a state of equilibrium after being disturbed. This is a concept now routinely used throughout physics and engineering. One can, for example, easily picture its application to car suspension systems. Like so many of Maxwell’s innovations, it has become so familiar that one wonders why nobody had thought of it before.

The other simplification arose, amazingly, when repulsion between molecules varied inversely as the 5th power of separation distance. When he put n = 5 in his equations, all the terms concerning the relative velocities of molecules cancelled out, leaving much simpler relationships. And there was a bonus: viscosity now became directly proportional to absolute temperature, in line with his own experimental results in the Kensington attic. This particular triumph was short-lived, as more accurate experiments by others showed that the relationship was not linear after all. But a later generation of experimenters found that some kinds of molecules do indeed seem to follow an inverse 5th power repulsion law. For those that do not, physicists still find Maxwell’s formulae useful as a starting point for more exact calculations.

Even with the simplifications, the mathematical obstacles were as formidable as those James had faced when tackling Saturn’s rings. He overcame them with such mastery that some scholars consider this the most inspiring of all his works. The young Ludwig Boltzmann, already working on his own first great paper, was entranced. The mastery was not easily achieved. At one stage James almost threw in the towel when some of his equations predicted perpetual motion currents in the earth’s atmosphere. He found his mistake, then had to search for and correct a further mistake in the revised calculations. Doggedness won the day. James was able to work out formulae not only for viscosity but for diffusion, heat conduction and other properties, which agreed with known experimental results. It was a seminal paper. He had not only corrected and extended his earlier work but had greatly strengthened the theory that gases (and, by extension, all forms of matter) were composed of molecules. Most of all, he had set the theory on a firm base, on which he, Boltzmann and others could build.

The work of the British Association’s committee on electrical standards had not stopped with the production of a standard of resistance. The next task on their agenda stemmed from James’ prediction of electromagnetic waves which travelled at a speed equal to the ratio of the electromagnetic and electrostatic units of charge. As we have seen, an earlier measurement of this ratio by Kohlrausch and Weber, once converted to the appropriate units, was very close to Fizeau’s measurement of the speed of light, thus supporting James’ theory that light itself was composed of electromagnetic waves. This result was so important that the evidence needed to be checked; a new experiment was urgently needed to corroborate Kohlrausch and Weber’s result. It would be a difficult experiment and at best the range of possible error would be a few percent, but it had to be done.

This time James’ chief collaborator was Charles Hockin, of St John’s College, Cambridge. They decided to try to balance the electrostatic attraction between two charged metal plates against the magnetic repulsion between two current-carrying coils, and built a balance arm apparatus to do this. For this method to work they needed a very high voltage source. The biggest batteries in Britain were owned by a Clapham wine merchant, John Peter Gassiot, who had acquired them for his private laboratory. Gassiot was delighted to act as host for the experiment and furnished his guests with a battery of 2600 cells, giving about 3000 volts.

James arranged to do the experiment during his 1868 spring visit to London. It was not easy work. First they had to take precautions to stop electricity leaking from the great battery through the laboratory floor. Then they had to become expert at taking readings at speed because the batteries ran down so quickly. When these problems had been overcome, the experiment gave a value for the ratio of the two units of charge, and hence for the speed of James’ waves, of 288,000 kilometres per second.

This was about 7% below the value which Kohlrausch and Weber had obtained for the electromagnetic/electrostatic units ratio and 8% below the speed of light as measured by Fizeau (the two results James had quoted in his paper). And it was 3% below a new measurement of the speed of light by Fizeau’s compatriot Foucault. James must have felt a tinge of disappointment that the correspondence was not closer but in logical terms the experiment was a success. His theory that light consisted of electromagnetic waves now stood on stronger ground because two independent experimental results gave predicted wave speeds which, given reasonable allowance for experimental error, corresponded to the measured speed of the light. We now know that the true speed of light is about mid-way between that predicted by James’ experiment and that predicted by Weber’s².

The nineteenth century was the age of the steam engine and great strides had been made in the understanding of heat, through the work of Carnot, Clausius, Joule, Thomson, Rankine and others. James wrote what was originally intended as nothing more than an elementary introduction to the subject: The Theory of Heat. It was indeed a good introduction to the established theory but also included a completely new formulation of the relationships between the main quantities: pressure, volume, temperature and entropy^j. By a geometrical argument, he expressed these relationships as differential equations in a form which turned out to be extremely useful. They are now part of the standard repertoire and are known as Maxwell relations.

The Theory of Heat also introduced readers to James’ most extraordinary invention: an imaginary, molecule-sized being who could make heat flow from a cold substance to a hot one, thereby defying the second law of thermodynamics. This was Maxwell’s demon³. The little creature soon took on legendary status and lived up to its name, perplexing the world’s best physicists for 60 years. William Thomson gets the credit for the name.

Playful though it was, James’ idea was also a profound ‘thought experiment’ of the kind that Einstein later made his own. The demon guards a small hole in the wall separating two compartments of a container filled with gas. He has a shutter over the hole which he can open when he wants to. Molecules in both compartments are moving in all directions. Their average speed (strictly, the average of the square of their speeds) determines the temperature of the gas (the faster the hotter) and, to start with, this is the same on both sides of the wall.

According to James’ own law for the distribution of velocities, some molecules are moving slower than the average speed and some faster. When the demon sees a fast molecule in the right compartment approaching the hole he opens the shutter briefly and lets it through to the left side. Similarly, he lets slow molecules pass from the left compartment to the right. The rest of the time he keeps the shutter closed.

With each exchange the average speed of molecules in the left compartment increases and that in the right compartment falls. But there will still be some molecules in the right compartment travelling faster than the average speed in the left and when one of these approaches the shutter the demon lets it through. In the same way, he continues to let slow molecules through from left to right. So the gas in the left compartment gets steadily hotter while that in the right compartment gets colder.

The demon is making heat flow from a colder gas on the right to the hotter gas on the left, thus defying the second law of thermodynamics, which says that heat cannot flow from a colder to a hotter body. By the same token, the demon is providing the means of making a perpetual motion machine: the temperature difference between the gases could be harnessed to make a machine do physical work; the machine would keep going until the temperature difference fell back to zero; we would then be back where we started and could repeat the process until all the heat energy in the gas had been converted into work⁴.

Of course, this cannot really happen. The interesting question is why not? James’ gave two explanations. The first was that the second law of thermodynamics is, at root, a statistical law. As he put it, the law is equivalent to the statement that if you throw a tumblerful of water into the sea, you cannot get the same tumblerful out again; it applies to molecules en masse, not to individuals. This was indeed correct, but his other explanation, although apparently light-hearted, turned out to be even more penetrating. He said that if we were sufficiently nimble fingered, like the demon, we could break the second law ‘only we can’t, not being clever enough’.

Why aren’t we as clever as the demon? To match him we would have to know the positions and velocities of all the molecules. Leo Szilard, in 1929, showed that the very act of acquiring information about a system increases its entropy in proportion to the amount of information gathered. As the entropy increases, less of the system’s total heat energy is available for doing work. To gather enough information to work the shutter effectively we would have to use up, or render inaccessible, an amount of energy at least equal to the work output of any machine that we could drive from the system. So we will never be clever enough to create perpetual motion.

In the experiment at King’s to establish a standard for electrical resistance, James had used a governor to keep the coil spinning at a constant rate. This had turned his thoughts to the way governors worked. In a steam engine governor, weights on a driven shaft are linked to a valve controlling the steam input. The further the weights fly out under centrifugal force, the smaller the valve opening becomes. If the engine starts to speed up the steam input is reduced, causing it to slow again; so the engine settles down to a controlled steady speed. James saw that the same principle could be applied to give precise and stable control of any kind of machine.

The key idea was negative feedback. To control the machine’s output to a desired value (which may vary over time), you continuously compare the actual output against the required output, and feed the difference back to the input in such a way as to make the output converge on the wanted value. James worked out the conditions for stability under various feedback arrangements, and examined the effects of damping and of changes in the driven load. He wrote up the results in a paper called On Governors. It was the first mathematical analysis of control systems and became the foundation of modern control theory.

Amazingly, the work attracted little attention until the 1940s, when control systems were urgently needed for military equipment during the second world war. Engineers were then pleased to find that Maxwell had already worked out the basis of the theory they needed. After the war Norbert Wiener took things further and developed the science of cybernetics.

James had a knack of finding ways in which the natural world followed mathematical principles. A delightful example was his paper On Hills and Dales. The earth’s surface has high areas or hills, each with a summit, and low areas, each with a bottom point, which James called an ‘immit’. There are also ridges, valleys or dales, and passes. He saw that the numbers of each of these features must be somehow related by mathematical rules and set about working them out. One of the simpler rules he found is that the number of summits is always one more than the number of passes. The branch of mathematics dealing with the spatial relationships of things, irrespective of their sizes or shapes, was then called the geometry of position. It was in its infancy and James was breaking new ground, helping to pave the way for what has become the deep and complex subject of topology. His results in On Hills and Dales are also relevant to meteorology; the formulae apply equally well to an air pressure system with its highs, lows, troughs and ridges. Moreover, James’ original ideas about the earth’s surface have themselves now developed into a serious branch of topology called global analysis.

James expanded the ideas which began in the paper he had written at King’s College on how to calculate forces in frameworks by using reciprocal diagrams, eventually extending the method to continuous media. The Royal Society of Edinburgh acclaimed this work by awarding him its Keith Medal. He also developed the underlying principle of duality, on which reciprocal diagrams depend, and went on to show how it could be applied to such diverse topics as electrical circuits and optics.

James wrote most of his Treatise on Electricity and Magnetism during the period at Glenlair but, as it was not published until 1873, we will leave its description until our next chapter. On the way to the Treatise he published a fairly short paper, Note on the Electromagnetic Theory of Light. Here he gave a more compact derivation and formulation of the main equations from his Dynamical Theory and produced arguments to show that rival theories from Wilhelm Weber and Bernhard Riemann, which both postulated action at a distance rather than through an energy-carrying field, could not be true because they defied the laws of conservation of energy. As always, he couldn’t resist poking a little fun:

From the assumptions of both these papers we may draw the conclusions, first, that action and reaction are not equal and opposite, and second, that apparatus may be constructed to generate any amount of work from its resources.

Being essentially three-dimensional, the electromagnetic field posed a problem in mathematical notation. In his papers, James had so far written each relation involving vectors as a triple set of equations: one part for each of the x, y and z directions. This was cumbersome and made it hard to see the wood for the trees. But at least it was a notation people understood: it was difficult enough for them to cope with a new kind of theory; to have expected them to learn a new notation at the same time would have been unreasonable.

Nowadays we use the compact notation of vector analysis, in which each triple (x, y, z) set of equations is replaced by a single vector equation, where a single symbol embodies the x, y and z components of a vector quantity like force or velocity. Its precursor was the system of ‘quaternions’ invented by the great Irish mathematician Sir William Rowan Hamilton (not to be confused with Sir William Hamilton who had taught James philosophy at Edinburgh University). Quaternions are more complicated than vectors because they each have four components: a scalar part, which is just a number, and a vector part with components in each of the x, y and z directions. Our modern vector system dispenses with the scalar part.

Quaternions were not easy to get to grips with and only a few enthusiasts wanted anything to do with them. The foremost of these was James’ friend P. G. Tait. But they had the advantage of great compactness. In some of James’ equations nine ordinary symbols could by replaced by two quaternion terms. James found that they also made the physical meaning of the equations clearer. He decided to include the shorthand quaternion representation in his Treatise, alongside the conventional longhand notation: ploughing with an ox and an ass together, as he put it.

To help with the physical interpretation of his equations, James coined the terms ‘curl’, ‘convergence’ and ‘gradient’ for use in quaternion representation. Curl and convergence represent two kinds of space variation of vectors and are the terms used today, except that convergence is replaced by its negative, ‘divergence’, or div for short. Gradient, abbreviated to grad, is also used today; it represents the direction and rate of change of a scalar quantity in space. James had started the process that was to give us the elegant and relatively simple system of modern vector analysis, which is now so widely used that we take it for granted. The job was completed around 20 years later by two outstanding physicists, the American Josiah Willard Gibbs and the Englishman Oliver Heaviside.

James set up his colour box again and more guests were invited to have a go at matching colours. He summarised his accumulated results in two short papers, one of which dealt with colour vision at different points in the retina. Here James reported his investigation of the ‘yellow spot’, a small yellow region near the centre of the retina. He had found that most, but not all, people have much reduced colour perception in that region. The exceptional people had a yellow spot that was so faintly yellow as to be imperceptible. He also found a way to detect whether a person has a perceptible yellow spot without examining the retina; this is now called the Maxwell spot test. Katherine was among those with no discernable yellow spot but by now it was clear that she was in a very small minority. As James explained to a friend:

I can exhibit the yellow spot to all who have it—and all have it except Col. Strange, F.R.S., my late father in law, and my wife—whether they be Negroes, Jews, Parsees, Armenians, Russians, Italians, Germans, Frenchmen, Poles, etc. Professor Pole, for instance, has it nearly as strong as me, though he is colour-blind; Mathison, also colour blind, being fair, has it less strongly marked.⁵

Among the Maxwells’ most frequent guests at Glenlair were his young Cay cousins: William, who had built the new bridge over the Urr, and Charles, who had embarked on a teaching career and was now mathematics master at Clifton College, near Bristol. Charles, especially, was like a young brother to James and a great favourite with both of them. Katherine named her pony, Charlie, after him, and it was with Charles that James had shared the elation he felt about his ‘great guns’ paper on electromagnetism. Things were going well for Charles; he was popular at the school and had been made a housemaster. But in 1869 there came the tragic news that he had died, bringing a great sense of loss to the Maxwell household.

And there was sadness from another quarter; James’ old mentor and friend James Forbes died at about the same time. When Forbes had left his chair of natural philosophy at Edinburgh in 1860 to become Principal of St Andrews University, James had applied for the Edinburgh post but been turned down in favour of his friend Tait. Now James was asked by several of the professors at St Andrews to consider standing for the principalship. They included Lewis Campbell, who was now Professor of Greek there. At first James was reluctant. He had left King’s because he wanted to live at Glenlair and the attractions of home life were just as great now. And he was not sure that he was cut out for the job, feeling that: ‘my proper line is in working not in governing, still less in reigning and letting others govern’. But he did believe passionately in good education; maybe he could do some good in the post. And his supporters were very enthusiastic.

He was persuaded first to visit St Andrews, a long day’s travel each way in those days, and then to let his name go forward. This was a politically sensitive appointment and much would hinge on the political influence of the referees put forward by candidates. James was a political novice. William Thomson’s public standing was great enough to transcend politics, but who else should he ask for letters of recommendation? He wrote to a London acquaintance:

I have paid so little attention to the political sympathies of scientific men that I do not know which of the scientific men I am acquainted with have the ear of the Government. If you can inform me, it would be of service to me.⁶

Given such candid naivety, it is perhaps not surprising that James did not get the post. It went to J. C. Shairp, professor of Latin at the University. Whatever his other merits, Shairp had two clear points in his favour. He was a supporter of Gladstone’s Liberal party, which came into power shortly before the appointment was made, and a friend of the Duke of Argyll, who was Chancellor of the University.

A surprising footnote to this episode is that St Andrews had also turned down James Prescott Joule, the outstanding experimental physicist who had established the equivalence of work and heat, for their professorship of natural philosophy. It seems that Joule had a slight personal deformity which disqualified him in the view of one of the electors⁷.

Except for his first appointment at Aberdeen, James’ experiences with Scottish Universities were, to say the least, discouraging: he was made redundant by Aberdeen, then rejected in turn by Edinburgh and St Andrews. But life at Glenlair was good and in any event his ‘proper line’ was ‘in working not in governing’.

After his Edinburgh rejection, he had been welcomed by King’s College, London. Now the sequence was repeated. In February 1871 he was asked by Cambridge University to accept an important new professorship in experimental physics. The Duke of Devonshire, who was Chancellor of the University, had offered a large sum of money for the building of a new laboratory for teaching and research, and the first professor would have the dual task of starting up the department and getting the laboratory built. Cambridge was being left behind in experimental science by several British universities and many continental ones. This was a great opportunity for Cambridge to move up with the leaders. To make the most of it they had to recruit an outstanding professor.

Ideally, they wanted a top scientist who already had experience in running a thriving teaching and research laboratory. The obvious choice was William Thomson. He was approached but did not want to leave Glasgow University, where he had over the years built up a superb research centre, starting in a converted wine cellar. Hermann Helmholtz was also sounded out but he was about to take up a prestigious job in Berlin and declined. James was third choice. The authorities at Cambridge probably regarded him as brilliant, but something of a maverick with his strange theory of electromagnetism. And although he had given demonstration experiments to students and done private research he had no direct experience of running a research laboratory.

As he had been when the St Andrews post was mooted, James was at first reluctant. At length he was persuaded, although only on the understanding that he might retire after a year if he wished. This condition did not imply any lack of commitment. He saw what a great opportunity the scheme offered for Cambridge and the country and felt the excitement, but was acutely conscious of his lack of experience in directing a big operation and wanted to be able to withdraw if he found he was not able to run the show well. James was appointed in March 1871 and moved with Katherine to Cambridge.