WHY DO WEATHER FORECASTERS GET IT WRONG?
Unpredictability and chaos
It has often been said that, while most countries have climates, Britain just has weather. If there was an index that measured the degree of fluctuation from rain to sunshine, windy to still, and warm to cold over short periods of time, the British Isles would surely top the league.
This is one reason why weather is the most popular topic of conversation, and why forecasts are a key feature of every news broadcast and achieve audience ratings unmatched in any other country.
You would think, though, that with so much weather about, forecasters would have got the hang of it by now. So why is it that weather forecasts can sometimes get it so horribly wrong?
The answer begins not in the sky but on a pool table.
Big breaks and flukes
You probably know the normal way to begin a game of pool. The white ball is placed at one end of the table, and is then struck as hard as possible to smash into the triangle of striped and spotted balls at the other end of the table.
The balls scatter seemingly at random across the table, and if you’re lucky one or two of them drop into the pockets.
Apart from the need to make a decent collision between the white and the colours, the opening shot is almost completely without skill. That is to say, there is no way of predicting with any confidence where all the balls will end up even if you attempt to hit the white with exactly the same strength and in the same direction each time. The tiniest change in the way you hit the ball or the way that the triangle of balls is set up leads to a different outcome.
It seems fair to describe the spreading out of the coloured balls at the start of a pool game as chaotic, and in fact chaos is exactly the term that mathematicians would use to describe it. Because it is a relatively new science, mathematicians are still a little bit vague about the precise definition of chaos, though definitions do exist, some of them extremely complex. However, there is one underlying theme to chaos that most mathematicians agree on. Something is chaotic if the tiniest of changes to the initial input can lead to a completely different, and unpredictable, outcome.
This effect of small errors having huge consequences has been known for a long time. Benjamin Franklin, one of the founding fathers of the USA, was responsible for this well-known quotation:
For the want of a nail, the shoe was lost;
for the want of a shoe the horse was lost;
and for the want of a horse the rider was lost,
being overtaken and slain by the enemy,
all for the want of care about a horseshoe nail.
Might it be that some battles – and the whole course of history thereafter – have been influenced by the tiny detail of whether a nail was missing from a horse’s shoe? Many would argue that they have.
A contest that is closer to most people’s experience is a game between two teams. As in battles, everyone knows that the outcome of sporting tussles can hinge on one small incident, such as a player being booked, or a ball deflecting at just the wrong angle. However, what cannot be predicted is what would have happened had the hinge moment gone the other way. When commentators say ‘the game finished onenil, but it should have been three-nil because of those two missed chances’ they are wrong. Suppose the first chance had gone in, instead of hitting the post. Although the score in the game at that point would have been 2-0, what would have happened thereafter, says chaos theory, would have been unpredictable. Yes, the team in front would have felt stronger at that point, but the different interactions and tactics thereafter could have led to a 5-0 win or a 2-2 draw, or just about any other result.
Perhaps one of the best illustrations of how sport can show chaotic tendencies – that is, small changes in the starting conditions producing large, unpredictable, changes in the outcome – comes from a computer program to simulate a cricket match. This program, written by Gordon Vince in the 1980s, allows you to enter the names and details of any two teams, and then to play a complete cricket match between them. In fact this program is really an extension of a dice game called Howzat, which was popular long before the computer era.
Every event in the match is simulated by functions with random elements – a computerised version of throwing dice for every incident. For example, from any ball a batsman might score runs, or he might be out, or nothing might happen (there’s plenty of that in cricket). The program is extremely realistic – bowlers become ‘tired’ and perform worse when they have been bowling for a long time, and batsmen become ‘nervous’ and are more at risk of getting out when their personal score approaches 100. The program produces a complete printout of what happens after every ball of a match, and the outputs look convincingly like real matches.
To generate a match from this program, you need to enter details of the two teams, including a strength factor for each of the players. You also enter a seed number, which acts as a random-number generator. In effect, the seed decides what the outcome will be of every die-throw throughout the game. A different seed leads to a completely different match.
As an experiment to see how predictable the result would be, we used the program to simulate a cricket match between England and the West Indies. Details of the two teams were entered, with each batsman being given a strength factor, a number between 5 and 40. This strength factor determined whether the batsman was likely to score a lot of runs or hardly any runs. Using a seed number of 444, the scores in the match looked like this:
West Indies 1st innings: 193
England 1st innings: 162
West Indies 2nd innings: 253
England 2nd innings: 187
A quick bit of adding up will confirm that the West Indies scored more runs overall than England in this match. In fact the West Indies won by 97 runs.
The match was then replayed using the same seed, and with all of the player details the same except for one. The ‘strength factor’ of one West Indies batsman was increased from its starting position of 23 to 25. Because this batsman’s batting strength had been increased, the overall strength of the West Indies team was now slightly higher than before. Everything else was unchanged. As a result, you might expect the West Indies to have won the game and by an even larger margin. Yet the outcome of the new match was as follows:
West Indies 1st innings: 244
England 1st innings: 525
West Indies 2nd innings: 332
England 2nd innings: 52 for 0
Even though they were relatively stronger than before, the West Indies’ performance actually got worse while England performed far better this time. In fact, to use the cricketing parlance, England won this match by ten wickets. This isn’t an unusual example. Tweaking any of the starting factors, sometimes by very small amounts, could have had a similarly huge impact on the final outcome.
Why should this be? Suppose that because the batsman is stronger than before, he scores a run at one point that he would otherwise not have scored. This brings his partner to face the bowler, and since the partner has a different batting style, the next ball has a different outcome – maybe the batsman is bowled out on this ball, which would otherwise not have happened. The resulting chain reaction of events causes the game to diverge increasingly from its original path until it becomes unrecognisable from its previous incarnation.
This system is behaving in a chaotic manner. Whatever his knowledge of the game, no expert could have predicted this outcome. Instead he would end up with egg on his face, muttering the old adage used in so many sports: ‘It’s a funny old game‘.
The pendulum and the magnet
Another quite different situation that illustrates how tiny variations in the starting point can lead to big changes in the outcome is a toy that used to be common on the desks of executives.
The pendulum-magnet is a mesmerising device. A ball hangs on a pendulum above a steel base. On the bottom of the ball is a magnet, and on the base are three more magnets, each of which is placed so that it will attract the ball towards it.
To start the motion, the ball is pushed aside and then released. Without the magnets, the ball would just swing to and fro, but because each magnet is pulling it, the ball instead swings all over the place, sometimes in a violent jerking motion, until it eventually comes to rest above one of the three magnets, which we’ll call A, B and C for old times’ sake.
The three base magnets can be arranged in a triangle in such a way that each of them ends up with the ball above it about one third of the time. However, just like the pool balls at the start of the chapter, it can prove very difficult to release the ball and predict with any confidence which magnet it will end next to in any particular turn. One time it will end above magnet A, but the next time, despite being released from apparently the same place, it ends instead above magnet B.
An insight into why this unpredictability arises came when mathematicians were able to produce a computer simulation of the pendulum. Since all the dimensions and forces involved were known, it was, as these things go, a reasonably straightforward task to write a computer model mapping out the path of the ball right the way through to its finishing point. That way, the end point of the ball could be calculated for any specific starting point.
The results were surprising. Suppose you map out some of the possible starting positions of the ball in a square grid. If the ball is released from the midpoint of each square on the grid you can work out which of the three magnets it will end above. Here is how part of the grid might look:
If the ball is released from anywhere in the zone on the left, it seems certain to end up at B, while the upper middle area or the right seems to be an ‘A’ zone.
However, if you zoom in on the B square that is highlighted, and divide it into smaller squares, then a whole new complexity is revealed within:
It turns out that, although the ball ends above B if you release it from the middle of this square, nearby squares can lead to other end positions. Nestling within what was a ‘B’ square is a cluster of Cs and a solitary A. And zooming into any of those squares reveals another pretty and yet unpredictable pattern made up of As, Bs and Cs. However far you zoom in, fresh patterns will emerge. No wonder that tiny errors in the starting position of the ball can mean that it ends up at a different magnet from the one that you predicted.
Note that not all of the zones will produce these chaotic results. There will be some solid regions where the endpoint will always, consistently, be A, B or C. These are, if you like, the predictable zones. Other areas, like the one described above, are chaotic zones.
The ever-changing patterns of As, Bs and Cs in the chaotic zones will have a familiar ring to them if you have read the previous chapter. In fact fractals and chaos are intimately linked together, which is why each of them has managed to infiltrate the other’s chapter.
Randomness: how can a computer simulate dice?
Most games programs on computers require the computer to do ‘random’ things. Any computer, therefore, needs to be able to produce on command numbers that are as unpredictable as the outcome of rolling dice. This is not as easy as it may sound, because the whole point of computers, of course, is that they are there to be predictable by following rules.
Although they are not capable of generating truly random numbers, all computers contain a formula that will produce ‘pseudo-random’ numbers – numbers that appear random even though they are generated by a precise sequence of calculations. Hundreds of different methods exist for generating such sequences, many of them requiring an initial seed number to start them off. This seed can either be entered by the user, or else taken from the computer’s clock (for example the number of seconds past the minute at the instant when the keyboard is struck).
Randomness is quite tricky to define, but a common way of testing for it is to check that (a) all numbers in the sequence appear roughly the same number of times as each other, and (b) the numbers do not follow any predictable pattern. The sequence 1, 2, 3, 4, 5, 6, 7, 8, 9, 0 passes the first test of randomness, but fails the second. 5, 8, 3, 1, 4, 5, 9, 4, 3, 7, 0 appears, at first glance anyway, to pass both tests, though it is only pseudo-random because it was in fact generated using a simple rule -can you work out the rule?
Chaos in the weather
Pool balls, missing nails and magnet-pendulums contain all the analogies we need in order to explain the problems in forecasting the weather. As with the magnet-pendulum toy, weather is created by a range of simple forces combining together. In the case of the weather, these forces are caused mainly by the sun’s heat and the earth’s rotation. Small changes in these forces can in some circumstances have enormous impacts on the weather.
In fact, one of the earliest discoveries of chaos came when a researcher called Edward Lorenz attempted to model the way in which weather patterns develop. Using a computer, he devised a relatively simple model to simulate how a weather system would evolve given certain starting conditions. The computer then generated reams of paper with numbers showing the changes in weather patterns.
When he wanted to rerun the simulation, Lorenz decided to save time by copying down the numbers halfway through the process and using those numbers as the starting values. To his amazement, however, the weather forecast changed, even though he had copied down the correct numbers to several decimal places.
It turned out that the numbers that the computer had printed out included rounding errors. What was printed as 17.427, for example, was really 17.42719163 in the computer’s memory. That tiny error was enough to produce enormous changes to the outcome of the weather forecast for a week later. This was genuine chaos at work – a simple system creating highly complicated and unpredictable patterns. Thanks to his observations, Lorenz later made the claim that a tiny perturbation such as a butterfly flapping its wings could, in the right circumstances, lead to a chain of events that would create a hurricane in Florida – in just the way that a missing nail could have brought down George Washington’s army.
Fortunately, even though a tiny change to the starting conditions can have unpredictable consequences, the overall weather patterns turn out normally to follow certain reasonably well-known routes. By running several different simulations, all using slightly different starting conditions, forecasters can see what the weather outcomes might be. Often, all of the outcomes will be similar to each other, indicating that the weather is in a predictable zone. Sometimes, however, small changes to the starting conditions lead to a wide range of weather predictions. Here, the weather has entered a chaotic zone. The longer in range the forecast, the more likely it is that a chaotic point will be reached, after which the forecast becomes little more than a guess. That’s one reason why British forecasts rarely extend beyond five days.
And, whatever the forecasting model says, there is always a chance of a freak outcome. Michael Fish will for ever be remembered as the weatherman who publicly reassured a viewer in October 1987 that a hurricane was not due that night. Twenty-four hours later, southern England had been flattened by the worst storm in living memory.
A sports commentator would have known exactly what to say. It’s a funny old climate.