High above the blue rippling waters of Hong Kong, the humid breeze whispering through the hills, the secret listeners in Batty’s Belvedere (as the out-station was called) had been chronicling a fresh war. After the surrender of Japan, civil war had broken out in China between the nationalist forces, led by Chiang Kai-shek, and the Communists, led by Mao Zedong. Signals intelligence (or ‘sigint’) – if good – is occasionally a means of gazing straight into the hearts of leaders. The sigint in this region was not good; efforts were made, but with the West so closely focused on every twitch in Eastern Europe, there were other regions where even the greatest expertise could not make that much headway.
The conflict in China was complicated by the weight of more powerful countries pressing in on it. The Soviet Union was occupying the lands of Manchuria, to the north; the Americans, under President Truman, were giving their overt backing to Chiang Kai-shek’s forces – they did not want to see the Communists grabbing this region. The British were in a position to closely observe; curiously, this was to be one of those points where they and the Americans fell out rather sharply.
Mao Zedong prevailed in 1949: the People’s Republic of China was founded. The Americans simply refused to acknowledge Mao’s ascension. Stalin wanted the kind of relationship whereby Mao adhered closely to Soviet thinking and desires. The British were – by contrast – a little subtler (or more serpentine, whichever way one might look at it) in their dealings with this new regime. Partly this was to do with trade between China and the United Kingdom, which was strong; and partly, perhaps, because of the unlovely history shared between the two nations. In this new post-colonial age, the British wanted to put diplomatic relations with previously oppressed and exploited peoples on a more elegant and civilised footing.
There was also the matter of the British colony of Hong Kong. This eruption of Communism – with the threat of expansionism that accompanied it – suddenly made Hong Kong – a tiny area surrounded by ideological foes – acutely vulnerable. So it was that foreign secretary Ernest Bevin informed the Americans that the government intended to give proper recognition to this new Chinese government. His argument to secretary of state Dean Acheson was this would be a way of making sure that China did not swaddle itself completely in Russia’s embrace. Acheson was extremely doubtful that it would achieve any such thing.
And when Britain did publicly announce its recognition of Mao, it turned out that Acheson was right: the People’s Republic of China was magisterially dismissive of Britain’s overtures. Ambassadors and diplomats were broadly ignored. Mao Zedong, initially, apparently only had time for the Russians. Again, while this was partly ideology, it is difficult not also to imagine that a driven moderniser like Mao (so driven, indeed, that his modernisations were to lead to famines that claimed the lives of some 30 million people) would have studied the history of Britain’s opium trading with China in the 19th century very closely. The humiliations heaped upon China by the British at the height of Victoria’s reign still resonate now; Mao must have felt that weight of history even more acutely.
The idea of the British having any long-term influence in the region became even more fantastical the following year: with the encouragement of Mao, the neighbouring territory of North Korea (a Communist regime run by Kim Il-Sung) launched an attack against the (American-backed) south of the country. It would prove to have astoundingly bloody consequences, for Koreans, Chinese, American and British troops alike.
The lightning assault across the region demarcated by the 38th parallel had been utterly unforeseen. There had been no prior intelligence: no indications of unusual troop movements, no brilliant cypher-breaks of crucial messages between military leaders. Neither the Americans nor the British, for all their extensive listening facilities, had picked up any warning. In fairness to Commander Travis and GCHQ, they could not be expected to gaze into a crystal ball over every last square mile on earth. And given that the Joint Intelligence Committee had directed the cryptanalysts to focus on the Soviet Union, it was clearly taking some time to build up an equal level of expertise in the Far East.
That said, the British and the Americans had not been completely oblivious to the dangers inherent in the region. In 1948, a CIA report observed that: ‘Eventual armed conflict between the North and South Korean governments appears probable… in the light of such recent events as Soviet withdrawal from North Korea, intensified improvements of North Korean roads leading south, People’s Army troop movements to areas nearer the 38th parallel, and from Manchuria to North Korea, and combined manoeuvres.’1 Yet equally, this was not the only area in the world on a potential razor’s edge. And there were still substantial US forces in Tokyo, occupying Japan after its apocalyptic defeat; the military there was dismissive of any North Korean threat, not least because they knew how heavily armed the South Koreans were. If anything, the Communists in the north appeared rather more vulnerable than the regime in the south.
Added to this, the messages given out by America had been rather easier for the Chinese and North Koreans to read: Dean Acheson had announced in 1950 that Korea lay outside America’s ‘ring of interests’. Kim Il-Sung saw an opportunity to seize the entire country; according to former Joint Intelligence Committee grandee Sir Percy Cradock, he received gentle though reserved encouragement from Stalin, along the lines that if the adventure backfired, Russia would immediately step back, and Kim Il-Sung would have to rely on China for help.
According to Sir Percy Cradock, the British in 1949 knew that their American allies had better intelligence on Korea; and indeed a small team from the Joint Intelligence Committee, on a visit to Washington DC that year, had asked to be included in on what they knew. But the Americans refused: a punishment for the British recognising Mao Zedong’s government. In the years to come, there would be further outbreaks of American petulance in the face of what they regarded as British stubbornness; fractures in an otherwise remarkably easy-going intelligence relationship. In this instance, no-one benefited, as the intelligence had either not been heeded or lacked urgency. North Korea launched its attack and the US and British were each as surprised as the other.
This moment in 1950 was – for some American military – the point at which they were braced for the worst. Russia now had the capability to destroy whole cities in sun-bright blasts; the question according to some was whether North Korea’s headlong attack over the 38th parallel was a means of diverting attention away from some larger plot? A Stalinist plot for Russia to lunge deep into Western Europe?
The notion that Communism could be a form of contagion – the ‘domino theory’ of one country after another falling – was still a little distance off (President Eisenhower cited ‘the domino effect’ in a speech in 1954 when talking of the convulsions in Indo-China). In 1950, President Truman and those around him in the Oval Office had now formulated a view that Stalin was effectively in control of every Communist state on earth; that nothing could be done by North Korea without his bidding. There were many in the military, too, who were convinced that Stalin was the great overlord, planning each move with scientific precision.
In the meantime, the British interceptors of GCHQ had been reporting back to the Joint Intelligence Committee, and the JIC issued this rather more measured judgement: ‘We believe North Korean aggression was originally launched not with the primary object of diverting American attention from Europe, or as a prelude to provocative action against our weak spots of the European or Middle East periphery, but as a limited operation within the ambit of an intensified drive to expel western influence from the whole of the Far East and South East Asia.’2
Yet the fever was spreading elsewhere. In the week of the invasion, a fervidly anti-Communist ‘Congress For Cultural Freedom’ was being held in West Berlin, within slingshot distance of the East Germans. The event was financed by the CIA. This ‘Congress’ put out a statement. ‘Indifference or neutrality in the face of such a challenge amounts to a betrayal of mankind and the abdication of the free mind,’ it ran. And at the same time secretary of state Dean Acheson outlined the entire purpose of US opposition to the Soviet Union. ‘All the problems of the United States are related to the problem of preserving its existence as the kind of country which we know and love.’
And as a part of the effort to preserve that way of life, troops were committed to Korea. This, however, was not strictly war at this stage; the troops were sent with the blessing of the relatively newly formed United Nations. And the UN was then able to call upon troops from other countries to help too. It was important to all parties that this should not be seen as a baldly ideological stand-off between the US and the USSR. Yet what ensued was a hideous conflict that in some ways more resembled the primitive charnel-house squalor of the First World War than a struggle taking place in the nuclear age.
Fast-evolving technology was only so much use; the intelligence services of both Britain and America lacked sufficient agents on the ground in Beijing and Pyongyang to provide ‘humint’, or human intelligence. The American codebreakers of Arlington Hall, however, were on the case. ‘By far the most important and effective sources of operational intelligence… were decrypts of enemy wireless transmissions by the vast signals operation established outside Washington for the purpose,’ wrote Sir Max Hastings. ‘But the available quantity of “sigint” was restricted by the enemy’s shortage of sophisticated communications equipment.’3
Nonetheless, intelligence was harvested, the operatives at Arlington Hall taking care to master the Korean language much in the way that over in Bedford, near Bletchley, a masterclass in Japanese for codebreakers had turned out so many apt linguists. Meanwhile, the British were said to have had some expertise with Chinese cyphers, which involved a somehow breakable form of one-time pad. For the Royal Air Force, swooping missions over Communist lines at least gave ideas of where and when bombing runs might be targeted; and in more traditional terms, it was found to be possible to listen in and intercept communications on Korean and Chinese telephone lines.
In the very early stages, America’s General Douglas MacArthur scored a remarkable military coup by getting his forces, via amphibious landings, deep behind the North Korean lines. But they were not just facing Kim Il-Sung’s army: Mao’s People’s Liberation Army were moving in, in their millions. Opening victories gave way to painful and bloody stalemates. It was during this conflict that the term ‘brainwashing’ came to be used: Western prisoners of war were said to have been subjected to relentless, months-long conditioning to do with the superiority of Communism over capitalism. The idea of hypnotised soldiers later found its popular cultural moment in the thriller The Manchurian Candidate. Since that time, there has been some revisionism about exactly how effective these ideological mind assaults were: some of the prisoners, it has been suggested, only gave way in order to try and make life a little quieter; their world views, upon release, appeared not to have been modified so greatly.
A rather more concrete result of the conflict, however, was a sharpening of the focus of the American cryptology efforts; Arlington Hall did not enjoy the centralised power of the codebreakers of Eastcote. Prior to 1950, there had been increasing amounts of confusion in the US military about the various different codebreaking branches – naval, military, air force – and an attempt at rationalisation combined with some severe cost-cutting. Out of the three old departments now came the Armed Forces Security Agency. The explosion of violence in Korea reversed the staffing freeze; hundreds more civilians – as well as service personnel – were recruited into the secret realm. The Pentagon’s anxiety that Korea might just be the long fuse leading to a much bigger, more terrible war at last made it see that it was important to invest in intelligence.
The difficulties were still vast; ever since all the Soviet encryption systems had been changed – and thus closed off to codebreakers in American and Britain – there had been large patches of intelligence dark matter, voids in intelligence which could not be filled with inspired guesswork. One of those patches of dark matter was communications between Stalin and Korea. The Armed Forces Security Agency quickly started to make up for this shortfall: having started out with only a handful of Korean and Chinese specialists, they soon recruited hundreds. On the ground over in Korea, signals experts had formidable local difficulties to deal with, including rough mountainous terrain – not only bad for interception, but also awkward in terms of transporting equipment. But the air forces were on hand; the brave crews of the planes making intelligence-gathering ferret flights swooped over enemy Korean territory, vacuuming up secrets and relaying them back, while staying ever alert for the presence of any Soviet fighters in the air.
For the codebreakers, another hurdle – negotiated it seems with some style – was not merely the Korean language (a few cryptologists were Korean themselves, the rest either were taught the language, or indeed taught themselves). It was the unfamiliarity of all the military terms used in that language. For instance, one of Bletchley Park’s great achievements throughout the war had been its card file index: a roomful of cards, the result of painstaking logging of every single German term for every single moving part of a weapon or vehicle, every term for every aspect of military operations or manoeuvres. This card index had grown over the space of several years; here, the codebreakers were having to start from scratch.
Happily, by this stage the codebreakers had found a few levers into Chinese communications; and so it was that they were able to follow the deployment of troops, and messages passed between foreign minister Zhou Enlai and various diplomats. The bank of knowledge expanded fast.
In other words, despite having faced code blackouts, underfunding and disorganisation, the US and UK codebreakers still managed in time of crisis to produce useable, valuable intelligence. This did not save the US cryptologists from facing caustic criticism over their stuttering start back in Washington DC, though, and the Korean War was in many ways to have repercussions on American and British codebreakers that can still be felt today.
One of the lessons the Americans learned was that they would benefit from following the example set by the British: ever more centralised codebreaking, as opposed to competing departments in different military wings working without co-ordination, overlapping and missing other elements. And this centralising move led to the dissolution of the AFSA in favour of the even more compact and rigidly controlled National Security Agency, which came into being in 1952 and is still very much with us today.
Budgets improved; and more money brought not only extra recruits – brilliant logicians and linguists – but also more of the spectacular technology still in development. America was making lofty leaps forward in the new science of computing, while Britain was not doing so badly considering the sparse money available. But more than this: the codebreakers – British and American – understood after being tested in the crisis of the Korean War – that theirs was a discipline that had to be self-contained. The intelligence produced by these brilliant brains could not be allowed to become the subject of inter-departmental wrestling matches. In America, the National Security Agency swiftly became as monolithic a feature on the intelligence landscape as the equally fresh CIA and the Federal Bureau of Investigation. In Britain, the codebreakers of Eastcote were able to keep themselves ever more at arm’s length from MI5 and MI6. They sent the fruits of their efforts instead to the Whitehall grandees of the Joint Intelligence Committee who co-ordinated and filtered intelligence from all departments.
Another parallel between the American and British operations was the sheer number of National Security Agency personnel who had had their formative codebreaking experiences during the war. Geopolitics and high diplomacy aside, this was also why – on the personal level – there was so much mutual respect and friendship between the two agencies across the Atlantic. There was a shared warmth, as well as professional, pride. And long afterwards, the US codebreakers continued to sing the praises of British operatives like Brigadier John Tiltman, using him and Hugh Foss and others as shining examples in lectures to younger codebreakers being inducted into the mysteries of this dark world.
The Korean War did not end – as many feared it would – with mushroom clouds. Instead, there was a rather more wearily familiar outcome. For the Koreans, both of the south and north, the conflict was devastating, resulting in somewhere in the region of a million deaths. The same was true of the Chinese army. American mortality came in at 37,000. The death rate of British personnel was not quite so heavy, but still numbered into many thousands. Many of the British soldiers in Korea were there as part of their National Service: teenagers conscripted into the forces with no choice in the matter.
Brian Hough, from Manchester, was aged 18 when he and his fellow soldiers set sail for Korea. The sights that greeted him at Busam, he said, were ‘from medieval times’. He knew nothing of Korea or indeed much about this part of the world; nothing of its history, or culture. The nature of the war came as the greatest shock. ‘It was trench warfare,’ he said. ‘We laid in the trenches and dug holes in the sides of the mountains every now and then and that’s where we lived. They were rat-infested. I think I was more afeared of them than I was of the Chinese.’4
This did not feel like a jet-age war. The young soldiers were forced to exist in sub-zero conditions, and when the enemy came, eyes were taken out with mortar blasts, and bullets caved in skulls. There was filth and squalor and blood.
Then, many young soldiers returned to Britain to find that no-one wanted to hear a word about their experiences. The country was still absorbing the World War; and the weariness was deep. This in part is why the Korean War scarcely figures in modern history: a blip, a fight in a far-away place, best forgotten. One such soldier was the future actor Maurice Micklewhite (later Sir Michael Caine). He recalled in his biography his ship sailing into Pusan harbour and the ‘stench’ of the place that could be registered ‘three miles out’. He and his fellow Royal Fusiliers were sent up to the 38th parallel, living in dugouts composed of mud and bamboo. They came under nightmarish bombardment from the opposing Chinese forces. ‘It was a strange time, something just outside the bounds of reality,’ he said.5 As well as the physical danger, the constant 24-hour noise meant that sleep became more and more impossible; and even if fatigue was reaching its height, he recalled, there was also the crawling physical disgust of knowing that huge rats would be running over your prone body.
From a wider perspective this fighting – and all the other countless skirmishes throughout those three years – were in part the result of all sides misinterpreting each other’s intentions disastrously; a theme that was the become a recurring leitmotif of the Cold War being fought out in proxy around the world.
Yet as American and British intelligence agencies made further efforts to fathom this confused landscape, there were greater signs of the codebreakers and interceptors drawing even closer together. A memo sent out in the US in 1951 addressed what seemed the very pressing question of Hong Kong: just how secure were those British operatives in Batty’s Belvedere? Just as uncountable hordes of the People’s Liberation Army were pouring into the mountains and plains of Korea, what was to stop them annexing this particularly vulnerable corner of British imperial rule? And what would be the plan if they did?
‘In the event of emergency withdrawal from present location,’ the memo read, ‘US is committed to providing relocation of British Hong Kong COMINT unit on US or US-controlled territory. This is in part quid pro quo arrangement in return for accommodation of our units on British or British-controlled territory plus others now in Europe…’6
This was now more than a simple comparing of codebreaking notes. ‘US now receives full intercept output of Hong Kong Unit which is important and does not duplicate US effort,’ the memo continued. But the spy chiefs were not wholly happy: as the start of the Korean War had demonstrated so very sharply, there were dark gaps in intelligence results from that part of the world. ‘Present combined US/UK intercept facilities in the Far East are far short of requirements,’ the memo continued. ‘Considered full advantage to be taken of British unit.’7 And should disaster have befallen the unit – akin to the 1942 fall of Singapore, when the codebreakers had to scramble to escape, leaving not a clue or a trace of their work behind – then the US idea was to transplant the entire operation to US-controlled Japan.
The Americans were aware that the British, like themselves, had been developing some rather neat technological advances; and soon after, the Eastcote codebreakers were told by their superiors that GCHQ would now be regarded as being fully in charge of electronic intelligence, as well as the more straightforward communications intelligence. As ever, a committee was formed: the Technical Radio Intercept Committee. It was about the future of sharing knowledge; the British were not to keep any electronic intelligence from their American friends.
‘GCHQ is entering the non-communications listening field,’ ran the memo from 1951. It was referring to the growth in computing, upon which the following chapter will focus. ‘They [the codebreakers] will be responsible for transmissions of information from the field to London… new ground units, manned by RAF or Army personnel, will report through GCHQ to the Technical Radio Intercept Committee.’ But their reports would have to go further. ‘Exchange of raw intercept information is not satisfactory,’ continued the memo. ‘US Navy [is] feeding logs to Royal Navy and getting only collated reports in return… [the] exchange of raw intercept information must be kept to top secret channels… Commander Loehnis and Mr Smith will attempt to see that TRIC passes raw intercept information to US Force and US Navy personnel in London.’ An element of pleading came in. ‘If US Navy does not start getting raw data,’ the memo continued, ‘they will stop giving raw data.’ It added that ‘Commander Loehnis [has] requested information on procedure to follow for obtaining intercept equipment in the US.’8
The balance of power in the hermetic world of decryption changed remorselessly, and fast. The British navy and air force were still of immeasurable use for the collection of intelligence, just as those colonial listening stations were strategically vital. The Americans were now in a position to give orders. But Britain still had lightning bolts of ungovernable genius; and those US scientists who had been working throughout the war on prototype computing technology would – by 1950 – observe with some admiration the vast leaps made by UK boffins who were still among their connections. The world of computers and codebreaking was a small one.