9

OUT OF THE SHADOWS

The latest man to be known by the letter C would occasionally cross the river from MI6’s drab headquarters in Lambeth to wander through the corridors of Whitehall where other government departments were still beavering away. The civil servants Colin McColl met would sometimes be surprised to see him. It was almost as if he was a largely forgotten uncle discovered in the corner at a family get-together. ‘I’d meet people – intelligent, knowledgeable people in Whitehall – they’d see me and they’d say things like “Are you still here?”’ McColl later recalled.1 As it entered the 1990s, the Secret Service appeared a little lost without the comfort blanket of the Cold War. Spying on the Soviet Union had never taken up more than half of its effort, but nevertheless the work of MI6 had been defined in the public mind – and to some extent in its own – by the world of Moscow Rules, Smiley and Karla and doubling, tripling agents.

In a grand, high-ceilinged room near Downing Street, the Chairman of the Joint Intelligence Committee had offered a celebratory glass of champagne to toast the end of the Soviet Union in August 1991. And with that, the war was over. McColl, a Sov Bloc veteran whose youthful sense of humour masked a sharp mind, had presided over victory. From his office high up in the building with a view across London, he would tell new recruits that they had joined at a fascinating time in which MI6 could not take its eye off the old threats but also had to look out for new dangers while maintaining Britain’s place in the world. The position of the service in government was also changing, he would explain. But he did not always succeed in reassuring all his staff that the service had a role in the new world. One of his younger officers remembers an encounter with the Chief at a reception in headquarters. McColl turned to him and said, ‘Well, what do you think we should do?’2 The question did not necessarily inspire confidence. With its old adversary seemingly out of action, those on the inside wondered what MI6 would find to do next. Those on the outside wondered whether it was even needed any more. An air of gloomy insecurity hung around the corridors of the British Secret Service.

The rabbit warren of Century House, with its peeling lino and Formica tables, had a forlorn air and the Treasury, like a lion circling a wounded beast, was on the prowl. There was distressing talk of even merging with ‘the other lot’ at MI5 (who were also looking for work and trying to wrestle responsibility for fighting the IRA away from the Metropolitan Police). The peace dividend, shaped as an axe, fell. The first compulsory redundancies were painfully served and poorly handled within MI6. Overall staff numbers of around 2,000 were to drop by 25 per cent, senior staff numbers by more.³ Stations were closed in Africa and the Far East; more emphasis was placed on inserting officers into countries rather than having them based there permanently. Every piece of turf had to be fought for using every ounce of Smiley-like cunning. The Foreign Office decided to claw back some money from MI6 by estimating how much desk space was used by spies operating out of embassies abroad and then insisting it be paid for. MI6 retaliated by calculating how much work its staff did to maintain their cover as Foreign Office diplomats. This, of course, came to more than the desk space and a truce was called. In perhaps the most telling sign that the service was being dragged out of the past, that creature very much in vogue in the 1990s, the management consultant, was even brought in to sniff around. These consultants were all carefully vetted, but even so their arrival was watched with utter bemusement by many old-timers. The free-booting era of the 1920s, when the Chief could pull an armful of gold coins out of his desk and hand them over to fund an operation, was being replaced by one of audits and value for money. A small attempt at privatisation proved disastrous when a group of long-serving office cleaners were sacked. This was blamed on pressure from the Treasury to outsource their work to a private company but more likely resulted from a failure to realise that, with the Cold War gone, the willingness of people (whether cleaners or the press) to accept the traditional constraints of secrecy was eroding rapidly. To much embarrassment, the cleaners took the spies to the cleaners by winning an employment tribunal and a healthy pay-out (along with plenty of publicity). The insecurity surrounding all the intelligence agencies occasionally manifested itself even within the stately confines of meetings of the Joint Intelligence Committee as representatives of each intelligence agency competed to try and knock out references in reports to the other agencies’ work and put in their own.4

Gerry Warner had risen to be deputy chief of MI6 and then the security and intelligence co-ordinator in the Cabinet Office, responsible for overseeing the community as a whole and its health. He found he needed to traipse round departments, even the Bank of England, and ask what they actually wanted. A performance-monitoring system was instituted in which policy-makers would tick a box if they found intelligence useful. Intelligence was becoming less precautionary than it had been in the Cold War when the focus was on looking for signs of conflict or having the tactical knowledge of how to fight the war if it started. In the old days an analyst could spend his whole career watching a Soviet tank division before receiving his pension. Intelligence was now drawing closer to day-to-day decision-making. ‘In many ways the intelligence we were providing was of more immediate use to politicians than the intelligence we had provided during the Cold War, because most of the intelligence provided during the Cold War was mostly of background interest – very important, very interesting – but there wasn’t anything we could do about it,’ argues Warner.⁵ And even when it came to the Russians, things were a little odd.

The British Ambassador in Moscow, Rodric Braithwaite, nearly choked on his breakfast when one morning he opened up a local newspaper to find an interview, carried on two successive days, with his chauffeur. For seventeen years Konstantin had driven the Ambassador’s Rolls-Royce in and out of the grand, menacing prerevolutionary mansion on the Moscow River with its view over to the Kremlin. He was now admitting that he had spied on Braithwaite and his predecessors for the KGB. ‘You might have warned me because this could cause me serious trouble back home,’ Braithwaite told Konstantin. ‘You know, questions in the House about the limpwristed Ambassador who failed to notice his driver works for the KGB. That sort of stuff.’

‘I couldn’t,’ replied Konstantin. ‘I’m a Russian patriot.’6

Times were changing. For a while in 1991 the old KGB was in disarray and appeared out of business. Braithwaite hosted a delegation of spy-hunters from MI5 who had come to meet their former adversaries. The group were greeted at Moscow’s airport by a KGB officer bearing roses, before enjoying a meeting at the Lubyanka in which both sides felt ‘like wild animals being presented with their prey in circumstances where they couldn’t eat it’. The British asked politely if the surveillance and harassment of members of Embassy staff in Moscow could be reduced and asked if the level of Russian espionage within Britain might be limited. They received the distinct impression that these ideas were ridiculous.⁷ Braithwaite also had a new MI6 head of station working in the secure bubble room in the cavernous Embassy-cum-Residence. It had not been the obvious choice for an ambitious officer but Moscow was embedded deep in John Scarlett’s psyche and the chance to be the first station chief to be ‘avowed’ or declared to the Russians at such an interesting time was simply too good an opportunity to miss. The idea was that the two countries would normalise their intelligence relations and act like other countries where the head of station did not do any spying per se but acted as a liaison with the local services for the passing of agreed information. Scarlett’s past running of Oleg Gordievsky, a traitor reviled deeply by KGB types, was either unknown to the Russians or conveniently overlooked. But the old world had not entirely passed.

Dressed in shabby clothes the Russian had knocked on the door of an American embassy in one of the Baltic States. He was turned away disappointed. Defectors whom Western intelligence would have fought tooth and nail for in the past were now appearing hopefully at their doors bearing armfuls of documents and expecting dollars and visas in return. They seemed ten a penny and most got nowhere. Next stop for this man was the local British Embassy where he explained to a female diplomat that he had top-secret KGB material. The British diplomat had been trained to deal with walk-ins and glanced at the material, which had lain among bread, sausages and clothes in his bag. Vasili Mitrokhin would be one of the select few considered valuable enough to warrant assistance. He was a former archivist for the KGB who had secretly copied out and then buried large chunks of the organisation’s secret operational history in the garden of his dacha. Understanding its potential, the British diplomat told him to return again soon at an agreed date, when he would meet an MI6 officer who would come over from London. At that meeting Mitrokhin produced 2,000 pages of his notes which included details of KGB illegals. This was enough to swing the argument in London. Displaying the kind of cheek which the old Robber Barons would have enjoyed, MI6 organised for the clandestine exfiltration of Mitrokhin and his family on the seventy-fifth anniversary of the Bolshevik Revolution. Scarlett was kept out of the operation, but a young MI6 officer dug up Mitrokhin’s voluminous files and carried them to the British Embassy where they were taken to Britain in six large trunks.8

The Russians talked about ‘no spy’ deals, but there was little trust on either side. A few of the old Cold Warriors at MI6 could not quite let go and continued to want to use the defectors to turn the SVR, the renamed foreign intelligence wing of KGB, inside out and to extract every last drop of blood in revenge for Philby. MI6 even recruited one junior Russian diplomat who appeared to be mentally ill. This was a result of ambitious officers hoping to hit their ‘performance targets’ and exaggerating their successes, one disaffected colleague thought.⁹ As early as 1992, there were the first signs that the SVR was also up to its old tricks with a couple discovered at Helsinki airport travelling under false identity papers, claiming they had been born in Croydon and Wembley in London. They looked a lot like old-fashioned Russian illegals.¹⁰

Scarlett’s time in Moscow did not end happily. The Russians nominated one of their senior officers to take up the counterpart position as declared head of station in London. MI5, still not quite able to come to terms with the new world, kicked up a fuss. He is a spy, they said. Of course he is a spy, the Foreign Office and MI6 replied, that is the whole point. But he has had shady dealings with Saddam Hussein’s Iraq, MI5 responded. It would be more of a surprise if someone who had worked for the KGB did not have a shady past, came the reply. The Russian intelligence officer was suffering from cancer and Moscow was keen for him to get to London for medical treatment, but his visa was blocked thanks to MI5. Moscow was understandably furious and decided on revenge. Scarlett was expelled under the public pretext of having recruited an agent in a military metallurgical firm. The expulsion was carried out not quietly but in a blaze of publicity, a photographer capturing an image of the publicity-shy spy in a car at the airport.

In Eastern Europe events were in some ways even stranger as MI6 officers would walk gingerly into the office of former foes and politely ask exactly what they had got up to in the past. MI6 and MI5 began a low-key role in reorganising intelligence services across what had been the Eastern bloc. This involved civilianising their services and teaching them ‘how to collect intelligence in a different environment without threatening to put someone against a wall’, as one Briton involved puts it.11 One issue was what to do with the twenty-odd illegal sleepers that the Czechs had secreted around the world. This included one or two who had been sent to Britain and now did not want to go home. It was agreed between London and Prague to leave them alone as they had never done any damage.¹² As far as anyone knows, they continue to live somewhere in suburban Britain, their neighbours none the wiser about their training in servicing dead drops and sending burst transmissions.

There were old debts to be repaid as well. Agents who had been spying in place had been paid by means of escrow accounts with an understanding that one day they would be able to draw on the funds (this was always preferred by MI6 to giving them money which they could flash around, drawing attention to themselves). One agent had read the Financial Times voraciously and insisted on telling his case officer exactly how he wanted his portfolio invested. Some came out of the Cold War with a million pounds for having betrayed secrets. The valuable agent codenamed Freed had died in Czechoslovakia of a heart attack in the mid-1970s. With warm relations now established with the Czech service, MI6 approached Václav Havel, the former dissident intellectual now running the country, and explained that there was a bank account with the money the agent had accumulated in his lengthy career spying for Her Majesty. Havel agreed to help find any remaining family. A daughter was eventually located and carefully approached. She had never guessed her father had been a spy for MI6. But, according to a British official present at a meeting, she did remember him once saying an odd thing to her: ‘I hope you marry a British officer.’ Now she understood why, and she was given details of the bank account.¹³

There was one final innovation, an intrusion of the modern world, welcome to some but disorienting, even terrifying to others. The Secret Service was not going to be secret any more. The timing, coming at the end of the Cold War, was largely coincidental. MI5 had already been avowed in 1989. The staff there had foisted the move on a reluctant government. They had been unhappy with having no real legal basis for their work tapping people’s phones and bugging their houses in the UK, other than a minister’s sign-off. They wanted to feel that they were working within the law, not around it. ‘It was not comfortable to be engaged in operations for which there was no proper legal cover,’ recalled Eliza Manningham-Buller. There was also the European factor. Individuals had tried to sue security services in Europe for their actions. The European Court of Human Rights had decreed that any security or intelligence service had to be on a legal footing and have a proper system of complaints. Britain had neither. MI6 would have to become ‘legal’.

There was also the rather expensive new office that MI6 was having constructed at growing expense in Vauxhall Cross. Like Century House, the building was on the south side of the river physically separating MI6 from the rest of government. But that was about the only thing the new place had in common with the old. It was flashy and very unsubtle. Gerry Warner pointed out that it would be hard to move to something that looked like an Odeon cinema and expect people not to ask what it was. The problem with Century House had been security. Not the location, which was widely known to everyone including bus conductors (‘spies alight here’ they would say as they reached the stop outside), nor even access, with its security guards who would wave people in without asking for any ID unless they did not recognise them. It was the fabric of the building itself that kept those concerned with its safety awake at night. The largely glass headquarters of the British Secret Service was housed on top of a petrol station. ‘God, we were really living on borrowed time,’ says McColl. The station was owned by Q8, the Kuwaiti petrol company. When Iraq invaded Kuwait in 1990 there was a joke doing the rounds that all the Iraqis would have to do was light a fuse to destroy the entire British Secret Service. The new architect-designed home in Vauxhall, known as Legoland by some, was secure but also a touch sterile, befitting the new era. Staff attitudes towards it were perhaps best illustrated during a special premiere for the new James Bond film The World is Not Enough in 1999, hosted inside the new headquarters. The previous year Dame Judi Dench who played M had been invited to Christmas lunch by the real C to help her gain some insight into her role. The actress who played Miss Moneypenny introduced the viewing. When the scene arrived in which a large explosion rocked the new MI6 headquarters, the assembled staff issued a loud cheer.14

The legislation which placed MI6 on a statutory footing was piloted through the House of Commons by Foreign Secretary Douglas Hurd, a former diplomat who had turned down an invitation to join the service in the 1950s. Serving as a diplomat out in China, he had occasionally counted railways wagons for MI6 and gazed askance at his Embassy colleagues. ‘They were odd folk by definition,’ he later recalled. ‘You found yourself maybe sitting alongside somebody who had a rather peculiar job description and you understood gradually what he or she was up to.’¹⁵ The act was passed in 1994. It answered a profound insecurity in the service. Before then, since it did not actually exist, MI6 could have been wound up or merged by the stroke of a pen and by the whim of any minister. It now had a secure foundation and at last there were fewer pretences and ums and ahs from ministers and officials when explaining exactly what bit of government they were talking about. But was there also a cost to coming out of the shadows?

There was ‘a little sorrow’ from those who had lived beneath the shroud of secrecy for so long, according to McColl. A generational divide marked attitudes within MI6 to its emergence, blinking, into the light. ‘The old people were used to the old system and they weren’t anti-avowal but they simply weren’t terribly interested in it really,’ remembers McColl, a product of Shergy’s Sov Bloc master race where secrecy was prized and something of a reluctant lifter of the veil. ‘It was always the slippery slide that we were worried about. That once you got on to the slope and you started opening things up you would run into problems over secrecy. And secrecy was and is absolutely central to the whole of our work because our work is about trust. It is about trust between the government and people running the service. It is the trust between the service and the people all over the world who are working for it, and many of them are taking great risks … they do that in the faith that we are really a secret service which means to say we are not talking and we’re not going out into the public and declaring ourselves. And I’ve always felt that that was one of our advantages. We were – and have been always – a secret service.’ The cautionary tale, McColl and others believed, was the American experience. They had watched the very public undressing of the CIA by Congress in the mid-1970s. ‘There was a sort of shudder that went through the intelligence world and it went through many of the people who were working for the Americans or working for us, because they were coming to us and saying “For God’s sake, look what’s happening in America” and “Is it going to happen to us?”’16 As MI6 stepped out, some of the older generation went around muttering to each other. ‘It will all end in tears,’ they said.

The fear was that avowal would strip the service of its mystique, a sheen of glamour and power built up largely by fiction. This had been sustained through secrecy since no one had any way of judging whether the fictional portrayal was on the mark or not. McColl traces its origins back to the late nineteenth century when Kipling and others wrote of brave British spies fighting the dastardly Russians in the Great Game over India, a tradition continued in John Buchan’s stories about the plucky Brits now defending the realm against the cunning Krauts. And then came Bond. ‘It keeps the name going doesn’t it,’ reckons McColl, who speaks freely about MI6 as a ‘brand’. ‘I mean, everybody watched Bond. And so why shouldn’t a little Bond rub off on our reputation?’ The brand, it is argued, does more than just make the service feel good about itself, it also helps with the recruitment of agents who are convinced they are dealing with an all-knowing and all-powerful organisation. If people in the Middle East want to believe that MI6 is pulling the strings behind the most unlikely events, is that really a problem if it means they will come to it when they need help (and do so in preference to another country)? Some believed that the myths of popular culture carried with them dangers. One former Foreign Secretary argued that the image of a service that always did exciting things, always won and was always right, created an exaggerated view of MI6’s potential which in turn fed into government and skewed views, particularly of inexperienced ministers, of what was possible and what was not.

The bleaker world of le Carré divided the service. ‘There were those who were furious with John le Carré because he depicts everybody as such disagreeable characters and they are always plotting against each other,’ recalls McColl, perhaps thinking of Daphne Park and her distaste for him. ‘We know we weren’t always as disagreeable as that and we certainly weren’t plotting against each other. So people got rather cross about it. But actually I thought it was terrific because, again, it carried the name that had been provided by Bond and John Buchan and everybody else. It gave us another couple of generations of being in some way special.’ But with the end of the Cold War, even the old-fashioned spy thriller suddenly looked like a museum piece and the authors were having to search for new plotlines. Who were the bad guys now?

While MI6 did not mind the mythologising of its work in fictional literature, it was firmly determined that its own secrets should remain under lock and key. There was a strict rule that staff could not write memoirs. ‘We have always put quite a big effort into discouraging our retired people from writing books,’ explains McColl. ‘I have a lot of sympathy for them, because if you have been banging round the world for most of your life, unless you have got a very big family or a lot of old friends in the UK you come back here, you retire to some little village somewhere and it is a lonely life. You haven’t got many roots and it’s very tempting to write up some of the adventures you were involved in.’ There was one spectacular case where MI6 failed to discourage one of its own disaffected officers from talking. Richard Tomlinson had joined as a high-flyer (although those who recruited him would later rue failing to take up his references). He was eager and enthusiastic. Perhaps a bit too eager and enthusiastic, thought some of his superiors, who wondered if his notions of how to behave might display a little too much influence from the Bond films. He was also seen as something of a loner. Tomlinson himself believed he was badly treated and not given any due process before being turfed out, which is also more than plausible given the way MI6 operated. Both sides would have cause to regret the breakdown in relations, though, as Tomlinson wrote a book exposing the ins and outs of his time in the service.

The death of Princess Diana in 1997 in a car accident in a tunnel in Paris led to the surfacing of one of Tomlinson’s most awkward allegations. He said he remembered a not dissimilar plan being discussed to get rid of Slobodan Milošević, the Serb leader, earlier in the decade, involving a bright light being flashed at the driver of a car to induce a crash. Could there have been a plot? Eventually an inquiry would be launched which would find no evidence for the claim that the crash was anything other than an accident. It did raise the question of whether there had been a plan for a murder in the Balkans. The inquiry found there had been only the flimsiest notion. A ‘creative’ officer had been worried about a particularly violent Serb (but not Milošević) being on the verge of riding an extreme nationalist agenda to power in elections and accelerating the genocide. The officer compared the situation to Adolf Hitler the year before he came to power, where many lives would have been saved had he been assassinated. ‘It seemed to me that we might be in the months running up to this extreme radical nationalist politician taking power. It seemed to me there might be an analogy with 1932 in Germany. So the question I was posing is whether we should have a plan to take action before the Hitler option actually took place,’ he later testified.17

A few ideas were put down in messy handwriting in the form of a contingency plan, including using special forces and internal Serbian elements to do the deed. ‘It is true that the ethos of the service was against assassination,’ the creative officer would later testify, amid evidence that new entrants to MI6 were told in their initial training that assassination was not countenanced and the subject was not up for debate. ‘Suddenly here I am confronted by a situation where we are dealing with a bloody civil war in the centre of Europe, where tens of thousands of innocent people are being killed. So it seemed to me appropriate that we should at least revisit the dictum of the services and see if we felt obliged to revise it in an exceptional case.’ When he handed the notes to his secretary, she typed them up with some surprise into a page and a half of A4. ‘I had never read or seen anything like it before,’ she later said. He decided to go round his immediate boss, whom he thought might be cold on the idea, and send it direct to the Controller for Eastern and Central Europe. His superiors stamped on the idea, hard. They told him MI6 was not in that business – it was unethical, they said, a view with which he disagreed – and instructed him to destroy all copies of the memo he had written. A senior officer stood over the secretary as she deleted it from the antiquated computer system and shredded paper copies. The days of Anthony Eden and George Kennedy Young planning to bump off Nasser were long gone. ‘We do not have a licence to kill,’ a Chief would later explain, although he hesitated when asked whether the service had ever had one.18

With a secure footing and a shiny new headquarters, MI6 just needed something to do. The shape of an intelligence service, in theory, should be dictated by the threats a country faces and by its concept of national security. But this is rarely the reality. Institutional inertia often means old structures persist even when the threats they were created for have long since passed. This was the case in the 1990s when it took a while for the contour of new threats to be discerned. The old world in which an intelligence service purely and aggressively served the ‘national’ interest by helping secure advantage against other states had not entirely passed away but was being complemented by more amorphous, less state-centric threats to security like international terrorism and groups smuggling and selling nuclear weapons technology.

The new agenda for the spies was in parts familiar and unfamiliar. The Balkans conflict caught MI6 off guard, but the service began to adapt, establishing small teams in the region and working more closely with the military and other partners. Secretly obtaining the negotiating positions of other countries was traditional territory, even if now the targets might be the Serbs in the Balkans or even European allies. Former MI6 officers claim that secret intelligence had a major impact in Britain’s negotiations for the important treaties negotiated that decade. Unsurprisingly politicians refuse to comment on that rather awkward possibility.¹⁹ Economic intelligence was a staple of the Cold War, trying to divine the reality of Soviet economic performance, but now it shaded into the greyer terrain of business and commercial secrets. What were the Germans planning when it came to interest rates and who was bribing whom for arms contracts in the Middle East? More time was spent on dealing with drug barons, working with Customs, and looking for those laundering money in the Caribbean. All of this meant dealing with other government departments and agencies and not just with the Foreign Office. In the old days, officers from MI6 would not even declare themselves within Whitehall, and the intelligence community sat at one side in its own clearly designated compartment. Now its officers were being seconded to other departments and MI6 was increasingly drawn away from pure intelligence gathering. The threats looked global, but MI6 had shrunk in size with the end of the Cold War so it was having to choose more carefully and work more closely with other countries and institutions and not just with the US. There were some Cold Warriors who simply could not make the transition, who could not see the point of it all and who could not operate without the old familiar bearings. In late 1993, McColl instituted what became known as the ‘Christmas Massacre’ in which a raft of older, senior directors were pushed out to be replaced with a younger generation. The following year a youngish new chief, David Spedding, who tellingly had risen through Middle East work rather than Sov Bloc, was appointed.

By the second half of the 1990s, a clearer sense of the new agenda was coming into view, one which occupied more comfortable terrain than the economic and commercial focus of the preceding years. The proliferation of weapons of mass destruction – biological, chemical and nuclear weapons – formed one part. This had been at the cutting edge of work within the service thanks to concerns over the Pakistani scientist A. Q. Khan. From the 1980s, a small group within MI6 had been tracking Khan, who had stolen plans for uranium enrichment while working in the Netherlands and then returned to his native Pakistan to build a vast procurement network to secure the specialised parts in order to build a bomb. Nuclear and other unconventional weapons programmes tend to inhabit the most secret nooks and crannies of reticent states. But the network of largely European businessmen supplying Pakistan offered one way for MI6 to try and get a handle on what Pakistan was up to. This also required talking to other countries, including some in Europe, about what they were seeing, as well as trying to piece together the bigger picture, a picture which looked alarming as the 1990s came to a close.

The second plank of the new agenda was the emergence of a new breed of international terrorism. Terrorism was not new. Palestinian hijackings and later Lockerbie had seen it rise on the agenda in the 1970s and 1980s. But it began to take on a new hue in the mid-1990s, and ambitious officers started to gravitate towards it as they would have done to Sov Bloc work before. There were studies of the threat of radical Islam, but few in MI5 and MI6 had a clear realisation of quite what was brewing, not least on their own doorstep. London, or as the French called it Londonistan, was becoming a home for dissidents and a hub for propaganda including for Osama bin Laden’s allies (the Al Qaeda leader’s claim of responsibility for the 1998 Embassy bombings in Africa came through a fax machine in North London). British officials claim they were alert to a looming threat. The more aggressive American officials dispute that, saying that Britain was only marginally ahead of other European states which showed little appetite for the issue and often seemed only to feign interest because of the American focus. ‘They thought we were as mad as March hares,’ recalls one CIA officer who frequently pushed European counterparts to do more.20

Richard Dearlove became chief of MI6 in 1999. With a few exceptions, the top job has usually gone to the most aggressive operational officers and Dearlove, always ambitious, fitted that mould. He had acquired a reputation as a moderniser keen to make MI6 relevant and to engage with the outside world and other departments. Intelligence had to be useful, he argued. It was no longer about collecting against static targets like the Russian military but had, in the new world, to be about doing things. ‘My career was very much defined by the Cold War,’ Dearlove later told an audience. ‘I cannot really exaggerate the extent to which our preoccupations of national security, hitherto so very firmly anchored, were cut adrift when the Cold War ended.’²¹ He had joined in 1966, a time when the wounds inflicted by Philby and Blake were all too clear. He had risen through Sov Bloc which tended to produce traditionalists, like John Scarlett, who emphasised the purity of intelligence gathering. But in 1987 Dearlove had secured the plum posting of Geneva. This was traditionally one of the key Sov Bloc observation and attack posts targeting the large number of Communists who passed through the city for international summits and UN meetings. During the years of transition at the end of the Cold War, Dearlove broadened out the targets for intelligence collection, integrating technology with human sources. One of Saddam Hussein’s half-brothers was posted there and used the city as Iraq’s European hub. The Libyans and Iranians were also very active, as were the Chinese. Many influential Middle Easterners had summer houses by the lake. This opened up new targets and issues, including the proliferation of weapons of mass destruction and international crime. Dearlove’s strategy was recognised as a successful model of how to address a wider range of security threats. He began to be seen as a leading moderniser and was sent to be head of station in Washington. He encountered a CIA unsure of its mission and haemorrhaging talented officers. It was also penetrated in the form of Aldrich Ames and was soon to go through its own painful polygraph-led molehunts. Dearlove, who had been educated in part in the US, built close relations which would become important later. His ambition for the top job was clear. So, just behind him, was that of John Scarlett, and the two men did not get on.

Scarlett’s and Dearlove’s rivalry was more than personal, it was also cultural, reflecting two strands in the service’s culture. ‘Roundheads and Cavaliers’ was how one colleague put it. The modernising against traditionalist divide had evolved from the time when Shergy was battling to instil a sense of professionalism against the haphazard, amateurish culture that had spawned Albania and the like. Subsequently a divide emerged between Shergy’s Moscow Men and the more adventurous Camel Drivers who focused on Africa and the Middle East and covert action rather than on pure intelligence gathering. To be successful, a Secret Service needs both types: operating in St Petersburg will always be different to operating in Oman, but there is also normally a dominant culture. At the end of the Cold War, there was some resentment at all the talk about the crisis that the Soviet bloc’s passing had caused. It had only ever been less than half the service’s work, grumbled those who plied their trade in the souks of Damascus or on the plains of Africa. Now the Sov Bloc master race came to be seen as relics of the past, relegated to the sidelines. They had become the traditionalists, and the modernisers were those who wanted to do things, work more closely with the rest of Whitehall and make sure their intelligence had impact rather than just collect it. Scarlett was a classic Moscow Man. ‘The buccaneering spirit can be overplayed because that’s part of the myth,’ he later noted. ‘Above all we need people who are disciplined.’22 After ascending to the top job in 1999, Dearlove was beginning to change the service when the world around the service suddenly changed.

John Scarlett had been in his new role as chair of the Joint Intelligence Committee for six days on the morning of 11 September 2001. His rivals at MI6 had hoped that his unusual appointment (unusual because a consumer rather than producer of intelligence normally occupied the post) would keep him out of the running to become chief of MI6 in the future. During a meeting in his office in neighbouring Downing Street someone walked in and told him to turn on the TV. The Twin Towers in New York were collapsing. The Prime Minister was already on his way back from the TUC conference in Brighton.

Within three hours of the attack, Scarlett and the head of MI5 Stephen Lander began to brief the Prime Minister at Downing Street (Dearlove was on his way back from Stockholm). Blair had already framed the attack in dramatic, almost apocalyptic terms in his own mind. It was, he thought, the first salvo in a battle over the future of the world between modernity and fanaticism.23 He immediately asked to see all the intelligence produced on Al Qaeda in the last year. He was handed thirty reports. That night he told the nation that Britain would stand ‘shoulder to shoulder with our American friends’.

Britain’s spy chiefs had known something was coming. ‘The fact that a large-scale terrorist event occurred was not a surprise,’ Dear-love said later. ‘The fear was that it would be an attack probably against American interests probably not in the mainland.’²⁴ ‘We had prior intelligence that summer of Al Qaeda planning a major attack,’ Eliza Manningham-Buller, then the number two to Stephen Lander, recalled. ‘We didn’t know, nor did the Americans, where it was going to take place.’²⁵ Nebulous reports had coagulated and then dissipated over the summer. In June, British and American intelligence held one of their joint summits. ‘The primary topic of discussion was a major terrorist event,’ according to Dearlove. ‘That was a routine meeting which turned into something not routine … There was an increase in chatter [intercepted communications], an increase in indicators.’²⁶ Everyone was fearful. It was as if they were walking through a long, pitch-black corridor knowing that somewhere around them an animal was preparing to pounce. That month, the British passed on details that a senior Al Qaeda figure was planning car-bomb attacks against US targets in Saudi Arabia in the coming weeks. Nothing happened.²⁷ A British report from 6 July read: ‘The most likely location for such an attack on western interests by UBL [Usama bin Laden] and those who share his agenda is the Gulf States, or the wider middle east.’28 A JIC report that month said that attacks were in their final stage of preparation.²⁹ An attack had not been a surprise but its target and scale were.

At the Downing Street briefing on 9/11, Scarlett and Lander told Blair it was the work of Osama bin Laden and that no government would need to have been involved.³⁰ Blair and his press secretary Alastair Campbell pressed them on how they could be so sure. Blair immediately feared that President Bush would be put under pressure to respond aggressively and they all knew that Washington would be looking for any evidence that Iraq, Libya or Iran were involved. A ‘ragged’ meeting of the emergency committee, COBRA, followed in a secure basement room beneath Whitehall. London’s airspace was closed and security at key buildings tightened.

Intelligence that flowed in overnight pointed even more strongly at bin Laden. The newly appointed Foreign Secretary Jack Straw said that the UK should not get ahead of the US. ‘He felt our best role was to stay close and to try and exercise influence privately,’ Alastair Campbell recorded in his diary.³¹ In Washington, there was an awareness that Britain wanted to be involved in any military response in Afghanistan. ‘Give them a role,’ Bush said to his aides on 13 September.³² The instinctive reaction of both Blair and his spies was the same: to get close to the Americans and find out what they were planning and to try and guide them away from any temptation towards unilateralism. Blair wrote the first of many private notes to the American President advising him on some of the diplomatic options.

A flock of spooks was despatched to Washington. Although US airspace would be closed for days, a special dispensation was made for a flight from Britain on 12 September. Richard Dearlove, Eliza Manningham-Buller and Francis Richards, head of GCHQ, all headed to an airbase where an old freight DC-10 was the only plane available to fly them across the Atlantic. The station commander would not allow it to take off. Airspace was closed, he explained. Dearlove let rip. The Prime Minister was calling the President to make sure it took off, he explained, and he would accept responsibility if there were any problems. It took off without clearance to land in the US and with the pilot keeping an eye on the fuel gauge in case he had to turn back. Some seats had been installed in the back, but the plane was largely empty. The small group of passengers remained silent over the journey, lost in their thoughts of what the previous day meant and where it would take them.

As they flew down the eastern seaboard the spies gazed out of the window at the smoke from the burning rubble of the Twin Towers curling up into the sky. When they arrived at Andrews Air Force Base outside Washington they were taken by motorcade to CIA headquarters at Langley. They were greeted at the doorstep by the normally ebullient but now sombre Director George Tenet. The delegation was too big to fit in Tenet’s personal dining room and so they used another executive room, soulless with blue walls and tables covered in crisp white linen. ‘There was an air of surrealism about the whole late-night gathering, as white-jacketed waiters moved quietly around the tables and served us food,’ recalled an American who was present.33 The Americans looked knackered, thought the British. There was not much intelligence to impart but the gesture of solidarity was appreciated by the Americans. ‘The message I was sent to Washington by Tony Blair to deliver’, Dearlove later said, ‘was that we would stand by the United States in their hour of need and we would bring to the table our capability and our assets.’³⁴ Blair’s Foreign Affairs Adviser David Manning, who had been stranded in New York, had joined the team. ‘I hope we can all agree that we should concentrate on Afghanistan and not be tempted to launch any attacks on Iraq,’ he said at the end of the dinner as the officials broke into small groups.³⁵ The relationship with the Americans, so close during the Cold War, had been fraying towards the end of the 1990s. The absence of a unifying threat and focus on more localised enemies like drug barons had meant that the intelligence services were not always pointing in the same direction. Britain had also edged closer to Europe. But with 9/11, old reflexes kicked back in. A CIA official told his counterpart that MI6 could play an important role in acting as a link to other European intelligence services.³⁶

The British contingent returned to their Embassy in Washington to talk late into the night before moving on to the Four Seasons Hotel where they were staying. Among the other guests there was Mahmood Ahmed, the head of Pakistan’s ISI, who had been in town by chance. He studiously avoided the British until they managed to doorstep him in the corridor. He seemed terrified by what it all meant, they thought. For the return journey they agreed to pick up some stranded VIPs. Among them were a group of parliamentarians including former Prime Minister John Major, who started asking why these Britons he did not recognise had a plane at their disposal. The spies drew the line, though, at picking up the Duchess of York, Sarah Ferguson, who was also stranded.

At CIA headquarters, a sense of unreality had haunted the empty corridors on 11 September as all but a small core of staff were evacuated into the car park amid fears of an incoming plane. Mike Scheuer, who had previously run the CIA’s unit tracking bin Laden, had turned on his TV just as the second plane hit. ‘We had a chance to stop this and we didn’t,’ was his reaction. Many who had worked on the Al Qaeda issue reflected that the day’s carnage was the direct result of having been constrained for so long from going after bin Laden with all the aggression he showed in targeting the United States. The insiders were aware in a way that others were not just how much had been known about bin Laden and his plans. They were determined not to repeat past mistakes. These individuals took the attack hard, feeling a sense of personal responsibility. Aggression would be the watchword in their response. ‘The analogy would be the junkyard dog that had been chained to the ground was now going to be let go and I just couldn’t wait,’ recalled the take-no-prisoners head of the CIA Counter-Terrorism Center, Cofer Black, who had personally had a run-in with bin Laden in Sudan. Later he would say, ‘There was a before 9/11 and an after 9/11. After 9/11 the gloves came off.’37

Five days after the attack, Black and CIA colleagues went to the bland, modern British Embassy in Washington late in the afternoon to brief senior MI6 officers still anxious about the American response. The British officials had requested the meeting and they expressed caution about taking actions which might destabilise the Middle East. ‘Our only concern is killing terrorists,’ Black said in his characteristically blunt style during a bleak three-hour meeting. ‘When we’re through with them, they will have flies walking across their eyeballs,’ he would tell President Bush, who relished the tough talk and the promise of swift retribution. ‘All rather bloodcurdling, isn’t it,’ one person present said to another CIA officer as the meeting at the British Embassy ended. Cofer Black told the same CIA officer they would probably all get indicted for the things they were about to do. ‘If you’re going to be an officer of the CIA you’re just going to have to appreciate that if you go on long enough and do a good enough job, at the end of your career, it should involve probably hiring a lawyer,’ he later explained.38 The colourful language and martial metaphors of Black and his cohorts jarred with the style of London’s more old-school spies. Back in Whitehall, every morning the spy chiefs would traipse in, ‘all in dark suits and carrying their battered briefcases,’ noted Alastair Campbell.³⁹ For Blair himself, disaster also spelt opportunity. Members of his own cabinet told him he was seen around the world as the only person who could restrain Bush. A headline in an American paper described one of his speeches as ‘a pitch for world leadership’. For the spy chiefs as well, the thrust into the centre of the action was enough to give them whiplash. Before 11 September 2001, the Prime Minister hardly saw his spy-masters and the Cabinet Secretary, Sir Richard Wilson, recalled doing his best to get them a slot in Blair’s diary in order to build a relationship. But now the position of the intelligence chiefs was transformed. Every day, the Prime Minister would turn to them first in the meetings. The chief of MI6 drew close. ‘The Prime Minister was in the air a great deal of the time going round the world. At least that’s what it felt like sitting at home,’ recalled the Cabinet Secretary later. ‘He had Sir Richard Dearlove with him. Richard Dearlove, who had previously, as it were, not had contact really with Number 10, seized his chance, quite understandably, and got to know the Prime Minister and the Prime Minister got to know him.’⁴⁰ Dearlove would become one of his closest advisers in the months to come, a reflection of the extent to which national security decisions surrounding terrorism and proliferation were informed by intelligence rather than the diplomacy of the Foreign Office, an institution that was seen as lacking the heft and agility required. Intelligence was now determining government policy day to day. The argument seemed resolved in favour of the modernisers with their call for intelligence to be ‘useful’ and as close as possible to the decision-making The events of 11 September 2001 answered the insecurities felt so deeply since the end of the Cold War. For Britain’s spies it was a chance to influence a prime minister directly and to be at the centre of power once more. For that Prime Minister, it was a chance to influence an American president and walk the world stage. No one in London had any conception of where the American President would lead them or at what cost.

In Afghanistan, MI6’s shadow warriors were returning to the fray but without their old partner. Massoud had been killed by the suicide bombers posing as TV crew two days before 9/11, the removal of their most troublesome adversary a gift from bin Laden to his Taliban allies. It was clear that the Taliban had not so much fought their way to power as bribed, bartered and intimidated their way. The coalition strategy would be to mirror that.

On 28 September, the Foreign Secretary approved the deployment of MI6 officers to the region.41 The UK still had people who had been involved with the mujahedeen in the 1980s and who had the language skills and regional expertise. Insiders say the response showed an MI6 strength but also a weakness. When it moved, MI6 could move very quickly, but it moved with all it had and this was not that much. A handful of officers with a budget of $7 million landed in the northeast of Afghanistan at the end of the month. Passing the rusting Russian vehicles blown up during the last jihad, they met General Mohammed Fahim of the Northern Alliance and began working with other contacts in the north and the south to build alliances, to secure support and to bribe as many Taliban commanders as they could to change sides or leave the fight.⁴²

Plucked overnight from retirement in the Cotswolds to be despatched to the wilds of the Panjshir, one of the Britons sent into Afghanistan after 9/11 to negotiate with the Northern Alliance was an old MI6 hand straight out of a John Buchan novel. Paul Bergne, then in his mid-sixties, was an expert archaeologist who spoke at least a dozen languages. He was a tall, gentle, veteran spy who had immersed himself in the culture of every country he visited. In the Foreign Office he was often referred to as ‘Greenmantle’ in reference to one of John Buchan’s novels in which the gentleman-adventurer Richard Hannay investigates an uprising in the Muslim world. Bergne was the type of polymath who inhabits the nooks and crannies of the service often largely forgotten until the next crisis hits their region of expertise. With a detached, quizzical air and an independent streak he had joined MI6 in 1959, first serving overseas in Vienna before becoming the in-house expert on Islam and Central Asia. After he retired in 1992 he was appointed Britain’s first ambassador to Uzbekistan and Tajikistan where he initially ran the Embassy from a messy, cheap hotel room. His second retirement, largely spent in academia, ended as he was asked to go to Afghanistan to persuade the Tajiks of the Northern Alliance to do more than use Western intervention to regain the upper hand in their old feuds, not least with the Pashtun. When the Northern Alliance took Bagram airbase, he had to intervene personally to stop them firing on British troops as they landed because no one had expected them. The Northern Alliance, he said, ‘came within an ace’ of opening fire on the Britons. ‘I asked the foreign minister not to take any hasty action, because he was extremely angry,’ Bergne later said. ‘They had been sorely tempted to open fire.’43

At the CIA, Cofer Black’s instructions to the team leader he was sending to Afghanistan were simple and grisly when it came to Al Qaeda’s leadership. ‘I want to see photos of their heads on pikes,’ Black told Gary Schroen. ‘I want bin Laden’s head shipped back in a box filled with dry ice. I want to be able to show bin Laden’s head to the President. I promised I would do that.’ It was the first time Schroen had heard a direct order to kill. ‘We can certainly kill bin Laden, but I don’t know where we’re going to find dry ice in Afghanistan,’ he replied. The strictures put in place after Larry Devlin’s testimony before Congress in the 1970s were gone, lifted by a president who demanded action and supported by an agency which did not want to let the American public down. ‘The authorities had changed,’ Black would explain. This was war. The pendulum was swinging once again for the CIA, this time violently.⁴⁴

Schroen had been pulled out of a retirement programme because he had served in South Asia on and off since the late 1970s and had been the link man to Massoud in the late 1990s. He got hold of an ageing Russian helicopter and $3 million in hundred-dollar bills and headed towards Northern Afghanistan, one of the worries being the danger of being shot down by a shoulder-fired missile of the type that the CIA had supplied years earlier. He was joined by a team of CIA paramilitary officers. ‘If they didn’t do paramilitary actions for a living, they’d probably be robbing banks,’ one CIA officer said of their ilk.⁴⁵ After they landed they hooked up with the Northern Alliance, still shell-shocked from the brutal murder of their leader. There was no master plan to co-ordinate the work of the CIA and MI6 teams. Everything was moving too fast. Everyone just needed to do what they could and make sure they did not run into each other. Conversations were simply about who had assets where. Most British contacts were with the Northern Alliance but with a sprinkling in the south, where the Americans were also struggling to find viable allies. The failure to find a partner in the south was slowing plans and causing problems. Soon the US and British teams were providing intelligence reports on Taliban front-line positions and mapping their defences for a bombing campaign. ‘Are you going to stay this time?’ the Afghan allies repeatedly asked the CIA’s overall Afghan commander when he arrived. ‘Yes. We are staying this time,’ he promised.46

At a meeting at a military bunker, the concerns of the Pakistani leadership were evident. The Taliban had been their proxy. The wily President Pervez Musharraf, who had seized power in a coup, told his assembled senior commanders that they had to make a choice. Abandon their old allies, the Taliban, or else they would be steamrollered by the US. To make the choice easier he moved key generals, including his ISI chief Mahmood Ahmed, the man who had looked so worried at the Four Seasons. Ahmed had been particularly close to Mullah Omar, the Taliban’s one-eyed leader, and wanted to preserve the group’s influence. ‘In terms of the policy, it really did turn 180 degrees,’ recalls Bob Grenier, then the CIA chief in Islamabad.⁴⁷ But, with the decision made, the Pakistanis explained to Grenier that they were deeply worried that the Northern Alliance would rout the Taliban and take Kabul. They warned the Americans that this would cause bloodshed and risk excluding the Pashtun population of the south. Disputes raged in Washington and especially in the CIA over whether or not to listen to this advice and restrain the Northern Alliance. Post-Massoud, the collection of leaders at the top of the Alliance did not look appealing. Most looked like thuggish warlords because that was what they were. Washington agreed to hold back from bombing the Taliban front-line positions until a more coherent Pashtun resistance could be organised in the south to complement the largely Tajik Northern Alliance.

This was a chance for the ageing Afghan jihadists of the 1980s to choose sides – the Taliban or the Americans and the Northern Alliance. Old faces re-emerged from the woodwork seeking one more shot at glory. The one-legged Abdul ‘Hollywood’ Haq, once fêted in Downing Street, managed to garner some support from Afghan-American businessmen and rode into battle once more heading from Peshawar to Jalalabad with a group of twenty supporters. An unarmed US Predator drone watched as he was surrounded by Taliban forces and executed. Hamid Karzai, another Pashtun whose father had fought the Taliban, was nearly killed by a misdirected bomb. He would be one of the only Pashtun leaders to sign up with the Americans. That would stand him in good stead. Some of the other commanders the US had once backed now sided with their enemies, Gulbuddin Hekmatyar among them. Another was Jalaluddin Haqqani, who had been Congressman Charlie Wilson’s favourite and yet became one of the most deadly adversaries of the new Afghan government and the closest local leader to Al Qaeda (while remaining so close to the ISI that it considered him one of its assets).

For a tense few days in October it looked as if air strikes were not having any impact and the bombers started to run out of targets. There was talk in the press about the ‘Afghan Winter’ approaching. But once fire was finally focused on the Taliban positions in the north they crumbled, allowing the Northern Alliance to sweep into Kabul. In all, only around 100 CIA and 300 US special forces were sent to Afghanistan, but within two and a half months of deploying on 27 September and by working with air power and local forces they were able to topple the Taliban. But where was the man whose head was supposed to be delivered on dry ice?

Britain’s SBS alongside MI6 was sent into the wild, mountainous Tora Bora region to hunt for bin Laden alongside US teams. Several members of a SBS team listened to bin Laden speak to his fighters on shortwave radios that had been captured. Two of them at one point spotted a tall figure in a camouflage jacket move south-east with a fifty-man protective detail and enter a cave through a hidden entrance. The Americans and British, accompanied by Afghans of dubious loyalty, were few in number and their request to the Pentagon for back-up was turned down.48 By failing to commit large numbers of ground troops to secure the border, the US allowed bin Laden to slip away into Pakistan. ‘Massoud liberated Afghanistan from the Soviets,’ declared his former aide Abdullah Anas. ‘Osama bin Laden gave it to the Americans in two weeks and then fled.’49 Bin Laden was able to regroup his organisation over the border in Pakistan.

Once again, the view in Washington was ‘job done in Afghanistan’. Soon the US began withdrawing special forces teams and preparing them for their next war. Nothing much had followed behind the military, no great aid or development or attempt at institution building. Some in Washington, ignoring the key role played by Afghan allies, drew the over-confident conclusion that small teams combined with air power could win wars without a large body of ground troops.

The rout of the Taliban meant that prisoners were swept up into makeshift and ageing jails, including many foreign fighters who had trained in the camps and fought against the coalition. In mid-December the MI6 officers who had been deployed to the region began to interview prisoners held by the Northern Alliance. In January they turned to interviewing those held by the Americans. In neither case was the decision to interview referred up to ministers in London. On 12 December it had been agreed in London that MI5 officers should also be sent out to interview prisoners who might possess intelligence on attacks against the UK. The first MI5 staff arrived at Bagram on 9 January 2002. The following day an MI6 officer conducted his first interview of a detainee held by the US. He reported back to London that there were aspects of the way the detainee had been handled by the US military before the interview that did not appear consistent with the Geneva Conventions. Two days after the interview he was sent instructions, copied to all MI5 and MI6 staff in Afghanistan, about how to deal with concerns over mistreatment. ‘Given that they are not within our custody or control, the law does not require you to intervene to prevent this,’ it explained, referring to signs of abuse. It went on to say that the Americans had to understand that the UK did not condone such mistreatment and that a complaint should be made to a senior US official if there was any coercion by the US in conjunction with an MI6 interview. The instructions ended with a no doubt happily received lawyerly warning that ‘acts carried out overseas in the course of your official duties [are] subject to UK criminal law. In other words, your actions incur criminal liability in the same way as if you were carrying out those acts in the UK’.50 The instructions, it was later found, were inadequate, in that they failed to require officers to report their concerns immediately to senior US officials and to London. In the next three weeks before he came home the MI6 officer saw no other signs of mistreatment. But it had not been an isolated incident.

The gloves had come off. On 7 February President Bush made clear that the US did not consider the Geneva Conventions applied to detainees arrested in Afghanistan and they were to be sent to Guantánamo Bay in Cuba. There were more signs of abuse. In April an MI6 officer was present at an interview conducted by the US military and raised concerns with a US officer. In July an MI5 officer said he heard a US official talking about ‘getting a detainee ready’, which appeared to involve sleep deprivation, hooding and stress positions. When he reported it to his senior management they did nothing. It would not be surprising if officers in the field felt they were getting little support from back home.

Britain’s closest ally was, as its vice-president Dick Cheney put it, working ‘the dark side’. The top Al Qaeda prisoners who were picked up in the War on Terror were taken on a journey to ‘black’ sites, secret locations in Eastern Europe and Asia where pliant governments had allowed the CIA to harbour secret facilities. There the agency could engage in acts everyone but the administration itself would describe as torture. The CIA was becoming, in effect, a military command as well as an intelligence agency which would be engaged in everything from interrogation of detainees to firing guided missiles to killing Al Qaeda leaders, creating an awkward situation for an ally who worked closely with it.

How much did British intelligence know about their cousins’ walk on the dark side? Very little, they claim, which in itself points to an intelligence failure. ‘It took us longer than it should have done to find out,’ accepts one former official. There were warning signs early on, however. ‘I was worried by the reaction that one heard – which was a very emotional reaction – in the weeks after 9/11 from a number of American officials,’ Dearlove said later. ‘We tried even at that stage to point out we had lived with a serious, different type of terrorist threat. We had gone down a track which had caused great problems. I’m thinking of internment in Northern Ireland. And we had learnt many hard lessons … our position was if you forfeit the moral high ground in confronting these types of problems you make it maybe much more difficult. You maybe gain a short-term advantage, you certainly lose long-term advantage.’51

Britain and America share most but not all secrets. The secret rendition programme in which high-value Al Qaeda suspects were tortured was highly classified, but it was still clear that something was happening to these prisoners in the years after 9/11 because they had disappeared. What’s more, the UK was happily receiving intelligence reports based on their interrogation. A ream of very interesting reports began to reach Britain in 2003. The material was gold dust but the Americans were saying very little about its source. Was it a newly minted agent inside Al Qaeda’s leadership? some in British intelligence wondered. Soon it became clear that material was coming from the self-confessed ‘mastermind’ of 9/11, Khalid Sheikh Mohammed, who was spewing out details of plans and plots and plotters. ‘I said to my staff, “Why is he talking?” because our experience of Irish prisoners and terrorists was that they never said anything,’ Eliza Manningham-Buller recalled.⁵²

Another intelligence chief wondered if Mohammed had been turned to work with the West. What had really happened was that he had been waterboarded, a technique which makes the body think it is drowning, a total of 183 times. ‘The Americans were very keen that people like us did not discover what they were doing,’ Manningham-Buller said. ‘One of the sad things is Cheney, Rumsfeld and Bush all watched 24,’ she added, referring to the TV drama in which an American spy uses extreme techniques to extract information and stop attacks.⁵³

As they processed the leads supplied from Washington, MI5 was relieved to find that it already knew about half the people Khalid Sheikh Mohammed had identified in the UK. But the remaining half were new leads which were hurriedly pursued. An urgent hunt began for one of those he helped identify named Dhiren Barot, a key Al Qaeda figure in the UK, who proved well trained enough to lose his surveillance team temporarily. Eventually he was picked up, but the FBI was furious he was not handed over to it.

When MI5 officers interviewing detainees in Guantánamo Bay became concerned over mistreatment in 2004, protests travelled through intelligence channels and also through Downing Street and the Foreign Office.54 But the spies also say that the leads that emerged out of torture in the black sites were important for national security (although not quite to the extent President Bush later claimed when he wrote that they directly prevented attacks on Canary Wharf and Big Ben). Ask them quietly and a few of those who have worked at the heart of the secret state will whisper that the idea that torture is never useful and always produces flawed intelligence is too easy a truth to cling to. They will say that Algeria, whose methods were brutal, was an important partner after 9/11, providing vital intelligence. They will say that you never really know how intelligence you are handed was produced. And, they ask, can you imagine the witch-hunt and the blame game if there was to be an attack on Britain tomorrow and it was found that intelligence on it from, say, Saudi Arabia had been declined because of concerns over its provenance? But while passive receipt of intelligence is one thing, the moral terrain becomes even more treacherous when you are watching interrogations on video monitors from neighbouring rooms (as Pakistani officials say occurred with some detainees) or sending questions over to the Americans to be put to someone you know has been spirited away somewhere secret, as happened with British resident Binyam Mohammed, held in Pakistan and then in a secret prison in Morocco. In his case there was sufficient concern over wrongdoing to justify opening a criminal investigation into complicity with torture. ‘No torture and there is no complicity with torture,’ John Scarlett said, defending MI6’s actions. ‘I have every confidence and always have had every confidence in the standards, the values, the integrity of our officers.’ The relationship with the Americans had nonetheless experienced tensions, he acknowledged. ‘Our American allies know that we are our own service, that we are here to work for the British interests and the United Kingdom,’ Scarlett later said of the relationship. ‘We’re not here to work for anybody else and we’re an independent service working to our own laws – nobody else’s – and to our own values.’⁵⁵ Trust between the two countries’ intelligence services began to erode as British inquiries unearthed and made public details of American techniques, breaching the ‘control’ principle in which a country which originates an intelligence report maintains control over where and how it is passed on or released. Some Americans fumed at the way in which European governments distanced themselves from American policy on detainees while at the same time using the intelligence produced. Speaking before Congress, Mike Scheuer, the former head of the CIA’s bin Laden unit, lashed out at ‘effete sanctimonious Europeans who take every bit of American protection offered them while publicly damning and seeking jail time for those who risk their lives to provide the protection’.56

The British and Americans have long had a classified no-spy agreement which says they do not collect secret intelligence on each other and do not recruit each other’s citizens as agents. This is partly about sovereignty and partly about the problems of parallel intelligence operations pranging into each other as they go after the same agents. The British can, however, occasionally run unilateral operations inside the US against other nationals with US approval (approval which is extremely rare, although some US officials suspect that Britain may have run unilateral operations without approval against the IRA at various points). The US does not have the ability to run approved unilateral operations in the UK, much to the annoyance of some CIA officers (very occasionally they have been caught evading this). After 9/11, some senior American intelligence officials pushed hard to run their own operations inside the UK to collect intelligence. London resisted strongly, fearing not least that the Americans might end up carrying out rendition operations as they did in other European countries like Italy in which suspects were snatched off the street. One former CIA official recalls a meeting at which the head of MI5, Eliza Manningham-Buller, never one to be shy of expressing her opinion, resisted US unilateral operations particularly strongly. ‘Try it and we’ll arrest you,’ the CIA man recalls her saying. Another CIA official at the same meeting came out irritated by what he saw as a patronising lecture about how Britain was a country of rules where the Magna Carta had been written and fumed to his colleague about being told what he could or could not do by a ‘medium-sized power’.⁵⁷

When a man in Birmingham is communicating with a terrorist planner in Karachi the distinction between foreign and domestic information quickly blurs. Streams of information from different sources need to be integrated and trails of data mined and manipulated to establish how someone might be linked to a network. Staff from MI5 and MI6 were thrown together in joint teams alongside colleagues from GCHQ who pluck terrorist communications out of the ether. MI5 found itself co-ordinating closely with the police with whom it used to have an often difficult relationship. The days of bitter rivalry between MI5 and MI6 had passed, although the odd clash still occurred. Most of the difficult entanglements, particularly the case of Binyam Mohammed, centred on the Security Service, MI5, rather than on MI6. After 9/11 senior MI5 officials had been keen to take the lead on all intelligence which related to threats to the UK. Even if detainees were held abroad, they wanted to carry out the interviews. In the past, MI6 officers used to joke that their counterparts should never be allowed beyond the Straits of Dover because their understanding of the ways of foreign intelligence services was so limited. There used to be a running joke about the telegrams MI6 officers in a faraway country received from MI5 in London which was looking for help from the local intelligence service in some investigation. ‘Please instruct your liaison …’ the telegrams would begin before going into the details of the request. ‘You weasel, you cajole, but you don’t instruct liaison partners,’ explained one old hand from MI6. But there was no Schadenfreude at the mounting allegations faced by its sister service, not least because it coloured the reputation of all of British intelligence. Paradoxically, public perception and reputation is remarkably important for those who work in the secret world. It is precisely because they cannot talk much about their work that they worry more about the ways in which it is perceived by the public.

MI5 also had to deal with its fictional portrayal in the BBC drama Spooks, which began broadcasting a few months after 9/11. ‘I don’t think the public think it’s like Spooks. I think they realise Spooks is fiction in the same way as they know that James Bond isn’t like MI6,’ says Manningham-Buller, who claims not to have watched the programme since the first series and the unfortunate demise of a female officer in a vat of boiling chip fat (the programme was thought to have led to a drop in female applicants because of its violence). ‘There are two regrets I have. One is that it portrays intelligence as simplistic, as a simple thing to be understood if you only do things in the right order – that things can be solved in forty minutes by six people. The second thing I regret about it is it portrays the service as having utter disregard for the law. Whereas we are very careful that everything we do has a proper legal basis.’58 The hits to the MI5 website, and particularly its recruitment pages, surge after every episode of the drama, but insiders are less sure that the Bondish image of gun-toting, rule-breaking secret agents is as helpful to a domestic security service trying to investigate its own citizens as it is to a foreign intelligence service, like MI6, out trying to do bad things to foreigners. If Spooks was like real life, one MI5 officer explained, the camera would cut to the officer at 3 a.m. still filling in his warrant form and all the associated paperwork before being allowed out the door.

At times the pace of a counter-terrorist investigation is not far off that portrayed in Spooks. Intelligence gathering can involve some of the same techniques as were used in the old Cold War world – human motivations are much the same when it comes to recruiting agents who know the secret intentions of an enemy. But the differences are also stark. Intelligence is much more time-sensitive. Discerning the Soviet order of battle could be pieced together slowly. Intelligence about a planned terrorist attack or the location of a terrorist leader requires immediate action. The shelf life of good intelligence in counter-terrorism may be days rather than years. The shelf life of an agent in the badlands of Pakistan, whom it might have taken a long time to recruit, may be much shorter as well.

MI5 officers in early 2004 watched grainy surveillance of a terrorist cell inspecting fertiliser for a bomb in a storage facility and listened to the men in their bugged cars and flats talk of the nightclubs and shopping centres they were planning to attack. One night in March, they recorded the driver of a Vauxhall Corsa picking up the ringleader of the plotters and another man in Crawley and driving them around before taking them back. The driver was identified in the log as ‘UM’ – unidentified male.⁵⁹ It was clear that ‘home-grown terrorism’ had arrived on British shores. These were second-generation Britons of Pakistani descent. It was a trend that had not been appreciated in the immediate aftermath of 9/11. In the 1990s, the radicals who had found a home in Britain were seen as plotting against their homelands of Algeria, Egypt and Saudi Arabia. Arab intelligence services would privately describe the UK as a ‘terrorist sanctuary’ to CIA officers, but despite those complaints a blind eye was turned by the British as long as individuals were not breaking any law or plotting against Britain. The 2004 investigation, codenamed ‘Crevice’ by the police, revealed for the first time that Britons would attack Britain, in this case with support from an Al Qaeda which had been ejected from Afghanistan but had been able to regroup in the wilds of the tribal areas of Pakistan. ‘Crevice was the moment when the lights went on and you could see the state of the kitchen,’ explained one intelligence official soon afterwards.60 There was worse to come. Home-grown radicalisation was now on the radar, but still no one expected British suicide-bombers to strike.

The first that staff at MI5 knew of what was happening on 7 July 2005 came as they watched their TV screens. Soon the images of a blown-apart bus opened like a tin can would be seared on their minds. The UK’s threat level had been lowered a few days earlier. No one had seen the four young men with their rucksacks coming. ‘We did not know it was a suicide bombing until the forensics began to come through,’ recalled Eliza Manningham-Buller. ‘So at the beginning we were trying to support the police in possibly finding the team who had done it, who for all we knew at that stage were still alive and capable of mounting another attack.’⁶¹ The aftermath was chaos. At one point closed-circuit TV from Luton, where the bombers had passed through, made it seem as if there was another person carrying a rucksack. Should they tell the public? Someone from MI5 thought they saw the same person from the CCTV around Westminster and Buckingham Palace. Everywhere was locked down. ‘It wasn’t until the evening when I got home quite late that the emotional impact of that day hit me,’ recollected Manningham-Buller later.⁶²

Eliza Manningham-Buller addressed her staff the next day inside MI5 headquarters at Thames House in Millbank. She said it had been a day they had always feared would happen but hoped would not. Many had been up all night and she said she was proud of them. They needed to continue to do what they had been trained to do. Brace yourselves, she also warned; by the end of the week there would be speculation about whether we were to blame for allowing the attack to take place. Don’t read the papers, get on with your job. The accusations would indeed come. The night of her speech, the credit card of Mohammed Siddique Khan was found close to where the bomb at Edgware Road tube station had detonated. MI5 investigators soon realised that they had come across him before. He was the unidentified male who had been driving the Vauxhall Corsa in March 2004. He had come under MI5 surveillance on multiple occasions as part of the Crevice investigation but had never been prioritised sufficiently to be followed up. There had been a number of leads dating back to 2001, including that Khan had attended training camps in the UK and Pakistan, but the different strands had never been woven together to understand that all the pieces of intelligence related to the same man. The explanation for not placing him higher up the list of targets was a lack of resources, especially as a new set of leads had arrived at Thames House in 2004 (including those generated by Khalid Sheikh Mohammed’s waterboarding). ‘We’re not the Stasi, we can’t cover everyone,’ explained one official defensively.63

Two weeks to the day after 7/7, there was another attack. This time it failed. ‘For me, it was worse,’ recalled Manningham-Buller, who was having a regular lunch with her directors when news came through. ‘Although nobody died, I had that feeling that if this is going to happen every fortnight, how are we going to be able to cope with this?’⁶⁴ The fear of an endless army of suicide bombers coming over the horizon filled everyone with apprehension. Authorities were unsure that they would be able to cope as they circled the date two weeks ahead and wondered if something would happen then. For the coming months and years, it felt like ‘trench warfare’. ‘It’s like the old game of Space Invaders,’ explained one person involved in the day-to-day work of counter-terrorism. ‘When you clear one screen of potential attackers, another simply appears to take its place.’⁶⁵ The days of playing investigations long, watching them patiently in order to build up enough evidence, were disappearing. That was too resource intensive. Disrupt and move on to the next network was the only game-plan available. ‘We had more than we could cope with and had to make some uncomfortable decisions on prioritisation,’ explained Manningham-Buller. Government money soon came flooding in and MI5 more than doubled in size and expanded into the regions. A transformation from the old counter-espionage agency of Arthur Martin to a modern counter-terrorist organisation which had begun with the tackling of the IRA in the 1990s was now under way.

MI5 launched a blitz, recruiting new human sources on an ‘industrial scale’, working its way through lists of people who could be approached. ‘We knew that we needed many more human sources and much greater coverage,’ said Manningham-Buller. These agents remain the lifeblood of intelligence work. For all the talk of ‘community intelligence’, insiders say that there is no substitute for agents within the organisations themselves. ‘Like the IRA you don’t get it [the intelligence] from housewives. To get to terrorist planners you need people to get to people in the middle. The sort of detailed information to stop an attack isn’t swilling around the mosque.’

After 7/7, Manningham-Buller approached the Chief of MI6 to ask him to lend her his best agent runners because running agents has traditionally been more of a domain of expertise for MI6 than MI5. At one point, the largest deployment of operational MI6 officers was not to any country overseas but within the UK. They would work alongside MI5 officers to try and recruit agents who could be sent into Pakistan’s tribal areas and elsewhere to infiltrate Al Qaeda and its allies. Getting into the heart of networks takes time. The ambition, just as the KGB had for Philby, is to find someone who can survive inside the enemy’s ranks, prosper and rise. The higher they rise, the more complex their moral choices may become. When does revealing a terrorist plan to your handler endanger your own life? When does going along with a terrorist plan involve innocent people dying? Agent motivation is infinitely variable. For some it is the desire to prevent violence in their community, for others it will be a grudge against someone they know. The approach to an agent is always the hardest moment. Typically an officer has between half a minute and a minute in which to keep someone interested or lose them for ever. A bit like an approach at a nightclub for a different type of assignation, a good opening line and the right manner can go a long way. When approaching someone to be an agent, the first ten seconds are typically lost anyway as the subject is in shock at the officer revealing their hand.

Amid the danger and excitement also lies the mundane. MI5 agents and their handlers can spend up to half a day conducting complex counter-surveillance routines to make sure they are not followed when they are preparing to meet. The initial thrill of feeling as if you are in your own spy film is quickly replaced by the boredom of having to jump on and off yet another bus but with the added edge that getting it wrong could cost someone their life. The process could end up with the agent climbling into a covert vehicle. These are hardly the height of luxury – some have no windows in the back and a small light illuminates what looks like carpet peeling off the walls. There are no seatbelts and no air conditioning or heating. In an emergency, a meeting can take place in a vehicle but otherwise the agent may be driven, by a circuitous route, to a lock-up somewhere where a half-dead yucca plant sits in one corner and his case officer waits with a cup of tea. The first question is always the same – ‘How long have you got?’ If the answer is only fifteen minutes then they will cut to the chase. If there is longer then there will be time to talk about the agent’s welfare and whether they have any security concerns or other needs. The idea is to make them feel confident. ‘The main aim is not to extract intelligence. If that is your top priority and all you do then it goes wrong,’ explains one former agent handler. The art lies in building a relationship and establishing empathy and sympathy while always remaining in control.66 A room full of MI5 agent handlers would look like a cross-section of British society from old men to young women, black and white, skinheads and men in suits, one for each occasion and for each type of agent.

In a modern warehouse somewhere in central London, a young Muslim man sits, observed by his MI5 handler. A small heater keeps the room warm and a cup of tea sits untouched on a coffee table. A driver stands by the vehicle just outside, ready to organise a fast getaway in case of any sign of trouble. ‘I don’t see myself as a spy in that sense anyway because I am just fulfilling my duty and my right as a Muslim citizen – you know keeping my eyes out,’ the Muslim agent working for MI5 explains.⁶⁷ The bearded man could still recall his first nerve-racking meeting with MI5, the fear that he might be a suspect, the relief quickly followed by a new anxiety when he was asked to inform. ‘I’m not going to go and spy on Mohammed buying halal sausages in the local store because that’s not what I do. I would never do that. But if I know there is a bunch of Muslims who intend to do something and I hear about that or find out about that in whatever way then, yes, I was more than willing to see them again and talk about that and if they [MI5] ask me, to query more about these individuals. I felt that that’s OK because these lads are not just the normal Mohammed or Abdullah praying in the mosque … I don’t want anyone to blow up a bomb next to me or blow my family up … so I don’t feel that this is against the Muslims in that sense. It is against the criminals.’68 The agent meets maybe three or four times a month with an MI5 officer, sometimes for half an hour, sometimes for two or three hours with a twenty-four-hour number for emergencies. ‘I never thought I would have personal contact with MI5 but it’s not like the TV. It’s not like the movies, yeah, so I don’t feel like I’m in some kind of Bourne Ultimatum … Some of them look a bit dodgy, some of them are ugly, some of them are nicer. You know, that’s how it goes you know. Men and women, big and fat. You know. Thin, chubby. Black and white.’

The agent’s work includes trying to stay close to people, although he maintained he was never pushed to do anything by MI5. ‘The pressure is not really from them. The pressure is when you are out there on your own … It’s not comfortable sitting there with a terrorist or a criminal or whatever it is. You don’t want to associate with these kind of people. So sometimes when you do sit there, yeah, it’s a little bit of adrenalin kick there … I don’t really see them as practising Muslims. I see them like any other criminal, any other gangster out there who is just using the name Islamic or Muslim … if someone’s going to blow up a place and I find out about it, I’m going to make sure someone is going to stop it because that’s harmful to Islam. Because that’s what now people are going to think, “Ah Muslim, the terrorist, Muslims they’re barbaric, Muslims they’re this, Muslims are that,” and that is not the true picture of Islam and if I can stop that before it happens then I am happy … there are limits to what I am willing to do. If I was asked, and I mean that, if I was asked to do something specific against specific Muslims and they’re not really any threat to society or threat to anyone then I would never do anything like that because that’s not right, because that’s not straight down like that. I would never do it.’⁶⁹

The more agents it recruited and the harder it looked, the more threats MI5 found. In 2003, it was watching thirty networks linked to terrorism; by 2004 it was fifty. By November 2006 it had suddenly climbed to 200 groups with 1,600 individuals. Thirty of these groups were actively planning attacks.⁷⁰ There was no telephone directory of Al Qaeda officials outlining its structure of the type Penkovsky had supplied decades earlier. The enemy was fluid and constantly evolving. It felt like chasing a will-o’-the-wisp.

At a lonely secret CIA base near the Afghan border with Pakistan, the real dangers of agent running became clear in December 2009 along with the ability of Al Qaeda to learn the old games of double agents. Al Qaeda is always on the look-out for plants from Western intelligence and demands references from known radicals. Recruits can be quickly exposed to extreme violence, perhaps witnessing the beheading of someone carrying a mobile phone, to send a message. Parts of Al Qaeda, especially those like the older generation of Egyptians who had grown up battling a state security service trained by the KGB, were well versed in counter-intelligence and the organisation has tried to send some people into MI5 as penetration agents (a greater concern might be someone who joins in good faith but is then pressured due to family or personal ties). By late 2009, Al Qaeda was able to replicate old-style Cold War techniques and run a double agent against the CIA. Jordanian intelligence approached a well-known radical asking him to go to Pakistan to contact Al Qaeda’s senior leadership. He was run jointly with the CIA, which had high hopes he might be able to locate Al Qaeda’s number two. But when he came across the border from Pakistan to a meeting at the CIA’s base in Khost, Afghanistan, he blew himself up. Among those killed was the commander of the CIA base, a brave, dedicated and experienced woman with a long track record of tackling Al Qaeda.

On 2 May 2011, the CIA would at least have its vengeance for her death and the three thousand others killed nearly ten years earlier. Osama bin Laden was tracked, through one of his couriers, to a compound in Abbottabad, Pakistan, using a combination of human and technical intelligence. A team of US Navy Seals landed by night and killed the Al Qaeda leader; his body was disposed of in the sea a few hours later. A man who had eluded western intelligence for so long and who had been assumed to be hiding in the wilds of Waziristan had been watching satellite TV in a comfortable house close to a military garrison. Pakistan was not informed in advance for fear of jeopardising the operation. The first MI6 knew of the mission was when its Chief was called by his American counterpart to explain what had just happened.

MI6 had reorganised after 9/11 and reshuffled its staff, opening new stations overseas, with Islamabad becoming its largest base. MI6’s uptick in funding was not as large as that for MI5, but it still struggled to recruit fast enough. Old hands were rehired to help out. It took a while for it to be clear how it could help most effectively in countering terrorist threats to the UK. Eventually it focused on chasing leads overseas and upstream to relieve the pressure domestically. This included maintaining the intelligence coverage of suspects as they moved from the UK overseas, particularly to Pakistan. It was no good following someone every day in Britain if you lost them the minute they set foot in Karachi. But knowing what they were doing required the help of a sometimes already stretched liaison partner who had to be carefully massaged. This was always particularly difficult with Pakistan’s mercurial ISI. Sometimes help could be enlisted overtly, sometimes through a secret relationship, having a local official on your payroll to check the records. One problem is reciprocity. A Pakistani general might offer his help in catching a British Al Qaeda suspect in his backyard – but only if a Baluchi nationalist in London is sent the other way in return. Working with the ISI was always complex, not least because it cared little for foreigners’ concerns over human rights. ‘We’re not worried about that,’ one Pakistani official explained when asked about the allegations of torture in the Western media. ‘We’re not afraid of the third degree.’71

The task of facing terrorism was daunting in terms both of scale and of the moral chicanes to be navigated. It required close working with allies, some of whom played by different rules. There were mistakes made and lessons learnt. It did though provide a new raison d’etre for the spies and seemed to answer the question of what they were for in the new world. But for British intelligence the years after 9/11 witnessed another crisis, one that shook them to the core and that exposed the deep and bitter tensions over their new role.