Dilly’s greatest triumph in cryptography and intelligence, which for him were always inseparable, was the breaking in October 1941 of the Enigma machine used by the German military intelligence service, the Abwehr. Little was known about the Abwehr organisation before the war. In March 1940, two members of the Radio Security Service (RSS), charged with intercepting illicit wireless in the UK, broke some simple hand ciphers from a Hamburg-controlled station, which at first aroused little interest. Finally Hugh Trevor-Roper and E. W. Gill, the latter a leading member of the Army’s First World War codebreaking organisation, had succeeded in attracting Bletchley Park’s attention and Oliver Strachey was asked to set up a section to be called Illicit Services Oliver Strachey (ISOS) in Elmer’s School to research this traffic. By December, Strachey had broken the main hand cipher, enabling Bletchley to give MI5 advance warning of the arrival of German spies. These individuals were rather different from the disorganised refugee fifth columnists I had inadvertently befriended in 1938, when there was still free access across the Channel. It was now apparent that the Abwehr was operating a considerable professional espionage network across Europe.
After the fall of France and Hitler’s plans to invade Britain, Operation Sea Lion, were drawn up, the agents being infiltrated were meant to be an advance guard to wait for the arrival of German troops and to report on morale. However, in 1941, when the threat had receded, there was a reorganisation of our security and the RSS (largely still amateur ‘ham’ wireless enthusiasts) was put under direct control of SIS. Dick White, a future head of both SIS and MI5, covering both espionage and counter-espionage, suggested that captured agents should be left in place and ‘turned’ to work as double agents. Camp 020 was set up for interrogation at Latchmere House, Ham Common, Richmond, to identify suitable double agents who could send back false information to their unsuspecting controllers.
MI5 managed the controlled agents and a London clearing house was set up to co-ordinate operational activities; it was known as the Twenty (XX) Committee, hence Double Cross, under John Masterman, a future vice-chancellor of Oxford University. The double agent was to send out his false information on his wireless transmitter as directed by his case officer, using his own hand cipher given to him by the Germans. Their controllers in neutral capitals, mainly Lisbon and Madrid, would receive and analyse the messages before transmitting the information to Berlin on the Abwehr Enigma machine.
Before this high-grade Enigma traffic was recognised for what it was and thought to be on the services machine, it was sent to Gordon Welchman in Hut 6 but he could make no headway and it was given to Dilly as unknown Enigma research. After Matapan the Italians mostly used their Hagelin machine; the Enigma machine traffic was infrequent and was turned over to routine production outside the Cottage, so that Dilly was able to give the Abwehr material his undivided attention. The Knox family had always been fascinated by spies and Kim’s ‘Great Game’. Dilly spent some time over at the School with his friend Oliver Strachey (the brother of his good Cambridge days friend Lytton) learning about the organisation of the Abwehr. First of all the many different spy networks had to be sorted out covering Spain, Portugal, the Balkans and Turkey to see how the ISOS hand-ciphered messages related to Dilly’s Enigma messages sent on from the neutral capitals to Berlin after December 1939. It was hoped that there would be good cribs from the previous hand-cipher traffic, which Dilly studied carefully. One member of Strachey’s section, who was proving to be very skilled at breaking ISOS hand ciphers, was Denys Page, a kindred spirit for Dilly as he was also a scholar of Greek poetry and interested in the Oxyrhynchus papyrus texts. Page would succeed Strachey as head of ISOS in 1942 and would collaborate with Dilly’s friend Edgar Lobel in publishing literary papyri after the war.
At this stage, however, Dilly’s easy-going relationship with ISOS had not been officially approved. Dilly kept Denniston informed of his progress on the Abwehr Enigma machine but Denniston, with his obsession for secrecy, seemingly disapproved of the way he was going about things. Denniston’s success about keeping secret his own activities in signals intelligence is shown by the Venlo incident. The Schellenberg document on the British secret service following the interrogation of Richard Stevens states that the head of the cipher and decipher section is unknown and implies that he ran a code-making and not a codebreaking department. Denniston knew how essential it was not to let the Germans know that their messages were being broken as the whole signals intelligence system which had been gradually built up would be lost. ‘Need to know’ restrictions were strictly enforced, but it is difficult to understand why Denniston considered that Dilly had no ‘need to know’ about ISOS, which was obviously so relevant to his work. Infuriated, Dilly sent Denniston a fierce note, again threatening resignation over what he saw as a ridiculous ‘hush-hush’ policy imposed by Denniston – almost certainly on the insistence of SIS – that prevented him seeing material which would help him break the Abwehr machine.
My dear Denniston,
As you, I think, are aware I have decided to attempt a scheme for the reconstitution of one or more outlying German Enigmas. Before proceeding further in the matter there are one or two points, relevant either to the matter itself or to my examination of points of attack, on which I must press for your assurances, and failing these, for your acceptance of my resignation …
In the event of success the whole traffic must be handled in ‘The Cottage’ or our nominees. This is a fundamental point in all research of an academic nature. Research, in fact, does not end till the person responsible has affixed his imprimatur on the last proof sheet …
We still have far too many intelligence sections, appearing to the casual observer, as mangy curs fighting over whatever bones are tossed to them, and (as far as circulation goes) burying their booty in grimy and schismatic indexes. Yet what they get is the material which assists the cryptographer in his researches and this he is wholly unable to see. Occasionally someone may hand him a slip of paper with references to a buried file, but this is not wanted. As in Broadway, he wants the document, all the documents, and nothing but the documents …
These burials of essential documents are, I believe, made in accordance with your policy of ‘hush-hush’ or concealment from workers in Bletchley Park of the results of their colleagues. Against this I protest on several grounds … Such action cripples the activities of the cryptographer who depends on ‘cribs’ … Such action wholly destroys any liaison or pride in the successes of colleagues …
I would urge with the utmost assurance that your action in directing any acts of concealment in Bletchley Park is wholly unconstitutional and ultra vires. Under the Official Secrets Act, I may discuss my work with anyone in the course of my duty. Of this I am the only judge, but it may perhaps be limited to the time of my arrival at, and departure from Bletchley Park. Outside Bletchley Park you may authorise me to discuss with anyone or refuse such authorisation. That is your constitutional prerogative.
When Dilly knew that he was near to a successful break, he wanted to know how the intelligence from the messages would be dealt with before the inevitable flow-chart appeared indicating who was in charge of what. Another letter to Denniston, a mixture of under- and overstatement says it all. He speaks of a ‘minor ailment’, which is why he is away for a few days; it was in fact a secondary cancer but few people were aware how ill he was. He wants to know where he stands and to try to ‘establish an independent line as I nearly secured at Matapan’. He tells Denniston: ‘I am almost despairing of making you see reason on the major issues. You owe the present solution, for it is near to that, to my interest in enemy intelligence as a whole.’ Now apparently Denniston has a ‘monstrous theory’ that when the messages are broken they are to be handed over to ISOS as the operative unit and it will ‘now be possible for Strachey to prevent my free inspection of his other material and use of the new’, Dilly complains.
As a scholar, for of all Bletchley Park I am by breeding, education, profession and general recognition almost the foremost scholar, to concede your monstrous theory of collecting material for others is impossible. By profession in all his contacts a scholar is bound to see his research through from the raw material to the final text. From 1920–1936 I was always able to proceed as a scholar, and I simply cannot understand, nor I imagine can the many other scholars at BP understand, your grocer’s theories of ‘window dressing’. Had these been applied to art scholarship, science, and philosophy, had the inventor no right to the development and publication of his discourses, we would still be in the Dark Ages.
The inevitable threat of resignation unless things improved follows and the letter ends, ‘a small grouse … Yours ever, A. D. Knox.’
Dilly’s irritation at the way in which his material was handled by intelligence officers writing reports to be passed on to the various services and government departments was a recurrent theme in his complaints. In a letter to Menzies, he insisted that the material should be passed on in its original format. ‘In my opinion, Bletchley Park should be a cryptographical bureau supplying its results straight and unadorned to intelligence sections at the various ministries,’ he said. ‘At present we are encumbered with “Intelligence Officers” who maul and conceal our results yet make no effort to check up on their arbitrary corrections.’
Above all, Dilly wanted to be in on the overall intelligence and suggested that heads of sections should be issued with regular bulletins with headings ‘Enemy Operations, Enemy Intelligence, Enemy Codes’. In anticipation of his wish being carried out, he went over to see his old friend Professor Edgar Lobel to discuss whether he was free to leave Oxford. As a former keeper of Western manuscripts in the Bodleian library, and as editor of the Oxyrhynchus papyrus texts, Lobel was renowned for his critical intelligence and concise abstracts. For good measure his wife, who was an editor of the Victoria County History, impeccable on sources, could accompany him. ‘She is a very nice and remarkable woman of the donness class,’ Dilly told Denniston. Although the double Lobel scoop did not materialise, Dilly must have been satisfied with Denniston’s circulation arrangements as the next communication with him was the report on the breaking of the Abwehr machine, dated 28 October 1941, and a request for the increase of Cottage staff.
Hopes of reliable cribs from ISOS messages, given that the messages encoded on the Enigma machine were a version of the actual text dictated to the double agent by the British case officer, had soon been dashed; but it was not immediately clear to Dilly why. However, once the various networks and their message indicators had been sorted out Dilly found the successful way in to breaking this unknown Enigma variation without textual cribs. Fortunately, nobody seemed to have told the Abwehr of the danger of having repeat indicators on a fixed Grundstellung, which the services had abandoned as long ago as 1938. Instead of three-letter indicators as in the services Enigma, Abwehr was seen to have four, the same number as the ‘K’ Enigma, which meant that there was a settable Umkehrwalze. When Dilly embarked on making key-blocks he found that the number of messages on any one day in a network was insufficient for evaluation and the boxing chains were too fragmentary to be of use. He decided, therefore, that if he could find two days where the same wheel order was used, and in such a way that the Grundstellung from one day could be got from rotating each wheel and the Umkehrwalze through the same number of places, he would be able to double the number of indicators on the key-block. This would be observed because the cipher pairings at each position of the Grundstellung would have a QWERTZU substitution relationship, if indeed the diagonal was, as in the commercial machine, his old friend QWERTZU.
Dilly accordingly went off to Hut 7 to enlist the help of Frederic Freeborn’s Hollerith card-sorting and tabulation section in searching for two such days. Then followed a true example of serendipity, as defined by Horace Walpole: ‘making discoveries by accident and Sagacity of things they were not in quest of’. Freeborn had been unable to find the two QWERTZU-related days’ settings required but when Dilly studied the results ‘he found what was wanting standing, like the abomination of desolation, precisely where it should not – on a single setting’. Then he had one of his ‘quick as lightning’ inspirations that he was dealing with a multi-turnover machine and that in one day’s key-block all three wheels and the Umkehrwalze had turned over between the first two letters of the indicator and again in its repeat position, i.e. (1) ABCD (2) BCDE (3) BCDF (4) BCDG (5) BCDH (6) CDEI. This phenomenon he called a ‘crab’, which he deemed to be useless for his purposes, but if there were four-wheel turnovers on both sides of the throw-on indicator key-block there would probably be many more cases of such turnovers on one side of the key-block alone and this possibility he called a ‘lobster’; this indeed was the brilliant idea which would break the Abwehr machine, thereafter always called the ‘Lobster Enigma’.
Dilly worked on far into the night but we always made sure that there was someone on in the backroom to make black coffee and find lost things for him; there was no filing system as such. It was Phyllida Cross who was there on this momentous lobster occasion and she tried her best to come to grips with it when he rushed through to tell her but she admitted that she didn’t understand a word. The next day, he excitedly instigated a lobster hunt. The method involved his well-tried boxing or ‘saga’ method for breaking key-blocks of indicators on the same Grundstellung with the additional QWERTZU bonus of the lobster turnover. Chains were made for positions 1–5, 2–6 and so on; if a cipher letter pairing was assumed in position 1, the chains gave deductions about other pairings in 1 and 5 and, if a lobster turnover existed between 1 and 2, there would be several pairings implied for 6; it was then easy to see whether these were consistent with the implications for the chain 2–6. Having confirmed the position for the lobster, the fun could begin guessing the indicators.
Dilly’s report to Denniston says: ‘The hunt was up and scent was good. One very fine Lobster among others was caught and after two days Miss Lever by very good and careful work, succeeded in an evaluation which contained sufficient non-carry units to ascertain the green wheel.’ It was obvious by the repeat patterns that many operators were choosing pronounceable indicators such as WEIN, DEIN, NEUN. In one column of a key-block there was an indicator cipher bigram of the order TR, NB or WQ in the third and fourth place, which meant that if there had been a lobster four-wheel turnover at that point the clear text would also have to be a bigram reverse on the QWERTZU keyboard and if it were pronounceable then SA was just about the only choice. My lucky guess was that an operator on the Balkan network had a girlfriend called Rosa, who really did have a lobster on her in the required place. Having hopefully fixed ROSA in the position on the indicator key-block, as it was all on the same setting, reciprocal values could be filled in and in the repeated ROSA indicator on the other side of the key-block and continuing backwards and forwards with evaluations from one side of the key-block to the other. The generated alphabets of text and cipher could then be ‘buttoned up’ and Dilly’s normal saga methods applied as there were ‘sufficient non-carrying units’ (positions without turnovers) to find the wheel wiring.
After Margaret Rock and I had evaluated a number of key-blocks, it became possible for Dilly to discover how many turnovers each of the wheels had. Each evaluation produced seven consecutive places at which it was shown whether the wheel had a turnover or not: if a turnover the position was marked by +, if not by -, so that the turnovers produced by an evaluation would be shown by a sequence such as + - + + - + +. With evaluations for several days, these sequences could be fitted together – in a process akin to dendrochronology, the ring-dating of trees by overlapping sequences – to give the complete sequence of turnovers, which became known as the wheel track. The wheel we called green had 11 turnovers, blue had 15 and red 17. It was now clear why there had been no success for Dilly in trying to get cribs from ISOS messages for rodding, which needed a good run on the right-hand wheel; the most one could hope for on the Abwehr machine was a run of four in two places on the green wheel. Dilly’s invention of the lobster process meant that a long crib was no longer necessary but having discovered the wheel wiring and wheel tracks through ‘lobstering’ the key-block, it still remained to discover how the letters on the wheel rings related to the turnovers and that would take longer, as it necessitated the breaking of a message.
The first message was actually broken on 8 December 1941. As usual Dilly gave credit to his girls and when Denniston told Menzies of the success, he wrote:
Knox has again justified his reputation as our most original investigator of Enigma problems. He has started on the reconstruction of the machine used by the German agents and possibly other German authorities. He read one message on December 8th. He attributes the success to two young girl members of his staff, Miss Rock and Miss Lever, and he gives them all the credit. He is of course the leader, but no doubt has selected and trained his staff to assist him in his somewhat unusual methods. You should understand that it will be some weeks, possibly months, before there will be a regular stream of these ISOS machine telegrams.
It was again an obligingly positioned lobster, as in the key-block break, which provided the break for the first Enigma Abwehr message. When Dilly first analysed the traffic, it was found through a ‘boil’ that it was possible that most of the messages began NRX (number–space), but, owing to the multi-turnover nature of the machine, guessed numbers yielded nothing. Then one day, a possible lobster was noted in the second and third position of the number following NRX and if there had actually been a four-wheel turnover at that point the only number with a lobster in the right place could be dREi (three). As luck would have it, we had actually hit the 300s serial number in the relevant month and so there were several possible lobsters to choose from in that position; for good measure now that the rods were made from the known wheel wiring a click was found somewhere in NRXDREI (Number–space–three) and there was a run on, which threw up right letters for the two other numbers in the 300 series. We did, as Denniston said, indeed employ ‘somewhat unusual methods’ in the Cottage and it was a real Wonderland situation when lobsters, starfish and beetles could all be coaxed to join in the dance.
Denniston must have found Dilly’s Carrollian logic particularly difficult to cope with. Lewis Carroll’s Sylvie and Bruno was a favourite book, especially the Professor’s seemingly inconsequential lecture beginning:
In Science … in fact in most things, it is usually best to begin at the beginning. In some things, of course, it’s better to begin at the end. For instance, if you wanted to paint a dog green, it might be better to begin with the tail, as it doesn’t bite that end.
Carroll had actually heard a child in a train make a similar remark when warned about pulling a dog’s tail. Dilly frequently left bits of his reasoning out in the same way, which we had to try and fill in for ourselves. Completing Dilly’s ellipses was a good training for cryptographic puzzle-solving.
Dilly’s original report to Denniston on the breaking of the Lobster Enigma machine told him that ‘the solution was based on a theory and observation and a procedure devised by the head of the section who had decided that as everything that has a middle has also a beginning and an end’. After the long night when he had the lobster inspiration, he was waiting for Margaret and myself in the morning at the Cottage door beside himself with excitement and said: ‘If two cows are crossing the road, there must be a point where there is only one and that’s what we must find.’ Unlike poor Denniston, we were well trained in Carrollian logic and could get the point. Apparently Dilly told Hut 6: ‘Give me a Lever and a Rock and I can move the universe.’ Perhaps that was our contribution to Archimedes. Dilly’s bright ideas sparked off like Catherine wheels and Margaret Rock might be trying to pin down the latest one while Mavis Lever was still struggling with yesterday’s, which might well be a winner and might have been forgotten in the excitement of today’s. It was not like that in the days when he had Gordon Welchman and Alan Turing as assistants, as they had too many bright ideas of their own to assist with Dilly’s.
Denniston did his best to come to grips with Dilly’s requirement of the type of staff he said in his report would be needed for the lobster production line. Up until this point, the staff consisted of Dilly and seventeen girls, of whom only Dilly himself and two of us girls, Margaret and myself, were German linguists and therefore capable of taking part in his lobster hunts. Dilly told Denniston he needed at least two more linguists, who must have time to learn, for serendipity’s sake, what we knew we wanted and what might be useful.
‘All hunters must know the tricks of the machine,’ Dilly said. ‘We must proceed as with the Italian Enigma by the careful study and correction of messages before they leave us. Any other system of arbitrary correction by those who do not understand the machine plans and cannot avail themselves of Morse corrections is repugnant and unthinkable.’
We did indeed have to make intercept corrections from time to time, when a Morse error could contradict an otherwise good lobster or throw-on. Of course, Dilly was already into this in Room 40 days, when interception was not of the high standard that now came to Bletchley Park from the ‘Y’ stations such as Beaumanor. He had invented a table of syllabic metre words to come to terms with Morse. I can only remember the beginning, ‘Gallantly and Furiously, he fought at Waterloo, against the Barbarian.’ We preferred just to run an eye down the Morse code chart on the wall.
Denniston clearly had personal talks with Dilly as to how his section should now develop and a letter on Christmas Day to Valentine Vivian, who was in overall charge of counter-espionage within SIS, takes the matter forward:
With regard to Knox’s success and the resultant labours, I would suggest that the series be issued as ISK [Illicit Services Knox]. Secondly, it will be necessary for the emending party to be reinforced to deal with some 50–100 extra telegrams per day and I suggest this is an opportunity to develop the ISOS hut on lines parallel to Hut 3.
Hut 3 was the intelligence-reporting section for the Hut 6 codebreakers working on the German army and air force messages and Denniston suggested that an SIS counter-espionage expert be put on each Hut 3 reporting shift to cover the material produced by Dilly’s section. It was not just the Abwehr Enigma messages that were to be designated ISK; Dilly’s section was in future to bear his name. Officially, it was referred to as Illicit Services Knox, matching up with Illicit Services Oliver Strachey, but it soon became generally known as Intelligence Services Knox, which greatly pleased him.
Panic set in at MI5 and Bletchley Park when in November 1941, just as Dilly’s Spy Enigma was being broken, Agatha Christie published a counter-espionage detective novel called N or M?, in which her protagonists Tommy and Tuppence attempted to track down two German secret agents believed to be in Britain. There was horror when it was discovered that the novel included a character called Major Bletchley. As Dilly was known to be a friend of Christie’s and had been party to her detection club rules, he was asked to find out what she knew, in order to prevent any further breaches of security. He invited her to tea at Courns Wood and over Olive’s scones it appeared that she had never heard of Bletchley Park and she clearly didn’t know Dilly worked there. It then emerged that once she had been stuck on the horrible station when changing trains and took revenge by giving the name of Bletchley to one of her least lovable characters. It was with great relief that another cup of tea was poured out for her. Intelligence Services Knox had not been compromised before it even started after all.