30

The Sio box

Central Bureau got a lucky break in January, thanks to a chance discovery by an Australian soldier in northern New Guinea.

The coastal towns of Lae and Finschhafen had been captured by the Allies, and signals units were operating in both locations. From here, the objective was to take the entire peninsula — the Huon peninsula, a craggy network of ridges, valleys, and cliffs enveloped in jungle. The Australian 9th Division fought its way along the coast towards Sio on the northern side of the peninsula. Dengue fever was common — in the month of December, 291 soldiers were evacuated sick.1

They were worn down and weakened by a series of obstacles: enemy outposts in pill-boxes; minefields; snipers; the hostile terrain of cliffs and mountain rivers; and mosquitoes. But it was a war of attrition, and the Japanese defenders had also been weakened by the Australian advance.

Fifty miles to the west, the Americans landed at Saidor, in order to cut off the Japanese army’s line of retreat westwards. Heavy rain filled the mountain rivers, making them impassable and stopping the Americans’ advance. Consequently, Japanese scouts discovered that their retreat had not been cut off. When the Australians approached Sio, they met little resistance, as most of the Japanese defenders had fled into the mountains, taking only the bare essentials for the long hike to Wewak. Only a few rear units stayed behind to slow the Australians’ advance. The Australians climbed a nearby cliff with rope ladders and stood on a precipice overlooking the town below, which was almost deserted. Two days later, on 13 January, they entered Nambariwa unopposed, and secured Sio two days after that. The neighbouring towns had served as a north-eastern base for the Japanese army but were now abandoned, the dead left unburied.

The Australian historian David Dexter described the scene:

The Sio-Nambariwa area contained large dumps of all kinds which the enemy had made no systematic attempt to destroy. It was found that Nambariwa was the principal enemy supply base for the Finschhaven area. Both banks of the river had been used to provide barge off-loading points, barge hideouts, or dump areas. A huge fuel dump, a barge workshop, and an engineer stores dump were found. In the Goaling’s upper reaches there were large bivouac and hospital areas. The various arms of the river had been used for hiding and off-loading barges and there were large dumps of various kinds. All principal dumps had suffered from aerial bombardment and six sunken barges were visible in one deep river...

Mopping up continued for the next six days. Patrols from Sio Mission killed eight Japanese and counted 16 bodies on the 16th. Nine Japanese were killed on the 17th and a sergeant-major was taken prisoner. Patrols up the Goaling found further huge dumps of equipment and weapons. In the afternoon Captain Pursehouse, the devoted Angau officer2 whose local knowledge had been most valuable, was killed by a lone sniper south of Sio lagoon after interrogating about 350 natives collected by the 2/17th Battalion.3

Before the Australian 9th arrived, the Japanese defenders at Sio, the Japanese 20th Division, had a problem that needed solving: they were in possession of a set of Mainline army codebooks.

There was a chance the defenders would be either killed or captured by the Australians. Whatever happened, the codebooks could not be allowed to fall into enemy hands. For this reason, a unit from the 20th Division was told to burn their codebooks. As standard procedure, they were also ordered to cut off the books’ red covers and hand them in later to prove that the books had been destroyed.

But it was the monsoon season. There was a strong wind blowing, and it was raining. The codebooks themselves were wet, and in the monsoon weather would not burn. Even if it was not raining, the Japanese unit was reluctant to burn the books, because a fire would create a column of smoke and might give away their position to the advancing Australian army.

They stuffed the codebooks into a metal strongbox, then dropped the box into a deep, watery pit in a nearby bog before retreating into the jungle.

When the Australian 9th Division entered Sio, they proceeded carefully, wary of Japanese stragglers and snipers, and concerned that the Japanese might have left land mines before they departed. Sappers — Australian combat engineers — scanned the area with minesweepers brought in from Finschhafen.

Two Australian Field Security soldiers, Richard Henry ‘Dick’ Smith and John Burke, got word from a patrol of the remnants of a nearby enemy headquarters, and organised a patrol to investigate, where they found various papers as well as the box in the nearby pit.4 After the army established that the object was not a land mine, they pulled it out of the pit, opened it, and discovered the books.5 The books were taken back down to Finschhafen and from there flown to Brisbane, where they were delivered to the Allied Translator and Interpreter Section, known as ATIS.

ATIS was basically the Australian military’s team of Japanese translators. They were located at a prisoner-of-war camp on the outskirts of Brisbane, in a large villa named ‘Tighnabruaich’ in the suburb of Indooroopilly.6 They were located at the camp because one of their duties was to interrogate prisoners of war.

As both organisations dealt with classified military information, ATIS and Central Bureau had a close working relationship. The few Japanese translators in Australia were in high demand, and since Central Bureau had its own in-house translation unit, the two organisations helped each other and even exchanged translators, depending on workload. So when the still-sodden codebooks arrived, ATIS knew exactly who to call.

Two Central Bureau translators, Charles E. Girhard and Hugh Erskine, came over to inspect the red-bound books. They immediately realised what the books were, and excitedly took them back to Nyrambla.

Girhard said of it, ‘What a mess! The books had been buried near a stream bed in a steel trunk that looked like a footlocker. There was so much mildew on the material that each page had to be dried in the large commercial cooking ovens in our kitchen.’7

At Nyrambla, the books were carefully pulled apart and each page dried — some in the kitchen ovens, others in the open air. The Sio box, it turned out, contained the Mainline army code that had so eluded Central Bureau. One of the books was ‘Rikugun Angosho Number Four’, the latest version of the Mainline army codebook. But it was even better than that. In addition to the main codebook, there were two additive books, a book with instructions on how to use all the other books, and descriptions of the message forms for a range of Mainline army codes, specifically codes with the following traffic identifiers: 7870, 2345, 6666, 5555, and 7777.8 Any Japanese army message sent with one of those numbers at the start was now completely readable to Central Bureau.

Until recently, the code-breakers had thought that each of those numbers represented a different code system. But Arlington Hall, liaising closely with Central Bureau through Sinkov, had studied them carefully and had concluded a few months earlier that they were all variants of the same code system, each one using a different indicator book and message form. (The form refers to how the words and other message items are arranged and ordered.) The contents of Sio box were proof that Arlington Hall was right about the Mainline army codes. It was just one code with variations, and now Central Bureau had the key. They communicated everything they learned to the US cryptanalysts at Arlington Hall, and to the British at New Delhi and Colombo.

The Japanese army was making heavy use of the Mainline army codes across the south-west Pacific, including for transmitting high-level orders and memos. Now Central Bureau changed its work priorities to focus on solving and translating as many Mainline army messages as it could.

One of the first Mainline messages that was decrypted at Nyrambla was about the Sio codebooks themselves. It was a report from the lieutenant whose job it had been to destroy the books, reporting to his superiors that he had done so. Here is his message:

No. 1 SEMPAKUDAN Adjutant Message No. 501, Part 1. As per No. 1 SEMPADUKAN Restricted Message No. 1, we returned only the covers of Rikugun Angosho No. 4, Additive Book OTSU No. 4 and GO No. 2 Special Conversion Square NO. 11, and burned their contents. The personnel of the code section took along the codebooks that were left, and during the change in direction of the overland route from SHIO to HANSA… (a)

(a) To be continued.9

The lieutenant’s lie would keep the discovery of the books in the Sio box a secret. The lieutenant had reported to Tokyo that he had taken those books and ‘burned their contents’. No, he hadn’t. He and his unit had disobeyed a direct order, in order to save their own lives. Central Bureau knew it was a lie, but Tokyo did not. This meant that not only did the bureau have the complete set of army Mainline codes, but Tokyo did not suspect it, and was therefore unlikely to issue an emergency code change.

With the Imperial Japanese Army active across northern New Guinea, there was a massive flow of army Mainline messages coming in from wireless units and sections across northern Australia and New Guinea. The only limit to the number of such messages that Central Bureau could solve each day was how fast they could work.

Sinkov’s IBM machines now proved their worth. Installed a few blocks away at Ascot Fire Station, the machines were operated by Central Bureau’s ‘Machine Section’, the head of which was US army Major Zach Halpin. The beauty of the machines was that they could do the work of an entire team of clerks, and could do it faster. In this case, the entire codebook, Rikugan Angosho Number Four, was entered into the machine. A message was entered onto a punch card; the relevant additives for that message were found in the additive book, and they were entered on a separate punch card. Both cards were then fed into the computer.

The machines would print two decoded versions of the message, one with the codebook entry numbers, and one with a romaji translation — a version in the English alphabet. With the IBMs running around the clock, this was industrial-level decryption.

The entire process was streamlined.10 Incoming messages arriving at the nearby airfields were first delivered to Pappy Clark’s traffic-analysis section. There, the analysts recorded what they needed — such as the identities of the senders and receivers, and other information in the preambles — before sending the messages to Sinkov’s cryptanalysis section. Since the Mainline code was completely broken and solved, the only thing the cryptanalysts had to do was find the additives for each message in the additive books. They would write out the additives and attach them to each message before sending them to the machine section, where the messages and additives would be punched into punch cards, laid in stacks, and fed into the computers.11 Before long, they were processing about 2000 Mainline messages per day.12

With the vast collection of mainline messages that could now be decrypted, the translators could not process them all. ATIS translators moved across to Nyrambla to help, as part of which Fabian sent two of his translators in Melbourne, Forrest Biard and Thomas R. Mackie, to Central Bureau. Biard and Mackie had much-needed expertise in translating and understanding more complex military messages, because FRUMEL had been dealing with the large, versatile JN-25 code for some time.

The codes that had preoccupied Central Bureau until now — the air-to-ground codes, weather codes, and the Water Transport code — were all based on relatively limited vocabularies and smaller books. As valuable as the Water Transport code was from an intelligence perspective, it was mostly about shipping movements, which was a relatively narrow topic. The newly broken Mainline army code, on the other hand, covered a wide range of topics, and its messages contained a huge amount of Japanese military jargon. Biard and Mackie’s knowledge of this jargon was invaluable.

The Mainline solution also provided a rare opportunity for the code-breakers to assess their own techniques. They retrieved and looked back over some of the work they had done in the past on the Mainline codes, using the Sio books to find out where they had made mistakes and where they had gotten it right.13 It was not the case that they had made no headway on the Mainline codes — there had been some progress — but it was slow, and it would have been some time before they solved it. Going back over that work, they discovered that there were intermittent mistakes throughout the process, from interception through to the code-breaker’s desk. Interceptors sometimes misheard; sometimes they wrote down the wrong Kana symbol; sometimes they wrote it down correctly, but their handwriting led to a mistake later. There were errors in transcribing, in filing, in stripping additives … mistakes that compounded themselves and slowed the solution process.

To the north-west of Rabaul is a small group of islands known as the Admiralty Islands, the largest of which is Manus Island. MacArthur knew that these islands would make an excellent air base, and that occupying them would help isolate the Japanese stronghold at Rabaul. Japanese forces had occupied the Admiralties earlier in the war, but recent evidence suggested that they had abandoned them. Allied reconnaissance planes flew overhead without trouble, and without seeing any signs of activity. But Central Bureau learned otherwise.

Fifty-three Wireless Section at Nadzab, which was monitoring the Admiralty Islands radio traffic closely, intercepted messages that revealed the presence of 4000 Japanese troops hiding on the islands, patiently waiting in ambush for the Allies to attempt a landing. With MacArthur intending to personally accompany the landing force, such an ambush could have been disastrous.

Central Bureau discovered the existence of the hidden units through decrypted Mainline army messages sent between Manus Island and Rabaul. These messages revealed that the Japanese army was expecting the Allies to try to capture the Admiralties in the near future, and had correctly guessed that the initial landing would occur on Manus Island. The decrypted messages further revealed that the army had stationed most of its troops on Manus Island to launch a surprise attack once the Allies made landfall. Pappy Clark estimated, with traffic analysis, that the Japanese army’s strength in the Admiralties was between about 3000 and 5000 troops.

Knowing of the trap and how it had been set, MacArthur instead landed on the smaller neighbouring island of Los Negros on 29 February. He came ashore from one of the frigates, inspected the airstrip at the small township of Momote, announced that the invasion had been successful, and then departed.

Before long, the wireless units at Finschhafen and Nadzab intercepted Mainline army messages from the Japanese commander on Manus Island, reporting that the Allies had landed, and declaring his intention to fight to the death.

All-out close combat began. The battle for the Admiralty Islands was bloody, with a heavy loss of life on both sides. Even with the advantage of pre-battle intelligence from Central Bureau and with continuing updates about radio signals from the Japanese army, the Americans found the going tough. At one point, the Japanese defenders broke through the American line and came within 15 feet of the makeshift command headquarters that had been established near the landing.14

Eventually, the Americans strengthened their hold on the Admiralty Islands, and did not hurry to defeat the enemy on Manus Island. Over the coming days and weeks, using their new codebooks for the army Mainline code, the wireless units at Nadzab and Finschhafen were able to read messages about the diminishing forces on Manus. They were able, therefore, to monitor the action from afar — not based on reports from their own command or from the Americans, but from enemy radio signals. As was the case in battle after battle in the Second World War, the Japanese forces refused to surrender. Instead, they followed the instructions of their commander on Manus Island, Colonel Ezaki, who sent a radio message at the start of the battle, ‘Be resolute to sacrifice your life for the emperor … and commit suicide in case capture is imminent.’15

Five days later, the wireless units intercepted a message that originated in plain Japanese from the Admiralty Islands, not encoded at all, which said, ‘The time of the last hour is drawing nearer … we are striving for the fatherland.’

The next day, 6 March, they read a message from Colonel Ezaki that said, ‘The troops opposing the landing are reduced to 800, with ordnance reduced by 2/3rds and ammunition reduced by 7/8ths.’

A week later (on 15 March), they read of ‘about 120 men left on Los Negros, SW of Momote strip.’

A week after that (23 March), they learned that ‘entire admiralty garrison Manus and Los Negros, reduced to 750 … prepared to die to the last man … please drop by air more hand grenades and mg ammunition if situation of the enemy permits.’

There were now few enemy messages from the direction of the Admiralty Islands. On 30 March, the wireless units detected a radio message from Tokyo announcing, ‘We regretfully report our forces on the Admiralties have been overcome while fighting to the utmost.’16

There were soon no more enemy radio signals coming from the Admiralty Islands.