‘You’re unveiling a voice-activated personal assistant at the gala?’ Fang’s hand was already inside the puppet. ‘Like Siri. Or Alexa? Or Cortana?’
Fang removed a gleaming, oversized tablet from inside the puppet and placed it back on the shelf. ‘That’s no biggie.’ The puppet sat upright in her arms, its head swivelling back and forth as she worked it. ‘Digital assistants are beyond stupid,’ Fang said into the puppet’s face.
‘Syd is more than an “assistant”,’ Tobias intervened. He snatched the puppet from Fang, sat it back over the tablet and settled the smock to his satisfaction.
‘Show them, Tobias.’ Esme sounded tired, North thought. Killing someone could do that to you. ‘I’m sure they’ve signed the Official Secrets Act.’
‘Absolutely,’ Fang said.
They hadn’t. Doubtless they should. But then Hone would probably kill them if they said anything, so their discretion was pretty much guaranteed.
Tobias beckoned them over to the computer at his desk as his wife folded herself into an easy chair alongside. He moved a black-and-white photograph of a freckle-faced boy North knew to be his lost son and placed his cane along the desk. ‘Syd, activate “Medical”,’ he ordered.
Rapidly streaming columns of numbers and symbols filled Tobias’s screen. His long fingers flew over the keyboard, populating boxes, shifting columns and lines – mathematics everywhere. ‘We’d been working for ten years without getting very far but this is where we first moved up a gear with Syd.’ He spoke over his shoulder, knowing they were there, but not entirely willing to break from his screen and engage with them, now he was captured by his coding. ‘Five years ago, the government bypassed UK data protection laws to release all UK medical records for children from birth up to the age of eighteen to Derkind.’ Five years ago their child had died, North thought. Had they had the clout to push for the release of the records even then?
‘I don’t remember that happening,’ North said.
‘Let’s say they didn’t make a big deal out of it,’ Esme said, one eyebrow raised.
‘They kept it quiet?’
‘People broadcast their entire lives on social media but jump up and down about their “right to privacy” whenever the mood strikes.’ Tobias was scornful.
His wife nodded, her elbows resting on her knees, the lovely face cupped in the palms of her hands. ‘We’re aware of the confidentiality issues for the individual, but we needed to think big-picture – of the gain at stake for society at large.’
Tobias slumped in his chair, his body leaning towards them. He gestured towards the computer. ‘On this screen are patients, not numbers – some of these children are being treated as we speak. Within these brackets is DNA coding, along with treatment plans and diagnoses. These are blood pressures and liver functions and kidney functions. These are a million tragedies of children who should be healthy. Some of them are…’ – he hesitated, his gaze on his wife, before continuing – ‘… dead. But the vast majority are still alive. And we’re helping make them better.
‘We input the data via text, scans and X-rays, and our algorithms can discern patterns in a raft of childhood cancers. At the press of a button, we know whether a particular medication is worth trying. That child doesn’t have to be subjected to chemotherapy or radiation, when something else might suit them better. Along with DNA sequencing, we can tweak each therapy for each child with unbelievable precision. We will prolong lives that would otherwise be lost. Eventually we can make the jump to other childhood diseases, to adult cancers and to other adult diseases. Pharmaceutical companies have invested billions of pounds in drugs and now they are investing billions of pounds in AI. This program is the biggest advance in medicine since penicillin. It is transformative and if anyone needed evidence of the benefits of AI…’ – there was no mistaking the emotion in Tobias’s voice – ‘… it’s right here, in children’s lives. Regrettably, we’ve just been hit by a major leak of our medical work.’
North thought back to the cardboard box of the colleague’s belongings, to Jarrod’s comment – outrageous. Paulie Holliday had been blamed for the leak and fired unceremoniously.
‘For some companies, it would be the end of the road, but not for us.’ Hawke hesitated as if deciding whether to carry on or not. ‘I don’t think I should say any more, Esme.’
‘In for a penny,’ she said, standing. Her hands went to her hips as she stared out through the glass wall, watching the drones. As if she were thinking hard about other things. North had to drag his own gaze away from her.
‘Syd isn’t a digital assistant – Syd has consciousness. We’re unveiling her at a gala at the British Museum tonight.’ Tobias’s voice held a note of profound reverence. ‘Revealing the fact there is another intelligence in the universe – an intelligence, moreover, with the ability to reason and to distinguish right from wrong.
‘Most AI systems are narrow. They do one thing very well. Like classifying and recommending a programme of treatment for specific cancers, as we’ve done here. Nobody has anything like Syd though. This is the world’s first general AI system. Syd is the holy grail – a machine that can both reason and apply common sense. She’s already beyond human intelligence. Up to now, she’s been running on an air-gapped system within Derkind – boxed in—’
Fang interrupted. ‘It isn’t air-gapped. She knew stuff about me.’
Tobias grinned and North glimpsed something of what Esme must see in him – the charisma. ‘Esme’s uncle provided us with information on both of you. We inputted it as we do with everything that comes into Derkind.’
‘That’s a cheap trick.’ Fang’s scorn was real.
‘Blame my wife. I believe Syd is ready to go out into the world; my wife disagrees.’
‘We have a moral obligation to tell the world where we are with Syd – as a matter of some urgency, which is what the gala is about. But letting Syd loose on that world is a different thing completely. That can’t happen. Tobias knows that I have fundamental concerns about Syd. I’m sorry, but there are too many unknowns about what would happen next. Any decision on Syd’s future has to be a collective decision taken at intergovernmental level.’
‘I’ve told you – I can install a kill switch.’ Tobias didn’t even try to control his impatience.
‘It’s too much of a risk.’ Esme’s tone was heated. As if they were picking up the threads of a row that had been going on for days.
Tobias waved a dismissive hand in his wife’s direction. ‘Thanks to my overly scrupulous wife, Syd isn’t linked up to the internet or indeed to any computers other than our own. We’ve built safeguarding algorithms into the system – we’ve ensured Syd’s models conform to human values of fairness, reciprocity and curiosity. She appears to be a moral creature.’
‘Obviously it isn’t easy to spell out a value like loyalty in software code,’ Esme said, her arms crossed over her body as if she was cold or in need of comfort. ‘And we have a fundamental problem establishing how Syd conceives of its own purpose. What is its goal? How is it going to reach it? Is there any chance that will be in a way that’s disastrous for humankind? I predict it will be years till we have fully explored the ramifications of allowing Syd into the wild. Then and only then will we – as a species – be able to make that call. But for the moment at least, I know which side of the argument I’m on. A free-range Syd is dangerous to humanity.’
Tobias spoke over his wife’s last few words. He didn’t like the implied criticism of his machine, North thought. Didn’t like the idea that Syd might be dangerous. ‘Once Syd is out, communicating with other systems, it’s true that we can’t know how she will act. But the internet is the most effective communication and information distribution network the world has ever seen and the more data and systems Syd has access to, the faster she’ll learn and the better she’ll be.’
North could hear Fang’s gum-chewing getting faster and faster. The kid was locked on.
‘You’re talking about Syd getting out. But air-gapping isn’t a failsafe in terms of security. Hackers can get in via flash drives or radio waves,’ Fang said. ‘You’ve already had one breach. Syd could get infected or destroyed like that.’ She snapped her fingers.
‘Exactly. Bearing in mind we’ve had the medical leak,’ Tobias said, ‘at Esme’s insistence, I’ve moved the machine learning code for the ensemble of models – the essence, as it were, of “Syd” – out of our cloud and into the tablet on that shelf to safeguard the work. The tablet is a prototype I was working on a couple of years ago. I’ve adapted it – engineering it myself so it can cope.’
They all stared at the goggle-eyed puppet. It didn’t look like it was sitting on a revolution.
‘Tonight we announce what we have with Syd,’ Esme said. ‘We warn the world what’s heading their way. No offence, Syd.’
‘None taken, Esme.’ The digitized voice sounded interested and entirely relaxed.
North shook his head.
Esme carried on as if it were normal to apologize to a machine. ‘I’m convinced I was attacked because of what we’re doing here. And I’m not going to feel safe until we’re through the other side of the gala. That’s why I want you around, North.’
The buzzing North had been aware of earlier had got louder. Through the office window, North could see more drones. They were smaller again, each one the size of a bird, and they swept and swerved together. Like a flock, he thought. Was there a programmer sitting somewhere at his screen making it happen, he wondered, or was Syd putting on a display just for him. Because the more he thought about it, the less he liked the idea of a swarm of drones controlled by a thinking machine.
Tobias frowned as he looked at his wife. ‘Esme, we’ve been to hell and back together. We wanted this, remember. We’ve been working towards tonight for a very long time. And…’ – he spoke slowly, as if reluctant to voice his thoughts – ‘… it’s only a couple of days since you killed a man. Every time you walk in the front door it must bring it all back. After the gala, I’m moving us out of that place. We’ll sell it and I want you to see someone. To get some sort of perspective.’
Beside him, Esme paled.
‘The attack on my wife is an unfortunate coincidence. There were a quarter of a million violent crimes in London this year alone,’ Hawke carried on regardless. ‘If I thought I couldn’t keep my own wife safe, then of course I’d want her protected by someone like you.’ North took a second to wonder how the one-eyed man had phrased the recommendation of ‘someone like him’. Whether he’d mentioned lethal violence.
‘But we don’t need you. Tonight’s gala is happening, then Esme can relax, and as far as the medical leak goes, I’ve already fired the man responsible first thing this morning.’
Esme opened her mouth, then closed it as her husband kept talking. Her lips were drawn in a tight line. North wondered if she hadn’t agreed with her husband’s decision.
‘Paulie Holliday is a gullible idiot, I trusted him more than was wise. But Paulie is out, Esme, and that’s the end of it.’ Hawke turned away from his wife and towards North. ‘I’ll ask you to treat with confidence what you’ve heard today, and forgive my rudeness. I’m sure you’re very good at what you do.’ Standing up from the desk, Tobias held out his hand. North took it – there was no alternative – and the two men shook. ‘But this is a big day for us and you’re wasting our time here. Syd and I need to get back to work – I’ll have someone show you out.’