This week's snazzy AI event in the RDS was a Glastonbury for geeks

Analysis

But there was a very worrying undercurrent to the discussion.

THE GREAT AND the good of the Artificial Intelligence world – or a lot of them, anyway – gathered in Dublin this week.

The government’s AI summit saw around 1,500 politicians, tech-types, investors and researchers gather in the RDS for a snazzy event. They were there to discuss how artificial intelligence could turbocharge Europe’s competitiveness. It was like Glastonbury for geeks. 

The headliner was Demis Hassabis.

The Google DeepMind co-founder and chair said that international standards for governance like those used in nuclear and biological industries, were needed for AI. He said recent incidents where AI agents went rogue and hacked third-party systems were “early warning shots”. 

European Commission vice-president Henna Virkkunen said these incidents undermine trust in AI and threaten our public safety. Taoiseach Micheal Martin said we need to balance innovation and opportunity, and adopt a trustworthy, human-centric approach to AI.

Minister for AI Niamh Smyth said adoption and trust need to go hand-in-hand; citizens need confidence that AI is being deployed responsibly. The summit saw the signing of the Dublin Declaration on Trustworthy, Safe and Sovereign Artificial Intelligence.

If there was one word that did a lot of heavy lifting it was trust. The word was everywhere, indicating that actual trust in AI systems is thin on the ground.

I guess it’s not surprising, given the near constant warnings from experts that artificial intelligence will take our lives or livelihoods. 

A divide on trust?

Europe’s political establishment and the technology industry appear to be converging on the need to make artificial intelligence trustworthy. But they are approaching the problem from opposite ends.

Microsoft’s chief executive Satya Nadella, wasn’t at the AI summit. But he wrote a post on his personal blog earlier in this week arguing that advanced AI systems can never be trusted. Why? Because AI systems can produce decisions that even their creators can’t fully explain. And this is totally unlike traditional software where engineers and testers can trace or recreate an error through lines of computer code.

If we don’t understand the internal logic, Nadella says it’s risky to give these systems access to confidential information, financial accounts, computer networks and the authority to take actions on our behalf.

Nadella’s solution is to treat AI like an employee who could always be making a mistake, breaking the rules or becoming compromised. We need to do everything to avoid trusting AI, he says. That means never relying on a single solution.

Permissions should be restricted. All activity should be monitored and decisions recorded. There should be an off switch.

“We must assume a model is compromised and contain it from the start,” he says. “Think of it like an emergency brake. An authorised person should always be able to pause or shut down a model mid-task. More advanced models will require more advanced containment technologies that we need to standardise on.”

This wariness is slightly at odds with what we’ve heard from European legislators.

The EU AI Act imposes obligations according to the risks associated with different AI applications and requires safeguards for the most powerful models. Europe wants human oversight, transparency, testing and accountability. Nadella is warning that human oversight is impossible. 

While legislators are skilled in writing regulation, the technologists that know AI understand just how unenforceable traditional rules will be in relation to testing their non-deterministic tools. Regulation traditionally assumes that products can be tested against standards.

You can crash-test a car, inspect a lift or check whether a kettle is likely to electrocute its owner. Artificial intelligence systems may behave differently at different times – even if the conditions are identical.

And so the challenge seems to be how to connect Europe’s legal requirements with AI’s slippery systems. How do you regulate, if you have to assume no possibility of trusting the process?

Nadella’s post ends with a striking quote.

The most trustworthy superintelligence system will not be the one with the model we trust most. It will be the one that enables us to trust the model the least.

We may need to accept that trust is something we place in each other, and we should stop expecting it from machines that can give a different answer to the same question every time we ask.

If the singularity occurs and artificial intelligence puts humanity at risk, who (or what) is to blame?

The software or the people who wrote and deployed the software?

Who was the real bad guy: Dr Frankenstein or the monster he created?