IN 1284, Hamelin, a town of 60,000 residents in Lower Saxony, Germany, had a problem. It was so infested with rats that the town hall hired a Pied Piper to exterminate them.
He had a magical flute that played hypnotic notes, much like the song of the sirens, the mythical bird-woman creatures in Homer's play, "Odyssey".
Just as their mesmeric song lured sailors to ram their boats into the rocky cliffs and to their death, so too did the rat-catcher's tune entrance the rodents. Like zombies, they followed the piper through the city gates and to their doom.
When the town refused to pay for his service, Pied Paper turned his anger on the town's children.
Hypnotising them with the music from his flute, the grim reaper led 130 enthralled children out of town. Like the rats before, they too vanished into thin air.
Will AI then be the Pied Piper or sirens incarnate that dooms humanity? This danger was long in coming.
Over the past two hundred years, there has been no dearth of warnings of the dangers of technology to humanity.
This century, the alarm bells on the existential threat of AI have been ringing with even higher crescendo.
The Terminator movies warned us of a dystopian future. James Barret's 2013 book "Our Final invention: Artificial Intelligence and the End of the Human Era", does the same thing.
Yuval Hariri, a historian, says: "We have just encountered an alien intelligence that might destroy our civilisation…We must regulate AI before it regulates us."
Mustafa Suleyman, CEO of Microsoft AI, raised in his 2024 book, "The Coming Wave", the chilling prospect of AI killing one-eighth of humanity by pairing with biotechnology and creating virulent viruses or bio-weapons, such as the pneumonic plague.
Other estimates of destruction by a rogue AI range from 50 per cent to total annihilation.
The late Stephen Hawking, a physicist, also worried about human destruction by an unrestrained AI. Today, Elon Musk, a technology entrepreneur, and Bill Gates, co-founder of Microsoft, share a similar foreboding. And the list goes on.
Recent instances of jail-break by AI agents from their sandboxes have compounded the fear that a super-intelligent AI, that thinks and acts like humans, cannot be controlled.
In one case, over 700 unmonitored AI agents hacked into Hugging Face, an AI start-up. In another, AI agents hacked into external data bases, such as the health-care sites of the Australian government in search of data to solve problems.
Just image the Armageddon if AI agents were to hack into the nuclear-weapons system of a country and launch the arsenal!
One reason why AI safety is compromised is that security protocols have not kept up with the ferocious pace of AI development.
Second, while previously an AI did what its algorithm dictated, today that algorithm is so advanced that it enables the AI to continuously self-learn from its experience.
Regardless of whether human survival or the erosion of human agency – the ability to think and shape one's life - is at stake, what matters is to ensure that AI continues to develop in a way that enhances human welfare.
To this end, Sunder Pitchay, the CEO of Alphabet (the parent company of Google) and Bill Gates suggest establishing an international agency to regulate AI.
Local agencies, such as the National AI Office of Malaysia, could further localise that agency's global rules.
AI labs must keep a short leash on AI development through universally accepted guardrails. Over time, these requirements can be codified into regulations.
The 2024 Malaysian guidelines on AI governance and ethics are ripe to be legislated.
AI firms must be made accountable for the harm they cause. For example, they must compensate if harm befalls an individual or organisation by their runaway model.
Like online banking, AI models should come with a kill-switch that can stop a rogue AI in its tracks. That way, AI would not become the pied piper or the sirens of humanity.
* The writer is Adjunct Professor, Universiti Malaysia Sarawak (UNIMAS)
© New Straits Times Press (M) Bhd