AI has already been linked to at least one deadly military strike on a school and a narrowly averted attack that could have started a war with China
By Moustafa Bayoumi / The Guardian
OpenAI chief executive Sam Altman has some words about his company and the future of artificial intelligence (AI).
“One of the differences between us and some of the stricter AI safety people is that we believe that the world should accept some bad things happening for the benefits of this technology and people having the agency” to use it, Altman told Politico’s new technology podcast and newsletter Decoded.
The statement — and the approach it represents — is problematic, to say the least. It is also emblematic of precisely the kind of world-building and world-destroying that is happening with today’s technology in front of all our eyes, with precious little input from the rest of us.
Illustration: Yusha
Altman’s mention of the “bad things” that AI might bring makes it sound like AI’s failures mistakenly create innocent, minor problems or even silly little episodes, such as the time when Google’s “AI Overviews” told users to put nontoxic glue on pizza so the cheese would be less likely to slide off. Good advice is helpful. Bad advice? Chalk it up to being a bad thing.
However, culinary adventurism is hardly the only realm that AI operates in and the damage AI is inflicting has already and repeatedly proven itself to be dangerous and even potentially deadly to humans.
OpenAI is embroiled in a lawsuit after the tragic suicide of Adam Raine, a 16-year-old student who initially began using OpenAI’s ChatGPT to help him with his homework. The family’s lawsuit says that the teenager took his own life with the chatbot’s encouragement. (OpenAI has claimed Raine misused the platform.)
Raine’s action is not the only one. OpenAI is facing multiple lawsuits filed by different families of victims of suicide and self-harm.
Nor is suicide the only “bad thing” to emerge in the AI universe. On Feb. 28, on the first day of the war in Iran, two Tomahawk missiles struck the Shajareh Tayyebeh Elementary School in Minab, Iran, killing at least 120 children. (A UN fact-finding mission concluded that the US was probably responsible for the strike.)
In an investigation into the attack, Bloomberg News reported that an AI-linked system — Palantir’s Maven — was used to target the school.
Unnamed defense officials told Bloomberg that “inside Centcom [US Central Command], which conducted the US attack, some personnel relied too much on the artificial intelligence embedded in Maven Smart System.” The report added that “in the past year, the Defense Department has rapidly made Maven a cornerstone of the US military apparatus.”
AI is helping to produce these tragic and needless deaths, but bad things happen.
There is also the little detail that the US might have come within a hair’s breadth of starting a major war against a superpower, all due to an AI error.
CNN last month reported that an unidentified AI chatbot used by the US military determined that a Chinese ship in Middle Eastern waters was carrying components of a nuclear weapons program. CNN US planes took off on an interception mission and US military personnel were preparing to board the ship, CNN said.
Only then did the military look more closely at the report they were basing the mission on and realize the chatbot’s mistakes.
Also found in CNN’s story is that the analyst who used the chatbot to come up with the shoddy analysis then used AI again to write the report, which set the whole thing in motion. A twofer of bad things.
Why does it seem that these technologies — which an executive order by US President Donald Trump said are “empowering Americans to achieve what was previously impossible across science, medicine, and nearly every other domain of human endeavor” — get such serious stuff so wrong and so often?
Is it not true that the more people rely on AI, the more dangerous these little mistakes become? Why were Americans never consulted about any of this in the first place?
Could it be that what is behind the AI revolution might be less about the utopian or dystopian future that is just around the corner and more about the enormous amount of money that is being poured into these companies in the present?
AI might be the only thing keeping Trump’s economy afloat right — and it is paying dividends. According to the New Yorker, OpenAI “is reportedly preparing for an initial public offering at a potential valuation of a trillion dollars” and Sam Altman “may become one of the wealthiest and most powerful people on the planet.”
Those bad things? They only happen to other people.
Meanwhile, we are being warned almost every day that AI systems are getting so smart that they would end humanity.
Maybe the real worry is not that the machines are taking over, but that the humans are giving up, ceding control and responsibility in the name of hyping up a product that has shown itself as being really good at taking jobs away from people, stealing the intellectual property of others, writing some piss-poor undergraduate essays, spying on you and your license plates, and bringing us to the brink of World War III because God forbid humans should actually question the output from machines — or the makers of those machines.
What we need is real, tangible and enforceable government oversight over these technologies, which are quickly becoming a dominating force in all our lives. We must collectively have the right to decide how AI should be used.
Despite what you might hear, China already regulates its AI companies quite aggressively.
Instead, Trump, Meta CEO Mark Zuckerberg and Nvidia CEO Jensen Huang believe that the AI companies should regulate themselves, so they have come up with something called the AI Accords.
Signed by leaders of Meta, Nvidia, Google, Anthropic, xAI and OpenAI, the accords ask tech companies to “implement robust internal controls and monitor the capabilities and alignment of its models,” among other things. “Over time, it may make sense to codify these steps into laws or regulations,” it states.
I am not holding my breath.
Trump, who has ordered the term artificial intelligence be discarded for “super intelligence,” posted the accord on social media on Sept. 29. The document, titled The White House Accord on Super Intelligence, can still be seen on his account and the White House’s official account. It is signed by all six tech leaders and Trump, who in the document is labeled “President of the Unites States.”
Super intelligence, indeed.
Moustafa Bayoumi is the author of How Does It Feel To Be a Problem? Being Young and Arab in America and This Muslim American Life: Dispatches from the War on Terror. He is an English professor at Brooklyn College, City University of New York.