Rennie Naidoo, Professor of Information Systems at the Wits School of Business Sciences.
Claims of superintelligence often overlook unresolved questions about what intelligence means. They can also divert attention from the performance of firms directing AI’s development.
I hear tech vendors, media, academics and students describe AI as becoming superintelligent. The conversation turns to machines surpassing humanity, escaping control, or deciding our future.
I tend to ask more ordinary questions. Who controls the infrastructure? Who approves deployment? Who checks the outputs? Who is accountable when something goes wrong?
These questions concern decisions being made now. AI needs no ambitions of its own to cause major harm.
Giving an unreliable AI system access, authority and insufficient oversight can be enough.
I am sceptical of superintelligence claims because I remain unconvinced that we have adequately settled on what intelligence means.
“AI” itself is an umbrella term that covers different techniques and systems for prediction, classification, language generation and action. Their capabilities and limitations vary. Treating them as one entity obscures those differences.
Does intelligence mean speed, recall, reasoning, understanding or sound judgement in unfamiliar situations?
The term “superintelligence” risks collapsing distinct capacities into one property that can simply be increased.
The gravest existential threat may arise when powerful technologies are governed by leaders whose ambition exceeds their judgement.
Recent exceptional performance by AI models on particular tasks does not settle that philosophical question.
Behind such performance are data centres, electricity, training data, software integrations, human evaluations and people correcting errors. Organisational usefulness also depends on reliable records, institutional knowledge and staff expertise.
Where should we locate the intelligence being praised: in the model alone, or in the arrangement making its performance possible?
Why attribute achievements produced through extensive infrastructure to a seemingly self-sufficient machine?
Present AI as unstoppable, and adoption becomes a race between superpowers and Big Tech firms; questions about evidence, cost and accountability can then sound like resistance to progress.
It also seems that firms promising extraordinary capabilities can position themselves as indispensable advisers on how to govern themselves.
The Y2K scare illustrates the power of such framing. Predictions of a “computer-induced apocalypse” arguably made a repairable engineering problem appear beyond human control. The date-handling problems back then required substantial remediation, testing and coordination, but had identifiable causes and practical responses.
Similarly, dramatic AI safety narratives can obscure human agency. Which failures are preventable? What corrective work is needed, who must do it and who is accountable if it is neglected?
And warnings about future existential risks should not exempt Big Tech leaders from scrutiny of present financial performance. These executives must still demonstrate what their spending delivers.
We should also be concerned when tech billionaires enjoy privileged access to political leaders. The industry’s technical expertise is relevant, but it does not entitle Big Tech to set the terms of public debate or policy.
The danger is regulatory capture: oversight serving those it should scrutinise. Influence over which risks receive attention, whose expertise counts and which responses appear feasible can shape regulation without companies writing every rule.
Tobacco and climate history offer warnings. Tobacco industry documents exposed efforts to weaken public health measures aimed at reducing smoking. Research on a major fossil fuel producer found public statements contradicting its own internal climate projections.
Similarly, an agenda centred on hypothetical superintelligence could divert attention from insecure products, surveillance, concentrated power, and the electricity and water demands of data centres.
While the future risks of AI deserve investigation, workers, affected communities, independent researchers and elected representatives must help set priorities. AI safety cannot become a private understanding between the tech and political elites.
In 2024, the US Cyber Safety Review Board found that a major cloud e-mail breach the previous year had been preventable and criticised a major Big Tech provider’s security culture. The firm subsequently committed to putting security first, even when that meant delaying new features.
Similarly, more recent incidents of AI agents behaving unexpectedly do not establish superintelligence. We should investigate excessive permissions, exposed credentials and inadequate supervision. The apparent loss of control may reveal poor cyber hygiene or unsafe deployment.
Describing an AI system as having “gone rogue” can also misplace agency. It turns a failure of design, deployment or oversight into a story about an autonomous machine acting on its own. Yet people gave the system its permissions, access and operating conditions.
Language that personifies AI can therefore do more than exaggerate its capabilities: it can make responsibility harder to locate precisely when accountability is most needed.
Regulators should require independent testing, incident reporting and consequences for preventable failures. Boards must establish who can halt launches and whether staff can challenge unsafe decisions.
Speculation about superintelligence should not distract from these immediate tests of leadership and oversight.
The Cuban Missile Crisis reminds us how decisions about power, conflict and technology can bring societies to the brink. Restraint, negotiation and reconsideration can help pull us back.
We need not settle on whether machines will become superintelligent to demand better judgement from those governing them today.
The gravest existential threat may arise when powerful technologies are governed by leaders whose ambition exceeds their judgement, and by democratic institutions too weak to hold them accountable.
Society has committed enormous capital, energy, infrastructure and public trust to AI. Neither apocalyptic warnings nor utopian assurances substitute for evidence of the benefits that justify those commitments.
The initial promise was extraordinary: AI would transform civilisation and deliver abundance, from curing every disease to solving the climate crisis. It just needed money. Now its potential dangers become another reason to ask for more money.
Big Tech leaders owe the public results. When will they demonstrate benefits that outweigh the full financial, social and environmental costs of their investments and fulfil the industry’s promise of abundance for all?