When AI kills someone, who’s to blame?

Wherever there is war, there are war crimes. Since the nineteenth century, the world has had rules governing the appropriate conduct of conflict, now codified in international law under the United Nations’ aegis, primarily in the Rome Statute of the International Criminal Court and the Geneva Conventions.

We have long tried to maintain these rules of engagement because otherwise war tends to be an excuse for depravity. Mass civilian death, slavery, torture, rape, ethnic cleansing and genocide are all incidents, typically, of war rather than any other circumstance. This reflects the dark side of human nature, never far below the surface of civilisation; we are capable of horrors and largely incapable of self-policing.

One manifestation of this is our propensity to use every weapon as soon as it’s invented, with no regard for the morality of doing so. However, humanity has also proved itself capable of being so horrified by certain weapons that it agrees they must not be used. Examples include poison gas, biological weapons, fragmentation mines and cluster bombs.

Related Article Block Placeholder Article ID: 1268035

We could add to this short list a newcomer: AI decision-support systems (AI DSS). We won’t, for the same reason we won’t get in AI’s way in any other industry, but its increasing prevalence is already posing some thorny questions about how the law can respond to the inevitable human carnage this technology will inflict.

Drones haven’t taken long to dominate modern battlefields. They are the primary weapon now on both sides of the Russia–Ukraine war, and have been extensively used by Israel in Gaza. They are fast rendering older weapons systems — such as navies and missile defences — obsolete.

AI-enabled systems, however, take the obvious advantage of so-called autonomous weapon systems (a euphemism for “unmanned killing machines”) to an infinitely higher plane of effectiveness. Weapons are no longer just operating without risk to human actors — they’re also being driven by algorithms.

Taking one example of an AI DSS in use right now: Lavender, operated by the Israel Defense Forces. It provides recommendations for human targets (i.e. it identifies individuals in Gaza for potential targeted killing) based on a scoring approach. That is, it gives a score out of 100 for the likelihood that a target meets the pre-set criteria (e.g. an adult male member of Hamas).

In the first two weeks of the Gaza war, Lavender was apparently recording a 90% accuracy rate based on double-checking by human operators. Satisfied that this was good enough, human checks were then stopped. Lavender became effectively the decision-maker for who was to be killed. This begs an obvious question: if Israeli drones (or manned weapons such as planes or tanks) kill civilians in Gaza because they’ve been misidentified by the AI DSS which is directing their attacks, who — if anyone — could be prosecuted for the war crimes these acts may constitute?

The problem is immediately obvious. Where AI has been deployed on a set-and-forget basis, and is therefore making discretionary decisions about the use of military force based on its own parameters, can it be said that any human person is culpable if it gets its choices wrong?

We have to be clear, for one thing, that the killing of civilians is not per se a war crime. Bombing that disproportionately kills civilians is, and targeting that doesn’t attempt to distinguish between combatants and civilians is. Both have unarguably been major features of Israel’s campaign in Gaza, which is partly why the International Criminal Court has issued arrest warrants for Israeli leaders Benjamin Netanyahu and Yoav Gallant.

International law scholars have been writing furiously during the past few years, grappling with the problem of how one fits the very human constructs of criminal law to choices made by non-human intelligence. The suggested pathways are largely twofold.

First, there’s the notion that there is criminal culpability in setting the AI machine loose without supervision — that is, those who switch it on, set its instructions and point it at the war zone are legally responsible for whatever it then does. In this sense, it is arguably analogous to throwing a hand grenade through a window with no knowledge of who might be in the room. You own the consequences.

Related Article Block Placeholder Article ID: 1255894

The other idea is that of supervisory responsibility: in the same way that a government leader like Netanyahu can be liable for war crimes committed by those under his overall direction and control (his soldiers) — a principle established at Nuremberg — the same can apply where the murderous inferior isn’t a person but a machine.

Both theories have gaping holes, because the way the law of war crimes has developed is an uncomfortable fit with non-human actors. At root, the problem is that all criminal law naturally assumes human involvement in both of the essential elements of crime: the actus reus (the physical act) and the mens rea (the mental element, such as intention or recklessness).

AI, we all know, does things it hasn’t been asked to do and its instructors don’t want it to do. That’s not because it has attained sentience but because it processes information so fast that it can mimic human decision-making behaviour with ever-increasing sophistication. It’s still a machine, just a terrifyingly amoral one.

We cannot prosecute AI for the war crimes it’s already committing; that’s nonsense. But we may be able to prosecute the humans pointing it at their enemies and licensing it to decide when to press the button marked “Fire”. It isn’t yet clear how, but we’d better urgently work that out.