Necro-Economics: Death by Algorithm
Why it's here
A chilling piece by Eyal Weizman in the London Review of Books, with a chilling title to match. It’s disturbing to read, but it shows how AI is being used by the Israeli military to select and kill targets in Gaza. Exactly how much the military relies on AI is unclear, but what’s undeniable is that it does.
What makes it chilling is that a machine, a non-human entity, now helps decide whether a so-called combatant lives or dies, and how many civilians may die with them. Under the laws of war, the principle of proportionality is supposed to limit how much “collateral damage,” the deaths of innocent civilians, can be justified in attacking a military target. In Weizman’s telling, the system turns that principle into arithmetic: a pre-set number of civilian deaths per target.
The moral maths has a history. In 2002, Daniel Reisner, then the Israeli army’s chief legal adviser, ran an experiment at a seminar for officers. Each wrote down, anonymously, how many civilians they thought it would be legal to kill in an attack on a particular target. The notes went into a military cap, and Reisner averaged them. The answer was 3.14: π. His lesson for the room:
As a lawyer I need numbers to work with.
The whole logic of putting a value on a human life is deeply disturbing. And it’s hard to deny how much these technologies have helped the Israeli army carry out what I believe is a genocide in Gaza.
While today’s debates about AI revolve around whether it will take our jobs, I think the far more consequential debate should be about these technologies deciding who gets to live and who gets to die. To be fair, Weizman himself cautions against making AI the whole story. Most of Gaza was destroyed by old-fashioned artillery, tanks, bulldozers and bombing so intense it might as well have been carpet-bombing. His point is that the technology serves mainly as cover, lending the killing an air of scientific legitimacy, and that the calculus behind it is far older than AI: colonial powers have always put a price on the lives of the colonised. But that’s precisely what worries me. AI doesn’t have to be the main weapon to be the most dangerous one, if it makes slaughter look rational.
Another disturbing element is how these systems were bent to give people in power what they wanted. As the philosopher Shannon Vallor argues, AI is a mirror. It reflects the choices and biases of its makers, and here, the lawyers, coders and soldiers who shaped this non-human entity, gave birth to it and granted it the power to play God. Not on a battlefield, because Gaza is not a battlefield, but over the fates of the innocent and the guilty alike.
It’s a preview of the world to come. And it isn’t just Israel: numerous reports describe Ukraine as a proving ground for AI in its defense against Russia. The world has changed. We just don’t know it yet. B.
From the piece
The devastating application of AI after 7 October was exposed by Abraham, NAZA’s co-director, in a series of groundbreaking articles for the online magazine +972, Local Call and the Guardian. In NAZA we hear some of his sources talk for the first time. They describe a system called Lavender, which was intended to establish – through phone location data, calls made and contacts saved – whether the user of a phone was a Hamas operative, if so how senior they were and what their home address might be. The system’s degree of certainty was only 85 per cent – and as one of NAZA’s interviewees admits, a baker near a Hamas base will have a phone full of contacts who are Hamas operatives – but this was enough to authorise their killing, along with their entire families. Another of Abraham’s exposés had to do with a system called ‘Where’s Daddy?’ It enables intelligence officers to listen to, among other things, conversations between family members, including a targeted person’s children, happy that their father is home. This technology makes remote killing intimately close. The killers listen to the everyday conversations of the people they are about to kill, and later to the moans and screams of the injured.
The commander of Unit 8200, the intelligence corps that designed the system, referred to ‘the synergy between humans and machines’ as ‘supercognition’. Intelligence officers and legal advisers rarely dispute the results it produces. One of the interviewees says: ‘You have to trust the system.’ And they do trust it, as is made clear by the fact that it takes, on average, just twenty seconds for a target to be authorised. The system allows for the responsibility for ‘industrialised annihilation’ to be shared between coders, operators, legal officers who pre-set the level of collateral damage, officers who approve the strike, officers who choose the bomb and the pilots or drone operators who launch it. ‘I am just a small screw in the system,’ one of the interviewees says, quoting Eichmann’s defence during his trial in Jerusalem. The system that begins with a phone number being put into a computer system ends with an entire family crushed under the rubble.
