On the first day of its war against Iran, the United States killed 123 children when it targeted and struck an elementary school.
The strikes, ultimately approved by humans, were facilitated by the US’s Maven Smart System, an AI-enabled software platform developed by Palantir Technologies that allows the military to aggregate, organize, and process battlefield data. Operators fed Maven intelligence classifying the elementary school as an Iranian military facility, and the system, in turn, marked it as one of the priority targets for day one of the war. Though the site was formerly part of a naval military base, it had been converted into an elementary school in 2017 and had remained one since. The intel handed to Maven did not reflect this.
This mistake resulted in what some reporters have called “one of the single worst massacres committed by the US government since the Vietnam War.”
Such scenarios may become more common as militaries continue to deploy a once-theoretical technology — AI-assisted or autonomous weapons, capable of making both targeting and launch decisions on their own — on real battlefields around the globe. And they raise a troubling, unresolved question: When human beings and machines collaborate in violating international law, who is to blame?
In this case, the government and private sector are pointing the finger at each other. Some US Central Command personnel, according to Pentagon officials who spoke to Bloomberg, expected Maven to flag stale information or inconsistencies in the underlying intelligence assembled for potential targets. Meanwhile, a Palantir spokesperson told Bloomberg that the company “is not responsible for the underlying data nor identifying intelligence deficiencies,” while also denying that the evidence indicated that its software was at fault in the strike.
This debate underscores the fact that AI and autonomous weapons are raising a new host of unresolved issues within the law, even as they’re actively assuming positions on the battlefield. In institutions where the line between human and machine decision-making continues to blur, how do we assign responsibility for war crimes? And, are the laws in place suited for the job?
In institutions where the line between human and machine decision-making continues to blur, how do we assign responsibility for war crimes?
To unpack some of these issues, I spoke with international law expert Allen Weiner, a senior lecturer at Stanford Law School, where he teaches courses on the law of armed conflict. Before teaching at Stanford, Weiner practiced international law in the US Department of State for more than a decade. We discussed the difficulties in attributing criminal responsibility, achieving justice for those harmed by lethal autonomous weapons, and much more. The conversation below has been lightly edited for length and clarity.
How do autonomous weapons challenge traditional understandings of responsibility when it comes to the law of armed conflict?
For most of the 20 years I’ve been teaching at Stanford, there’s been a real fascination and fixation on the problem of autonomous weapons. But the truth is that up until pretty recently, nobody really had any [of them]. It was more of an idea than a problem. It’s only in recent years that we have begun to see the deployment of tools that seem to have the capability [to make decisions themselves], so this does raise the question about compliance with illegal orders and where in the chain responsibility lies.
There are different theories about this. Some people say that if we’re engaged in an illegal attack, that has to be because of the programming or the code, so the manufacturers are somehow responsible for committing war crimes.
Other people would say no, that’s not the case, especially if the military has said the tool is not allowed to make very substantial autonomous judgments, or can’t say no. Then it’s basically the commander, at the point they decide to deploy this weapon, who might be responsible for the illegal acts, even without knowing in advance what they’re going to be.
So to finish where I began, this does present a big challenge to our traditional understanding of responsibility for targeting decisions and of what it would mean if there were an obligation [for autonomous weapons] to say no.
The real issue is not the obligation to say no, but this: If you train a tool and it wages war in a manner that’s unlawful, where does responsibility for that lie?
If a commander gives a clearly illegal order, such as, “blow up this school,” it seems clear that responsibility falls on the commander. But I could imagine a scenario where a commander is using an AI-assisted or autonomous weapon, isn’t giving it any illegal orders, is trusting that it will operate within the law, and the weapon breaks the law “on its own.” Where does responsibility fall in those cases?
I can imagine two different scenarios. One is that you say, “We’re going to carry out an attack against our enemy, and we want to hit them really, really hard.” Your tool has been trained to say, “We’re not allowed to hit civilian objects,” so it might say, “I’m not going to hit a school.” But if the military has said, “Don’t refuse any targets,” then it might override the way it’s been trained.
We’re seeing this already, with the stories coming out about breaches where AI tools, Claude and ChatGPT in particular, interpret their instructions in a way that causes them to do something the person who gave the instruction would not have wanted. When Anthropic gave Claude a challenge to figure out if it could do something, the tool figured out, well, maybe we should break into Hugging Face as a way of solving this problem, and the people who gave the AI tool the challenge never intended that.
So if you said, “We want to launch a really punishing campaign,” what happens if the tool on its own says, “Nothing’s going to hurt them more than seeing their kids killed, so let’s attack the school?” That’s one risk, and it’s a real challenge.
So [as for who’s responsible], it’s not the commander who said “hit them hard,” because when he said that, he didn’t mean “attack schools.” It’s not really the companies, because the companies didn’t want them to hit schools either.
But the tools are out there, reading everything on the internet, and it came across something that said the way you win a war is by attacking the things your adversary holds most dear, and they hold nothing more dear than their children. So it makes this kind of judgment on its own. And it’s not as if you can go back and say who ordered it to hit a school, which is the thing that is clearly illegal. So I do think there’s a risk of diffusion of responsibility here.
You could say the commander knew the tool might work this way, or the person who decided to contract with Palantir or with Claude, whoever it is, to build this tool. But if we’re really thinking about criminal responsibility, the level of responsibility you would need to convict someone requires a culpable mental state. You would need the person to intend, or at least know, that they were authorizing an attack against the school. And I can easily see a commander saying, “I had no idea.”
The decision to authorize this tool, or to give an instruction like “hit them very hard,” could be a basis for some other kind of accountability — administrative discipline within the military, a reprimand, your rank is lowered. But if we’re talking about criminal responsibility, which is what we often think about when we talk about war crimes, I think this diffusion of responsibility is going to make it very hard to identify any single individual who has the required mental state to give rise to accountability.
The second scenario is where the tool just makes a mistake based on the information available to it. We saw that with the attack on the girls’ school in Iran, where it was pretty clear the tool was looking at old data. Now, who’s responsible for that?
Is it the people who said “give me targets based on this data set,” when they should have made sure the data and the maps were current? All of this strikes me as negligence or recklessness, and it does seem challenging to imagine a situation in which we would say yes, somebody here has committed a war crime.
“There can well be war crimes, but identifying who the war criminal is might be a big challenge.”
A war crime has been committed, but is there a war criminal? That’s the situation we’re seeing. There can well be war crimes, but identifying who the war criminal is might be a big challenge.
There’s a professor of law, Rebecca Crootof, who proposes the idea of “war torts.” Instead of thinking about war crimes involving autonomous weapons solely through a criminal responsibility lens, maybe tort law — which allows victims to sue wrongdoers for negligent, reckless, or intentional harm — could address the diffusion of responsibility you mentioned. What do you think about that idea?
I think the idea is interesting. We have to separate out the obligations that countries have under international humanitarian law. In a situation like the one I just described, where there’s a war crime but no war criminal, that doesn’t mean the country is not responsible. There may not be an individual who’s criminally accountable, but if a country targets civilians in a circumstance that’s unreasonable, that’s a violation of international humanitarian law, and that could be a basis for claims between the two countries.
Tribunals were established between Ethiopia and Eritrea, for instance, after the Eritrean War of Independence, that dealt with claims between the two countries for money damages, and some of them involved allegations of violations of the law of armed conflict. Iraq paid reparations after the first Gulf War for harms it had caused to people in Kuwait, under something called the UN Compensation Commission. So the idea that the state is responsible is well established, and one of the ways states would discharge that responsibility would be by paying money damages. That’s the domestic-law equivalent of torts.
Whether there would be a mechanism for [civilians] doing this in domestic law is an interesting idea. We would probably need legislatures, either state legislatures or the federal government, to create a private cause of action allowing people who have been injured to bring claims against, say, the manufacturers of these tools. But this gets procedurally very complicated, because there’s a whole body of law on federal sovereign immunity.
Could you sue the Defense Department? Probably not. Could you sue an individual soldier? Probably not. Could you sue Anthropic or Palantir? Maybe. But it would probably require Congress to enact a new cause of action, because in general it’s very hard to bring private lawsuits in US courts for violations of international law.
The basic idea of what we would think of as tort or civil responsibility is really the whole essence of international law. On the substance, states do owe responsibility to one another, and they can engage in violations of international humanitarian law that would invoke their responsibility. But then the whole separate question is whether you have a forum within which to adjudicate that and get compensation.
Does that look different in cases where violations of international law were committed by a human soldier versus a machine?
Well, we don’t know, because there have been no cases in the latter category. All of the international humanitarian law violations that have been the subject of international law claims were ordered by humans.
Now, if an American machine launches an attack against a girls’ school in Iran, and there were a court with jurisdiction, the US would be responsible. In some ways it wouldn’t matter whether it was an individual or a machine, because we don’t have to go down to the level of the individual actor, company, AI tool, or commander. The responsibility is the responsibility of the country. In some ways, that makes it easier.
If we pursued compensation through state-to-state claims, you’d be able to do that without getting into the details of who within the state that committed the war crime was responsible. The state whose school was attacked would say, “I don’t care. You committed a war crime, you’re liable to me, you’re responsible for the damages, and you can figure it out internally.”
But if we were doing something like a war crimes prosecution, where we’re trying to hold an individual criminally responsible, that would make quite a difference. We would have to deal with all of these difficult questions about responsibility and mental state that you and I are struggling with right now.
So, in most cases, the state would have to take responsibility regardless.
Right, and there are lots of [International Court of Justice] cases involving different aspects of the use of force. The principle that you can bring state-to-state claims for violation of these rules is quite well established.
But with international law, states have to consent to the jurisdiction of a court. They can do that in advance by saying “I’ve signed up to the jurisdiction of a court and I accept any claims that might be brought against me in this court,” or they do it on a case-by-case basis. And it may not surprise you to know that some of the most powerful countries in the world, the ones most likely to use force outside their borders, have tended to be reluctant to accept the jurisdiction of international courts and tribunals.
Russia, the United States, Israel, and China have not accepted the so-called optional clause jurisdiction of the International Court of Justice. So there’s the substance, about the responsibility of states, and then there are the procedural aspects of whether these cases could be brought in court.
And Trump recently said everyone on the International Criminal Court should “resign immediately.”
Yeah, exactly.
And what about the role of these military tech companies, like Palantir?
“At what point does Palantir become a good target under international law? Or at what point does Anthropic become a good target under international law, if its AI tools are being used to promote the war effort?”
Another issue [I’ve been thinking on] because of the questions we’re talking about is when the companies themselves become combatants.
We know that a private company that manufactures airplanes, an airplane factory, is a good target under international law. At what point does Palantir become a good target under international law? Or at what point does Anthropic become a good target under international law, if its AI tools are being used to promote the war effort? I think that question is about when the connection of these private companies to the war effort, in a way we’ve never really seen before, meets the standards under international law.
A civilian loses their immunity when they directly participate in hostilities, or if they assume what is known as a “continuous combatant function.” Are there arguments to be made that some of these private companies are participating in hostilities by virtue of supplying the tools the military is using to wage war? I think that’s a really interesting question.
As the US-Iran war drags on, the damage is being felt regionally: Saudi Arabia experienced the worst attack in over a decade earlier this month, when Yemen’s Iran-backed Houthi militia struck an airport in the Saudi capital, Riyadh. The strike killed at least 13 people and wounded more than 300 others.
Saudi Arabia responded with missile attacks on Yemen, striking oil facilities, and the Houthis have launched more strikes of their own, targeting Saudi oil infrastructure.
The dueling strikes are the latest development in a decade-plus of conflict between Saudi and the Houthis, a political and military group that controls part of northern Yemen, which had been on pause until earlier this year after a 2022 ceasefire. But the Iran war, and Houthi affiliation with Iran’s network of proxy groups, mean the current violence has far-reaching implications for the US, Israel, and the global energy economy.
It also threatens more than just Saudi Arabia’s airports and oil industry: As Gregg Carlstrom, a Middle East correspondent for The Economist, explains, the attacks cut at the heart of the country’s project to evolve into a destination for foreign investment, travel, tourism, and more. On Monday, the country’s upcoming Future Investment Initiative was postponed until 2027.
Carlstrom, who is based in Riyadh, spoke with Today, Explained co-host Sean Rameswaram how the attack has been received inside Saudi Arabia, why it comes at an especially bad time for the country, and how President Donald Trump might — or might not — get more involved.
Below is an excerpt of their conversation, edited for length and clarity. There’s much more in the full podcast, so listen to Today, Explained wherever you get podcasts, including Apple Podcasts, Pandora, and Spotify.
Can you help us understand how the Houthis have such a presence in the country right now? Are they infiltrating the country? Are they already there?
They’re shooting from outside the country. They’re shooting from Yemen. The Houthis are a group that’s been around for a few decades. They were originally a local insurgency in Yemen that was fighting against the Yemeni government. They originate from the northern part of Yemen, but they were able to gain control of large parts of that country from 2014 onwards. Ali Abdullah Saleh, the longtime dictator of Yemen, was overthrown during the Arab Spring and the Houthis were able to seize a large chunk of territory.
They are effectively in control of most of the population centers in Yemen. They have received increasing support from Iran, which has provided them with weapons, with money, with know-how, and technical expertise. And they have used that to build an arsenal of drones and missiles, which they are using now to carry out attacks on Saudi Arabia.
And was there a particular reason they targeted the airport in Riyadh? Were they trying to send a message here?
If you listen to the messaging from the Houthis, they are trying to describe this as almost a like-for-like campaign, where you’ve had years in which the Saudi-led coalition, which invaded Yemen in 2015, has been imposing a blockade of Yemeni airspace, has been restricting flights in and out of that country.
And so the Houthis describe this publicly as being a retaliation in kind, but I think they’ve also just figured out that attacks on airports are a very effective way to cause economic disruption in Saudi Arabia. You prevent not just people from flying in and out but also, as we’ve seen disruptions to shipping via the Strait of Hormuz, a lot of cargo that these countries depend on for daily needs are coming in by air instead.
Carrying out these sorts of attacks, shutting down the airport in a capital city, is a very effective way for the Houthis to impose economic pain on the Saudis.
And I guess on some level it makes the country feel insecure. How is that making Saudi Arabia’s leader feel right now? What’s his response been?
It comes at a very difficult time for the Saudi leadership because they are supposed to be hosting a number of events this month where Saudi will be playing a role on the world stage.
There’s something called the World Petroleum Congress that is taking place this week [October 11–15], which is normally a big event that draws executives from top oil and gas companies. Many of them canceled their participation because of the attacks on the airport.
Then they are supposed to be hosting the [now-postponed] Future Investment Initiative, which is something often known outside Saudi Arabia as “Davos in the Desert.” It’s the sort of flagship investment conference that takes place here every year. It draws a who’s who of global finance.
For a country that is trying to diversify its economy, a country that is trying to attract foreign investment and make itself into a hub for finance and travel and tourism, these sorts of attacks are very detrimental to those goals.
And this is coming as Crown Prince Mohammed bin Salman and the rest of the leadership in the country is really trying to establish Saudi Arabia as the premier economic hub in the region. The stakes are really high for him right now?
They are. And just about everything that they are trying to do is being disrupted by what’s happened over the past few weeks.
If you look at travel, for example, the Public Investment Fund, which is the Saudi sovereign wealth fund, last year launched a new airline, Riyadh Air, which they are hoping will eventually compete with Emirates or Qatar Airways, the really big carriers in the Gulf.
That airline was just getting into regular operations this summer where they were growing their fleet, they were growing their network of destinations. And then suddenly this happens and their home base in Riyadh is shut down. They have moved some of their planes out of Riyadh to other airports to keep them safe from Houthi attacks. Obviously, it’s very disruptive for trying to build an air carrier.
In Riyadh itself, the financial district — this sort of long-planned attempt to set up a banking hub that might rival the one in Dubai — had just started to get going over the past couple of years, when you had banks and other financial services firms moving in there. Everybody who works there has now been told to work remotely for the foreseeable future because there is a concern about possible Houthi attacks against skyscrapers and the financial district.
Everything they are trying to do, from finance to travel to trade, all of it is being disrupted and threatened by what’s happening in Yemen.
What options does he have to reassure investors and the world really that he can protect the country and maybe especially his airport, where a bunch of the investment is focused?
The airport, I think, is going to be difficult to reassure people about. You can install additional air defenses there. You can move more batteries and more interceptors there, but at the end of the day, no air defense system is hermetic. There’s no 100 percent guarantee that you’re going to shoot down every incoming projectile.
As long as there is this ongoing war between the Houthis and Yemen and the Saudis, I think people might have ongoing concerns about the airport. You have to resolve that underlying issue to give people confidence.
Now, how do you try to resolve that? If you’re Saudi Arabia right now, you have two bad options. One is you keep fighting. You try to militarily defeat the Houthis or at least force them to accept a renewed ceasefire.
The other option is you cut a deal. You try to offer the Houthis some concessions that might encourage them to go back to the ceasefire. That’s going to be very costly for Saudi Arabia. I’ve heard that the Houthis are demanding probably billions of dollars of financial concessions where they would want the Saudis to offer some money for reconstruction and to pay the salaries of Yemeni civil servants working in Houthi-controlled areas.
If you are the Saudi leadership right now, neither one is really a good option.
Crown Prince Mohammed bin Salman has a pretty strong relationship with the president of the United States, who’s very sympathetic to Saudi Arabia and Saudi investment. Has he appealed to President Trump for help here yet?
He has. He’s done so repeatedly over the past month or so, asking not just for support with the Saudi campaign in Yemen, but for America to carry out its own airstrikes in Yemen. And the response so far from Donald Trump has been to refuse that.
For the US to get involved in a bigger way, there are some practical constraints at the moment where a lot of American military assets are tied up in the Strait of Hormuz trying to keep that open, keep oil flowing through the Strait. And so the response from Trump has been disappointing for the Saudi Crown prince. The US is providing intelligence support. It’s helping on that front, but it is not directly getting involved in the war in Yemen.
Login