- Tech leaders including Elon Musk and the cofounders of DeepMind signed a pledge last week to never develop "lethal autonomous weapons."
- The letter argued that morally the decision to take a life should never be delegated to a machine and that automated weaponry could be disastrous.
- An Oxford academic, Dr. Mariarosaria Taddeo, told Business Insider that while the pledge's intentions were good, it missed the real threat posed by artificial intelligence in warfare.
- She believes the deployment of AI in cybersecurity has flown under the radar — with potentially damaging consequences.
Promising never to make killer robots is a good thing.
That's what tech leaders, including Elon Musk and the cofounders of Google's artificial-intelligence company, DeepMind, did last week by signing a pledge at the International Joint Conference on Artificial Intelligence.
They stated that they would never develop "lethal autonomous weapons," citing two big reasons. The first is that it would be morally wrong in their view to delegate the decision to kill a human to a machine.
Second, they believe that autonomous weapons could be "dangerously destabilizing for every country and individual."
Swearing off killer robots is missing the point
Business Insider spoke with Dr. Mariarosaria Taddeo, of the Oxford Internet Institute, who expressed some concerns about the pledge.
"It's commendable — it's a good initiative," she said. "But I think they go in with too simplistic an approach."
"It does not mention more imminent and impactful uses of AI in the context of international conflicts," Taddeo added.
"My worry is that by focusing just on the extreme case, the killer robots who are taking over the world and this sort of thing, they distract us. They distract the attention and distract the debate from more nuanced but yet fundamental aspects that need to be addressed."
Is AI on the battlefield less scary than in computers?
The US military makes a distinction between AI in motion (i.e., AI that is applied to a robot) and AI at rest, which is found in software.
Killer robots would fall into the category of AI in motion, and some countries already deploy this hardware application of AI. The US Navy received a self-piloting warship, and Israel has drones capable of identifying and attacking targets autonomously, though at the moment they require a human to give the go-ahead.
But AI at rest is what Taddeo thinks needs more scrutiny — namely the use of AI for national cyberdefense.
"Cyberconflicts are escalating in frequency, impact, and sophistication," she said. "States increasingly rely on them, and AI is a new capability that states are starting to use in this context."
The WannaCry virus, which attacked the UK's National Health Service in 2017, has been linked to North Korea, and the UK and US governments collectively blamed Russia for the NotPetya ransomware attack, which took more than $1.2 billion.
Taddeo said throwing AI defense systems into the mix could seriously escalate the nature of cyberwar.
"AI at rest is basically able to defend the systems in which it is deployed, but also to autonomously target and respond to an attack that comes from another machine," she said. "If you take this in the context of interstate conflict, this can cause a lot of damage. Hopefully, it will not lead to the killings of human beings, but it might easily cause conflict escalations, serious damage to national critical infrastructure."
There is no mention of this kind of AI in the IJCAI pledge, which Taddeo considers a glaring omission.
"AI is not just about robotics," she said. "AI is also about the cyber, the nonphysical. And this does not make it less problematic."
AI systems at war with each other could pose a big problem
Today, AI systems attacking each other don't cause physical damage, but Taddeo warns this could change.
"The more our societies rely on AI, the more it's likely that attacks that occur between AI systems will have physical damage," she said. "In March of this year the US announced that Russia had been attacking national critical infrastructure for months. So suppose one can cause a national blackout, or tamper with an air-control system."
"If we start having AI systems which can attack autonomously and defend autonomously, it's easy that we find ourselves in an escalating dynamic for which we don't have control," she added. In an article for Nature, Taddeo warned of the risk of a "cyber arms race."
"While states are already deploying this aggressive AI, there is no regulation," she said. "There are no norms about state behavior in cyberspace. And we don't know where to begin."
In 2004, the United Nations assembled a group of experts to understand and define the principles of how states should behave in cyberspace, but in 2017 they failed to reach any kind of consensus.
Still, she thinks the agreement not to make killer robots is a good thing. "Do not get me wrong — it's a nice gesture," she said. "It's a gesture I don't think it will have massive impact in terms of policymaking and regulations. And they are addressing a risk, and there's nothing wrong with that. But the problem is bigger."
Join the conversation about this story »
NOW WATCH: This hands-free crutch takes the strain off your hands, wrists and arms