The US Air Force can now equip autonomous drones with face recognition technology, raising fears that they could be used to find and kill specified people.
The drones will be employed by special operations forces for intelligence gathering and for missions in foreign countries, according to Ā and Seattle-based firm The companyās software, based on machine learning, is designed to work on a drone that is piloting itself, with limited or no communication with a human.
The contract has been fulfilled, but it isnāt known whether the software has yet been rolled out or what drones might be running it. RealNetworks said it was unable to comment.
Advertisement
A big area of concern, says at the University of California, Berkeley, lies in the contract document saying the software will āopen the opportunity for real-time autonomous response by the robotā.
āHard to see what else it refers to, other than lethal action,ā he says.
In 2017, Russell released a viral video presenting a fictional future in which small quadcopter drones called that were armed with explosive charges hunted down specific individuals using face recognition. The new project appears to make this a practical possibility, he says.
āThe difference between a lethal autonomous weapon, and a weapon capable of assassinating individuals by facial recognition, is just one of software,ā says Russell. āItās probably not possible to detect or prevent software updates that would convert one to the other.ā
The US policy on lethal autonomous weapons, which the country in a conference in the Netherlands earlier this month, doesnāt require direct human supervision, but calls for āappropriate levels of human judgmentā. The Pentagon hasnāt explained what exactly this means.
Russell says a drone finding, identifying and attacking someone from a list of targets might be allowed under these guidelines. āSome might argue itās not the weapon thatās selecting the target, but the human operator āpre-approvingā.ā
, director of the artificial intelligence governance project at the Center for Strategic and International Studies think tank in Washington DC, doesnāt believe the technology is intended for robotic assassination. āI sincerely doubt this has anything to do with lethal autonomous capabilities,ā he says. āI have never met anyone in the DoD who is even slightly interested in using facial recognition for autonomous anti-personnel targeting capabilities.ā
A droneās recognition of a face might not be the sole factor used in identifying whether someone is a person of interest. The military and police services also use other methods, such as theĀ known to be associated with someone. But regardless of whether the drones act against a target or just raise an alert, there is a risk they could identify the wrong person.
In theory, automated face recognition should be capable of identifying subjects with a high degree of confidence, says at the University of Lincoln, UK. āThe world standard test of face recognition algorithms now puts the top algorithms as having a false positive rate ā that is, incorrectly identifying the wrong person ā of less than 0.5 per cent.ā
However, these results rely on high-quality images. Achieving good results with a small drone camera at long range and in variable lighting would be more challenging, she says. āThe algorithm would need to be specifically trained to function well under these conditions.ā
There may be other issues, too, making misidentification more likely. Some algorithms are less effective at distinguishing between people of certain ethnicities, says Ritchie.
āThe main point here is that it is so important for users of face recognition technology to understand the capabilities of the algorithm they are using,ā she says.
A spokesperson for the says: āDetails such as those involving safety or other operational measuresĀ involving this capability are not releasable with respect to operationalĀ security considerations. However, DoD personnel are directed to exercise appropriate levels of judgement and care, while remaining responsible for the development, deployment, and use of AI capabilities.ā
