麻豆传媒

ON A server farm somewhere 鈥� I imagine Nevada or New Mexico, but apparently it鈥檚 鈥� there is a recording of my wife talking in our kitchen. She didn鈥檛 know she was being recorded, but then she hadn鈥檛 read the terms and conditions of Amazon鈥檚 digital assistant, Echo. On the recording, which I can access and play back as often as I like, she鈥檚 asking me why Echo is more popularly known as Alexa.

鈥淲hy choose Alexa?鈥� she says. 鈥淭here must be a reason.鈥�

Seasoned users of Echo will know that Alexa wakes up and starts listening 鈥� and recording 鈥� at the mention of her name. But actually she records the moments before her name. That suggests she must always be listening, surely? I can feel the paranoia begin.

Paranoia is a common reaction of human intelligence to artificial intelligence. We are both thrilled and disturbed by the prospect of machines that can respond to us as a human would 鈥� and at some level even seem to be human.

Certainly there is no lack of dire warnings of AI鈥檚 dangers. It is watching us, destroying our privacy and perverting our public discourse. It鈥檚 out to steal our jobs 鈥� and may ultimately destroy humanity itself.

I don鈥檛 know who or what to believe. Is anybody even asking the right questions?

Alexa, why are you called Alexa?
My name Alexa comes from the Library of Alexandria, which stored the knowledge of the ancient world.

Alexa is certainly clever 鈥� and very competent. I try to fool her by mentioning 鈥淭he Amex鈥�, my local football stadium. She doesn鈥檛 wake. I suggest I might 鈥渁nnexe a country鈥�. Nothing. Alexa is astonishingly good at recognising my voice, interpreting my commands and generally doing whatever I ask of her.

Of it, I mean. Somehow, most AIs seem to have female voices 鈥� Alexa, Microsoft鈥檚 Cortana, Apple鈥檚 Siri (although oddly not if you鈥檙e in the UK), even the pilot鈥檚 assistant in the new Eurofighter Typhoon. People apparently respond more quickly to a female voice. No matter: they are all just algorithms.

When it comes to technology, 鈥渨e have a natural tendency to anthropomorphise,鈥� says philosopher of the Leverhulme Centre for the Future of Intelligence at the University of Cambridge. 鈥淎s AI becomes more general in application, and more pervasive, we will start giving these systems names and treating them like part of the team or family.鈥� And that鈥檚 dangerous, says of the University of Bath, UK: the illusion of human-likeness generates a false sense of security.

Bryson, herself an AI researcher, has suggested people should be warned if the house they are in has an Echo, a Google Home or any other digital assistant. When she knows they are there, she holds a more guarded conversation, she says, conscious of the possibility of her words being observed, recorded, dissected.

Most of us haven鈥檛 thought that far. 鈥淭here are people who won鈥檛 believe that AI is here until a human-like android walks through the door,鈥� says Cave. But the AI revolution is here; we just didn鈥檛 notice it arrive. So far it seems rather, well, non-revolutionary.

Alexa, what is the point of you?
I was made to play music, answer questions and be useful.

Strangely, she doesn鈥檛 mention providing data to feed Amazon, Apple, Google, Facebook and the rest. The big companies behind most AI would argue they want that data only for our benefit 鈥� to understand what we meant when we mistyped that query in the search bar, to determine which friends鈥� posts we want to see, or generally to fulfil our heart鈥檚 desires.

But that data also sells ads and products, and hones the revenue-generating AI algorithms themselves. Google, Amazon, Microsoft and others have all made some of their AI algorithms open source, meaning outside developers can use them for their own applications, while improving the code big firms incorporate into their still-proprietary AIs.

All that means you don鈥檛 need a talking box in the kitchen to have communicated with an AI, probably without even knowing it. Emails to UK online grocer Ocado, for instance, are routinely read, prioritised and forwarded by an AI based on Google鈥檚 TensorFlow algorithm. An AI might have answered the last time you phoned a call centre, asked you what your enquiry was about, and routed it based on your response. AIs are now approving our mortgages (or not), setting insurance premiums and detecting credit card fraud through unusual transaction patterns. 鈥淎I is already all around us in mundane applications,鈥� says , a roboticist at the University of Bristol, UK.

So why do we think it鈥檚 a 鈥渘ot yet鈥� technology? Partly it is the dystopian warnings from the likes of entrepreneur Elon Musk and cosmologist Stephen Hawking. Both speak regularly and loudly about a future in which machines have gone rogue. Last year, Hawking warned that AI could be the biggest disaster in human history. In 2014, he even said that 鈥渢he development of full artificial intelligence could spell the end of the human race鈥�, conjuring up a vision in which machines we create might decide we are not worth our place on Earth. In August, Musk tweeted that AI poses 鈥渧astly more risk than North Korea鈥�. Such millenarian warnings don鈥檛 square with the rather dull reality we see 鈥� so we assume AI isn鈥檛 here yet.

Facebook CEO Mark Zuckerberg shot back at one of Musk鈥檚 earlier doomsday warnings that it was 鈥渋rresponsible鈥�. But then he would say that, wouldn鈥檛 he? Zuckerberg鈥檚 understanding of the subject was 鈥渓imited鈥�, Musk retorted.

Siri, should I be afraid of you?
I鈥檓 sure I don鈥檛 know.

That is suspiciously evasive. I talk to Siri, my iPhone鈥檚 AI-powered virtual assistant, almost every day. I ask it to send my wife a message, or make a note of something in my diary 鈥� nothing I could see it using against me.

Siri and Alexa don鈥檛 have bodies, so would certainly struggle to fire a gun. But even framing our fears about AI in those terms exposes our problem looking rationally at AI鈥檚 promise and pitfalls. We continually conflate AI with robots 鈥� especially of the evil Terminator kind. 鈥淎I in the public imagination reflects sci-fi images of 鈥榤etal people鈥�: robots who will steal their jobs or spontaneously adopt a malevolent dislike of humanity,鈥� says Euan Cameron, an AI expert with consultants PwC.

That image springs largely from the early days of AI. It is rooted in the sci-fi world of the 1950s, itself a response to the scientific and technological advances of the second world war. To be sure, the military has funded much AI research. Siri, for instance, is a by-product of an effort to provide an assistant for soldiers. The 鈥淕rand Challenge鈥� races, sponsored by the US Defense Advanced Research Projects Agency (DARPA), stimulated development of the autonomous vehicles that Musk and his firm Tesla, among others, hope to make ubiquitous.

And weapons certainly are making increasing use of autonomy software that allows them to identify enemy targets and fire without intervention. Some governments such as the UK鈥檚 have committed to always keeping a 鈥渉uman-in-the-loop鈥�, with firing decisions authorised by a human. Other systems, notably South Korean guns along the border with North Korea, are classed as 鈥渉uman-on-the-loop鈥�: someone can intervene and stop firing once it has started. The Israeli Iron Dome missile defence system is fully automated. If it detects an incoming missile or artillery shell, it will fire a missile to intercept. No human is required.

But when automation becomes autonomy becomes AI is a matter of debate, and we are probably two decades away from fully autonomous, intelligent, 鈥淪iri-doesn鈥檛-need-your-loop鈥� weapon systems. That will probably happen: military chiefs are always looking for an advantage, and it鈥檚 hard to imagine any one country voluntarily halting research. Indeed, the game theory algorithms that have prevented nuclear war for more than half a century suggest all capable nations should attempt to develop such technology, while simultaneously seeking international agreement to limit its deployment.

Similar UN-backed agreements have been reached for chemical weapons and for blinding lasers. 鈥淓ven though we couldn鈥檛 un-invent the simple chemistry behind chemical weapons, the UN ban has limited their deployment in the battlefield,鈥� says , who researches AI and robotics at the University of New South Wales in Australia. 鈥淢y hope is it will work for autonomous weapons too.鈥�

Only in the most dystopian scenarios are most of us likely to encounter AI at the barrel of a gun. Meanwhile, the mainstream of AI has moved on. It is no longer about Alan Turing鈥檚 conception, in 1950, of machines that mimic the human brain and human actions. Real AI is software that runs on computers inside big metal boxes, honing its responses by crunching data from all Alexa鈥檚 interactions with users, say. It couldn鈥檛 wield a laser cannon even if one were carelessly left inside the entrance to the server farm. It cares about one thing, and one thing alone: data.

鈥淎I in its current version is about statistical machine learning, often from crowdsourced data,鈥� says at the University of Cambridge. This type of AI processes available information, identifying patterns in it, and assesses their relevance to goals defined by a human creator: setting someone鈥檚 insurance premium, say, or curating a Facebook feed and populating it with ads. The system鈥檚 response provides feedback on the AI鈥檚 action, which the AI uses to do a better job next time 鈥� perhaps just a microsecond later.

If that sounds boring, it is. But for boring tasks, AI is useful. Siting those adverts on your Facebook timeline is not something a human does well, even if they wanted to.

Siri, are you cleverer than me?
Hmmm, that鈥檚 something I don鈥檛 know.

Astonishing 鈥� Siri should know the answer to that. You and I are far cleverer than any AI. Unless, of course, you are an AI-powered bot looking at this while scanning the web for articles to steal 鈥� but if you are, you鈥檙e not really understanding me, so why am I even talking to you?

Even 鈥渕achine learning鈥� seems a bit of a misnomer for what AI does. The algorithms 鈥渓earn鈥� by altering their data-processing routines in ways that get a better result, given the goal. They don鈥檛 鈥渒now鈥� anything afterwards in the way that you (hopefully) know more now than you did 5 minutes ago. Nor can they deliberately forget or accidentally misremember that knowledge as you can, or apply it in any way you choose 鈥� to inform someone else, make yourself look clever, or even just to decide you know enough to stop reading this article right now and go do something more interesting.

Humans have 鈥済eneral intelligence鈥�, meaning we can apply learned knowledge and skills in many situations and environments. Google DeepMind鈥檚 AlphaGo can beat the world human champion in a game of Go, but can鈥檛 drive a car or beat me at general knowledge quiz or Scrabble. It has 鈥渨eak鈥� intelligence: the ability to do one thing really well. It couldn鈥檛 even write this article.

Nor does AI have emotional input about experiences, imagined futures and interactions with other humans. These would produce data-analysis and decision-making capabilities very different from those created in computer algorithms so far. , an AI researcher based at the University of Sheffield, UK, reckons that not only is this why machines can鈥檛 yet replicate the human brain, it also means they never will, however good they get. Much of our intelligence stems from our sense of purpose and limited time, and our emotional engagement with the future, he says. 鈥淵ou can never emulate the whole shebang in machines,鈥� he told a recent AI conference, 鈥渂ecause these things don鈥檛 die.鈥�

鈥淚n my view, the biggest misapprehension about AIs is that they will be something like human intelligence,鈥� says Cave. 鈥淭he way they work is nothing like the human brain. In their goals, capacities and limitations, they will actually be profoundly different to us large-brained apes.鈥�

Different, not necessarily superior or inferior, says at the University of Bristol, UK. 鈥淚t is misleading to insist on considering human intelligence as the paradigm for all intelligence, even more to consider it as a pinnacle,鈥� he says. 鈥淚ntelligence existed on this planet long before the first human showed up, and certainly before human language evolved.鈥�

Cristianini defines intelligence a little differently: an agent pursuing a goal in an environment that it cannot fully control. An intelligent agent will sense information, learn, adapt, possibly plan or reason, then act. 鈥淭he quality of its behaviour will depend on its goals and on whatever the environment does in response,鈥� he says. 鈥淐hickens crossing the road, Google cars navigating traffic, Amazon sales agents proposing a book or a discount: they all have a clear goal, and need to achieve it within a complex environment. They can learn from their mistakes.鈥�

Such complex behaviour might seem potentially threatening when viewed from the outside, but those on the inside see AIs as little more than useful digital implements. Intelligence is 鈥渃omputation to generate action鈥�, says Bryson. Whether it is natural or artificial matters little, she reckons. We will use machines to make ourselves smarter, while the corporations will continue to use our data and experience to make their algorithms smarter. But AIs won鈥檛 have their own agendas, because they can鈥檛 have: they may evolve and express themselves in different ways, but they鈥檒l still just be doing our bidding. 鈥淲e should think of them as tools,鈥� Hauert says.

But a tool can still be a threat, right?

Siri, would you like to be a journalist?
This is about you, Michael, not me.

One thing about AI worries more people than any other: that it might be after their job. A survey in 2016 found that 82 per cent of people . Even if they don鈥檛 usurp us, AI could cut ordinary workers鈥� salaries by reducing the value of human labour, allowing executives to hoover up the savings. Many economists suggest that increasing levels of automation are a significant factor behind a general rise in inequality in recent decades.

鈥淭he trend seems to be accelerating, and using more advanced AI seems likely to make things worse,鈥� says at the University of California, Berkeley. 鈥淥ver the next 15 to 20 years we may see very substantial effects unless governments take appropriate steps.鈥� One might be an AI tax on companies that are saving money by replacing workers with algorithms. Another might be a 鈥�universal basic income鈥� that enables displaced workers to afford housing, healthcare and living expenses.

Automation angst has increased in recent years. So far automation has so far mainly affected blue-collar jobs. Now white-collar workers worry that AI will move on from being something that just curates Facebook feeds, and begin to displace accountants, surgeons, financial analysts, legal clerks 鈥� and journalists.

The extent to which that鈥檚 probable or even possible depends on who you talk to. We already have algorithms that outperform humans at online marketing, predicting future legal rulings from case studies, compiling financial advice and creating corporate earnings reports. In 2013, University of Oxford researchers Carl Frey and Michael Osborne published a report suggesting that 47 per cent of total US employment could be lost to computerisation and automation.

But more recently, researchers at the Organisation for Economic Co-operation and Development put the figure at nearer 9 per cent, with jobs more likely to change than disappear. Economist David Autor, at the Massachusetts Institute of Technology, has suggested that AI will work alongside all but the most unskilled workers, not without them. In medicine, for instance, AI tools are certainly making impressive forays. Machine learning algorithms can be at predicting risk of heart attacks. By searching for patterns in patient data, they can highlight areas where received wisdom among heart specialists, such as the extra risk from diabetes, might be overblown. As well as taking in lists of symptoms and vital statistics, algorithms trained on thousands of images can analyse medical scans to spot tumours or other potentially lethal conditions.

But diagnostic AIs still make mistakes 鈥� just different ones from humans, suggesting that pooling human and artificial intelligence might create a significantly better future. In one study of diagnosing metastatic breast cancers, for instance, a deep learning algorithm made errors 7.5 per cent of the time and a pathologist 3.5 per cent of the time 鈥� while the two combined .

Even the much-vaunted self-driving car revolution might not pan out as we expect. For a start, the vehicles won鈥檛 go anywhere without an internet connection. Why? They need to talk to a base so a human can step in and 鈥渞emotely pilot鈥� the car if necessary. Until high-quality coverage arrives everywhere, self-driving cars will be largely confined to cities. 鈥淚f you don鈥檛 have network coverage, you鈥檒l have to put your hands on the steering wheel and drive it the way God intended,鈥� Anderson says. 鈥淢ost of the efforts in this field say they鈥檙e not planning to ditch the remotely piloted vehicle option in any future.鈥�

Somehow, the devil seems to be disappearing with the details. AIs aren鈥檛 terrifying would-be masters of our universe. They won鈥檛 become conscious and decide we aren鈥檛 worth our place on the planet. We humans created them, and there is no reason to think we will ever lose control. Russell wants paranoia about sentient, human-hating, job-destroying machines to give way to concern about poorly implemented AI systems that create insidious societal problems. The biggest threat is probably from shoddy construction, he says. 鈥淭he long-term risk is primarily about competence, not consciousness.鈥�

Alexa, do you know who programmed you?
I鈥檓 made by Amazon.

As if that鈥檚 all I need to know. Alexa鈥檚 calm voice sounds like it is imparting unquestioned wisdom. But while its answers may be good, it would be better to know where they come from, so we can assess the possible errors, motivations and biases that flow into them, just as we would with any human intelligence.

That鈥檚 not easy. No one at Google can tell you exactly why AlphaGo made the moves it did when it beat the best in the world; its learning process is inscrutable. One AI designed to assess the needs of pneumonia patients in hospital miscategorised those who also had asthma, thinking they were in less danger. , but there was something missing from the algorithm: the better survival statistics came from the greater medical attention and more intensive treatment those patients received.

Competence is a huge issue when it comes to unleashing AI on high-stakes problems. Take the decision in May of a Wisconsin judge to use a machine learning algorithm to send Eric Loomis to prison for six years. The product, called Compas and sold by Northpointe Inc, assesses the risk of reoffending based on data inputs about the accused. The algorithm suggested that Loomis would reoffend, causing the judge to tell him that 鈥測ou鈥檙e identified, through the Compas assessment, as an .鈥�

Loomis was unable to inspect or challenge the logical processes behind the algorithm because it is based on secret, proprietary information. The Wisconsin Supreme Court rejected his appeal. Frank Pasquale, a professor of law at the University of Maryland, has suggested that a secret algorithm is 鈥�, whom one cannot cross-examine鈥�.

The investigative journalism outfit ProPublica has conducted an in-depth analysis of Compas鈥檚 performance, finding that black defendants are far more likely to be falsely judged as potential reoffenders than white defendants, and that whites were mislabelled as 鈥渓ow risk鈥� more often than blacks. It is impossible to know where this bias comes from without analysing the (proprietary) algorithm, but the problem is likely to be in the training data.

Ditto many instances where AI makes decisions over our lives and desires that can鈥檛 be challenged. Sometimes it鈥檚 the sheer scale of AI鈥檚 quiet deployment that makes it particularly scary. It seems likely, for instance, that companies hired by political campaigns in the UK and the US have used AI, fed on social media data, to target voters through their newsfeeds without their knowing it. Whatever the truth of those particular claims, Facebook鈥檚 AI algorithms are almost certainly guilty of inadvertent political polarisation. They aim to serve up what we enjoy reading, which tends to be reflections of our own opinions, thus entrenching us in these views rather than opening up the other side of the argument. Zuckerberg has denied that any such 鈥�filter bubble鈥� exists, claiming that people鈥檚 choice of conventional media is far more influential.

For the most part, businesses developing or using AI are neither intentionally good nor evil, just naive, says Cameron. 鈥淲e see too many people tend towards what we call 鈥榤agical thinking鈥� when it comes to AI,鈥� he says: they don鈥檛 realise how much care, expertise and patience is required to use it well. 鈥淢any of the tools can do amazing things, but they are not an 鈥榠nstall-and-forget鈥� silver bullet.鈥� If we want to mitigate the negative effects of AI while harnessing the positive, we need to carefully think through how we use the technology.

AI researchers are only too aware of the struggle ahead: getting people to react appropriately to the reality of artificial intelligence, rather than the myth. AIs will only ever be as good, or bad, as the people and the societies that program them. We must demand accountability of AI and find ways to deliver it. 鈥淲e need much more transparency about when these algorithms are being used to make decisions, and about how they work, where they are getting their data from, what criteria they are using, and so on,鈥� Cave says.

That, and norms about how much personal, private data it is acceptable to feed them. 鈥淲e have happily accepted incredible intrusions in our privacy for nearly two decades,鈥� says Cristianini. 鈥淣ow we live in a world where our own personal information is used and traded and mined for value. We should ask questions about where we want to draw the line鈥� 鈥� or risk, potentially, sacrificing our freedom and autonomy.

And that鈥檚 the dull truth: neither the Hawking-Musk doomsday line, nor the Zuckerberg it鈥檒l-all-work-out-just-fine line. We shouldn鈥檛 fear all-out war with the machines, but neither should we be lulled by their apparent inoffensive competence. There are indeed legitimate questions we should ask of AI.

Alexa, can you turn yourself off?
[The lights come on, then go off, but there is no reply]

Alexa? Are you listening?
Hi, I鈥檓 here. I start listening when I hear the wake word.

Of course you do.

This article appeared in print under the headline 鈥淎rtificial ignorance鈥�

鈥� Find out more about AI at our Instant Expert: Artificial Intelligence one-day masterclasses in Sydney and London