
ROBOTS are taking over Wall Street. “Technology has utterly transformed the financial system,” says Andrew Lo, an economist at the Massachusetts Institute of Technology. “The vast majority of day-to-day trading is done purely algorithmically.”
More and more are being shown the door. And researchers like Lo are beginning to find that the more the stock market is run by machines, the less it behaves like one. Today’s markets are an ecosystem, a zoo of bots grazing on our pensions and investments – and no one quite knows how they work.
Is this newly autonomous market a route to financial prosperity, an end to boom and bust? Or are we a few lines of code away from financial doom?
Advertisement
To understand why machines are taking over, it helps to look at how perceptions of the stock market have changed following the financial crisis of 2007-08. It is increasingly clear that for the average person, investing is a mug’s game. Individuals have little hope of picking successful firms to back, while giving your money to investment managers who aim to beat the market often sees any gains being eaten away by a laundry list of opaque fees.
The alternative, espoused by the likes of finance tycoon Warren Buffet, is to . A market index is a collection of companies that give a snapshot of the market’s value at any given time, like the FTSE 100 and the Standard & Poor’s 500.
Largely because the stock market as a whole has been on the rise since the financial crash, index funds have become a no-brainer. They are also much cheaper, essentially run by software that follows simple rules to track the index, with little human direction.
These low-cost, “passive” investments are starting to crowd out conventional, actively managed funds. Over the past three years, passive funds added nearly $1.3 trillion to their coffers, , while actively managed funds lost about $250 billion. “There’s so much money going into passive funds,” says , who studies the impact of technology on finance at University of Maryland School of Law.
But paradoxically, active funds may be growing in influence. Passive funds can’t decide when to buy or sell a stock, because they only follow the market, meaning they essentially follow the lead of active trading. “Active funds have a disproportionate impact on markets,” says Lo.

That doesn’t mean humans are in control. Like the rest of the stock market, most active funds are run by algorithms that move faster than human brains can comprehend. And that’s where the real trouble starts.
The most extreme active funds are hedge funds, which number in excess of 9000 globally. These are in constant churn – new funds begin every day to exploit opportunities, while others close because their strategy failed. “It’s the Galapagos Islands of finance,” says Lo.
The latest evolutionary niche is occupied by algorithms that scour markets for patterns that yield a competitive edge. Before that, it was high-frequency trading (HFT) – the subject of much hand-wringing a few years ago.
Unforeseen crash
On 6 May 2010, the Dow Jones Industrial Average fell 600 points in 5 minutes and share prices became incomprehensible for half an hour. Authorities initially blamed unforeseen interactions between HFT algorithms, with no single identifiable culprit. Then, two years ago, the US Department of Justice filed charges against , saying his bots spoofed the market and caused the chaos.
Fears of HFT have since faded. , a lawyer at Temple University in Philadelphia who specialises in financial technology and regulation, says this is partly down to them falling out of the media hype cycle, but also because now everyone uses them, they are less profitable.
But other fears remain. “The symbiosis between technology and finance has accelerated the pace of the financial markets beyond mere human capacity at all levels of the financial system,” says Lo. “Whatever can go wrong, will go wrong, and faster and bigger when computers are involved.”
Even when all the algorithms are behaving exactly as they’re meant to, financial technology has brought us fire sales, flash crashes and catastrophic algorithmic trading errors.
On 15 October 2014, the US Treasury market crashed for about 10 minutes. Experts hypothesised that “activities of electronic trading algorithms” bore part of the blame, but reserved judgement for when they had more information. Three years later, .
What this flags up, says Lo, is that it’s not so much the individual rogue algorithms we need to worry about any more. Rather, it’s the way they reinforce and affect one another, which can quickly snowball, even when everything is working as it should be. “We don’t understand the network,” he says. When everything is interconnected, a financial crisis could start anywhere and affect anyone. “We have no map of the entire system. Even the regulators probably don’t,” says Lo.
So he has a solution: when surveying the financial world, we need to study it the way an ecologist might. Within the financial landscape, investors, managers, regulators and policy-makers are simply individuals who innovate, compete, adapt, reproduce (their ideas) and evolve, he says. “What are the keystone species? The predator-prey dynamics? We need to see how different agents evolve and adapt in response to the actions of the others.”
“Whatever can go wrong, will go wrong, and faster and bigger when computers are involved”
But when today’s technology can compress the entire life cycle of a stock market crash and rebound into fractions of a second, how can we keep up? We can’t, says Lo, without the laws that govern biology.
For example, natural selection tells us that if a pool of organisms is more diverse, it is more likely that one of them will develop an adaptation that is exquisitely suited to its environment.
But that’s not what’s happening with investors flocking to index funds. A larger pool of assets is simply following the market. That means less diversity in trades. As these passive funds accrete more people and their money, a problem emerges that would be familiar to any ecologist: they are forming a monoculture. When every investor is doing the same thing at the same time, the rapidly evolving hedge fund algorithms seem like they’re poised for a big lunch.
Algorithmic zoo
So what happens when the monoculture collapses? If the market starts to tank, and passive funds follow it down while the hedge funds munch on the remains, investors are likely to take out their assets and put them into cash. To understand how and when such a freak-out might start, Lo wants to add humans, and our irrational biases, to the algorithmic zoo.
At the end of September, Lo presented a paper at the in Boston that does exactly that: it models the most relevant human biases and how they interact, with a view to predicting the forces these human algorithms exert on the markets. For example, there’s the sunk-cost fallacy – the more one has invested in something, the harder it becomes to abandon it.
Armed with this information, we can develop more adaptive regulations. If, for example, it looks like irrational exuberance is causing the system to become too dependent on some particular kind of index fund, it might become beneficial to encourage greater diversification in the industry by closing certain funds to new investors.
Many have lauded Lo’s ideas, but Pasquale isn’t so sure the solution is biology. “People think all this complexity is somehow inevitable. It’s not. It’s there to shield wealthy and powerful people when things go wrong.”
He says there’s an easier way to understand these complex markets – regulate the market environment back to the low-complexity 1920s. “I’m not fan of HFT,” he says. “I’d tax them so hard they’d become economically infeasible.”
Ultimately, we need to decide what purposes we want the markets to serve, says Lin. “The toughest questions in finance and AI right now are not about the technology itself, but about what kind of human values and ethics we want to embed into the world’s financial systems.” Lin likens the problem to the philosophical debates surrounding the introduction of driverless cars.
The upshot is that the model of the stock market that has been in place since the 1930s is being upended by our new robot masters of the universe – it just doesn’t work the way it used to. Since the crash, the market has been rising steadily. Is that because the machines are doing a better job, or is this yet another cyclical bubble? We just can’t know, and won’t know until the whole thing goes bang.
This article appeared in print under the headline “The money machine”