AS THE coronavirus pandemic swept across the US, it brought with it an unprecedented economic crisis. As firms shut down and people stayed home, the country’s unemployment rate adding fuel to a political fire already raging in a tumultuous election year.
That much is well known. But the stories of many of those who lost their livelihoods and sought help exposed a slower-burn technological crisis. as they attempted to deal with the flood of people applying for welfare benefits – and hardly anyone around knew how to fix things.
Advertisement
It is far from an isolated problem. Tangled webs of computer code built up over decades, often written in programming languages now rarely taught or understood, underpin IT systems across the world, in government departments, banks, airlines, hospitals and more. Coronavirus taught us a lot about how the systems we had assumed would assist and protect us can fail in a crisis. As the fallout continues, it is becoming ever clearer that we need to revisit the computer code that underpins many aspects of our societies before disaster strikes.
Thousands of different programming languages exist, performing the same basic job: translating real-world commands such as “import this data” or “run this calculation” into the strings of binary 1s and 0s that encode information in computer processors and memory chips. Certain ones dominate (see “Top five languages”), but new languages pop up as requirements change. Google developed the Go language, for example, to streamline the development of massive applications running across hundreds of servers in the cloud. “There’s still a rich space out there where people are exploring new ideas and trying to make things better,” says at the Massachusetts Institute of Technology.
As new languages become favoured, so others fall out of use or find a different purpose. Fortran, for example, was developed by IBM in the 1950s for general business use. It went out of favour in corporate circles, but is still prized by physicists for its mathematical chops, thanks to its ability to run many operations in parallel at breakneck speeds.
Other languages stick around, unfashionable, but too deeply embedded in computing systems to get rid of. COBOL, or the common business-oriented language, is a prime example. When first released in 1959, it was aimed at allowing corporations to program business software on large mainframe systems. It was wildly successful. “The staying power of COBOL is the fact that it’s easy to use,” says Barry Baker at IBM in New York.
With the advent of personal computing and the internet, COBOL lost ground to newer, more flexible general-purpose languages, but roughly 220 billion lines of COBOL code still support the systems behind businesses and other institutions worldwide. Perhaps most significantly, it underpins huge chunks of the world’s financial sector. According to Reuters, run on COBOL and 95 per cent of ATM transactions still rely on the language.
COBOL cowboys
The extensive use of COBOL in welfare processing systems was seemingly behind the US unemployment benefit fiasco. Universities have stopped teaching this programming language, and when state governments needed to scale up their systems quickly to deal with the surge in demand, skilled labour was in short supply. New Jersey governor Phil Murphy , and a group of ageing coders dubbed the swung into action. IBM released a free COBOL training course. Despite this attempt to improve matters, a survey by the Economic Policy Institute think tank in Washington DC has found that reliance on COBOL caused real problems. For every 10 successful applicants in the initial phase of the covid-induced US jobs crisis, three or four others didn’t receive their benefits. Many people went months without income.
Part of the problem is that, while most programmers could learn COBOL in a few weeks, picking up its vocabulary and grammar is only part of the challenge. Mastering how a coding language is used in practice, and its common styles and patterns or idioms, is no less important. Most computing languages have large libraries of ready-made snippets of code that streamline the programming process. Understanding how to draw on this literary canon is as much a key to fluency in a programming language as it is in any spoken language. Opaque turns of phrase, plus coding conventions that can vary significantly between domains or even organisations, make deciphering a specific bit of software difficult for an outsider.
“You hear these stories of people rehiring this old guy in his 70s who’s retired,” says at the University of Oxford. “You’re not getting this guy back because he knows COBOL, you’re likely getting someone who has worked on that particular piece of software in the past.”
Building complex software from scratch is expensive and time-consuming, so code is also frequently reused and adapted. This means earlier decisions become deeply embedded in software that runs present-day systems. Over time, “dark shadows” start to appear in the labyrinths of code built up at large organisations, says , director of the Information Innovation Office at the US Defense Advanced Research Projects Agency (DARPA). “There’s certain components that the programmers dare not touch,” he says. “There’s fear and superstition.”
And the strain is showing. A 2019 report from the US Government Accountability Office (GAO) whose creaking code is expensive to maintain and increasingly prone to serious failures or hacking. These include those that underpin the federal Social Security Administration, that keep the Air Force’s planes battle-ready and even those that operate major dams and power stations. “Think about how many people, how much infrastructure, how much capital is downriver from a dam,” says Carol Harris at the GAO, who wrote the report. “Imagine if that were hacked or went offline and the dam went through a catastrophic release.”
It isn’t just government bodies affected by this. Last year, the UK Financial Conduct Authority said it had received at financial institutions in 2018/19 – a dramatic increase on the previous year as banks, trying to compete with finance start-ups, raced to add new features to their systems, some of which have code dating back to the 1970s. In the US in 2017, cybercriminals stole data on 148 million consumers from credit rating agency Equifax. The company forked out $700 million in fines and settlements and a US House of Representatives report accused it of relying on legacy systems with known security risks. Ancient, sprawling legacy systems have caused regular IT failures at airlines in the US and elsewhere, too.
Fake it till you make it
Fixing the problem isn’t easy. When the Commonwealth Bank of Australia replaced its core COBOL platform with software developed by the German company SAP in its ABAP language in 2012, the switch took five years and cost $750 million. The GAO report highlighted a US Internal Revenue Service system in urgent need of modernisation. The upgrade would cost $1.6 billion, for an operation normally requiring just $5.5 million a year to run. “The payback period on that is just huge,” says Harris. “This is why, in many cases, you wind up with these systems that just sit in the corner quietly and diligently doing their job until they break.”
“Over time, dark shadows start to appear in tangled webs of code built up at large organisations”
Then there is the risk factor. When the UK’s TSB Bank attempted to upgrade to newer banking software in 2018, many customers were locked out of their accounts for a week, and CEO Paul Pester his job. Often, too, important business rules that govern how a company operates are embedded in software and, if not properly documented, can be forgotten as employees retire. “If you were to replace the system you might actually lose that corporate memory that is embodied in that code,” says Scherlis.
That is why most “modernisation” efforts in corporate IT focus on surface details, says Tom Winstanley at NTT Data UK, which helps upgrade legacy software. Rather than updating core systems, many businesses adopt a “fake it till you make it” approach of adding new features such as e-commerce websites or flashy web apps – like adding new floors to a building that is crumbling rather than repairing the foundations.
The coronavirus pandemic has laid bare the short-sightedness of that approach, says Winstanley. As entire workforces shift to working from home, many of his clients are scrambling to enable remote access to ageing office computer systems, and dramatic shifts in business models are forcing painful overhauls. “It’s only when you start needing to change fundamentally core policies that this stuff really comes to the surface,” he says. “It’s been hidden by a veneer of transformation.”
Some help is at hand. Owing to the huge amounts of legacy software operating on its hardware, IBM has built a way for customers to map their sprawling systems, which spits out a visual diagram of how different software modules and components work together. It has also developed an AI-based tool that can recommend the most efficient strategy for modernising a company’s software.
Such tools can help new IT staff get up to speed and shine light on those dark shadows in legacy code, says Baker. “It’s shown to be really valuable for these clients that have maybe atrophied in their ability to keep doing the care and feeding of those applications,” he says.
DARPA, meanwhile, has to develop ways to recover lost code knowledge. It has called for proposals for “advanced automated program understanding techniques”, which analyse software to tease out the assumptions and architectural decisions that went into making it, and how individual modules are interlinked. That allows developers to tweak individual bits of a system without knock-on effects that might otherwise bring down the whole house of cards.
Heading off the legacy problem doesn’t need fancy tools, though. Building software tests that check if later edits will introduce errors is an effective, but often neglected practice, says Scott Ford, founder of Corgibytes, a company that specialises in remodelling legacy systems. An even more rigorous approach is test-driven development, where developers build tests before they start coding. “You’re building your safety net as you go,” says Ford. Open-source principles, where developers share, reuse and modify each other’s code, can give more people an incentive to maintain it.
“Choosing software or a programming language is a bet on how long it will stay in fashion”
Ford is part of a burgeoning movement of “menders” that aims to change a prevailing culture that regards code maintenance as second-rate work. But there is only so hard you can fight decay. Even the act of choosing a programming language or software product is tantamount to placing a bet on how long it will stay in fashion. “As soon as a developer pushes something out and somebody else has to come along and maintain it… it is legacy,” says Baker.
Any system in regular use will face pressure to change, says Ford, sometimes in unpredictable and sudden ways, as with the US unemployment systems. “They assume that it was working yesterday, it’ll continue to work tomorrow,” says Ford. “But that pressure is going to show up; so if you don’t have the human infrastructure in place to be able to respond to it, you can be caught by surprise.”
There are no silver bullets. Staying on top of the legacy software problem means investing in maintenance and adopting best practices when building new systems. IT managers will always face trade-offs between the robustness of their software, its cost and how quickly they can deploy new services, says Winstanley. But every shortcut makes it trickier to modify code later, resulting in the build-up of “technical debt”.
“You need to find the space in your planning to pay down that debt just like any other debt that you’re taking on as a business,” he says. “If you just let that pile up, at some point, the debt is overwhelming and you go bankrupt.” With our shaky computer systems, it is increasingly looking like payback time.
Top five languages
Software consultancy company TIOBE publishes a of the world’s most popular programming languages, based on factors including search engine results mentioning the languages and courses teaching them. This is October 2020’s top five.
C – 16.95 per cent of listings
The go-to for programs requiring speed and efficiency, such as operating systems, robotics controllers and trading algorithms, C has been at number 1 or 2 of the most prevalent programming languages for at least the past four decades.
Java – 12.56%
The leading language for most of the past two decades, Java is a child of the world wide web, and is a workhorse for mobile and web applications and games.
Python – 11.28%
Listed as only the 21st most popular language as recently as 2000, Python’s versatile and easy-to-learn vocabulary has seen it gain popularity lately for everything from web applications to artificial intelligence systems.
C++ – 6.94%
An extension of C, used to code operating systems, browsers and games, C++ was for a period in the 1990s the number 1 language, but has since slipped down the rankings.
C# – 4.16%
Pronounced “c-sharp”, this is another extension of C, developed to incorporate similar principles to Java, and it has similar spheres of application.
A glitch in the science?
“Legacy code” (see main story) is also a big problem in academia, where old, poorly maintained computer programs can be prone to bugs that throw off results, says Caroline Jay, research director at the UK-based Software Sustainability Institute, which advocates for better programming training for scientists.
Researchers have to write software for everything from data analysis to modelling natural processes, but most learn these skills in an ad hoc way. They are perennially short of money and time to properly sustain their code. “Being a scientist is a full-time job,” says Jay. “It’s really difficult to be an expert in both of those areas.”
The tendency to repurpose code written by other researchers can also cast a long shadow. Last year, scientists discovered a glitch in a tool to predict nuclear magnetic resonance spectra, a key method used to characterise chemicals. The tool, published in a 2014 paper, .
The software has been cited by other researchers more than 150 times. It isn’t clear how many of those teams actually used the tool, or how many results were thrown out by the glitch.
