IT ALL started with a sticky note. Harold Thimbleby was visiting one of his students in hospital when, amid the flowers, grapes and cards, he noticed an infusion pump in the corner, a device used to feed fluids and drugs into a patient’s blood vessels. On the pump was a note that read “don’t press these buttons” – an awkward warning suggesting hospital staff might be having trouble using it as intended.
Technology is everywhere in healthcare, and it is a potential source of bugs as deadly as any virus that might stalk a hospital corridor. That much was driven home in the UK earlier this month, when health secretary Jeremy Hunt announced to Parliament that a computer glitch meant an estimated 450,000 women who should have been invited to breast cancer screening appointments since 2009 had not been. “Tragically, there are likely to be some people in this group who would have been alive today if the failure had not happened,” Hunt said.
Advertisement
Until now, Thimbleby, a computer scientist at Swansea University, UK, has been something of a lone voice with his warnings about the dangers of misused technology in healthcare. But if his analysis is right, it is a problem that goes far beyond just cancer screening, and it could be putting thousands of lives at risk every year in UK hospitals alone.
That’s the bad news. The good news is that by taking a thorough look at hospital technology and drawing lessons from other industries where safety is critical, we have a chance to squash these bugs for good.
Hospitals are filled with spaghetti heaps of wiring connecting different digital systems. There are computers that control MRI scanners and X-ray machines; more modest circuits that control pumps and microscopes; and databases and screens that record and display test results and patient records. It isn’t so different from any modern workplace – except lives are at stake if things go awry.
Just how significant the cancer screening error was remains to be seen. The women involved were in their late 60s, when the high risk of false positives means the merits of routine breast cancer screening are still being evaluated. But given our reliance on technology, any poor designs or flaws in computer code that make it hard for humans to use medical devices and systems accurately could have major consequences. So how often do they lead to serious harm or death?
In the UK, official figures from the National Health Service (NHS) show that there are that result in serious harm or death each year. However, those figures are collected by voluntary self-reporting, which, the evidence suggests, only captures about 7 to 15 per cent of such incidents. That means the real figure is probably more like 100,000. That would be roughly in proportion with the US, where preventable errors cause in hospitals, making them the third highest cause of death after heart disease and cancer.
“Computer bugs could kill or seriously harm 2000 people a year in UK hospitals”
Discerning how many of those errors involve a computer bug is no simple matter. There are no official figures and precious few large-scale studies. Â鶹´«Ă˝ contacted various NHS bodies and none said they kept data on technology design issues that led to patient harm.
Thimbleby’s encounter with the sticky note in 2002 got him wrestling with the problem, and he began to get a sense of its scale. These studies provided wildly differing numbers. On the one hand, a 2006 study found that almost 25 per cent of some 176,000 medication errors reported to the United States Pharmacopeia voluntary incident reporting database . On the other, a study conducted at around the same time looked at a sample of 43,000 incidents voluntarily reported in one Australian state and found that , or 0.2 per cent, were linked to computers. That could be an underestimate: finding out whether computers were involved is probably not the first thing on a doctor’s mind after a serious incident.
Thimbleby suggests we should conservatively assume that 1 per cent of hospital errors that cause serious harm or death involve computer bugs or other technological flaws. That works out at 2000 cases a year in the UK alone. In a paper released in February, Thimbleby and Martyn Thomas, an independent software consultant, . They compare its impact with the 2017 Grenfell Tower fire in London that killed 71 people and resulted in a public inquiry.
The duo have unearthed what they say are worrying specific examples of problems with medical devices. In some infusion pumps, for instance, pressing a key makes an unexpected digit on the display change. This might result in a doctor accidentally programming in the wrong flow rate and a patient getting too much or too little medicine or fluids. Another issue is “key bounce”, where pressing a button once results in the digit being entered twice or more. These are clear examples of bugs, they say. The makers of the devices dispute the claims.
This is potentially a big deal because doctors put their faith in these systems. “You need to trust the machines you’re using,” says Martin Elliott, a former paediatric surgeon and now a professor at University College London. Even small errors can be dangerous, for example when measuring extremely precise doses of medication for babies and children. “Everything is filtered through electronics,” he says. Most doctors and nurses simply assume that medical technology is reliable.
We need a change akin to what happened with cars some 50 years ago, says Thimbleby. Until then, people generally accepted that cars were dangerous and sometimes killed people. Then in 1966, the book Unsafe At Any Speed by activist Ralph Nader accused car-makers of being resistant to safety features like seat belts. This helped prompt a shift whereby the onus was put on manufacturers to make cars safer.
Not so long ago, Thimbleby’s son had a car accident. “If he’d had the same accident in the 1960s, he would have been killed,” he says. “But now cars have air bags and seat belts and crumple zones.”
“You don’t test a house by jumping on the roof. You make it safe by design”
Shifting things in a similar way so that the design of medical devices prevents as many errors as possible wouldn’t be so hard to achieve. This is the sort of thing Thimbleby studies in his day job. It might mean adjusting displays using a scroller like you would find on a mouse rather than a 10-digit keypad, for example.
An important part of any such shift would be the agencies that regulate healthcare devices, such as the Medicines and Healthcare products Regulatory Agency (MHRA) in the UK. Devices and products are regulated in different ways: condoms aren’t as tightly controlled as artificial hips, for instance. For devices like infusion pumps, manufacturers must obtain a CE mark, which involves paying a small fee and submitting the results of safety tests. But those tests are performed by the manufacturers themselves before the results are submitted to an approved healthcare services firm that the MHRA appoints to review them. There is no requirement for physical testing by an external body.
An MHRA spokesperson says that problems with devices can be reported through its , a web portal. But Thimbleby is calling for tighter regulation and better oversight of the medical technology design process, perhaps employing software to automatically ensure the devices function as intended.
And even if individual devices were foolproof, there is a deeper problem: how information is processed. This is illustrated by the 2015 trial of two nurses from a UK hospital who . The case focused on data from glucometers, instruments used to measure a patient’s blood sugar levels. It turned out that records of these readings, held centrally in the hospital’s computer system, differed from the handwritten notes of measurements that nurses added to patients’ records.
The trial determined that the instruments themselves were working properly. But Thimbleby, who served as an expert witness on the case, noted that nurses frequently had trouble scanning patient IDs, so they would scan their own ID cards to take a reading on the glucometers. That meant patient results would not be stored correctly in the hospital’s electronic records. Manual intervention could fix that – but there was no protocol setting out how this should be done. The judge in the case determined that the evidence against the nurses was unreliable, and they were discharged.
How such “informatics” problems crop up in clinical settings can be subtle. A screen with test results that requires a doctor to scroll down might mean some get missed. A small delay in filing electronic reports could cause an error of diagnosis. Typos in records may mean crucial information is not returned when a doctor makes enquiries.
Subtle these problems may be, but we are becoming more aware of them. at the Australian Institute of Health Innovation in Sydney has studied problems caused by healthcare IT for years. A 2015 study she led examined self-reported incidents that involved IT and had an observable impact on the delivery of care. There were recorded by 87 doctors in one Australian state between May 2012 and November 2013. One of these involved a patient who was injected with double the intended dose of a medication because of a delay scanning in drug dosage information.
Thomas says we should not have to live with such risk, at least when it comes to individual pieces of software in hospitals. He advocates what he calls an engineering approach to software design. It’s like building a house. You don’t build it and check it doesn’t fall down by jumping on the roof. You design it using formulae so that you know it can’t fall down.
There’s plenty of precedent for software built like that in another safety critical industry: air traffic control. Take the iFACTS system used in British airports that allows controllers to see how flight trajectories will pan out up to 18 minutes into the future. It is crucial that the forecasts are right as a bug could beget a collision. To that end, iFACTS is built so that whenever it is updated, it can be tested quickly via a series of mathematical proofs to confirm it is still functioning as intended. In theory, this makes programming errors less likely. The same approach is used in various safety-critical contexts, including military aircraft systems – but not generally healthcare.
That may be because, for healthcare tech, there is no competitive advantage to employing this method and no regulatory motivation to change the status quo. “Until there is a real liability cost that comes with buggy software, nothing will change,” says Thomas, who formerly owned the company that developed iFACTS.
In the UK, there has been one isolated effort to improve the situation. In the early 2000s, a project run by Maureen Baker, a former chair of the Royal College of General Practitioners, asked clinicians to systematically review all kinds of hospital procedures involving IT and identify the steps where errors could creep in. Then the project team developed fixes or standard procedures, now known as the NHS clinical safety management system, that would minimise the risk of error. For instance, doctors’ drug dictionaries didn’t map exactly to those held at pharmacies, potentially resulting in the wrong drug being dispensed. Baker’s team fixed this by introducing software that highlighted any mismatches.
A 2013 review by Magrabi found that Baker’s was the first and only healthcare analysis to oversee whole systems of people and technology. “That’s still the case,” says Magrabi.
Small wonder, perhaps, given how many devices, databases, doctors and nurses there are even in one hospital. Some think the way to simplify the problem is not to focus on technology, but people: to find ways to guide human interactions with technology to minimise the risk of mistakes.
One solution that sounds laughably simple is to make lists. In 2009, the surgeon and author Atul Gawande published The Checklist Manifesto, which describes how following step-by-step plans in complex situations can help avoid catastrophe.
A 2009 study co-authored by Gawande found that using a caused the rate of death to almost halve from 1.5 per cent to 0.8 per cent for patients having surgery in eight hospitals around the world. The study examined outcomes for 7500 patients, roughly half of whom were treated after the checklist approach was brought in.
Checklists and balances
Checklists aren’t commonly used to second-guess IT systems at the moment, but they could be. Doctors are already trained to conduct a common-sense check of dosage calculations. A checklist reminding them to do this at a crucial moment could avoid errors involving infusion pumps, for example.
Used too extensively, however, checklists risk becoming a drain on staff. Identifying those critical situations that could most benefit might be easier if we borrow another trick from the aviation industry. The Line Operations Safety Audit is a procedure used to log how an airline crew, their tools and computer systems work together. It can be used to spot differences between real life and an ideal workflow, including errors. The insights can be fed back into training.
In 2017, a group of medical students tested in an English hospital. They found plenty of problems, including some related to the unavailability of equipment, trip hazards posed by cables in operating theatres and patient confidentiality issues when information was left visible on a computer screen.
Audits are no silver scalpel for cutting out errors, either. But by pairing them and checklists with smarter engineering, and by raising awareness of the possible risks, we might just be able to limit the damage caused by hospital bugs. At the very least, it has got to be better than a sticky note.
This article appeared in print under the headline “Fatal System Error”
