麻豆传媒

THE departed spirit of an 18th-century English cleric is suddenly starting to
turn up in a whole range of odd places. Fortunately, he is a friendly ghost.
Take Microsoft鈥檚 business software, Office 97鈥攊f you鈥檙e having a spot of
trouble, he appears with just the right tips to help you out. Officially,
Microsoft鈥檚 helper is called Office Assistant but his real name is the Reverend
Thomas Bayes.

Bayes鈥檚 ghostly presence can be seen everywhere from hospitals to industrial
research labs. It鈥檚 all because of a mathematical formula Bayes came up with 200
years ago to answer a vital question with all kinds of applications: how should
we change our beliefs in the light of new evidence? In the Office Assistant,
that means the software changing its choice of tips as it learns more about what
the user is doing. For doctors, it means helping them to decide how convinced
they should be that a drug works, following results from a clinical trial. For
engineers, it鈥檚 about using their past experience to create more efficient
engines.

With new applications surfacing at an ever-increasing rate, Bayes鈥檚 theorem
is one of the hottest topics around in statistics. In Nottingham this summer,
the annual conference of Britain鈥檚 Royal Statistical Society (RSS) was devoted
to it. All of which begs a question: if it鈥檚 so useful, why has it taken 200
years to be recognised?

One reason is that applying Bayes鈥檚 theorem often requires beefy computing
power that has only recently become available. But more importantly, the theorem
has spent years struggling to overcome a big intellectual hurdle in the minds of
scientists. For it deals with something that is anathema to traditional views of
蝉肠颈别苍肠别鈥攕耻产箩别肠迟颈惫颈迟测.

At its most fundamental level, the theorem is completely uncontroversial, a
direct outcome of the axioms of probability (see 鈥淏ayes in action鈥�). It simply shows how the
chances of event X happening are altered by the occurrence of another event, Y.
Used in this form, the theorem can be used to solve any number of mundane
probability problems, such as the chances of plucking a black card from a pack,
given that you have already taken one.

But there is another, altogether more powerful, way of thinking about Bayes鈥檚
theorem. For X can be your belief in a theory鈥攕ay, the idea that a new
drug can treat cancer鈥攚hile Y can be evidence from a clinical trial.
Bayes鈥檚 theorem then turns into a recipe for showing how you should update your
original, 鈥減rior鈥� belief in X in the light of the new evidence Y.

No hard evidence

So why the controversy? The problem is that Bayes鈥檚 theorem demands that you
provide a value for the prior probability of a theory being correct, even if
there is no hard evidence either way. And if there is no hard evidence to go on,
there is only one way to do this: an educated guess.

The idea of subjective guesstimates entering the scientific process
explicitly is certainly shocking. Why should it be necessary? After all, isn鈥檛
every statistics textbook filled with entirely objective measures of whether or
not a new experimental result is 鈥渟ignificant鈥�?

Not according to advocates of Bayes, who point to the best-known of these
textbook measures, the 鈥減-value鈥�. This puts a value on the probability of
obtaining as least as impressive results, assuming that random chance was
responsible. According to convention, if a p-value is less than 1 in 20, the
chances of the results being due to fluke alone are so low that the result is
objectively 鈥渟ignificant鈥�.

But wait: where did this 1 in 20 figure come from? It turns out that the
figure is not based on anything more rigorous and objective than the fact that
everyone has used it for years. These p-values, Bayesians insist, are just as
subjective as any other test of 鈥渟ignificance鈥�: the textbooks sweep the
subjectivity under the carpet, rather that addressing it upfront.

But Bayesians have a far more damning criticism of these classical hypothesis
tests. 鈥淭hey don鈥檛 answer the question we are interested in,鈥� says Tony O鈥橦agan
of the University of Nottingham, and organiser of this year鈥檚 RSS conference.
Imagine, he says, that you鈥檙e taking part in a game where you have to guess how
many heads will appear in 30 tosses of a coin. You expect around 15, but if just
10 appear, what do you conclude? Were you just unlucky, or is the coin
biased?

鈥淭he Bayesian response is to work out the probability that the rate of heads
really is less than 50 per cent, given the evidence you have seen鈥攁
natural and meaningful reply to the question,鈥� says O鈥橦agan. 鈥淚n contrast, a
classical hypothesis test simply does not answer the question. The p-value tells
you only what results you would have seen, assuming that the coin was fair.
That鈥檚 a statement about the data, not about the coin.鈥�

But Bayes does more than answer the right question, says O鈥橦agan. It also
allows us to bring in any information we might already have about the coin being
biased, via the prior probability. This could be subjective鈥攆or example,
how shady the game鈥檚 organiser seems鈥攐r it could be based on talking to
others about how they fared playing the game.

Either way, it makes sense to exploit everything that bears upon the question
one is trying to answer, says O鈥橦agan. 鈥淧rior information is an integral part of
Bayesian statistics, but is absent from classical methods. To a Bayesian, every
problem is unique and has its own context.鈥�

Red herring

As for the charge that Bayesian inference threatens to wreck science through
its reliance on subjective prior probabilities, advocates of its use point out
that deep down, every scientist is a Bayesian anyway. Take extrasensory
perception (ESP). Over the years, dozens of studies claim to have found an
effect, and the cumulative p-value is now well below 1 in a
million鈥攊ncredibly 鈥渟ignificant鈥� according to conventional statistics. Yet
despite this 鈥渙bjective鈥� result, most scientists still remain deeply sceptical
about ESP鈥攂ecause, as a Bayesian would put it, they attach a very low
prior probability to the existence of ESP.

P-values, in contrast, do not allow any such scepticism to be applied: they
simply spit out a result based on what would have been seen if random chance
were responsible. But what if the experiments were flawed, or someone cheated?
There鈥檚 no room for these possibilities in the calculations of p-values.

Bayesians also point out that the 鈥渟ubjectivity鈥� criticism is something of a
red herring, as the precise choice of prior probability often doesn鈥檛 matter:
mathematically, the theorem shows that as data comes in, everyone from sceptics
to enthusiasts ends up being directed to the same conclusion. The only
difference is that those who put a low prior probability on a theory being true
will need more data to become convinced鈥攚hich is just how real science
works anyway.

It sounds so reasonable that you could be forgiven for wondering why
scientists have stuck to p-values for so long. Dennis Lindley, emeritus
professor of statistics at the University of Warwick, one of the world鈥檚 most
influential pioneers of Bayesian methods, has a simple if cynical, explanation:
鈥淧eople like conventional hypothesis tests because it鈥檚 so easy to get
significant results from them,鈥� he says. 鈥淢any papers based on the conventional
p-value approach are going to be wrong.鈥�

This has particularly worrying implications in medicine, where the merits of
new treatments are often judged by p-values. A salutary example of how p-values
can mask the truth centres on recent claims that magnesium injections could help
save the lives of heart attack victims.

In 1991, the British Medical Journal carried a statistical analysis
of trials of magnesium injections revealing a whopping 55 per cent reduction in
the odds of death. Quite why magnesium should work this well wasn鈥檛 at all
obvious, but with a p-value of just 0.001 the implication seemed clear: the
success of magnesium could not be a fluke. By 1993, doctors were hailing
magnesium as an 鈥渆ffective, safe, simple and inexpensive intervention鈥�. But two
years later, the results of a huge trial involving 58 000 patients were
announced鈥攁nd the amazing abilities of magnesium to save lives had
disappeared.

According to David Spiegelhalter of the Medical Research Council鈥檚
Biostatistics Unit in Cambridge, the case highlights the need to include some
measure of plausibility. 鈥淚n drug regulation, it seems reasonable to adopt a
fairly sceptical approach to likely benefits,鈥� he says. 鈥淭he main effect of
using a Bayesian analysis would probably be to be more conservative, and
therefore some ineffective drugs might be stopped.鈥�

Backing for Spiegelhalter鈥檚 claim comes from recent studies of the
clot-busting drug anistreplase. In 1992, the medical world was astonished by a
study in the Grampian area of Scotland showing that if GPs gave suspected heart
attack victims the drug at home, the chances of survival were 49 per cent better
than if they were given the drug at hospital.

Few doctors doubted that there would be some improvement, but the sheer size
of the improvement stunned many. Could GPs, with none of the sophisticated
diagnostic and treatment methods available in hospitals, really do so much
better? Despite such doubts, the study鈥檚 p-value of just 0.04 seemed to imply
that the results could not be a fluke.

Stuart Pocock of the Medical Statistics Unit at the London School of Hygiene
and Tropical Medicine was less sure: 鈥淪uch a great benefit from giving the same
treatment earlier simply doesn鈥檛 make sense.鈥� So, together with Spiegelhalter,
Pocock reanalysed the Grampian trial results using Bayesian methods.

Based on their own experience and evidence from other studies of clot-busting
drugs, they decided that an improvement of between 15 and 20 per cent was much
more plausible, while they viewed both zero and 40 per cent improvement as
unlikely.

Capturing their opinion with a mathematical prior probability curve that
peaked around 17 per cent improvement, they then used Bayes鈥檚 theorem to work
out just how much their views should be modified by the Grampian study. They
found that the data were consistent with a 25 per cent improvement鈥攕till
impressive, but half that implied by the p-value result.

Real result

So, five years after the original study, who has proved right? According to
Keith Fox of the University of Edinburgh鈥檚 Cardiovascular Research Unit,
experience world-wide now points to an improvement from early use of
clot-busters of between 20 and 25 per cent鈥攋ust as the Bayesian analysis
suggested. 鈥淭he original Grampian results do seem to have been particularly
fortuitous鈥攁nd the Bayesian result helped warn of this,鈥� says Fox.

The ability of Bayes鈥檚 theorem to reveal the true significance of new
findings is now being increasingly recognised by regulatory authorities: both
the European Commission and US Food and Drug Administration have recently agreed
to accept Bayesian analysis of drug trials. But Bayes鈥檚 recipe for updating
knowledge in the light of experience is finding applications beyond the
laboratory bench.

In Microsoft鈥檚 Office Assistant, the theorem is used to help the software
spot when users are running into trouble, and what help to give them. By
studying how people behave when they run into different types of problem,
Microsoft engineers were able to work out the chances that users need specific
types of help.

Using this information, stored as prior probabilities inside the software,
Office Assistant can then analyse what a user is doing, apply Bayes鈥檚 theorem
and duly come up with tips that have the highest probability of proving useful.
According to Microsoft engineer Eric Kovitz, the resulting 鈥渋ntelligence鈥� has
gone down very well with consumers.

The ability of Bayes鈥檚 theorem to exploit past experience is also being used
at Ricardo Consulting Engineers on Britain鈥檚 southern coast to find faster ways
of optimising engine performance. The conventional way is to carry out scores of
tests on the engine and create a mathematical model showing how, say, fuel
consumption is affected by injection timing. Then a combination of mathematical
optimisation methods and engineers鈥� judgment is brought in to tweak the model,
and thus optimise the engine.

According to Ricardo鈥檚 project leader Tony Pilley, one big problem with this
is that engineers have to wait for all the tests before they have a model to
tweak. Another problem with this method is that it only exploits a fraction of
what engineers already know. 鈥淥ur idea was to try to use Bayesian methods to
capture the information and experience that is in people鈥檚 heads and to get to
the optimum as quickly as possible.鈥�

To do this, Deborah Mowll and Derek Robinson at the University of Sussex
created a computerised image of the relationship between different variables in
3D form. Engineers could alter the shape of the plots until it reflected their
beliefs.

To begin with, these beliefs were simply priors, based on past experience.
But as test results came in, Mowll and Robinson used their computer system to
change the shape of the graphs according to Bayes鈥檚 theorem, updating them in
the light of the data.

In tests earlier this year, the Ricardo team created a model of an engine
that had already been optimised, and then checked to see how quickly it took the
Bayesian method to reach it. Creating the optimised engine from scratch required
over 130 tests. The Bayesian method, using both new and existing data, led two
engineers to it in fewer than 20 tests.

As Bayes鈥檚 theorem begins to permeate other branches of science and
technology, Bayesians admit that there are still some tough problems to be
solved鈥攏otably the need to work out accurate prior probabilities. But,
says O鈥橦agan, the effort must be made. 鈥淏ayes鈥檚 theorem is the right way to make
sense of data because it makes best use of everything you know.鈥�

* * *

Bayes in action

Bayes鈥檚 theorem shows how to update your belief in a
particular theory in the light of new data. Mathematically, it states:

Odds (Theory, given observed data) = LR x Odds(Theory)

where Odds (Theory, given observed data) are the odds on the correctness of
the theory, given the new data, and Odds (Theory) are the so-called prior odds
that the theory is correct鈥攖hat is, a measure of the plausibility of the
theory before the new data emerged. Precisely how that new data changes your
beliefs is captured by the so-called 鈥淟ikelihood Ratio鈥� (LR), which is made up
of two factors:

LR =
Prob (getting the data, given theory is correct)/Prob (getting the data, given theory is wrong)

Conventional significance tests, in contrast, dispense with all this,
concentrating instead on just one factor: the 鈥減-value鈥�. This is the probability
of getting at least as impressive data as that seen, assuming fluke alone is to
blame. But by ignoring all the other possible explanations of the
data鈥攑oor experimental design, for example鈥攁nd also the plausibility
(or otherwise) of the results being a fluke, p-values typically greatly
exaggerate the true significance of results.