You’re being tricked – by your own medical mind. Pretest probabilities mean more to your clinical work than you might think. While sensitivity, specificity and P value, the ‘gold standard’ of statistical validity, is not as universally applicable as many of us, including many scientist, assume.
This is an attempt of explaing pretest probabilities, the basis of Bayesian statistics and why it is important to our everyday clinical work. Don’t fall asleep yet, but read through and see what you think.
This article in Nature: Scientific method: Statistical errors discusses the p-value and why it is often used incorrectly. The article is well worth a read as it explains in simple words why the pre-test probability of a hypothesis being true or false will often mean far more than the p-value of the end result. What they’re touching on is Bayesian statistics vs Frequentist statistics. Saywhat?!?
Frequentists vs Bayesians
Frequentist statistics is what we usually refer to as statistics: pure numbers. “Objective” numbers referring to a normal frequency distribution – symbolised by the Bell curve. But to apply it correctly in real life settings, you often need to adjust your numbers.
Enter Bayesian statistics. It deals with conditioned risk – how the risk is conditioned by pre-test probabilities and facts that will affect the outcome and skew it compared to pure frequentist statistics.
Pretty much a fancy way of working out what we try to do every day in medicine: Adjust general, categorical statistics to your specific patient. Sort of like “subjective” numbers. It will skew your Bell curve. And it will help you navigate in the world of statistics by adding relevant information that gives new perspective to your statistics. A conditioned probability is simply a probability based on some extra background information.
Bayes and common sense
Bayesian statistics has a wide application in medicine – and maybe we should use it even more. Not that it’s a new idea, Thomas Bayes worked on his theories in the 18th century – but it seems computers and new thinking helped bring his theories back into practice.
The most basic concept is the one where you tune your statistics to your patient. Risk of ischemic heart disease has a certain prevalence in the general population, but for a particular patient diabetes, gender, smoking etc skews that probability. And when a patient comes in with general symptoms of illness, the normally rare diagnosis of malaria springs to mind if the patient has been traveling in malaria endemic areas. Both are basic examples of pre-test probabilities ‘conditioning’ your statistics.
Often, we don’t need to apply the whole Bayesian formal framework, but just the Bayesian way of thinking. Most of the time, that just means common sense. In a way, Bayesian statistics is the rules and math of common sense. But more powerful than that, Bayesian statistics sometimes end up giving surprising results where common sense can lead you astray.
Back to the p. Remember pre-test conditions
One area where common sense might trick you, is tests with low pre-test probabilities, like the Nature article talks about. It is important to keep in mind that a very unlikely hypothesis will need more than a positive test result and a nice, low p to prove them. Consider this experiment by the excellent XKCD. You need to summon your nerd powers to get this:
OK, you didn’t get it. Luckily, the XKCD cartoon is explained by real nerds here. The essence is that the chance of the sun exploding is extremely low compared to the chance that the test is false positive (the dice throw: 1/36th of the time).
Bayes in everyday medicine
For a similar problem in the medical world, consider a setting similar to what we encounter in clinical practice every day:
Suppose a drug test is 99% sensitive and 99% specific. That is, the test will produce 99% true positive results for drug users and 99% true negative results for non-drug users. Suppose that 0.5% of people are users of the drug. If a randomly selected individual tests positive, what is the probability he or she is a user?
Normally, we would trust that test. The patient tested positive, so he took the drug. Or did he? Bayesian statistics can help tune the answer. The real, Bayesian, probability is – surprisingly – only 33%.
It follows the same logic as the XKCD cartoon. Because drug use occurs less frequent than the test being false positive, your end result is fraught with risk of error. There is a 1% chance of a false positive test result, but only a 0.5% chance the person being tested is a drug user. So the false positive will come out twice as often as you will encounter an actual drug user. Wikipedia explains it in detail here. Bayes’ thinking would dictate that you account for this pre-test flaw in your assessment of the test result.
Ace of Bayes
Still, we tend to get tricked to believe the answer when the sensitivity/specificity is looking good, or if the p’s are all nice and low. Bayes helps correcting this and skew the result by taking into account relevant pre-test conditions – hopefully skewing your answer towards something more meaningful.
In the settings mentioned above, Bayesian statistics can help put a number on our clinical/common sense judgement, to weight different diagnosis against each other. I find the concept of Bayesian statistics fascinating, and a useful way of thinking to try to navigate through tests, articles and statistics trying to apply it all to my patients and clinical practice.
For an easy-to-understand introduction to Bayesian statistics, have a look at this online book: Think Bayes. It’s not written for doctors, but the first few chapters, before it goes deeper into programming, is an excellent description of Bayes theorem and the logic behind it and how to apply it.
Also, to see a great example of how Bayesian statistics can come up with surprising, but correct, answers – answers that common sense just won’t understand – read the Monty Hall problem. The Monty Hall problem will drive you mad.