Interesting that they mention the medical case, when there's some psychological work around the idea that we should present these cases in terms of natural frequencies instead of Bayes' theorem.
The natural frequencies approach is to say "if 10000 people take the test, 100 will have cancer. Of them, 99 will get an accurate positive test, and 1 will have a false negative test. Of the other 9900, 99 will receive a false positive, and 9801 will receive a correct negative. What are the odds that someone who has a positive test has cancer?"
It turns out that doctors and other professionals whose core professional competency doesn't concern probability do terribly when presented with percentages and Bayes theorem, but can handle natural frequencies quite well (here's one quick summary: http://opinionator.blogs.nytimes.com/2010/04/25/chances-are/...).
As is obvious, this isn't an argument that Bayes' theorem is wrong--it's a theorem after all. It's an argument about which types of reasoning people can be easily taught.
It's true that when the question is formed in frequentist terms, the answer is much more intuitive. But is that how the problem occurs in real life? The doctor doesn't see ten thousand people take a test; they see a person take a test, and get either a positive or negative result. The traditional way of forming the problem seems closer to actual experience: 'your patient tested positive. you know how accurate the test is, and how common the disease is; how likely is it that the result is a false positive?'
I'm not quite sure what you're saying. Doctors don't observe probabilities or enormous frequencies. Either way, there are good odds that this is information that someone is communicating to them, not the result of their personal experience.
If I may rephrase (and steelman) the parent's point:
Reality does not neatly format itself for easy plug-and-chug into your formulas. To appropriately respond to reality, you must be good at recognizing when there's a mapping to a well-tested formula, technique, or phenomenon.
Therefore, if you require problems to be phrased in a way such that that's already done, that means you're not good at that domain; blaming the phrasing of the problem is missing the point.
Indeed, 90% of the mental work lies in recognizing such isomorphisms, not in cranking through the algorithm once it's recognized, and this is a hard skill to teach. (Schools that teach how to attack word problems have to rely on crude word-match techniques to identify e.g. when you want to subtract vs add vs divide.)
> The prescription put forward is simple. Essentially, we should all be using natural frequencies to express and think about uncertain events. Conditional probabilities are used in the first of the following statements; natural frequencies in the second (both are quoted from the book):
> The probability that one of these women [asymptomatic, aged 40 to 50, from a particular region, participating in mammography screening] has breast cancer is 0.8 percent. If a woman has breast cancer, the probability is 90 percent that she will have a positive mammogram. If a woman does not have breast cancer, the probability is 7 percent that she will still have a positive mammogram.
> Imagine a woman who has a positive mammogram. What is the probability that she actually has breast cancer?
> Eight out of every 1,000 women have breast cancer. Of these 8 women with breast cancer, 7 will have a positive mammogram. Of the remaining 992 women who don't have breast cancer, some 70 will still have a positive mammogram. Imagine a sample of women who have positive mammograms in screening. How many of these women actually have breast cancer?
Doctors observe a result of the test, and know the basic probabilities (in the example, 99% test accuracy, 1% of population have the disease). The problem is that they [often] draw incorrect conclusions from those observations (99% test accuracy and you tested positive? well then you likely - 99% - have the disease, right?).
The question formed as 'your one patient tested positively' is more immediately relevant, I'd think. The correspondence with actual practice is obvious. The question formed as 'out of 10000 ...' could be remembered as a quirk of statistics, but not actually recalled when someone tests positively for cancer.
Of course, doctors do not randomly assign tests to patients. Their prior that a patient has a disease is a lot higher than the background frequency of it occurring.
Getting them to estimate their prior would be interesting.
Except when they do. For example, when 100% of men above a certain age are screened for prostate cancer, or 100% of women above a certain age are screened for breast cancer. Both cases spawned major public health campaigns to encourage screening, followed years later by recommendations AGAINST 100% screening, based on the high degree of false positives and unnecessary treatment.
Other cases that come to mind:
-- doctors who offer "full body scans" as a part of an executive physical; you're pretty much guaranteed to turn up something that is 2 sigma away from the population norm, somewhere in the body, on such a scan
-- spinal x-rays for back pain. Doctors almost always find something abnormal, and use that to justify the back pain and treat aggressively. But, we don't really have a good prior; if you x-rayed 1000 people off the street, would we find similar abnormalities frequently?
It depends. Some tests are applied without prior suspicion, so you deal with exactly the background frequency. With others, the disease in question is so rare that false positives will dominate even if the doctor has serious suspicions. The second case is the reason for the "think horses not zebras" aphorism.
Doing a few explicit Bayesian calculations can help one internalize just how much important is that often forgotten P(A) factor is.
The same applies, by the way, to the antiterrorism security theater - many support it just because they have no intuition (or idea) about base rates.
I think GP's point is that in the case of interacting with an individual patient, the Bayesian conception of probability as a quantification of degree of belief is actually more intuitive than the frequentist conception of probability as a relative frequency of outcomes under repeated hypothetical experimentation.
I suspect that this is simply a sense of scale, with the freq-vs-bayes aspect playing only a minor role.
The normalization that happens when you use percentages and 0.0 -> 1.0 probabilities is often useful, but it can sometimes obscure the magnitude of some relationships. It's easier to understand the sense of scale when you say "1 out of every 100,000 people" (which is easily extended to "a handful of people in a large city"). The same information is presented as "0.001% of the population" requires the reader to do more mental math if they want to understand how many people could be affected.
Knowing how to interpret false-positives and false-negatives is very important, but doctors are busy people who have to memorize a lot of data. It is probably better for patients if they can at least remember if something is "common" vs "rare" when they need to make a quick decision.
It seems likely that this will help some doctors in at least simple cases, but natural frequencies don't 'compose' well if you are doing multiple tests; you can't use them to compute posterior probabilities when combining a half-dozen different tests, at least not without involving impractically huge numbers or fractional people.
Moreover, the trend in modern medicine is towards combining binary/categorical tests with a range of distributions based on age, gender, race, genetic factors, vitals, continuous lab result values, etc. Yes, we can theoretically delegate some or all of that to software, but that is true for medical diagnosis in general; while we're not there, doctors must understand and perform these calculations.
So it seems fine to use natural frequencies to help out, but building Bayesian intuitions early and often seems a better path.
Maybe that the natural frequency "normatized" for some people may be completely different from the actual frequency that apply to them, because the people that researched the frequency did their grouping badly.
I'm not sure if that's relevant. It seems like either you know the actual frequency for their subgroup, in which case you redo the numbers in the example, or you don't in which case you can't accommodate that regardless of how you calculate.
A natural frequency is different way of expressing and reasoning about the same information as percentages.
Well, me neither. If you get the frequency wrong, you'll also get your priors wrong. Natural frequency does not compose well, but that is another problem.
Doctors and professionals are probably not as concerned about Bayes because cancer isn't identically or independently distributed across the population.
Also, endocrinology diagnostic platforms usually don't have identical accuracy rates for positive and negative results, varying by test, machine, controls used, etc.
The reason why doctors and other professionals find it difficult to understand is because they are paid for treating people. As Upton Sinclair observed “It is difficult to get a man to understand something, when his salary depends on his not understanding it.”
The natural frequencies approach is to say "if 10000 people take the test, 100 will have cancer. Of them, 99 will get an accurate positive test, and 1 will have a false negative test. Of the other 9900, 99 will receive a false positive, and 9801 will receive a correct negative. What are the odds that someone who has a positive test has cancer?"
It turns out that doctors and other professionals whose core professional competency doesn't concern probability do terribly when presented with percentages and Bayes theorem, but can handle natural frequencies quite well (here's one quick summary: http://opinionator.blogs.nytimes.com/2010/04/25/chances-are/...).
As is obvious, this isn't an argument that Bayes' theorem is wrong--it's a theorem after all. It's an argument about which types of reasoning people can be easily taught.