I'm a tedious broken record about the fact that the base rate of most cancers means that extraordinarily-accurate-seeming screening tests have surprisingly untenable false positive rates. Like, a 99% accurate test for liver cancer might be almost worthless: because the base rate is so low, 99% of positives will be false.
And a false positive screening result is not innocuous: it incurs costs in a variety of different ways, including human health.
This is an important and unintuitive reality about all kinds of tests. You have to take into account the base rate - the likelihood that you have the disease given what you know prior to the test.
If you're otherwise healthy and would have a 1/1,000,000 chance of having the disease before the test, and then you test positive with a test that is 99% accurate, you are ~100x more likely to have the disease than before - but that's still only 1/10,000 - not at all 99% likely, even though the test was "99% accurate"
That said, I think with this knowledge the test still confers helpful information. I might decide to spend $1000 on an additional diagnostic, even knowing that I'm still very likely to be negative. Depends on how wealthy I am, and how serious the disease is, and what the treatments for it are.
> I might decide to spend $1000 on an additional diagnostic, even knowing that I'm still very likely to be negative.
The problem is that diagnostics aren't necessarily risk-free. For example, there's a non-zero risk of death while getting a colonoscopy, to the point that false positives from unnecessary testing can increase all-cause mortality for patients.
There's also a real psychic cost to even a tentative positive result, which is why I'm much more likely to get an actual colonoscopy than a Cologuard test.
I had the same thought. I got the colonoscopy and I don’t regret it at all. And for anyone thinking about it or stressing about it, the prep was not that bad. If you’re over 45 just do it.
I had put it off for several years because everyone kept telling me how awful the prep was. My doctor kept pestering me to agree to do it so I eventually relented.
Turns out, I needed to go to the bathroom frequently during the day which was an annoyance but I never had anything close to an "accident" nor did I feel any strong urge to evacuate at night. So the whole experience turned out to be a huge nothing-burger and I had a few polyps that got sniped that weren't cancerous - so now I have peace of mind that I didn't have before.
Case I read in the JAMA which I get because I signed up during covid.
Late 20's Hispanic lady shows up in the ER with what they think is probably food poisoning. But they do a CT. Which shows changes in her liver which probably is a fatty liver. But they do a biopsy just in case. Biopsy results in a bleed which requires a transfusion and 4 days in the hospital. Biopsy result, fatty liver.
There is also a nontrivial chance of every CT scan to cause fatal cancer. It's a ton of radiation[1], and ERs love pushing it indiscriminately on vulnerable people.
Surprisingly few ER docs anywhere in the world have even a rudimentary understanding of the risks of CT scanning patients. There's a lot of information around about this, but my own first hand (anecdotal) experience is that I've had ER docs try to convince me that it's basically the same as an X-Ray and act like I'm a crazy person when I explain that it's orders of magnitude higher and cumulative over a lifetime. On one hand, it's not their job to care about your long term health - they need to rule out an emergency and get you out the door as quickly as possible - but it's very concerning.
It's a bit like how general practitioners aren't taught about nutrition at all, so give out really poor advice for heart disease patients (the leading cause of mortality in Western economies).
Edit: For comparison, a chest X-ray is around 0.1mSv, a chest CT at 6.1mSv, so a factor of 61 between (https://www.radiologyinfo.org/en/info/safety-xray ). Compared to natural exposure (usually 1 to 3mSv/a) however, a chest CT isn't that bad at 2 to 3 years natural dose, 2 polar flights or 1 year of living at higher altitude or Ramsar (https://aerb.gov.in/images/PDF/image/34086353.pdf ). Acute one-time dose damage has been shown above 100mSv, below that there is no damage shown, only statistical extrapolations.
So I'd say that the risk of using a CT right away should be lower than the risk of overlooking a bleed or a clot in an emergency, where time is of the essence and the dance of "let's do an X-ray first..." might kill more patients than the cancers caused by those CTs.
> On one hand, it's not their job to care about your long term health - they need to rule out an emergency and get you out the door as quickly as possible - but it's very concerning.
Yeah, but what's the alternative when you have a stroke? They need to understand it's type and the mistake here is likely to be fatal since they require opposite treatments.
The article seems to suggest the false positive rate is only 38%:
The trial followed 25,000 adults from the US and Canada over a year, with nearly one in 100 getting a positive result. For 62% of these cases, cancer was later confirmed.
(It also had a false negative rate of 1%:)
The test correctly ruled out cancer in over 99% of those who tested negative.
If the stats were as good as the hyperbole in the article, it would clearly state the only 2 metrics that really matter: predictive value positive (what's the actual probability that you really have cancer if you test positive) and predictive value negative (what's the actual probability that you're cancer free if you test negative). As tptacek points out, these metrics don't just depend on the sensitivity and specificity of the test, but they are highly dependent on the underlying prevalence of the disease, and why broad-based testing for relatively rare diseases often results in horrible PVP and PVN metrics.
Based on your quoted sections, we can infer:
1. About 250 people got a positive result ("nearly one in 100")
2. Of those 250 people, 155 (62%) actually had cancer, 95 did not.
3. About 24,750 people got a negative test result.
4. Assuming a false negative rate of 1% (the quote says "over 99%") it means of those 24,750 people, about 248 actually did have cancer, while about 24,502 did not.
When you write it out like that (and I know I'm making some rounding assumptions on the numbers), it means the test missed the majority of people who had cancer while subjecting over 1/3 of those who tested positive to fear and further expense.
so possibly saving lives and late stage cancer care level medical expenses 2/3 of positive results vs fear and lighter medical care 1/3 of the time. is this not a win?
Nope, there is another important thing that matters: some of the cancers tested are really hard to detect early by other means, and very lethal when discovered late.
I would not be surprised if out of the 155 people who got detected early, about 50 lives were saved that would otherwise be lost.
That is quite a difference in the real world. Even if the statistics stays the same, the health consequences are very different when you test for something banal vs. for pancreatic cancer.
Careful: the stats you're reading are all-cancers, cancers aren't uniformly prevalent, and the specific cancers you're referring to might (and probably do) have much, much worse screening outcomes than the aggregate.
I don't see where this "probably" comes from; it could well be the other way round. It is a new technology and its weak and strong points / applications may differ significantly from what we currently use.
You're overfocused on the "technology" and underfocused on the base rate of the cancers you're concerned about. Just do the math. What you need to know is "just how accurate would this test need to be in order for most of the positives it generates to be true positives". The numbers will be surprising.
Say that you are hunting the elusive snipe, one bird in a million. With standard techniques, you will have a lot of false positives.
But if you learn that the elusive snipe gives off a weird radio signal that other birds don't, your hunt will be a lot shorter.
Same with relatively rare cancers. If you can detect some very specific molecule or structure, your test will be quite reliable anyway. That is why I don't get your use of "probably". Unless you are really familiar with the underlying biochemistry, the probabilities cannot be guessed.
There is absolutely no reason why tests for rare diseases should have high false positive rates. In many other diseases, they don't. For example (although the underlying technology differs), Down syndrome is rare, but its detection barely has any false positives. You can test the entire pregnant population for Downs reliably, and many countries already do that.
I'm not saying that you couldn't develop a super-reliable test for a rare cancer! The point isn't that these cancers are impossible to screen for. The point is that the numbers given in this article do not suggest that such a test has been developed in this case.
It surely does not make sense to screen the entire population for pancreatic cancer, but we already do have a screening for pancreatic cancer in Czechia for "at risk population".
Let's say you do have a positive test for pancreatic cancer. Overall 5 year survival rate 12%, but other than with other cancers, people continue to die after that. Basically, it is almost a death sentence if it is a true positive. Early detection will increase your odds a bit, and prolong your remaining expected lifetime, but even stage 1 pancreatic cancer, only 17% survive to 10 years. Let's say you are one of the 99% of false positives, because everyone gets tested in this hypothetical scenario. Let's say imaging and biopsy looks clean. No symptoms (which you typically don't have until stage 3 with pancreatic cancer, where it is far too late anyways). With the aforementioned odds, what would you do?
Panic? Certainly, given that if it is a real positive, you might as well order your headstone.
Get surgery to remove your pancreas? Well, just the anesthesia as a 0.1% chance of killing you, the surgery might kill 0.3% in total. No pancreas means you will instantly have diabetes, which cuts your life expectancy by 20 years.
Start chemotherapy? Chemo is very dangerous, and there is no chemo mixture known to be effective against pancreatic cancer, usually you just go with the aggressive stuff. It is hard to come by numbers as to how many healthy people a round of chemo would kill, but in cancer patients, it seems that at least 2% and up to a quarter die in the 4 weeks following chemotherapy (https://www.nature.com/articles/s41408-023-00956-x ). And chemotherapy itself has a risk of causing cancers later on.
Start radiation therapy? Well, you don't have a solid tumor to irradiate, so that is not an option anyways. But if done, it would increase your cancer risk as well as damage the irradiated organ (in that case probably your pancreas).
So in all, from 100 positive tests you have 99 false positives in this scenario. If just one of those 99 false positives dies of any of the aforementioned causes, the test has already killed more people than the cancer ever would have. Even if no doctor would do surgery, chemotherapy or radiation treatment on those hypothetical false positives, the psychological effects are still there and maybe already too deadly.
So it is a very complex calculation to decide whether a test is harmful or good. Especially in extreme types of cancer.
"Let's say you are one of the 99% of false positives, because everyone gets tested in this hypothetical scenario."
This alone is a disqualifier for your scenario. A test with 99 per cent of false positives will not be widely used, if at all. (And the original Galleri test that the article was about is nowhere near to that value, and it is not intended to be used in low-risk populations anyway.)
I am all for wargaming situations, but come up with some realistic parameters, not "Luxembourg decided to invade and conquer the USA" scenarios.
> Nope, there is another important thing that matters: some of the cancers tested are really hard to detect early by other means, and very lethal when discovered late.
You are arguing for testing everyone there. If you cannot detect them by other means, you need to test for them this way. And do it for everyone. You have already set up the unrealistic wargaming scenario. You picked pancreatic cancer as your example where you do have to test every 6 months at least, because if you do it more rarely, the disease progression is so fast that testing is useless. There are no specific risk groups for pancreatic cancer beyond a slight risk increase by "the usual all-cancer risk factors". Nothing to pick a test group by.
And a 99% overall false positive rate is easy to achieve, lot's of tests that are in use have this property if you just test everyone very frequently. Each instance of testing has an inherent risk of being a false positive, and if you repeat that for each person, their personal false-positive risk of course goes up with it. All tests that are used frequently have an asymptotic 100% false positive rate.
Are you mistaking me for someone else? I never said or even implied that.
"And a 99% overall false positive rate is easy to achieve,"
Not in the real world, any such experiment will be shut down long before the asymptotic behavior kicks in. Real healthcare does not have unlimited resources to play such games. That is why I don't want to wargame them, it is "Luxembourg attacks the US scenario".
"There are no specific risk groups for pancreatic cancer"
This is just incorrect, people with chronic pancreatitis have massively increased risk of developing pancreatic cancer (16x IIRC). There also seems to be a hereditary factor.
Czech healthcare system, in fact, has a limited pancreatic cancer screening program since 2024, for people who were identified as high-risk.
Prolonging the expected lifetime by several years nontrivially improves chances of surviving until better drugs are found, and ultimately long term survival. Our ability to cure cancers is not constant, we're getting better at it every day.
Even so. Current first-line treatment for pancreatic cancer is surgery, because chemo doesn't really help a lot. Chemo alone is useless in this case. So any kind of treatment that does have a hope of treating anything involves removing the pancreas.
Take those 99% false positives. If you just remove the pancreas from everyone, you remove 20 years of lifetime through severe diabetes. In terms of lost life expectancy, you killed up to 25 people. Surgery complications might kill one more. In all, totally not worth it, because even if you manage to save everyone of those 1% true positives, you still killed more than 20 (statistical) people.
And the detection rate might be increased by more testing. But it needs to be a whole lot more, and it won't help. Usually pancreatic cancer is detected in stage 3 or 4, when it becomes symptomatic, 5 year survival rate below 10% (let's make it 5% for easier maths). The progression from stage 1 to stage 3 takes less than a year if untreated. So you would need to test everyone every 6 months to get detections into the stage 1 and stage 2 cases, that are more treatable. Let's assume you get everyone down to stage 1, with a survival rate of roughly 50% at 5 years, 15% at 10 years. We get a miracle cure developed after 10 years where everyone who is treated survives. So basically we get those 15% 10-year-survivors all to survive to their normal life expectancy (minus 20 because no more pancreas). Averaging they get an extra 10 years each.
Pancreatic cancer is diagnosed in 0.025% of the population each year. In the US at 300Mio., thats 750k in 10 years. With our theoretical miracle cure after 10 years for 15%, that is a gain of 1.125Mio years lifetime. A 1 hour time needed for testing per each of 300Mio people twice a year for 10 years already wastes 685k years of lifetime, so half the gain already. That calculation is already in "not worth it" territory if the waiting time for the blood-draw appointment is increased. That calculation is already off if you calculate the additional strain on the healthcare system, and the additional deaths that will cause.
> If the stats were as good as the hyperbole in the article, it would clearly state the only 2 metrics that really matter: predictive value positive (what's the actual probability that you really have cancer if you test positive) and predictive value negative (what's the actual probability that you're cancer free if you test negative). As tptacek points out, these metrics don't just depend on the sensitivity and specificity of the test
This is a bizarre thing to say in response to... a clear statement of the positive and negative predictive value. PPV is 62% and NPV is "over 99%".
Your calculations don't appear to have any connection to your criticism. You're trying to back into sensitivity ("the test missed the majority of people who had cancer") from reported PPV and NPV, while complaining that sensitivity is misleading and honest reporting would have stated the PPV and NPV.
Definitely keep at it. Also, we should know the accuracy of comparable tests.
Reading the article, I'm still not sure about the accuracy, and don't have the time to carefully parse the whole article. I see at least the following statements (there may be more):
The trial followed 25,000 adults from the US and Canada over a year, with nearly one in 100 getting a positive result. For 62% of these cases, cancer was later confirmed.
and
The test correctly ruled out cancer in over 99% of those who tested negative.
In this case, it is worthwhile. Adjusting for the base rate, the test is correct when it indicates cancer 62% of the time. That's almost certainly worth the false positives.
No, the factors causing the false positives are usually correlated. The most common cause is mutations causing unrelated proteins to mimic an epitope (binding site) of the biomarker protein. Another cause is analytical interference where some other molecule absorbs the wavelength used to measure the assay.
Running the test using multiple different labs helps eliminate contamination and handling errors but most false positives are due to genetics and long term environmental factors.
But if you got one test saying "Detected a fragment of pancreatic cancer DNA" two days later another saying "Didn't see any cancerous DNA fragments" what would the actual effect on the doctor or patient be? Who'll take the risk and say "the first was a false positive"?
Also, these tests are a grand a pop if I'm reading it directly (which I may not be)
> The doctors who perform these tests are trained to understand this type of stuff. They're not just blindly accepting any single test result
They're "trained to understand this type of stuff" in the sense that it will get a mention in medical school. Overwhelmingly, they aren't "trained to understand this type of stuff" in the sense that if you pose them a simple problem of this type, they'll be able to calculate the answers.
From An Intuitive Explanation of Bayes’ Theorem [0]:
> Next, suppose I told you that most doctors get the same wrong answer on this problem – usually, only around 15% of doctors get it right. (“Really? 15%? Is that a real number, or an urban legend based on an Internet poll?” It’s a real number. See Casscells, Schoenberger, and Grayboys 1978; Eddy 1982; Gigerenzer and Hoffrage 1995; and many other studies. It’s a surprising result which is easy to replicate, so it’s been extensively replicated.)
There are Silicon Valley startups increasingly selling these directly to consumers, so I wouldn't assume tests like this are always intermediated by a medical professional.
This isn't perscription drugs, this is information?
To answer your question directly I don't have strong feelings about it. I would prefer if they were freely accessible, other than maybe things like antibiotics that have clear externalities.
I'm okay with insurance companies saying they won't pay for them unless a doctor writes a script, but to gate keep relatively harmless drugs the way we do isn't my favorite policy. There are many times I would love to get a drug but don't want to talk to a doctor about it. This is one of the reasons companies like KHealth exist - they're just rent seeking on a silly system. You can get anything you want, you just have to jump through hoops, but all those hoops just make doctors richer and health care cost more.
Exciting results from blood test for 50 cancers (Oct, 2025)
The trial followed 25,000 adults from the US and Canada over a year, with nearly one in 100 getting a positive result. For 62% of these cases, cancer was later confirmed.
Lead researcher Dr Nima Nabavizadeh, associate professor of radiation medicine at Oregon Health & Science University, said the data showed that the test could "fundamentally change" their approach to cancer screening.
Galleri promises to detect multiple cancers—but new evidence casts doubt on this much hyped blood test (August 2024))
The NHS is currently running a £150m trial of the test, funded by Grail and involving more than 100 000 participants in England.
...
Documents leaked to The BMJ indicate that the criteria being used, unpublished until now, are unsuitable to justify a new national screening programme aimed at saving lives.
They show that even Mike Richards, the chair of the independent UK National Screening Committee, has privately voiced “serious concerns” to Amanda Pritchard, NHS England’s chief executive, about the trial and its ability to provide sufficient evidence “on whether the benefits of testing outweigh any potential harms and at reasonable cost.”
The non-invasive followup for people with positive test results would knock out a lot of the false-positives. At least, thats what I understand of "the usual result of a positive test result for a serious illness, is that a repeat test does not confirm it"
That said, at what level of risk of follow up diagnostic would you baulk? Any procedure which requires a general is bad news, and if you are over 70 its a lot more bad.
Their advertised sensitivity and specificity put them in the ballpark of what other liquid biopsies advertise. The ones I know of target high-risk cohorts where the benefits of other screenings already outweigh the risks of taking them. It doesn't make sense for the average person to be getting periodic full chest CT scans for instance, but it might for a decades-long smoker.
Sure, that's a concern. But for screenings like this the ultimate metric is all-cause mortality (perhaps adjusted for costs and quality of life). It will take several years before we have a clear signal on that.
It's possible (if not likely) that paying for this test made you more likely to die. There are two mechanisms for this:
1) You could develop cancer tomorrow, notice the symptoms, and assume it's not worth getting screened again because you were just supposedly cleared of cancer. A common logical fallacy for our human brains is that we think, "Oh, I just got a test, so it's less likely I have cancer today," which is not how probability works.
2) You could have gotten a false positive, which would have led to unnecessary additional screening. Many methods of cancer screening have some risk, whether from anesthesia, infection, or further false positives leading to unnecessary treatment.
> A common logical fallacy for our human brains is that we think, "Oh, I just got a test, so it's less likely I have cancer today," which is not how probability works.
Given my interactions with my doctors and their tests, they did seem to hold this belief, too.
In any case, the real mistake people make is failing to update their beliefs when new evidence (symptoms) appears. Rookie mistake. My doctors love to do (or rather, not do) this. Who needs differential diagnosis and re-evaluation when they can just keep the diagnosis and continue receiving a fuckton of money after you based on a secore-based system we have here?
I am not going to avoid any reasonable treatment/screen because of it. It was intended to catch asymptomatic cancer. Additional invasive screenings are voluntary and like all treatments they carry risk. I weigh all treatments based on their risks at the time.
For everyday people increased screening of all types has risks, but overall the benefits massively outweigh the risks. If I was a frail 80yo, I might see the risk profile differently.
In my career I've encountered many people who "don't want to know" about medical tests of any kind. I'm not one of those people. Minimally invasive screens early and often please.
> For everyday people increased screening of all types has risks, but overall the benefits massively outweigh the risks.
This is just not how math works, and it's why we still need doctors to order tests -- to protect people from themselves. You clearly don't know what you don't know, but you have a huge amount of confidence that you do, apparently.
There are many people, myself included, who don’t want to be patronized or “protected from ourselves” and prevented from accessing data about our bodies.
An MRI, blood test, continuous glucose monitor, etc. carry essentially no intrinsic risk. It’s ridiculous that we need prescriptions for such things.
What I do or don’t do with that data is my prerogative.
> Crucially, three-quarters of cancers detected were those which have no screening programme such as ovarian, liver, stomach, bladder and pancreatic cancer.
That's exciting. The first step of many. The bar is so low because it remains to be seen if this earlier detection prevents deaths. And of course we still don't know the root causes or triggers, which promotes the same thing occurring a few years later. Remission is not solved, we need cure still.
This test isn’t reliable for many cancers until they reach stage 3 or 4. Better than nothing, but it will give many people a false confidence that they don’t have cancer.
My doctor requested one for me, and I just got it in the mail (USA), which is nice for me.
I suspect that this will remain a niche product. It would, in theory, be applicable to most people on earth. Such wide spread screenings simply aren't viable in a world of massive compliance costs and subsidized health care.
It took an act of god for simple COVID testing to become somewhat inexpensive.
I look forward to getting this test every few years until it is killed by our regulatory apparatus.
They always could! Diagnostics are a bit different than therapeutics, which need that big Phase 3 trial before you can start selling a new therapy.
There are two regulatory regimes for diagnostics in the US:
1) Lab developed tests (LDTs) licensed under the CLIA legislation (I think from the 1980s). These are verified by a Lab Director with a professional license in diagnostics, that allows them to sign out clinical results from the test. There are professional organizations that perform regular inspections of the lab, its condition, its paperwork, its tests, the SOPs, and the internal validations that have been performed at the lab director's direction to assess performance of the tests. These are limited to a single site, the kits for the test can not be sold except for Research Use Only, and if a second site wants to start doing a similar test the lab director at the other site needs to do all the same validation all over again as at the first site.
2) FDA approval for diagnostic medical devices. These can be simple and straightforward for Class 1/2 devices, which do not directly provide medical advice but mere physical readouts (to greatly bastardize the distinction between Class 1/2 and 3). Or the device approval can be quite complex for Class 3 devices, and would require a huge trial like the one described here. If you want to sell the device for others to use, rather than just testing as service, you want to go this route. Though there are still single-site "devices" especially for DNA sequencing tests, that want the FDA label.
Neither of these will result in getting reimbursement for a test. For that, you need to pursue coverage determinations from all the payers, basically one on one. For complex sequencing tests like this that mostly affects older populations, getting CMS coverage (Medicare) can pave the path for others. For other conditions... well... get all your trials and papers together and hope that your patient population is super sympathetic or you can show the insurance company some savings.
A large clinical trial like this one can help with getting coverage for the test, but it has to either show a big medical benefit, or show economic benefits within five years for the payor. Or ideally both. Early cancer detection has potential for this, but I have not heard optimistic things up until now at Galleri's chance for reimbursement any time soon.
I think it’ll become an add on to an annual physical blood panel for the 50+ age cohort. Volume might push prices down faster.
I paid the $950 rate (it’s occasionally discounted to $800, such as now until the end of the year, and you might be able to use FSA/HSA funds depending on plan administrator) and thought it was worth it (to detect potentially asymptomatic early stage cancer).
I'm a tedious broken record about the fact that the base rate of most cancers means that extraordinarily-accurate-seeming screening tests have surprisingly untenable false positive rates. Like, a 99% accurate test for liver cancer might be almost worthless: because the base rate is so low, 99% of positives will be false.
And a false positive screening result is not innocuous: it incurs costs in a variety of different ways, including human health.
This is an important and unintuitive reality about all kinds of tests. You have to take into account the base rate - the likelihood that you have the disease given what you know prior to the test.
If you're otherwise healthy and would have a 1/1,000,000 chance of having the disease before the test, and then you test positive with a test that is 99% accurate, you are ~100x more likely to have the disease than before - but that's still only 1/10,000 - not at all 99% likely, even though the test was "99% accurate"
That said, I think with this knowledge the test still confers helpful information. I might decide to spend $1000 on an additional diagnostic, even knowing that I'm still very likely to be negative. Depends on how wealthy I am, and how serious the disease is, and what the treatments for it are.
> I might decide to spend $1000 on an additional diagnostic, even knowing that I'm still very likely to be negative.
The problem is that diagnostics aren't necessarily risk-free. For example, there's a non-zero risk of death while getting a colonoscopy, to the point that false positives from unnecessary testing can increase all-cause mortality for patients.
There's also a real psychic cost to even a tentative positive result, which is why I'm much more likely to get an actual colonoscopy than a Cologuard test.
I had the same thought. I got the colonoscopy and I don’t regret it at all. And for anyone thinking about it or stressing about it, the prep was not that bad. If you’re over 45 just do it.
I had put it off for several years because everyone kept telling me how awful the prep was. My doctor kept pestering me to agree to do it so I eventually relented.
Turns out, I needed to go to the bathroom frequently during the day which was an annoyance but I never had anything close to an "accident" nor did I feel any strong urge to evacuate at night. So the whole experience turned out to be a huge nothing-burger and I had a few polyps that got sniped that weren't cancerous - so now I have peace of mind that I didn't have before.
Case I read in the JAMA which I get because I signed up during covid.
Late 20's Hispanic lady shows up in the ER with what they think is probably food poisoning. But they do a CT. Which shows changes in her liver which probably is a fatty liver. But they do a biopsy just in case. Biopsy results in a bleed which requires a transfusion and 4 days in the hospital. Biopsy result, fatty liver.
There is also a nontrivial chance of every CT scan to cause fatal cancer. It's a ton of radiation[1], and ERs love pushing it indiscriminately on vulnerable people.
[1] https://www.radiologyinfo.org/en/info/safety-xray
This.
Surprisingly few ER docs anywhere in the world have even a rudimentary understanding of the risks of CT scanning patients. There's a lot of information around about this, but my own first hand (anecdotal) experience is that I've had ER docs try to convince me that it's basically the same as an X-Ray and act like I'm a crazy person when I explain that it's orders of magnitude higher and cumulative over a lifetime. On one hand, it's not their job to care about your long term health - they need to rule out an emergency and get you out the door as quickly as possible - but it's very concerning.
It's a bit like how general practitioners aren't taught about nutrition at all, so give out really poor advice for heart disease patients (the leading cause of mortality in Western economies).
Radiation damage being cumulative over long periods is an assumption for radiation safety. In reality, it is probably less harmful than that.
https://en.wikipedia.org/wiki/Linear_no-threshold_model
Edit: For comparison, a chest X-ray is around 0.1mSv, a chest CT at 6.1mSv, so a factor of 61 between (https://www.radiologyinfo.org/en/info/safety-xray ). Compared to natural exposure (usually 1 to 3mSv/a) however, a chest CT isn't that bad at 2 to 3 years natural dose, 2 polar flights or 1 year of living at higher altitude or Ramsar (https://aerb.gov.in/images/PDF/image/34086353.pdf ). Acute one-time dose damage has been shown above 100mSv, below that there is no damage shown, only statistical extrapolations.
So I'd say that the risk of using a CT right away should be lower than the risk of overlooking a bleed or a clot in an emergency, where time is of the essence and the dance of "let's do an X-ray first..." might kill more patients than the cancers caused by those CTs.
> On one hand, it's not their job to care about your long term health - they need to rule out an emergency and get you out the door as quickly as possible - but it's very concerning.
Yeah, but what's the alternative when you have a stroke? They need to understand it's type and the mistake here is likely to be fatal since they require opposite treatments.
The article seems to suggest the false positive rate is only 38%:
The trial followed 25,000 adults from the US and Canada over a year, with nearly one in 100 getting a positive result. For 62% of these cases, cancer was later confirmed.
(It also had a false negative rate of 1%:)
The test correctly ruled out cancer in over 99% of those who tested negative.
If the stats were as good as the hyperbole in the article, it would clearly state the only 2 metrics that really matter: predictive value positive (what's the actual probability that you really have cancer if you test positive) and predictive value negative (what's the actual probability that you're cancer free if you test negative). As tptacek points out, these metrics don't just depend on the sensitivity and specificity of the test, but they are highly dependent on the underlying prevalence of the disease, and why broad-based testing for relatively rare diseases often results in horrible PVP and PVN metrics.
Based on your quoted sections, we can infer:
1. About 250 people got a positive result ("nearly one in 100")
2. Of those 250 people, 155 (62%) actually had cancer, 95 did not.
3. About 24,750 people got a negative test result.
4. Assuming a false negative rate of 1% (the quote says "over 99%") it means of those 24,750 people, about 248 actually did have cancer, while about 24,502 did not.
When you write it out like that (and I know I'm making some rounding assumptions on the numbers), it means the test missed the majority of people who had cancer while subjecting over 1/3 of those who tested positive to fear and further expense.
so possibly saving lives and late stage cancer care level medical expenses 2/3 of positive results vs fear and lighter medical care 1/3 of the time. is this not a win?
"only 2 metrics that really matter"
Nope, there is another important thing that matters: some of the cancers tested are really hard to detect early by other means, and very lethal when discovered late.
I would not be surprised if out of the 155 people who got detected early, about 50 lives were saved that would otherwise be lost.
That is quite a difference in the real world. Even if the statistics stays the same, the health consequences are very different when you test for something banal vs. for pancreatic cancer.
Careful: the stats you're reading are all-cancers, cancers aren't uniformly prevalent, and the specific cancers you're referring to might (and probably do) have much, much worse screening outcomes than the aggregate.
"and probably do"
Why probably?
I don't see where this "probably" comes from; it could well be the other way round. It is a new technology and its weak and strong points / applications may differ significantly from what we currently use.
You're overfocused on the "technology" and underfocused on the base rate of the cancers you're concerned about. Just do the math. What you need to know is "just how accurate would this test need to be in order for most of the positives it generates to be true positives". The numbers will be surprising.
Both the technology and the base rate DO matter.
Say that you are hunting the elusive snipe, one bird in a million. With standard techniques, you will have a lot of false positives.
But if you learn that the elusive snipe gives off a weird radio signal that other birds don't, your hunt will be a lot shorter.
Same with relatively rare cancers. If you can detect some very specific molecule or structure, your test will be quite reliable anyway. That is why I don't get your use of "probably". Unless you are really familiar with the underlying biochemistry, the probabilities cannot be guessed.
There is absolutely no reason why tests for rare diseases should have high false positive rates. In many other diseases, they don't. For example (although the underlying technology differs), Down syndrome is rare, but its detection barely has any false positives. You can test the entire pregnant population for Downs reliably, and many countries already do that.
I'm not saying that you couldn't develop a super-reliable test for a rare cancer! The point isn't that these cancers are impossible to screen for. The point is that the numbers given in this article do not suggest that such a test has been developed in this case.
It surely does not make sense to screen the entire population for pancreatic cancer, but we already do have a screening for pancreatic cancer in Czechia for "at risk population".
https://www.cgs-cls.cz/screening/program-vyhledavani-rakovin...
For such programs, a blood test would be a huge boon and they could even expand the coverage a bit.
Maybe. Let's do a thought experiment.
Let's say you do have a positive test for pancreatic cancer. Overall 5 year survival rate 12%, but other than with other cancers, people continue to die after that. Basically, it is almost a death sentence if it is a true positive. Early detection will increase your odds a bit, and prolong your remaining expected lifetime, but even stage 1 pancreatic cancer, only 17% survive to 10 years. Let's say you are one of the 99% of false positives, because everyone gets tested in this hypothetical scenario. Let's say imaging and biopsy looks clean. No symptoms (which you typically don't have until stage 3 with pancreatic cancer, where it is far too late anyways). With the aforementioned odds, what would you do?
Panic? Certainly, given that if it is a real positive, you might as well order your headstone.
Panic more? Maybe people with those news will change their behaviour and engage in risky activities, get depressed, or attempt suicide (https://jamanetwork.com/journals/jamanetworkopen/fullarticle... ). All of which will kill some of those people.
Get surgery to remove your pancreas? Well, just the anesthesia as a 0.1% chance of killing you, the surgery might kill 0.3% in total. No pancreas means you will instantly have diabetes, which cuts your life expectancy by 20 years.
Start chemotherapy? Chemo is very dangerous, and there is no chemo mixture known to be effective against pancreatic cancer, usually you just go with the aggressive stuff. It is hard to come by numbers as to how many healthy people a round of chemo would kill, but in cancer patients, it seems that at least 2% and up to a quarter die in the 4 weeks following chemotherapy (https://www.nature.com/articles/s41408-023-00956-x ). And chemotherapy itself has a risk of causing cancers later on.
Start radiation therapy? Well, you don't have a solid tumor to irradiate, so that is not an option anyways. But if done, it would increase your cancer risk as well as damage the irradiated organ (in that case probably your pancreas).
So in all, from 100 positive tests you have 99 false positives in this scenario. If just one of those 99 false positives dies of any of the aforementioned causes, the test has already killed more people than the cancer ever would have. Even if no doctor would do surgery, chemotherapy or radiation treatment on those hypothetical false positives, the psychological effects are still there and maybe already too deadly.
So it is a very complex calculation to decide whether a test is harmful or good. Especially in extreme types of cancer.
"Let's say you are one of the 99% of false positives, because everyone gets tested in this hypothetical scenario."
This alone is a disqualifier for your scenario. A test with 99 per cent of false positives will not be widely used, if at all. (And the original Galleri test that the article was about is nowhere near to that value, and it is not intended to be used in low-risk populations anyway.)
I am all for wargaming situations, but come up with some realistic parameters, not "Luxembourg decided to invade and conquer the USA" scenarios.
> Nope, there is another important thing that matters: some of the cancers tested are really hard to detect early by other means, and very lethal when discovered late.
You are arguing for testing everyone there. If you cannot detect them by other means, you need to test for them this way. And do it for everyone. You have already set up the unrealistic wargaming scenario. You picked pancreatic cancer as your example where you do have to test every 6 months at least, because if you do it more rarely, the disease progression is so fast that testing is useless. There are no specific risk groups for pancreatic cancer beyond a slight risk increase by "the usual all-cancer risk factors". Nothing to pick a test group by.
And a 99% overall false positive rate is easy to achieve, lot's of tests that are in use have this property if you just test everyone very frequently. Each instance of testing has an inherent risk of being a false positive, and if you repeat that for each person, their personal false-positive risk of course goes up with it. All tests that are used frequently have an asymptotic 100% false positive rate.
"You are arguing for testing everyone there."
Are you mistaking me for someone else? I never said or even implied that.
"And a 99% overall false positive rate is easy to achieve,"
Not in the real world, any such experiment will be shut down long before the asymptotic behavior kicks in. Real healthcare does not have unlimited resources to play such games. That is why I don't want to wargame them, it is "Luxembourg attacks the US scenario".
"There are no specific risk groups for pancreatic cancer"
This is just incorrect, people with chronic pancreatitis have massively increased risk of developing pancreatic cancer (16x IIRC). There also seems to be a hereditary factor.
Czech healthcare system, in fact, has a limited pancreatic cancer screening program since 2024, for people who were identified as high-risk.
https://www.cgs-cls.cz/screening/program-vyhledavani-rakovin...
Prolonging the expected lifetime by several years nontrivially improves chances of surviving until better drugs are found, and ultimately long term survival. Our ability to cure cancers is not constant, we're getting better at it every day.
Even so. Current first-line treatment for pancreatic cancer is surgery, because chemo doesn't really help a lot. Chemo alone is useless in this case. So any kind of treatment that does have a hope of treating anything involves removing the pancreas.
Take those 99% false positives. If you just remove the pancreas from everyone, you remove 20 years of lifetime through severe diabetes. In terms of lost life expectancy, you killed up to 25 people. Surgery complications might kill one more. In all, totally not worth it, because even if you manage to save everyone of those 1% true positives, you still killed more than 20 (statistical) people.
And the detection rate might be increased by more testing. But it needs to be a whole lot more, and it won't help. Usually pancreatic cancer is detected in stage 3 or 4, when it becomes symptomatic, 5 year survival rate below 10% (let's make it 5% for easier maths). The progression from stage 1 to stage 3 takes less than a year if untreated. So you would need to test everyone every 6 months to get detections into the stage 1 and stage 2 cases, that are more treatable. Let's assume you get everyone down to stage 1, with a survival rate of roughly 50% at 5 years, 15% at 10 years. We get a miracle cure developed after 10 years where everyone who is treated survives. So basically we get those 15% 10-year-survivors all to survive to their normal life expectancy (minus 20 because no more pancreas). Averaging they get an extra 10 years each.
Pancreatic cancer is diagnosed in 0.025% of the population each year. In the US at 300Mio., thats 750k in 10 years. With our theoretical miracle cure after 10 years for 15%, that is a gain of 1.125Mio years lifetime. A 1 hour time needed for testing per each of 300Mio people twice a year for 10 years already wastes 685k years of lifetime, so half the gain already. That calculation is already in "not worth it" territory if the waiting time for the blood-draw appointment is increased. That calculation is already off if you calculate the additional strain on the healthcare system, and the additional deaths that will cause.
> If the stats were as good as the hyperbole in the article, it would clearly state the only 2 metrics that really matter: predictive value positive (what's the actual probability that you really have cancer if you test positive) and predictive value negative (what's the actual probability that you're cancer free if you test negative). As tptacek points out, these metrics don't just depend on the sensitivity and specificity of the test
This is a bizarre thing to say in response to... a clear statement of the positive and negative predictive value. PPV is 62% and NPV is "over 99%".
Your calculations don't appear to have any connection to your criticism. You're trying to back into sensitivity ("the test missed the majority of people who had cancer") from reported PPV and NPV, while complaining that sensitivity is misleading and honest reporting would have stated the PPV and NPV.
This website is a good analysis of various diagnostic tests:
https://thennt.com/thennt-explained/
Also why I don't bother with PSA tests until I start getting symptoms: https://thennt.com/nnt/psa-test-to-screen-for-prostate-cance...
Definitely keep at it. Also, we should know the accuracy of comparable tests.
Reading the article, I'm still not sure about the accuracy, and don't have the time to carefully parse the whole article. I see at least the following statements (there may be more):
The trial followed 25,000 adults from the US and Canada over a year, with nearly one in 100 getting a positive result. For 62% of these cases, cancer was later confirmed.
and
The test correctly ruled out cancer in over 99% of those who tested negative.
In this case, it is worthwhile. Adjusting for the base rate, the test is correct when it indicates cancer 62% of the time. That's almost certainly worth the false positives.
What if you just do these testa every few days? Would this minimize error rate?
Probably not. They are not likely independent. Whatever it is about you that caused the false positive yesterday might likely cause it tomorrow.
No, the factors causing the false positives are usually correlated. The most common cause is mutations causing unrelated proteins to mimic an epitope (binding site) of the biomarker protein. Another cause is analytical interference where some other molecule absorbs the wavelength used to measure the assay.
Running the test using multiple different labs helps eliminate contamination and handling errors but most false positives are due to genetics and long term environmental factors.
But if you got one test saying "Detected a fragment of pancreatic cancer DNA" two days later another saying "Didn't see any cancerous DNA fragments" what would the actual effect on the doctor or patient be? Who'll take the risk and say "the first was a false positive"?
Also, these tests are a grand a pop if I'm reading it directly (which I may not be)
The doctors who perform these tests are trained to understand this type of stuff. They're not just blindly accepting any single test result
> The doctors who perform these tests are trained to understand this type of stuff. They're not just blindly accepting any single test result
They're "trained to understand this type of stuff" in the sense that it will get a mention in medical school. Overwhelmingly, they aren't "trained to understand this type of stuff" in the sense that if you pose them a simple problem of this type, they'll be able to calculate the answers.
From An Intuitive Explanation of Bayes’ Theorem [0]:
> Next, suppose I told you that most doctors get the same wrong answer on this problem – usually, only around 15% of doctors get it right. (“Really? 15%? Is that a real number, or an urban legend based on an Internet poll?” It’s a real number. See Casscells, Schoenberger, and Grayboys 1978; Eddy 1982; Gigerenzer and Hoffrage 1995; and many other studies. It’s a surprising result which is easy to replicate, so it’s been extensively replicated.)
[0] https://www.yudkowsky.net/rational/bayes
There are Silicon Valley startups increasingly selling these directly to consumers, so I wouldn't assume tests like this are always intermediated by a medical professional.
Edit: There is in fact another comment on this thread of someone doing exactly this: https://news.ycombinator.com/item?id=45652535
One of the multitude of things that makes me frustrated about the medical establishment in the US is how they insist on gatekeeping test results.
Do you feel that we shouldn't have gatekeepers for prescription drugs either?
This isn't perscription drugs, this is information?
To answer your question directly I don't have strong feelings about it. I would prefer if they were freely accessible, other than maybe things like antibiotics that have clear externalities.
I'm okay with insurance companies saying they won't pay for them unless a doctor writes a script, but to gate keep relatively harmless drugs the way we do isn't my favorite policy. There are many times I would love to get a drug but don't want to talk to a doctor about it. This is one of the reasons companies like KHealth exist - they're just rent seeking on a silly system. You can get anything you want, you just have to jump through hoops, but all those hoops just make doctors richer and health care cost more.
Exciting results from blood test for 50 cancers (Oct, 2025)
~ https://www.bbc.com/news/articles/c205g21n1zzoA year ago:
Galleri promises to detect multiple cancers—but new evidence casts doubt on this much hyped blood test (August 2024))
~ https://www.bmj.com/content/386/bmj.q1706Company site: https://www.galleri.com/what-is-galleri/types-of-cancer-dete...
So 250 came back positive. What about the 24,750? Are they confirmed cancer free?
>The test correctly ruled out cancer in over 99% of those who tested negative.
Forgive me if I'm wrong, but isn't this the textbook example of understanding false negatives in testing people at scale?
Yeah that's classic Taleb right there
More like classic Bayes. Taleb's pop-sci came a few decades later.
I think you mean a few centuries.
Not convinced the Grail approach is safe and effective for low-risk cohorts given the false-positive rate. Follow-up diagnostics are not risk-free.
The non-invasive followup for people with positive test results would knock out a lot of the false-positives. At least, thats what I understand of "the usual result of a positive test result for a serious illness, is that a repeat test does not confirm it"
That said, at what level of risk of follow up diagnostic would you baulk? Any procedure which requires a general is bad news, and if you are over 70 its a lot more bad.
Their advertised sensitivity and specificity put them in the ballpark of what other liquid biopsies advertise. The ones I know of target high-risk cohorts where the benefits of other screenings already outweigh the risks of taking them. It doesn't make sense for the average person to be getting periodic full chest CT scans for instance, but it might for a decades-long smoker.
Sure, that's a concern. But for screenings like this the ultimate metric is all-cause mortality (perhaps adjusted for costs and quality of life). It will take several years before we have a clear signal on that.
Party pooper?
https://pmc.ncbi.nlm.nih.gov/articles/PMC11886625/
I did this recently. Paid out of pocket, but it was worth it for the peace of mind.
It's not perfect but it's easy/fast and a good way to screen for big problems.
It's possible (if not likely) that paying for this test made you more likely to die. There are two mechanisms for this:
1) You could develop cancer tomorrow, notice the symptoms, and assume it's not worth getting screened again because you were just supposedly cleared of cancer. A common logical fallacy for our human brains is that we think, "Oh, I just got a test, so it's less likely I have cancer today," which is not how probability works.
2) You could have gotten a false positive, which would have led to unnecessary additional screening. Many methods of cancer screening have some risk, whether from anesthesia, infection, or further false positives leading to unnecessary treatment.
> A common logical fallacy for our human brains is that we think, "Oh, I just got a test, so it's less likely I have cancer today," which is not how probability works.
Given my interactions with my doctors and their tests, they did seem to hold this belief, too.
In any case, the real mistake people make is failing to update their beliefs when new evidence (symptoms) appears. Rookie mistake. My doctors love to do (or rather, not do) this. Who needs differential diagnosis and re-evaluation when they can just keep the diagnosis and continue receiving a fuckton of money after you based on a secore-based system we have here?
Hyperbole.
I am not going to avoid any reasonable treatment/screen because of it. It was intended to catch asymptomatic cancer. Additional invasive screenings are voluntary and like all treatments they carry risk. I weigh all treatments based on their risks at the time.
For everyday people increased screening of all types has risks, but overall the benefits massively outweigh the risks. If I was a frail 80yo, I might see the risk profile differently.
In my career I've encountered many people who "don't want to know" about medical tests of any kind. I'm not one of those people. Minimally invasive screens early and often please.
> For everyday people increased screening of all types has risks, but overall the benefits massively outweigh the risks.
This is just not how math works, and it's why we still need doctors to order tests -- to protect people from themselves. You clearly don't know what you don't know, but you have a huge amount of confidence that you do, apparently.
Here's a list of different types of cancer screenings and where the risk/benefit falls: https://publichealth.jhu.edu/2023/balancing-the-benefits-and...
The risk of any cancer screening has to be calculated with variables like:
- how risky is the test?
- what are the risks of a false positive?
- how does a true negative affect the person's behavior in the future?
- what is the likelihood that the patient has asymptomatic cancer, based on risks like genetics and age?
- how difficult is the cancer to treat in different stages?
Without looking at all of those things, you don't know if the test is going to increase or decrease all-cause mortality risk.
There are many people, myself included, who don’t want to be patronized or “protected from ourselves” and prevented from accessing data about our bodies.
An MRI, blood test, continuous glucose monitor, etc. carry essentially no intrinsic risk. It’s ridiculous that we need prescriptions for such things.
What I do or don’t do with that data is my prerogative.
> Crucially, three-quarters of cancers detected were those which have no screening programme such as ovarian, liver, stomach, bladder and pancreatic cancer.
That's exciting. The first step of many. The bar is so low because it remains to be seen if this earlier detection prevents deaths. And of course we still don't know the root causes or triggers, which promotes the same thing occurring a few years later. Remission is not solved, we need cure still.
Function Health offers this test as an $899 add-on to their subscription which is $499. It’s on sale a few times a year at $365.
I’m about to order it.
This test isn’t reliable for many cancers until they reach stage 3 or 4. Better than nothing, but it will give many people a false confidence that they don’t have cancer.
Is it better than the status quo? How many people know they have stage 1 cancer anyway?
As opposed to…?
If they don’t take the test, then presumably they also think they don’t have cancer.
My doctor requested one for me, and I just got it in the mail (USA), which is nice for me.
I suspect that this will remain a niche product. It would, in theory, be applicable to most people on earth. Such wide spread screenings simply aren't viable in a world of massive compliance costs and subsidized health care.
It took an act of god for simple COVID testing to become somewhat inexpensive.
I look forward to getting this test every few years until it is killed by our regulatory apparatus.
Doctors can simply request these now?
The article made it read like it was some trial that just completed.
Does anyone know if (or when) these are regularly available in Canada? And the costs associated?
They always could! Diagnostics are a bit different than therapeutics, which need that big Phase 3 trial before you can start selling a new therapy.
There are two regulatory regimes for diagnostics in the US:
1) Lab developed tests (LDTs) licensed under the CLIA legislation (I think from the 1980s). These are verified by a Lab Director with a professional license in diagnostics, that allows them to sign out clinical results from the test. There are professional organizations that perform regular inspections of the lab, its condition, its paperwork, its tests, the SOPs, and the internal validations that have been performed at the lab director's direction to assess performance of the tests. These are limited to a single site, the kits for the test can not be sold except for Research Use Only, and if a second site wants to start doing a similar test the lab director at the other site needs to do all the same validation all over again as at the first site.
2) FDA approval for diagnostic medical devices. These can be simple and straightforward for Class 1/2 devices, which do not directly provide medical advice but mere physical readouts (to greatly bastardize the distinction between Class 1/2 and 3). Or the device approval can be quite complex for Class 3 devices, and would require a huge trial like the one described here. If you want to sell the device for others to use, rather than just testing as service, you want to go this route. Though there are still single-site "devices" especially for DNA sequencing tests, that want the FDA label.
Neither of these will result in getting reimbursement for a test. For that, you need to pursue coverage determinations from all the payers, basically one on one. For complex sequencing tests like this that mostly affects older populations, getting CMS coverage (Medicare) can pave the path for others. For other conditions... well... get all your trials and papers together and hope that your patient population is super sympathetic or you can show the insurance company some savings.
A large clinical trial like this one can help with getting coverage for the test, but it has to either show a big medical benefit, or show economic benefits within five years for the payor. Or ideally both. Early cancer detection has potential for this, but I have not heard optimistic things up until now at Galleri's chance for reimbursement any time soon.
It could be that GP participated in a study, or simply had a cool doctor willing to pull a few strings.
I think it’ll become an add on to an annual physical blood panel for the 50+ age cohort. Volume might push prices down faster.
I paid the $950 rate (it’s occasionally discounted to $800, such as now until the end of the year, and you might be able to use FSA/HSA funds depending on plan administrator) and thought it was worth it (to detect potentially asymptomatic early stage cancer).
(no affiliation)
cfDNA-based tests have received heavy investment, mainly because they are easy to market.
It’s not a false promise — rather, a technology that is still far from perfect because of existing technical barriers.
Neato