It's worth noting that, in the absence of pre-registration, one should should assume that the body of literature (and any systematic review or meta-analysis that relies on it) could be significantly influenced by the file drawer effect.
Anyway, I looked at a recent systematic review from those search results (https://pmc.ncbi.nlm.nih.gov/articles/PMC8622150/), and it found exactly one double-blind RCT (https://pmc.ncbi.nlm.nih.gov/articles/PMC5794882/) that seemed to support what the FDA is saying here. It had a fairly short duration (12 weeks) and a small cohort (48). I'm not medical expert but I do read medical literature as an amateur, and I'm pretty sure this is nowhere close to the standard of evidence for establishing safety and efficacy that the FDA used to demand. It feels like we may be reverting all the way back to the evidentiary standards that allowed crap like thalidomide onto the market.
Sample size doesn't tell you everything about a study. You don't need to throw 40,000 people out of an airplane to determine it's safer with a parachute.
It does tell you something. But that kind of sample size is arguably underpowered for anything but a preliminary study. It's the statistical equivalent of Hubble before it had its corrective optics installed: still fine for seeing big unsubtle things like whether parachutes are warranted when jumping from airplanes, but unable to resolve all the details you want to know before concluding that a medication is safe and effective.
Participants 92 aircraft passengers aged 18 and over were screened for participation. 23 agreed to be enrolled and were randomized.
Intervention Jumping from an aircraft (airplane or helicopter) with a parachute versus an empty backpack (unblinded).
Main outcome measures Composite of death or major traumatic injury (defined by an Injury Severity Score over 15) upon impact with the ground measured immediately after landing.
Another favorite of mine along these lines is "Cigarette smoking: an underused tool in high-performance endurance training". (https://pmc.ncbi.nlm.nih.gov/articles/PMC3001541/) This one might actually be quite pertinent in this case, because the FDA's decision appears to rely heavily on exactly the kind of reasoning that this article satirizes.
There's an old joke about the lack of randomized controlled trials for parachutes. The joke is deployed when people complain about the lack of formal studies for things whose benefit is obvious.
Then somebody went ahead and did it, just to be funny. But you can't actually do a randomized controlled trial on parachutes, so you get a third layer of joke, about studies that don't actually prove anything.
I reviewed the linked studies of folinic acid treatment in ASD, and they uniformly say that larger studies are needed before considering this as a widely available treatment. The main issues that need to be worked out are:
* who this applies to (some studies suggest genotype and autism subtype matter for getting positive outcomes)
* what the side effects are (12 week studies of <100 people are not enough to safely deploy this as a long-term treatment at scale)
* how this compares to behavioral treatment (ABA and sensory interventions have reliable positive outcomes as well)
I think there's a useful signal there, but we need to be cautious rolling things out at a national scale without bigger studies.
This gets brought up often and while true isn't very useful. We know jumping out of plane is dangerous because we have a good understanding of the physics and we have many many comparable examples (jumping out of buildings etc). We understand the mechanism of action of parachutes, so we also know for example a tiny little parachute made of paper towel won't work, and we don't need to test it to know that.
Do we understand the cause of autism or how this supposed cure works?
The key is effect size. We didn't know the exact method of action when we started injecting comatose diabetic children with insulin, but when the coma patient sits up and starts chatting with you, you can establish that a drug is effective with very small sample sizes.
Sure, but even there you need some reasonable sample size. i.e. a sample of 1 won't be sufficient and more importantly it needs to be reproducible. Inject 1 patient and they wake up, but if you inject 10 others and nothing happens and it may not be effective (not that it's definitely not effective). This would require a then a larger sample size.
Autism isn't as cut and dry as this. Falling out of planes and diabetic comas are truly black-and-white outcomes, but ASD assessment involves behavioral testing of social and verbal ability, which has high variability between and within individuals. The linked studies about folinic acid are not finding a binary conversion of of ASD kids from non-verbal to verbal, rather they are detecting X% increase in Y behavioral test, which is subtle and something that is found with behavioral treatment. I would love to live in a world where we discover a treatment for ASD that is as clearly successful as insulin for diabetes, but we're not there yet.
The evidence is from a study with N=40. Not 40,000. 40.