This is so, so interesting. Thanks so much for posting this!
The author already touched upon a few hypotheses why the industry has arrived at this state. There's one I'd like to highlight: The author, now having gone through this experience, would actually be in a perfect position to found their own CRO and implement all their efficiency learnings there. However, they probably look back upon this experience and think it was super painful and would prefer to do fun stuff like developing software instead.
That, for me, is the main reason. There are smart, structured, pragmatic people who got acquainted with this industry, but they choose to leave it.
And that's sad!
So the solution would be to find a group of smart, structured, pragmatic people with a high tolerance for bureaucratic pain and give them the resources to make this industry more efficient.
And here's my plug: That's exactly what we're doing at OpenRegulatory [1], albeit for medical device software and not for clinical trials. Among other things, we've published all our document templates for free (also on GitHub), host a free Slack community for companies to help each other out, and offer a compliance SaaS with transparent and fair pricing. It's still a long way, but it's a start :)
Your company is very interesting. I briefly looked over the website, but couldn't tell what country(ies?) you operate in? My assumption is the regulatory requirements of each country are dramatically different, so seeing the pricing in not $ makes me think it's unlikely you offer services in the US, for instance.
As jtwaleson already correctly replied (thanks!), we're based in Germany and are mostly focused on EU regulations. The "good" news is that if you gain regulatory approval within the EU you can bring your device to market in any EU country.
There's quite an overlap with US (FDA) regulations but we're less experienced there.
Most startups (at least in my bubble?) focus on one of those two areas.
Great summary! And yet, for all their problems, CROs are sometimes still the fastest alternative. We did a clinical trial for a software platform a few years ago, and hoped to partner with a prestigious hospital system to run the trial.
The hospital partner was 100% on board—our protocol just had to pass their internal review board (IRB). IRB is a black box, and they could only give us a vague estimate of how long this would take. We had a huge team, and every week we sat around waiting cost about $200k.
Finally the client couldn't take the uncertainty and burn rate anymore, and they reached out to a CRO they had used in the past. The CRO's IRB had a fixed timeline (2 weeks?), and we were onboarding patients a short time later.
Yeah the lesson with getting IRB approval if you want it done in less than 6 months is to always use an external IRB (if possible). Institutional IRBs are sloooooowwww.
Any particular issues? I am most opposed to the rigid 30-day cycles and the need to spend one just to get on the agenda. But in terms of the questions asked and the due diligence required, I have not found our IRBs to be onerous
We did keep working on the platform, but from the perspective of the client, we already had the MVP for the trial. So to them we were working on “nice to haves” for 200k/wk
I worked as Director of Engineering at a company for 10 years that built a SaaS for clinical trial document management.
There is a weird perversion of incentives in the industry driven under the guise of "quality" and "safety" that from a software engineer's perspective, looked pretty bad to me because of how manual and error prone the process is.
The problem is that the people who have the knowledge aren't incentived to make it more efficient or cost effective; the purpose of many of these folks is to simply extract rent from the system and blame regulations and "patient safety" concerns.
At an MES startup, one of the industry stalwarts (starts with V) was more or less using mafia tactics to force the startup to use their software or they would voice no confidence with pharma companies where big V was already entrenched.
One of my passion projects would be to build an open-core clinical trial document management system with a focus on automation and templatization of common processes and workflows. It's insane how much waste this system has.
The logistical inefficiencies mentioned in this article are mind boggling when compared to what we've come to expect in different industries. We all know healthcare is always a little behind the curve, but with clinical trials, we're talking decades.
For example: most clinical studies have no way to track samples going from sites to labs beyond a hope, prayer, and constellation of unconnected spreadsheets. The status quo is to write a 50 page lab manual and include a wall of text and bullets outlining a list of multiple stakeholders that need to be emailed a tracking number each time a sample is shipped. Not to mention there are days you really shouldn't ship samples. Labs have no idea when samples are going to show up. Seriously, samples just show up, they open the box expecting a piece of paper to identify the samples.
An incredible amount of perfectly good, usable samples get LOST or DESTROYED due to the fact that there's no way to track any of these samples or coordinate stakeholders. We're talking critical samples, like blood from kids with cancer. It's absolutely ridiculous.
Compare this sob story to the fact that I know in realtime where all the dumb shit I bought on Amazon is. The cognitive dissonance here is beyond comparison!
+ + +
About 8 years ago my brother and I were given a window into this problem. Being Amazon FBA sellers ourselves, we thought, "hey now that can't be too hard to fix" and thus began the wild journey of Slope [0]. What we learned working on this problem is that clinical research sites literally get crapped on left and right with ridiculous contracts & protocols written by people who don't understand the clinical setting or logistics in general. With each research site running 30-50 clinical studies and each of those studies having its own catalog of kits, shippers, and other inventory, it's no wonder why clinical research is so expensive! And this is just supply chain folks.
Slope started by making an inventory and sample management software to address the needs of sites and have worked backwards into selling this platform to Sponsors and CROs. While I have your attention, we're hiring a head of engineering now and will have a JD up for a product manager next week [1]. Thanks for reading!
People complain about industries with bureaucracy but nobody wants to take the hit and actually work in those industries. They want magically for their pay/benefits/culture incentives to align.
Exactly - like the OP you replied to - Slope are trying to hire a Head of engineering for a salary that competent mid/senior level engineers should be easily getting. It's no tech industry, and the work is less fun
I'm one of those that switched from tech into an industry that is steeped in bureaucracy, secrecy, red-tape, and whatever other term you can find. It's been about 10 years, but I occasionally lurk on HN to see people in tech are up to.
For several reasons, but basically I was burned out, and wanted a new challenge. Working in biotech manufacturing was also a way to make something that helps people.
Call me cynical, but this could apply in most any industry:
Suppose that one tries to push back on inefficiency in generalーsomeone naive but well-intentioned, like myself, who is tired of the Long March of interminable meetings. In fact, I expressed my concerns about the proliferation of unproductive meetings in the clinical department to the C-suite executives (with whom I remain on good terms!) and, together, we carefully planned a series of reforms to company culture designed to reduce meeting burden. We hoped to implement a soft cap on the number of meeting participants and the length of a meeting, to normalize efficient behavior such as leaving meetings midway if you no longer need to participate in the remainder, starting meetings on time without 15 minutes of small talk, and a mandated review of all recurring meetings with the intention of pruning overall meeting load by >50%.
Ironically, on the morning that we planned on roll out these reforms, we received urgent and vehement protests from a member of middle management who was notorious for obsessively overscheduling meetings with huge participant lists just for the sake of “inclusion” and “engagement.” Because this particular manager was currently playing a crucial role in managing the rollout of several clinical trials at the same time, executive management judged that the risk of alienating him was too high, and to this day I believe these reforms remain unimplemented (although I hear he has since left the company, so perhaps my slides will eventually see the light of day).
I worked in clinical development and the rough estimate of phase 3 clinical trial costs (this is a few years back) was $15,000 per patient per year.
Depending on the trial design it can be less (less frequent follow up, less invasive testing) or a lot more (say doing multiple MRIs or other expensive imaging).
It’s kind of mind blowing when you think about the really big trials for medications like cholesterol. Trials are often 30,000 or more patients, in two trials for 2-3 years.
You’re roughly talking about $2B or more just for phase 3 (ignoring all the earlier trials and preclinical work).
Another huge cost is comparator products. Some trials budgets I have seen spent 100-200+ million purchasing the drugs they are trying to outperform. This is for trials with <1000 patients.
I worked at a LVAD company (think artificial heart) that was doing a trial, they had to buy dozens (hundreds?) of their competitors LVADs, at $120k a piece
There's a reason why we ended up putting our trials together essentially by trial and error! You learn a lot by writing your own clinical protocol, interacting with the FDA etc. It's an intimidating process at the outset, but if you can save $200k on a first-in-man study by not contracting a CRO, that's gonna extend your runway by a lot!
I don't completely disagree with your point, but for one's clinical regulatory strategy, perhaps "trial and error" isn't the best branding to appeal to investigators, patients, regulators or investors.
Perhaps I was being too self deprecating! We worked closely with the FDA through multiple QSub meetings to make sure it was up to snuff, and did a lot of legwork on the backend to make sure we understood what they were asking for. 3 or 4 revisions later, we had an approval!
When we started putting the trial together, we were still self funded. In medical device, to get to series A you need human data, so by doing the trial protocol ourselves, we avoid the chicken and egg problem of needing money in order to raise money. Once you get human data, your valuation skyrockets.
This is a very naive. As a techie that became a physician and I'm now back in the tech world, it took me a long time to understand a fundamental difference in approach. The medical world moves very very slowly and it seems very very inefficent to a tech person. It is a necessary evil. In medical world, lives are at stake. The ethos is first, do no harm. It is inherently and arguably necessarily slow to change.
The ethos in tech is just fundamentally different. Move quickly, break things. Its an Agile mindset of quick changes, pivots, rapid advances. Inefficiencies get identified and are tackled.
Medical world is different. The inefficiences are a feature not a bug. They are there to exactly do what they are doing, slow the rate of change. Clinical trials is one place where both worlds collide, the fast pace technical innovation world is about to interface to the slowly changing medical world. All of these guys frustrations is exactly about that.
He might think the solutiom is to speed up clinical trials phases. It isn't. Even if you did, sooner or later you will still come to another traffic stop of slow changes and inefficiencies because the medical world moves at a different pace, and necessarily so. When lives are directly involved the burden of proof to enact changes have to necessarily be very very high.
And what techies do not understand and probably keep asking is "doee it have to be so?". Yes it does. We didnt arrive here by accident. Even if we started healthcsre industry anew, it will inevitably end up in this slow-changing equilibrium. When lives are directly involved, better to leave things as is, than change and make worse. The burden of proof for even the slightest of changes is extremely high. what's optimal for health is indeed a slowly changing world. It is not for lack of talent or intelligence, as this author naively suggests. If you want to solve problems with quickly, don't go near that world. Stick to the world of optimizing ads
Can you share specific things that need to happen slowly and why that slowness helps protect study participants?
Some things mentioned in the article clearly increase risk to participants: manually re-assembling translated documents, using custom software and paper forms, and diluting important documentation with boilerplate. Other things seem to purely increase cost: starting meetings with smalltalk, overcharging for server hosting, and cronyism.
To add to this comment. An analogy would be trying to drive highway speeds on a crowded downtown roads with heavy foot traffic, street vendors, open stores. Noticing inefficiencies in how cars are passing through is missing the point. The point of highways is to move people quickly. The point of crowded downtowns streets is to move people slowly so they can shop, interact and spend money.
If this efficiency-optimizing author were to take his SV mindset to tackle thise inefficiencies at industry scake, what this author will quickly find out is that it is an impossible task. The industry itself is necessarily slow and its not by accident. It can be fast when it absolutely needs to be e.g Covid vaccines. That alone should let you know those inefficiencies are only inefficiencies to you because you are naive to what the industry is actually optimizing for.
At best, you sound like the impatient driver honking on a crowded street, aggressively driving into every available opening thinking you're optimizing for efficiency. You arent.
This isn’t the government, it doesn’t need to be painfully slow to ensure it continues working properly. I’m having a hard time responding in a constructive manner because it’s a bit infuriating to hear someone make excuses for such a corrupt industry. Have you ever worked with one of these companies? I just don’t see how it’s possible for someone to see it firsthand and then come to the conclusion that the inefficiencies and high rate of error in data collection/management are not just preferential, but even required outcomes.
I will note that you addressed none of the points made in the article. My assumption is that either you feel the need to attack the authors conclusion, having no defense of your own, or you’ve foregone reading the text altogether.
I read the text. The author sees the trees, misses the forest. At the micro level, the inefficiences are glaringly obvious and pervasive. But they exist and continue to exist for a reason. It isn't because people are stupid or corrupt, it is because that's what the industry optinizes and incentvizes for. That doesn't mean things can't get better or there aren't things to improve on, it's just that this author doesn't seem to understand the big picture yet which makes me very dubious of such a person's ability to execute any meaningful changes
Just to be clear on this point — you think that the healthcare industry optimizes for inefficiency?
Edit: After looking at your post history, I realize you just want to tell people they are wrong and you are smarter than them. Don’t bother responding, I don’t care what you have to say.
yes tech people are often ill-informed about medical world. And i do end up posting a lot on topics where such gross misinformation is on full display. I read tons and tons of posts on here, I respond to a very tiny few. And I stay away from commenting on areas outside of my competence. And yes, the ones I comment on are predominantly, ones where people with incomplete (or totally absent) perspective of a field that's outside their area, feel overly confident enough to comment on those areas. So many of my posts is indeed calling out the naivety of such posts.
It's only an "inefficiency" from the perspective of the person who doesn't understand what the system is optimizing for. This author is ML engineer commenting on a field he is relatively new to and doesn't yet fully understand.
To a person wanting to drive highway speeds in a crowded downtown street, everything would seem 'inefficient'. In reality, they aren't. The objectives are just different. Once you understand the larger objective, you start seeing what the true problems are, you start understanding that some things that look like inefficiencies at first glance, are actually features serving a purpose within a larger context, that you initially didn't understand.
As someone fairly familiar with the medical world-- IME HN is generally very far off the mark about medical related issues. They don't understand that regulations in this industry are written in blood, and they often fail to realize how incredibly complex these problems are. Another issue is people also need to be able to trust healthcare-- if people felt the way they do about Facebook towards makers of antibiotics we'd have a huge issue(moreso than we already do with the current number of medicine skeptics). The Doctors running these trials can't afford to make these mistake, nor do they want to considering they generally do have a lot of face to face time with their 'customers' unlike in tech.
Thank you for explaining what I am trying to say. Reading comments on HN on anything medical (or how they can use tech to improve medicine), I just have to continue shaking my head at the ignorant hubris on display, and the confidence with they are said. There's a reason all the big boys in tech who have tried making incursions into health industry have left with their tails tucked despite their considerable talent and money. The tech world attitude and ethos can not work in the medical world.
I worked on one in the 90's that was $50,000 a day for three years. The drug company, assuming the trial was successful, had about 4 years to make all its money back before the patent expired. Testing blood, urine, and stool for multiple things is expensive.
Intellectually, I can understand the need for control groups, but I still think it's immoral. When you stare at a spreadsheet and see 70% of the control group is dead because some random number generator sorted them there like hell's own sorting hat, and the 98% of the people getting the drug are alive, you have no business talking about statistics. That graph will haunt me til my dying day.
You didn't kill 70% of the control group, you saved 98% of the experimental group.
Also, with the appropriate trial design, you can stop the control and transfer patients once you see these big differences. Same thing should happen if you see the opposite, e.g. killing 70% of your experimental group.
If that was truly the case, then - as the parent hinted - it was the trial design that was the problem, and not the practice of having control groups.
It is pretty common to stop trials early when it is obvious that the treatment works. Conversely, it is easy to think that "yea of course the new treatment works" despite the evidence not being there. The need for robust analysis must also be respected.
Yes, exactly. Control groups are incredibly important for ensuring good quality of clinical studies. It's a technique that solves multiple "calibration" problems, including:
- being able to draw causal conclusions
- being able to adjust against placebo effects
A lot of clever people have tried to come up with ways of doing away with control groups. But ultimately, the best we can achieve is to stop early, as soon as the trial has a clear outcome. I do perhaps think this has become more common in recent times though, so perhaps the study you were involved in was at a time when early stopping wasn't really "the done thing".
But it still beats "studies" done 100 years ago, when you might give someone a cough mixture, see that they improved (if they died, let's just ignore that), and conclude that it was the cough mixture that did it!
> the best we can achieve is to stop early, as soon as the trial has a clear outcome
That must be really hard: if you wait for 95% confidence, you are selecting for 5% noise. If you repeatedly re-measure for 95% confidence, you strongly select for random noise.
Not many medical advancements provide such an anomalously strong signal (98% survival versus 30% survival).
> If you repeatedly re-measure for 95% confidence, you strongly select for random noise.
You can't use standard methods for early stopping - as you rightly point out, you get gibberish if you naively keep peeking at a growing data set. Instead, you have to use statistical methods that explicitly adjust for the repeated sampling in early stopping trials. This does make early stopping more complicated to analyse than a trial with a pre-determined duration.
No, in fact it is quite common to do "open label" clinical studies in which patients know what drug they are getting and no one is given the current standard of care. This is especially common in cancer, where the standard of care is so poor, and in rare diseases, where the patient population isn't large enough to admit such a comprehensive study.
It is harder to have the same statistical confidence of efficacy and safety in such studies, but clinical researchers try to address those issues by varying e.g. dosage quantities, time in between doses, etc.
Source: place I work is currently doing studies of this nature, and in general such studies seem to be well-understood and accepted by the FDA.
Randomization is independent from blinding. Open label studies can be either randomized or not, and controlled or not.
The FDA will usually push for blinding if possible (sometimes it is not). They will also usually push for a randomization zed control with standard of care. They way the FDA views it (and I agree), is that control patients are not harmed because they are still getting the same care they would outside of the trial.
It is usually unethical to have a treatment free arm. However, this has its own problem, where if you keep using equivalence comparisons, the end of the chain might not actually be any better
For such a large effect it is quite possible to implement an adaptive trial design with unbalanced arms and interim analysis for efficacy. But you have to ask for this, the FDA will not necessarily suggest it directly.
I don't feel guilt, I still feel rage. I didn't determine any of the parameters or rules of the trial so I have no guilt on me. I just observed that the control group is dead because of some belief that they provided value when we already knew exactly what was going to happen to them.
If you can prevent harm to others, then not doing so is just being an a$$. The trolley problem is just counting souls and really doesn't happen much in real life. The true problem is thinking the only choices are us/them versus everyone. I'm not a doctor so I have no idea how their ethics applies to their professional decisions.
This does not happen in a vacuum - it is not about this group at all. It is all about the future and if this will put a stop to a lot more people dying in the future. If it is only going to save 20 people a year, it hardly matters, but if 1,000,000 people will be saved every year, that's different story.
I don't know how bit the groups were, but if you have 500 people in the control group and 150 die, and you save 1,000,000 per year, that is a great trade-off.
Plus, when you go into this experiment, you know you it is an experiment and know in advance it is a 50/50 proposition that you will get the drug/placebo, but at least you have a 50% chance. Better than no chance. And personally, which normally I hate personal experiences, but personally, I wouldn't mind giving up my life in a test if I know I might have a 50% chance of getting the drug, but if not, I save 1,000,000 people a year (or whatever it is).
It's like being in the army. If you are the commander, the general, you might have to send 5,000 men charging up a hill that you know is well-defended and 80% will die, but you are using it as a diversionary tactic and take 80,000 men towards your real objective and you win it, then too bad for the 5,000 men. It's just the way things are. And if someone doesn't have the stomach to be the general, then they shouldn't be the general and look for another line of work.
Control groups exist because they get us closer to global optimality.
We can't eliminate suffering and dying of untreated people but it's generally considered ethical to eliminate suffering and dying of many while withholding that treatment from a subgroup of the study ("allowing them to die" by not taking action, rather than the "causing them to die" by taking action).
as long as that selection is done randomly and "fairly" I think it's an entirely acceptable risk. There have been trials that were truly badly run, and people died and suffered more than necessary, due to mistakes (often amateur ones) in the protocol. I'm more concerned about those types of deaths.
If that was the case, the study actually was unethical.
Some designs do include interim analyses and stopping rules.
However in my view the real scandal is we are still using NHST for clinical trials. We should be continuously updating a sensible prior for the effect size and approving the drug/stopping the trial when we have sufficient evidence one way the other.
As it is, the results of many underpowered trials are essentially thrown away because p > .05, which is stupid. This says nothing about the balance of evidence for the efficacy of an intervention… only the inertia and innumeracy of many clinicians preserves the tradition.
It's possible to stop a trial because the new drug is much more effective than the old drug. It's also possible to stop a trial because the new drug's significantly worse than the old. We've done both.
Also, I hope that no one ever tells you that you have to do a drug schema change because "too many kids are dying."
This is incredibly important if you want to understand healthcare. Every little thing is expensive in a way that is hard to imagine before you get into the field.
How many industries could these sentences descrive, stripped of context?
Or more easily, are there any industries where this is not the norm?
> an industry which is severely lacking in human capital and fully captured by bureaucratic tendencies
> going through this process is an extended exercise in sheer absurdity.
> meaningless bloat
> a farcical “data-driven” scoring process
> there was an intermediary clinical organization we attempted to work with which was so obviously attempting to grift money from our coffers, the entire relationship escalated into the level of back-and-forth legal threats
> One cannot help but get the impression that this is an industry which is, at best, severely lacking in anything that could be called “human capital” and, in reality, simply deeply unserious
> despite the very large amount of boilerplate content that every single actor in this ecosystem has spent thousands of hours of manpower developing, there seem to have been very few actual gains made in simplifying common tasks or processes!
... Et cetera.
To me, this sounds a lot like processes in education, teaching, energy, environmental protection, agriculture, research, arms, media, law, IT, etc.
I don't believe we can afford to remain this stupid, however much money the inefficiencies are making for those at the top.
I had the same experience as the author when I first started learning about clinical trials from the perspective of the software world.
The amount of money invested in clinical trials and lack of visibility is staggering. We've companies that spend the entirety of their $100Ms in funding and not realized until month(s) after the fact when their accruals caught up!
However, similar to what others have alluded, Chesterton's Fence is a relevant concept here (https://fs.blog/chestertons-fence/). It's critical to understand the rational reasons why the industry is so apparently messed up before you can fix them affectively. I wouldn't be working on the problem we're tackling today, if it weren't for my cofounder's deep knowledge across several disciplines in the industry.
One thing that continues to make a meaningful impression on me is the mission driven motivation behind so many people working in the industry of all levels. Consistently from the chief scientist to the entry level accountant you hear people talk about how they tie their work to the patients. Many software companies say they are "changing lives," many life science workers can actually point to saving them. It's been a big motivating factor for my cofounder and I to realize the same companies we are helping extend R&D runway are the ones making cancer drugs for some of those close to us.
The author doesn't really understand the details of the operations of clinical trials, though this is a good overview of the kinds of outcomes that show up. I worked on operations software for clinical trials for about a decade, and there are (mostly necessary) regulatory drivers a large fraction of the complexity he complains about. The first order effects of the regulations are pretty straightforward, but when you get into second, third, and fourth order impacts, you start seeing some of the outcomes he's describing.
I totally understand the frustrations experienced by the author of this post. I am a nurse and my exposure to the Clinical Trial Industry came from a more bottoms up entry point. Its mind boggling to the the total costs CRO's charge for the services compared to the eventual experience for the staff and patients taking part in these studies.
CRO's are in dire need of disruption. I also agree about the incestuous industry hence the very slow progress being observed. It's hard to show any progress when it's the same people moving around the same companies and implementing ideas that are somewhat outdated.
Also, from a cost perspective, i think the future is also coming to a place where Clinical trials get outsourced outside the US.
Sites and PI's outside the US should be empowered to run this clinical trials as they provide a more cost effective solution with regards to operational costs and also access to an ever growing number of patients, reducing clinical trials timelines that lead to increased costs.
I understand the need for CRO's to maintain quality, and they can do this by building out the right relationships and partnerships with overseas contractors, but i believe in the long run, this is the solution.
I believe this so much that i launched my own CRO company. Infiuss Health. The goal of this company is to unlock the potential for studies to the run by CROs and smaller life science companies outside the US.
We are starting with Africa. As soon as we can unlock this population that has traditionally just been taboo to life science companies, imagine the lengths we could go with with access other geographies.
I used to be in enterprise software sales and consulting and have been involved for RFIs/RFPs across multiple industries. I don't know a single person who thought the process was sensible or efficient. Unless you have an person whose job is dedicated to responding to the questions, it's basically just a game of passing a hot coal around until you have something you can submit.
I work in an engineering consultancy organisation.
A lot of the big tenders are incredibly wasteful, with lots of money being wasted on them.
We have an aggressive sales pipeline process. We prune our sales pipeline hard, and ensure we are spending money where we can win.
Result, we win about 70% of tenders that go to the end - we save money by recognising where we can't win and walking away early.
On a related note, we are great fans of the 'pre-proposal'. You should be able to get the gist of any proposal down in 2 sides. That should be enough to get buy in from stakeholders on all sides. Only when everyone agrees on the pre-proposal do you then write the full proposal - so you save effort (and make the client happy because they don't have huge amounts of boilerplate to read!)
While there are lots of new biotech tools, most of the illnesses left to treat are caused by aging, when the human system falls apart from the interaction of old parts, therefore super hard to treat.
At this point I gave up waiting for a medical solution for my chronic illness, and have a better hope that I can live 15-20 years with it and rejuvenate my body after that time period.
One point in there - data management. It varies in cost because on the upper end, that includes support for data change requests for the life of the trial. Zillions of people involved, human error, need paper trails and sign offs to fix most things.
This is usually downstream of the CROs, or, at least contracted out. What's most interesting to me about the post is that the author doesn't mention any efficiencies gained by moving to electronic data collection. But, I also understand why it isn't mentioned -- I work for an EDC/eCOA provider, and our biggest competitor for new business isn't our competitors in the space, it is paper-based data collection.
Nicely written piece highlighting the value of in-house expertise and capabilities, providing a lesson that translates very well imo to why government projects should have more in house expertise and not contract everything to the private sector.
Best explanation I've read online. Most attempts at describing why boil down to the circular "clinical trials are very expensive because the process of getting a new drug approved is lengthy, complex and expensive".
One thing not mentioned: people are paid to participate in clinical trials. For example, in a vaccine clinical trial I know about, participants are paid over $1000 (in dribs and drabs over the course of the trial, as a retention mechanism.)
With tens of thousands of participants, this cost adds up.
Patient compensation is intentionally limited to prevent economic incentives from interfering with informed consent; pay people too much and they may participate in research that they would have passed on. Also, speaking from experience, patient compensation is a very small amount (5% or less) of overall cost. The vast majority goes to clinical staff costs, room costs, hospital overhead, labs etc.
> Patient compensation is intentionally limited to prevent economic incentives from interfering with informed consent; pay people too much and they may participate in research that they would have passed on.
I'm not sure I understand the logic here. How does agreement suddenly become less informed or less consensual because the participant gets paid more? Sure, if they get paid more, they might be willing to accept more risk or do something more unpleasant. But that seems fair, not immoral. Why would it be more fair to say "yes, perhaps this trial is risky, but we only want participants who don't demand to be compensated much for the risk they take"? If anything, maybe that approach leads to less informed consent by filtering out the people who actually understand the risk and would have only agreed to participant in exchange for more compensation!
Should we also be morally offended any time an employer raises wages in order to attract more workers?
Professors historically were prohibited from relationships with students because it creates the appearance (and high likelihood) of favoritism. And it probably pisses parents off a lot because the age difference seems creepy.
"Power" is a relatively new argument. These days the student has the power to utterly destroy the professor. The professor has at best the charismatic "power" equivalent to any number of smooth pickup artists occupying college dorms, of getting a temporary good time.
I think it's still a fair question. The money is intended to be an incentive. If the trial is ethical and has a positive risk benefit ratio, there is no harm from coercion.
Okay, but what if a person sincerely does not want to participate in a medical trial? But then when he sees the dollar figure, he thinks "well, shit, I hate doing this, but I can't afford to turn that much money down."
Same reason we don't allow the purchase of kidneys, and the same reason we think it's unethical for a millionaire with a tophat and a monocle to offer two homeless guys a wad of money to the winner if they fight each other. Sure, it's an adult making a decision of their own free will- but it's obviously unethical.
That's true. Good Clinical Practice guidelines say that Institutional Review Boards must evaluate compensation is not too high otherwise it constitutes "undue influence".
...but I think that's a terrible idea. I understand that, to come up with a far-fetched example, high compensation could result in some weird edge cases where, for example, a Phase I trial with healthy volunteers and a non-preventive drug candidate (e.g. an antibiotic rather than a COVID vaccine), but you're confident that drug is going to kill 99% of the volunteers so you offer $10 million to each volunteer. A totally informed volunteer might view this as something akin to a life insurance policy that pays out to the family even upon suicide. I don't personally see an ethical problem with that so long as volunteers are really well-informed, but I could see how others and the law would be uncomfortable with the ethics of such a situation.
But aside from that far-fetched situation, it seems harmful to limit compensation... especially if it's true that patient compensation is a small part of overall cost. Perhaps if every trial doubled compensation to 10% of today's overall costs, absolute overall costs would decrease because less effort has to go into patient recruitment.
I do see one real-world issue with high compensation, but it has nothing to do with ethics. It has to do with statistical interpretation. If you start dangling too much money to participate in a trial, you may change the kind of people who participate in the trial. These individuals might not be representative of the actual users of the drug. The problem is similar to running a drug trial in the US versus Japan. The US has a pretty diverse population with respect to genetics. Japan, in contrast, is almost entirely ethnic Japanese. If you were to somehow succeed in using Japanese trial data to market an acne cream in the US, the same drug that cleared up Japanese faces might result in Venezuelans having their noses fall off!
However, I'm cautiously optimistic that higher compensation could make trials more representative. In my mind, I could see how African-Americans, who tend to participate in trials at a lower rate than their Caucasian fellow humans, would, if only for the shameful history of the Tuskegee Syphilis Study, require greater compensation to take the risk of participating in a trial.
I would think the worst of such edge cases should be prohibited by law anyway. A set of more subjects is presumably a superset of a set with less subjects. So any issues can be addressed by screening them better. I suppose there could be a greater problem with fraud as people claim to have issues when they don't.
It just seems unfair to subjects to me. Not to mention paternalistic. Every other actor in the process gets enriched based on (limitless it seems) market rates for their contribution, except for the subjects who take on all the risk.
As you said, at the end of the day it's an ethical guideline that must be upheld, more than something based in economics. We do compensate participants as well as we're allowed within these guidelines!
I agree with your points, but compensation must be approved by the independent institutional review board, who is accountable to national ethical review boards and thus acts very conservatively.
That's not a vaccine trial though, right? I'm figuring those need more participants and have more retention problems, as well as less risk of informed consent problems.
Interesting article, but the lack of mention of 'patent' or 'intellectual property' sort of stands out. There may be many out-of-patent drugs that are effective treatments for various conditions, but typically clinical trials for alternative uses of out-of-patent drugs are not pursued as anyone could then manufacture and sell them.
This is not new - it dates back to the origins of the modern for-profit pharmaceutical industry, such as with Bayer's discovery of the anti-bacterial action of sulfanilimide (patented 1909) in the early 1930s (by which time patent had expired). Bayer made a derivative of sulfanilimide, i.e. Prontosil (patented in 1932) and tried to convince the world that this was the more effective product. In reality, it just broke down to sulfanilimide once ingested, that being the active species.
As far as clinical trials in the USA, there have been numerous scandals regarding lack of consent, failure to halt trials immediately on indication of severe side effects, etc. Major stories about this came out in 2005, 2008, etc. Then there's the world of unregulated third-world clincal trial testing, and actual falsification of clinical trial data by contractors hired to run trials for various Big Pharma outfits, such as this from 2022.
> "Tellus, which has worked with big-name pharma sponsors in the past like Pfizer, Gilead, Takeda, Boehringer Ingelheim and Eli Lilly, initially came under scrutiny for two trials related to opioid dependency, two for irritable bowel syndrome, and one for diabetes."
That's just organized crime 101: hire a contractor to do the dirty work, keep your own hands clean, claim ignorance, act shocked, promise reforms, etc.
Best solution I've heard of is a move to open-source (patent-free) drug discovery with federally financed clinical trials. Since almost all new drugs are discovered at universities with federal funds (see remdecivir, mRNA vaccines, etc.), before being exclusively licensed to private interests, the quickest way to that is to revoke Bayh-Dole and state that universities must non-exclusively license their discoveries to any parties for a flat fee.
So many things in this story, and in corporate life in general, arise from fear and selfishness and shortsightedness, which take many forms. The process of bidding, and proposals, and counterbidding described here, the meetings and trade studies and boilerplate text, are CYA efforts. They aren't there to help you make the right decision. They're there to prove you did everything anybody reasonably could be expected to do, to make the right decision. Every coward who passes through the process raises that CYA line of reasonability a little higher.
The inefficiencies that no one fixes are mutually supporting; in a project this size, a little inefficiency doesn't matter, never mind that it's the whole pile of them that makes the project this size. Fixing something is risky and difficult and you may break something and you don't get anything for it. Going along works. Is there an order of magnitude worth of bloat that can be trimmed here? Shoot, maybe two. But is there any gain in doing that? Would a bid an order of magnitude (or two!) too small even be taken seriously? If anything, such a bid would be a negative thing -- if it didn't work out, for whatever reason, it would be easy to accuse the guy who went with the cheapest bidder.
To say "I personally reviewed this process, and I am willing to say, on my own judgement, that it's good" takes an almost unbelievable level of courage, in a world so ready and willing to judge you for any mistake. And yet, people doing exactly that is the only reasonable solution. The only thing that will make good sense prevail is people standing up for good sense. It is the only thing that can possibly prevent... this.
It isn't just medicine. It's every industry. It's every community. It's politics writ large and HOAs writ small. I guarantee, where you work, in your own back yard, you can find examples of this problem. The substitution of the system for the hard work of understanding and the courage to say what you think and see. CYA over common sense.
What should be done?
The first thing to realize is that the fear that motivates this behavior is entirely justified. People who start revolutions get shot at. It's a fact as old as history. Look, in the world of nutrition, at people like Atkins and Taubes, who were ridiculed and ostracized for years before their ideas became mainstream. Gary Fettke got into legal trouble. That can absolutely happen to you, for nothing more than advocating an unconventional view on what you believe are solid, perhaps even scientific, grounds. (Pardon the example if you don't agree with these folks -- the fact remains that for each of them, the personal cost of telling the truth as they saw it, and trying to fix something they thought was broken, was, at least temporarily, high.)
But the second thing to realize is that the fear that motivates this behavior is largely unjustified. If you try to make things better, people usually appreciate it and try to help. Sometimes the resistance you're so worried about doesn't exist. And when it does exist, it's often weaker than you might expect.
Two contradicting realizations? Consider the case of Martin Luther King Jr., whose case was basic, and obviously moral, and inspiring, and who nonetheless had every reasonable thing to fear--whose enemies were absolutely real, and dangerous, and persecuted and ultimately killed him (point 1!), but who was also taking on the absolutely massive and invincible dragon of the generations-entrenched psychological direction of an entiremassivecountry, and who substantially won that fight! (point 2)
Do you want to change medicine?
Look at Dr. Garrison Bliss, a primary care physician who hated the way primary care was done, figured out how to do it another way, and started a movement in which I-don't-know-how-many-thousands-maybe-millions of people get to do it that better way.
Look at Dr. Jason Fung, a doctor who chased a "what if" curiosity about type 2 diabetes right into a revolution that is right now on the edge of saving millions upon millions of people.
These guys could easily have gotten run over by the massive industries they were taking on, but for one reason or another, they just . . . did what they thought was right, and the worst just . . . never happened.
Stepping away from medicine for a moment, do you know what else used to be this way? Taxis. I remember back when there were never enough taxis, and you had to wait for them for a long time, and they were so expensive and unpleasant that it was a running joke in pop culture. For the same basket of reasons -- corruption and cowardice and short-sightedness and selfishness and not my problem. And you know, when Uber started to change it, they were definitely maybe doing something illegal in a lot of places. Were they going to get legally slammed for it? I sometimes wondered. But the laws obviously made the world a worse place, and the alternative was so nice by comparison, and here we are -- in a world which still has its problems, but obscenely, irrationally, how-did-it-ever-get-this-bad taxi service isn't one of them.
It's not your job to fix the world. (And it turns out that even if you did become dictator-for-life, fixing the world would remain surprisingly hard and require surprisingly many of the same character traits -- so there is no point in waiting for official permission.) It's not your job to fix medicine. It's not your job to fix education. It is your job to fix what's right in front of you. Want to be the SpaceEX of CROs? Sure sounds like somebody can. Maybe it's you. Too big? Fix your own clinical trial. Too big? Fix some of the inefficiencies. I'm glad you tried to fix the meeting thing. Keep trying. Fix one meeting.
PG's advice about startups -- "just don't quit" -- is surprisingly apropos to any revolution. It seems black and impossible right up until it doesn't anymore. If you find something important enough to be worth that to you, and if you just keep swimming, if you don't give up and constantly pivot to trying something new when you find the way blocked, constantly figure out what to do about problems, it's astonishing what one person can accomplish.
You may die in the process. In the fifteenth and sixteenth century, the Catholic Church was in this place -- corrupt and ineffective and unbelievably powerful and invincible and had been ruling the world for literally hundreds of years. Enough people laughed at it and hated it for it to be a running cultural joke: the fat monks and corrupt bishops of Robin Hood are exactly this joke, and the story is about how an outlaw can obviously be the better person. The story is about exactly this problem, and it was popular in the 15th century because that was what was on everyone's mind! And yet even such story did not propose a solution. It settled for talking about the problem. There were no solutions.
Alternatives were unthinkable. Every now and again, someone would try what I recommend, they'd try to fix their little piece and make things better, and they'd be crushed by the great power and die and fail. Nobody knew Tyndale would be different -- and all he was trying to do was get better knowledge to more people. Nobody knew Luther would be different -- all he was trying to do was start a debate and get some theological reform. Their ambitions were small, and people before them had died both for less and for the exact same things, and the fact that they tried anyway is why they were able to succeed, and why we have the world we have today. (Pardon the example if you don't see this particular history this way -- you can find other examples where many failures preceded success, where fears are justified, and yet where taking that chance results in slaying absurdly powerful dragons.)
I myself am convinced that someone like that, doing something like that, is the only reason we ever have any nice things at all. The CYA process sure isn't producing them, I can tell you that much for sure! And so to answer the question of "Why things are this way", well -- in that particular corner of the world, nobody ever did that. What's to be done? Do it. :)
I worked at a small-to-mid-size clinical research organization for a hot minute. That company had its own set of - let's say "unique" - problems*, but the author touched on many of the issues I saw in my short experience at a CRO. I was in an odd (for me), pseudo-managerial, HR-type role at the company (a role in which I'll be the first to admit I wasn't good at) so perhaps I can offer some additional perspective.
But first, I want to give some background on an important part of the CRO world: the "clinical research associate" or "CRA". CRAs are are the CRO's meat-and-potatoes, boots-on-the-ground, core, etc. after contracts have been signed. A CRA is the person who actually goes to clinical trial "sites" (e.g. a doctor's office, a hospital, or, even a dedicated facility for clinical trial participants). The CRA goes to each trial site, talks to the doc, makes sure participant recruitment is going well, checks/writes documentation, and follows up as the trial goes on. In other words, the performance of the CRAs strongly & directly affects the performance of the clinical trial.
Anyway, my top issues with clinical trials today are:
(1) Poor patient recruitment - this is a major problem. There is, of course, lots of variability between trials and CROs with respect to recruitment, but it should be telling that "trial rescue" is often done (at least where I worked) as a result of the incumbent CRO's inability to recruit enough patients.
(2) (Related) Limiting pay to study participants: although it's up to the institutional review board's interpretation (IRB, an ostensibly independent entity that's supposed to sign off on the trial's ethics), good clinical practice (GCP) guidelines states that paying participants too much would be "undue influence". I have no idea how this plays out in practice, but I think it's a bad guideline. Most of the other parts of GCP are good things - stuff like informed consent and not coercing subjects - but this one is (again, in my opinion) probably raising trial costs through recruitment delays or total non-recruitment, even accounting for the extra cost of paying participants more.
(3) Clinical Research Associate (CRA) training & hiring: this isn't unique to the CRO industry but it seems like a bigger problem than in other industries. I've seen US median salary ranges from $70,000 to $120,00 from publicly available sources but my time in the industry rarely demonstrated numbers that high. Although CRAs don't treat patients directly, it is a role that requires medical knowledge, attention to detailed paperwork, and good communication skills. There seemed to be a lot of cutting corners on training & hiring CRAs - e.g. hiring highly experienced CRAs from India and putting them into entry-level roles in the US on the cheap. I don't mean to say there aren't great experienced CRAs from India who are also great when dropped into an entry-level CRA role but doing so as a matter of unofficial policy is a bad idea: India's drug industry has a different regulatory environment (although there is significant cooperation between FDA and Indian manufacturing facilities) and, perhaps more importantly, there will be a wide variation in communication skills simply because english is a second language (or, in the case of India in particular, "Indian english" has taken on a life of its own).
(4) Over-promising, under-delivering - true of any contractor even outside the CRA industry. Even when working on a fixed price basis, there are perpetual "change orders" and arguments over change orders.
Side-note: if you want to learn more about the process by which drugs come to market, I strongly recommend New Drugs by Lawrence T. Friedhoff. It's short and the HN crowd won't have any problem understanding. Amazon link: https://www.amazon.com/New-Drugs-Insiders-Scientists-Investo...
* "Unique" problems in the sense that they were personality-based. I won't go into specifics nor do I wish to speak negatively of my prior employer who, to their credit, took a chance on me that just happened to not work out. But I will give some general advice: if, during a job interview or in early conversations, senior management insults his/her own family and dumpsters their own employees, you're going to have a bad time.
An interesting company I talked to a while back working on (1) in the spaced is deep6.ai. Interesting approach and there is a lot of work that can be done here sourcing patients more effectively. I think they said something like only 10% of studies are able to be carried out due to lack of patient recruitment.
I would add to your list of problems: Decades of easy-money milk-the-cow style companies (Often CROs) with very outdated systems with folks sittings in cheap countries to do some data change requests and manual tests. Truely horrible and outdated tech, mixed with bad salaries, high turnover and bad management.
I used to work for a company were because of timezone issues a wrong date was entered, CRA requested correction (edit manually, sign, scan) and wrong assumptions etc we had to change 7 times the whole thing and send CRA onsite and recorrect the mess. In short: Inefficient mess.
I left that industry quickly and can say I that others are better. Lab tracking is way better with diagnostic samples with private labs or when mixed with good logistics partners. It is not everywhere as bad as this.
Get that covid one on to the childhood vaccination schedule? Emergency use authorization, unknown long term effects, more harm than good for this age bracket, no problem.
The author already touched upon a few hypotheses why the industry has arrived at this state. There's one I'd like to highlight: The author, now having gone through this experience, would actually be in a perfect position to found their own CRO and implement all their efficiency learnings there. However, they probably look back upon this experience and think it was super painful and would prefer to do fun stuff like developing software instead.
That, for me, is the main reason. There are smart, structured, pragmatic people who got acquainted with this industry, but they choose to leave it.
And that's sad!
So the solution would be to find a group of smart, structured, pragmatic people with a high tolerance for bureaucratic pain and give them the resources to make this industry more efficient.
And here's my plug: That's exactly what we're doing at OpenRegulatory [1], albeit for medical device software and not for clinical trials. Among other things, we've published all our document templates for free (also on GitHub), host a free Slack community for companies to help each other out, and offer a compliance SaaS with transparent and fair pricing. It's still a long way, but it's a start :)
[1] https://openregulatory.com