Honestly, I think we are saying something at least similar, but in the case of intersection with the legal system, I thin there's something else going on. Apologies, but I'm running with a tangent.
Bite mark analysis is a great example. It isn't just that CSI: Market Segmentation Unit convinces people science can do something it can't. The legal world (outside of some lonely voices to which nobody listened until pretty recently) was telling juries it was valid. Defense attorneys might have taken the odd run at expert testimony in a particular case if it looked promising, but their job is to win particular cases, not take principled stands in general. Meanwhile, the rest of the attorneys in the room (people with reputations for being serious, whom you might expect to have carefully considered the value of different types of evidence, given their careers) were taking bite mark analysis seriously. What, exactly, is a juror in that situation to do?
It is worth asking if the CSI effect is just a cultural feedback loop. Criminal law[1] has always wanted magic unassailable evidence. It just makes everything so much easier. Fiction sometimes still portrays fingerprint evidence as close to infallible, but who has spent generations treating it as such in the context it for which it matters most?
So at the risk of sounding cynical, I'll note that doubts about bite marks, fiber analysis, and now even fingerprints are legally acceptable now that a new magic guilt-certifier is available.
[1] I'm being somewhat flip with language. All of these topics are much more complicated than portrayed. Please take my broad generalities about legal topics in the spirit offered.
But aren't prosecutors and lawyers really a part of the general public when it comes to forensics? They have an "expert" telling them that they have magic, and they have Hollywood telling them that this kind of magic exists (and, as you point out, they have a good incentive to believe in magic - but it's not just the lawyers, it's the jurors as well - it makes their life much easier, too!).
But the same applies here! The general public wants to believe, for example, that if terrorists use encryption, some really smart guys from NSA or wherever can "hack" it somehow (but nobody else). Or at least that we could regulate encryption such that only the "hackable" varieties are available to terrorists. It makes them feel safer. And so you have 44% supporting legislation to mandate backdoors.
> But aren't prosecutors and lawyers really a part of the general public when it comes to forensics?
In this case, no, they aren't. A better comparison would be to a general contractor hiring an electrician. A prosecutor is likely to have no more formal education or hands-on experience with forensic science. But, as professional "consumers" of expert testimony, and a motivation (professional duty) to win, you can't evaluate their scientific ignorance the same way you'd evaluate a juror's. I don't think the reason bite mark analysis lasted so long is because Judge Goodsense watched cop shows, any more than you can say a GC can't be blamed for shoddy wiring because they weren't a member of the Electrician's Local.
To swap contexts, nontechnical people who work with programmers may not be able to give you the big-O cost of a loop, but they tend to know when a programmer is bullshitting about something.
This goes to my contention that the CSI effect isn't just 'I believe it 'cause I saw it on the toob'. Education, lived experience, class, personality and other priors play in to it. In the case of criminal evidence admissibility, I think there is at least motivated reasoning as there is true belief in methods. The motivation is in part the same as with how plea deals have evolved - the current criminal justice system would collapse if trials were needed for more than a tiny fraction of cases. Magic evidence is a performance hack.
I don't disagree that ambient falsehoods matter. I am saying that comparing motivated professionals in a field with random MOPs misses a lot of the picture, and that even your MOP is going to react to ambient falsehoods they can't evaluate differently, based on priors.
> This goes to my contention that the CSI effect isn't just 'I believe it 'cause I saw it on the toob'.
I don't think that is the case in general, either.
The problem is that "I saw it on the toob" has secondary effects, that attenuate as they propagate, but can still go a long way. Basically, you might not believe that DNA testing is 100% correct because you saw it on TV; but you you believe it to be 100% correct because "everybody knows it", and the reason why they do is because "everybody knows it", and so on, until you actually get to a much smaller minority who believe it because they personally saw it on TV.
(In general, it's amazing how many popular "common sense" things are blatantly false, but they keep getting repeated and propagated as part of ambient noise of society, because "everybody knows it".)
Bite mark analysis is a great example. It isn't just that CSI: Market Segmentation Unit convinces people science can do something it can't. The legal world (outside of some lonely voices to which nobody listened until pretty recently) was telling juries it was valid. Defense attorneys might have taken the odd run at expert testimony in a particular case if it looked promising, but their job is to win particular cases, not take principled stands in general. Meanwhile, the rest of the attorneys in the room (people with reputations for being serious, whom you might expect to have carefully considered the value of different types of evidence, given their careers) were taking bite mark analysis seriously. What, exactly, is a juror in that situation to do?
It is worth asking if the CSI effect is just a cultural feedback loop. Criminal law[1] has always wanted magic unassailable evidence. It just makes everything so much easier. Fiction sometimes still portrays fingerprint evidence as close to infallible, but who has spent generations treating it as such in the context it for which it matters most?
So at the risk of sounding cynical, I'll note that doubts about bite marks, fiber analysis, and now even fingerprints are legally acceptable now that a new magic guilt-certifier is available.
[1] I'm being somewhat flip with language. All of these topics are much more complicated than portrayed. Please take my broad generalities about legal topics in the spirit offered.