Very interesting. But it’s worth keeping in mind that the most common suppression tactic was to ask the researcher not to publish (9%), and we have no idea in how many cases it worked (since the unpublished trials didn’t make it into this study)…
Or trials alre registered, but then published with a different main result than the one that was registered as the main result (so-called outcome switching). This should be noted in peer review, but often it's not. And if you point it out to the journal later they should publish a correction, but they don't.
Because all of that has always been what science promised to justify its hold on funding and power while not delivering any of the accountability it claimed.
For me, if anything is surprising here, it's that it wasn't worse.
Other issues in behavioral health studies:
Self reporting can be unreliable.
The types of assumptions you make shape the questions you ask which can bias results. (I.E. "Which color is it? Black? Or white?" "Neither. It's yellow." "I don't have a category for that. I will record that as white.")
Honesty of participants for sensitive topics like sexual behavior and drug use that could have serious consequences for them.
> Honesty of participants for sensitive topics like sexual behavior and drug use that could have serious consequences for them.
One way to help get around that is to ask the respondents to flip a coin—heads they answer truthfully, tails they answer falsely. This gives the respondents plausible deniability and the true underlying rates can still be estimated statistically from the raw results.
I don't understand. Either you record the result of the coin flip, in which case you can easily determine each participant's actual answer -- or you don't, in which case the data collected is useless.
An easy way to see this is if you suppose there's a question for which the truthful answer is "Yes" for 100% of participants. After the coin flip, you will get 50% "Yes" and 50% "No", which is the exact same result you would get if the truthful answer were "No" for 100% of participants.
Here's where I saw it. And yes, I admit sloppiness with my word choice.
"Differently from the usual Direct Response technique, in RR, respondents toss coins to determine whether they will respond to the question or just mark “yes”. This still allows admission rates to be calculated, yet it guarantees full anonymity to respondents because no one can tell whether an individual respondent answered “yes” to the question or because of chance."
This has been obvious for decades. A very interesting deep dive would be one showing just how firewalls have not been established which would mark particularly trustworthy studies.
If it takes 1 unit of effort to ask a question, it takes 10 to answer if you know the answer and the audience knows the prereqs. It takes 100 units to answer if the audience needs a quick refresher and 1000 units to answer if the audience is missing fundamentals. Likewise, it takes 1000 units to do the research if you don't know the answer and 1,000,000 units if nobody knows the answer and it needs to be discovered. Your epistemology must take these ratios into account or it is trivial for a bad actor to fabricate truths by "just asking questions," creating unreasonable expectations of a quick and easy answer, and then inviting you to infer the truth of their malicious narrative from the absence of a quick and easy answer to the question they just posed.
Absolutely. For as long as the internet has been around, there has been this notion that someone asking questions is not only entitled to an explanation, but that it must fit into a brief response, the asker must be able to understand the explanation, and they must be thoroughly convinced by it. If none of these criteria are met, the asker concludes that the explanation does not exist or is wrong.
It is a fallacy that is quite obvious with just a little thought. Example: "Someone prove to me in a written reply here that Fermat's Last Theorem is true. If you can't, then I submit that it's false." Complete nonsense.
The flip side is that it's as easy to blow someone off by sneering at "just asking questions" when they either honestly don't know, or where they disagree with you but honestly think you can't answer the questions.
And sometimes people ask questions because they know very well what someone is insinuating, but are trying to force them to actually say it outright.
Or you can write the answer once and copy it infinitely.
I understand the feeling because I frequently tell the vaccine-hesitant why they should vaccinate, but it's not nearly that bad if you actually try to inform people.
Also, you're not accounting for the reverse, where something is popular but wrong and asserts that it should never ever have to explain why it's right.
Finally, a more useful heuristic is to listen to explanations from people who can and do tell you how things work. The BS artists generally either have models that directly contradict what we know of the world or have no coherent model at all.
Incidentally, the above post is an example of this, is something I've explained many times, and took me nowhere near 10 times as long as your post to write unless you can type significantly faster than 120 WPM.
Yeah, but you're posting on a forum that rewards effort and thoughtful explanations.
Also, I've gotten down the rabbit hole of debating with eg vaccine skeptics IRL, and sometimes giving a details-based answers helps, sometimes it doesn't. Some people just stick to their arguments like a shield, and will just cycle between arguments if you knock one of them down. Explaining that eg mRNA has a very short half-life and is unlikely to have long term side-effects doesn't really help.
> Yeah, but you're posting on a forum that rewards effort and thoughtful explanations.
Right now. I also mentioned posting among those who are against vaccines which is... not.
> Also, I've gotten down the rabbit hole of debating with eg vaccine skeptics IRL, and sometimes giving a details-based answers helps, sometimes it doesn't.
I don't think it's possible to convince everyone, ever. Some people have distorted learning and/or experience that may or may not even be possible to untangle.
But as you pointed out, some can be convinced. Also, only a small minority of people ever post, leaving stuff out there with no explanation of why it's wrong is perhaps even more dangerous.
It is difficult to gauge whether a discussion or explanation will be useful, especially when it involves a complete stranger. If one is determined not to be convinced by an explanation, they always have the option to declare your explanation insufficient.
I disagree that it's necessary to explain anything at all in one's spare time for free. Proper explanations require work--not just the work of forming an answer, but of learning the material in the first place. They're not something that anyone is entitled to just because they asked a bunch of questions. If the latter were the case, we could all get college educations for free just by asking questions to the right people. But it doesn't work that way. Nobody owes you anything.
Right, that's why it's necessary to do it yourself if you want your opinion to be heard.
> If the latter were the case, we could all get college educations for free just by asking questions to the right people.
Honestly, you basically can do that at this point with the info on the internet. You might not get the credential itself, but you absolutely can get all the knowledge and then some with things like Khan Academy, MIT OpenCourseware, etc.
Again, you're assuming that a full explanation is necessary simply because someone might want to be heard, which isn't true. Moreover, you haven't given much explanation for why you think that's true, which supports my point.
Regarding things like OpenCourseware, etc., that's not about anyone being obligated to support their position to people asking questions, which is what we're discussing. I never said people can't consult video or a book to learn more, but it's not the same thing as demanding answers from any expert and obligating them to explain. You're basically changing the subject now, and you've proved my points pretty well, so I think we're done here.
When I say it's necessary, I'm not talking about obligating others to do things, I'm simply saying it's a fact. It's necessary or people will remain uninformed.
Given that you've badly misconstrued what I'm saying, then randomly declared victory despite not really even being in the same book, let alone on the same page as me... yeah we're done here.
But as noted, we both still tried to explain our points. Actions speak louder than words, etc.
Nobody said people shouldn't be able to ask questions about health issues or vaccines.
But they don't automatically deserve to be taken seriously, especially if taking them seriously requires way more effort than they put in when they asked their questions.
You’re actually asking to be patronized. It’s ok to ask hard questions with complex answers, even when articulating the question is easy. Giving a subject matter expert permission to dodge questions because you are not an expert is a road to ruin.
