Ugh, wow, somehow I missed all this. I guess he joins the ranks of the scientists who made important contributions and then leveraged that recognition into a platform for unhinged diatribes.
Please don't lazily conclude that he's gone crazy because it doesn't align with your prior beliefs. His work on Covid was just as rigorous as anything else he's done, but it's been unfairly villainized by the political left in the USA. If you disagree with his conclusions on a topic, you'd do well to have better reasoning than "the experts said the opposite".
Ioannidis' work during Covid raised him in my esteem. It's rare to see someone in academics who is willing to set their own reputation on fire in search of truth.
“Most Published Research Findings Are False” —> “Most Published COVID-19 Research Findings Are False” -> “Uh oh, I did a wrongthink, let’s backtrack at bit”.
Yes, sort of. Ioannidis published a serosurvey during COVID that computed a lower fatality rate than the prior estimates. Serosurveys are a better way to compute this value because they capture a lot of cases which were so mild people didn't know they were infected, or thought it wasn't COVID. The public health establishment wanted to use an IFR as high as possible e.g. the ridiculous Verity et al estimates from Jan 2020 of a 1% IFR were still in use more than a year later despite there being almost no data in Jan 2020, because high IFR = COVID is more important = more power for public health.
If IFR is low then a lot of the assumptions that justified lockdowns are invalidated (the models and assumptions were wrong anyway for other reasons, but IFR is just another). So Ioannidis was a bit of a class traitor in that regard and got hammered a lot.
The claim he's a conspiracy theorist isn't supported, it's just the usual ad hominem nonsense (not that there's anything wrong with pointing out genuine conspiracies against the public! That's usually called journalism!). Wikipedia gives four citations for this claim and none of them show him proposing a conspiracy, just arguing that when used properly data showed COVID was less serious than others were claiming. One of the citations is actually of an article written by Ioannidis himself. So Wikipedia is corrupt as per usual. Grokipedia's article is significantly less biased and more accurate.
He published a serosurvey that claimed to have found a signal in a positivity rate that was within the 95% CI of the false-positive rate of the test (and thus indistinguishable from zero to within the usual p < 5%). He wasn't necessarily wrong in all his conclusions, but neither were the other researchers that he rightly criticized for their own statistical gymnastics earlier.
That said, I'd put both his serosurvey and the conduct he criticized in "Most Published Research Findings Are False" in a different category from the management science paper discussed here. Those seem mostly explainable by good-faith wishful thinking and motivated reasoning to me, while that paper seems hard to explain except as a knowing fraud.
> He wasn't necessarily wrong in all his conclusions, but neither were the other researchers that he rightly criticized for their own statistical gymnastics earlier.
In hindsight, I can't see any plausible argument for an IFR actually anywhere near 1%. So how were the other researchers "not necessarily wrong"? Perhaps their results were justified by the evidence available at the time, but that still doesn't validate the conclusion.
I mean that in the context of "Most Published Research Findings Are False", he criticized work (unrelated to COVID, since that didn't exist yet) that used incorrect statistical methods even if its final conclusions happened to be correct. He was right to do so, just as Gelman was right to criticize his serosurvey--it's nice when you get the right answer by luck, but that doesn't help you or anyone else get the right answer next time.
It's also hard to determine whether that serosurvey (or any other study) got the right answer. The IFR is typically observed to decrease over the course of a pandemic. For example, the IFR for COVID is much lower now than in 2020 even among unvaccinated patients, since they almost certainly acquired natural immunity in prior infections. So high-quality later surveys showing lower IFR don't say much about the IFR back in 2020.
There were people saying right at the time in 2020 that the 1% IFR was nonsense and far too high. It wasn't something that only became visible in hindsight.
Epidemiology tends to conflate IFR and CFR, that's one of the issues Ioannidis was highlighting in his work. IFR estimates do decline over time but they decline even in the absence of natural immunity buildup, because doctors start becoming aware of more mild cases where the patient recovered without being detected. That leads to a higher number of infections with the same number of fatalities, hence lower IFR computed even retroactively, but there's no biological change happening. It's just a case of data collection limits.
That problem is what motivated the serosurvey. A theoretically perfect serosurvey doesn't have such issues. So, one would expect it to calculate a lower IFR and be a valuable type of study to do well. Part of the background of that work and why it was controversial is large parts of the public health community didn't actually want to know the true IFR because they knew it would be much lower than their initial back-of-the-envelope calculations based on e.g. news reports from China. Surveys like that should have been commissioned by governments at scale, with enough data to resolve any possible complaint, but weren't because public health bodies are just not incentivized that way. Ioannidis didn't play ball and the pro lockdown camp gave him a public beating. I think he was much closer to reality than they were, though. The whole saga spoke to the very warped incentives that come into play the moment you put the word "public" in front of something.
Yeah I remember reading that article at the time. Agree they're in different categories. I think Gellman's summary wasn't really supportable. It's far too harsh - he's demanding an apology because the data set used for measuring test accuracy wasn't large enough to rule out the possibility that there were no COVID cases in the entire sample, and he doesn't personally think some explanations were clear enough. But this argument relies heavily on a worst case assumption about the FP rate of the test, one which is ruled out by prior evidence (we know there were indeed people infected with SARS-CoV-2 in that region in that time).
There's the other angle of selective outrage. The case for lockdowns was being promoted based on, amongst other things, the idea that PCR tests have a false positive rate of exactly zero, always, under all conditions. This belief is nonsense although I've encountered wet lab researchers who believe it - apparently this is how they are trained. In one case I argued with the researcher for a bit and discovered he didn't know what Ct threshold COVID labs were using; after I told him he went white and admitted that it was far too high, and that he hadn't known they were doing that.
Gellman's demands for an apology seem very different in this light. Ioannidis et al not only took test FP rates into account in their calculations but directly measured them to cross-check the manufacturer's claims. Nearly every other COVID paper I read simply assumed FPs don't exist at all, or used bizarre circular reasoning like "we know this test has an FP rate of zero because it detects every case perfectly when we define a case as a positive test result". I wrote about it at the time because this problem was so prevalent:
I think Gellman realized after the fact that he was being over the top in his assessment because the article has been amended since with numerous "P.S." paragraphs which walk back some of his own rhetoric. He's not a bad writer but in this case I think the overwhelming peer pressure inside academia to conform to the public health narratives got to even him. If the cost of pointing out problems in your field is that every paper you write has to be considered perfect by every possible critic from that point on, it's just another way to stop people flagging problems.
Ioannidis corrected for false positives with a point estimate rather than the confidence interval. That's better than not correcting, but not defensible when that's the biggest source of statistical uncertainty in the whole calculation. Obviously true zero can be excluded by other information (people had already tested positive by PCR), but if we want p < 5% in any meaningful sense then his serosurvey provided no new information. I think it was still an interesting and publishable result, but the correct interpretation is something like Figure 1 from Gelman's
I don't think Gelman walked anything back in his P.S. paragraphs. The only part I see that could be mistaken for that is his statement that "'not statistically significant' is not the same thing as 'no effect'", but that's trivially obvious to anyone with training in statistics. I read that as a clarification for people without that background.
We'd already discussed PCR specificity ad nauseam, at
These test accuracies mattered a lot while trying to forecast the pandemic, but in retrospect one can simply look at the excess mortality, no tests required. So it's odd to still be arguing about that after all the overrun hospitals, morgues, etc.
Does the IFR matter? The public thinks lives are infinitely valuable. Lives that the public pays attention to. 0.1% or 1%, it doesn’t really matter, right, it gets multiplied by infinity in an ROI calculation. Or whatever so called “objective” criteria people try to concoct for policymaking. I like Ioannidis’s work, and his results about serotypes (or whatever) were good, but it was being co-opted to make a mostly political policy (some Republicans: compulsory public interaction during a pandemic and uncharitably, compulsory transmission of a disease) look “objective.”
I don’t think the general idea of co-opting is hard to understand, it’s quite easy to understand. But there is a certain personality type, common among people who earn a living by telling Claude what to do, out there with a defect to have to “prove” people on the Internet “wrong,” and these people are constantly, blithely mobilized to further someone’s political cause who truly doesn’t give a fuck about them. Ioannidis is such a personality type, and as you can see, a victim.
> The public thinks lives are infinitely valuable.
In rhetoric, yes. (At least, except when people are given the opportunity to appear virtuous by claiming that they would sacrifice themselves for others.)
In actions and revealed preferences, not so much.
It would be rather difficult to be a functional human being if one took that principle completely seriously, to its logical conclusion.
I can't recall ever hearing any calls for compulsory public interaction, only calls to stop forbidding various forms of public interaction.
The SHOW UP act was congressional republicans forcing the end of telework for federal workers, not for any rational basis. Teachers in Texas and Florida, where Republicans run things, staff were faced with show up in person (no remote learning) or quit.
Interesting idea. How do you distinguish between critical and uncritical citation? It’s also a little thorny—if your related work section is just describing published work (which is a common form of reviewer-proofing), is that a critical or uncritical citation? It seems a little harsh to ding a paper for that.
That's one of the issues that causes a bit of work.
Citations would need to be judged with context. Let's say paper X is nowadays known to be tainted. If a tainted work is cited just for completeness, it's not an issue, e.g. "the method has been used in [a,b,c,d,x]"
If the tainted work is cited critically, even better: e.g. "X claimed to show that..., but y and z could not replicate the results".
But if it is just taken for granted at face value, then the taint-label should propagate: e.g. ".. has been previously proved by x and thus our results are very important...".
I was resigned to running cat6e up three floors because there was only coax and I needed a wifi AP up there. Came across the moca solution and it's great. I get flawless 2.5gbe from the basement switch to the third floor over coax. It's basically a little device that connects at each end of the coax and cat6 goes in and out.
Cat 6 would be better though so I could run POE from the basement switch to power the wifi AP, and instead I need to go do a much more complicated switch (cat6) -> moca adapter + power brick to power moca adapter -> coax -> moca adapter + power brick (cat6) -> POE injector (with power brick) -> wifi AP. SO I'm adding at least three power bricks to the setup, which is annoying. Otherwise it would be one cat6 drawing POE from the switch and powering the AP.
You can run power over coax! You can buy power-injecting splitters that were used to power old analog cameras. They basically just connect the cable to the 12V, sometimes directly but usually through some current-limiting safety switch.
MoCA devices have a 100 Ohm internal resistor at the end to limit the cable echoes, so they are not affected by the DC on the cable.
It's worth remembering that UK coax is typically lower quality than that used in the US where these are designed to be used, due to UK coax only needing to transmit terrestrial TV compared to cable in the US.
+1 on MOCA 2 being excellent to solve gaps in wiring. We bought a 6000 sqft 2001 house built with in-wall RJ11, lots of coax runs and some Cat5e runs (but not enough). Due to the size the house, the electrical, HVAC and cabling is roughly divided into two halves with separate electrical panels, HVAC pads, etc.
Unfortunately, all the RJ11 and alarm wiring runs to a closet in one half while all the coax and Cat5e run to a closet in the other half - with no RJ11 endpoints near the Cat5e/Coax closet and not Cat5e/Coax endpoints near the RJ11 closet (sigh). I tried Powerline data and it only works well in adjacent rooms and not at all between the halves due to separate electrical panels. Fortunately, there were a lot of coax runs set up for two separate nets (18-inch satellite and a huge attic antenna for OTA broadcast). So, by repurposing the now-unneeded antenna coax, MOCA 2.5 gbps mostly saved the day by filling in where the Cat5e should have gone but didn't.
That’s a strange argument. There are plenty of stochastic processes that have perfectly acceptable guarantees. A good example is Karger’s min-cut algorithm. You might not know what you get on any given single run, but you know EXACTLY what you’re going to get when you crank up the number of trials.
Nobody can tell you what you are going to get when you run an LLM once. Nobody can tell you what you’re going to get when you run it N times. There are, in fact, no guarantees at all. Nobody even really knows why it can solve some problems and why it can’t solve other except maybe it memorized the answer at some point. But this is not how they are marketed.
They are marketed as wondrous inventions that can SOLVE EVERYTHING. This is obviously not true. You can verify it yourself, with a simple deterministic problem: generate an arithmetic expression of length N. As you increase N, the probability that an LLM can solve it drops to zero.
Ok, fine. This kind of problem is not a good fit for an LLM. But which is? And after you’ve found a problem that seems like a good fit, how do you know? Did you test it systematically? The big LLM vendors are fudging the numbers. They’re testing on the training set, they’re using ad hoc measurements and so on. But don’t take my word for it. There’s lots of great literature out there that probes the eccentricities of these models; for some reason this work rarely makes its way into the HN echo chamber.
Now I’m not saying these things are broken and useless. Far from it. I use them every day. But I don’t trust anything they produce, because there are no guarantees, and I have been burned many times. If you have not been burned, you’re either exceptionally lucky, you are asking it to solve homework assignments, or you are ignoring the pain.
Excel bugs are not the same thing. Most of those problems can be found trivially. You can find them because Excel is a language with clear rules (just not clear to those particular users). The problem with Excel is that people aren’t looking for bugs.
> But I don’t trust anything they produce, because there are no guarantees
> Did you test it systematically?
Yes! That is exactly the right way to use them. For example, when I'm vibe coding I don't ask it to write code. I ask it to write unit tests. THEN I verify that the test is actually testing for the right things with my own eyeballs. THEN I ask it to write code that passes the unit tests.
Same with even text formatting. Sometimes I ask it to write a pydantic script to validate text inputs of "x" format. Often writing the text to specify the format is itself a major undertaking. Then once the script is working I ask for the text, and tell it to use the script to validate it. After that I can know that I can expect deterministic results, though it often takes a few tries for it to pass the validator.
You CAN get deterministic results, you just have to adapt your expectations to match what the tool is capable of instead of expecting your hammer to magically be a great screwdriver.
I do agree that the SOLVE EVERYTHING crowd are severely misguided, but so are the SOLVE NOTHING crowd. It's a tool, just use it properly and all will be well.
There’s a pretty rich literature around this style of pedagogy going back for decades and it is certainly not a new idea. My preferred formulation is Vygotsky’s “zone of proximal development” [1], which is the set of activities that a student can do with assistance from a teacher but not on their own. Keeping a student in the ZPD is pretty easy in a one-on-one setting, and can be done informally, but it is much harder when teaching a group of students (like a class). The. Latter requires a lot more planning, and often leans on tricks like “scaffolded” assignments that let the more advanced students zoom ahead while still providing support to students with a more rudimentary understanding.
Direct instruction sounds similar but in my reading I think the emphasis is more on small, clearly defined tasks. Clarity is always good, but I am not sure that I agree that smallness is. There are times, particularly when students are confused, that little steps are important. But it is also easy for students to lose sight of the goals when they are asked to do countless little steps. I largely tuned out during my elementary school years because class seemed to be entirely about pointless minutiae.
By contrast, project work is often highly motivational for students, especially when projects align with student interests. A good project keeps a student directly in their ZPD, because when they need your help, they ask. Lessons that normally need a lot of motivation to keep students interested just arise naturally.
I’m curious: what do you see as unnecessary about the CLI? Or, to put it another way, in what way should the CLI be changed so that the only remaining difficulties are the necessary ones?
I'm not qualified to give a complete answer, but I think two main issues are the proliferation of flags in standard tools (e.g. ls has a lot of flags for sorting behavior) and the extreme preference for plain text. Text is very useful, but a lot of semantic information gets discarded. Representing structured data is painful, stdin/stdout/stderr are all in one place, window resizing makes a mess sometimes (even "write at end of line" isn't given), and so on. I'm definitely not qualified to describe just how to fix these issues, though.
I think you hit the nail on the head. Plaintext is universal in a way that nothing else really is. Outputting structured data means that consumers would have to process structured data. That definitely raises the difficulty of the programming. It’s not an easy problem, but I also do not have any good ideas.
Ditto. I found that people whose attitude was “let’s just try it” tended to be a lot more capable and effective. Nevertheless the prevailing wisdom when I was in IT was that if you had a problem that didn’t have an obvious solution, you had to purchase the solution.
At the university level in the US, few faculty get any kind of training before they are expected to start teaching. And the teaching requirement is more or less “do no harm.” If you’re at a research university, which includes many publicly funded universities, then your career trajectory is based almost exclusively on your research output. I could go on, but it suffices to say that it’s not surprising that the teaching could be better.
That said, most institutions have teacher training resources for faculty. I was fortunate to be able to work intensely with a mentor for a summer, and it improved my teaching dramatically. Still, teaching is hard. Students sometimes know—but often don’t know—what is best for their learning. It’s easy to conflate student satisfaction with teaching effectiveness. The former is definitely an important ingredient, but there’s a lot more to it, and a really effective teacher knows when to employ tools (eg quizzes) that students really do not like.
I am frequently amused by the thought that here we have a bunch of people who have paid tons of money, set aside a significant fraction of their time, and nominally want to learn a subject that they signed up for; and yet, they still won’t sit down and actually do the reading unless they are going to be quizzes on it.
> the thought that here we have a bunch of people who have paid tons of money, set aside a significant fraction of their time, and nominally want to learn a subject that they signed up for; and yet, they still won’t sit down and actually do the reading unless they are going to be quizzes on it.
How often have they put down the money, as opposed to their parents?
How often do they actually care about learning the subject, as opposed to be able to credibly represent (e.g. to employers) that they have learned the subject?
How often is the nominally set-aside time actually an inconvenience? (Generally, they would either be at leisure or at the kind of unskilled work their parents would be disappointed by, right?) My recollection of university is that there was hardly any actual obligation to spend the time on anything specific aside from exams and midterms, as long as you were figuring out some way or other to do well enough on those.
I suppose I should have said “nominally want to learn” etc, but I think you are right: most students simply want the credential. I maintain that this is still a strange attitude, since at some point, some employer is going to ask you to do some skilled work in exchange for money. If you can’t do the work, you are not worth the money, credentials be damned. On the other hand, I routinely see unqualified people making a hash out of things and nobody really seems to care. Maybe the trick is not to be noticably bad at your job. Still, this all strikes me as a bad way to live when learning and doing good work is both interesting and enjoyable.
I sort of did the opposite of you. I hiked the AT before I started my career. The interesting thing is that it has always given me a reserve to draw on when I am feeling burned out or upset like the original poster. I can always say “you know what? I was my happiest when I was living in a tent.” It’s a reminder that I don’t actually need much to be happy, and that thought helps keep me centered on what goals I choose and whether my pursuits are worth it.
Like you, I was also massively burned out on tech after the pandemic. I had a very stressful work experience combined with some family medical crises. I ended up just taking some time off to do some woodworking. I understand that I was in a very privileged position to be able to do this. But after taking my mind off of daily tech worries and focusing on what I enjoyed doing, I found that my thoughts naturally gravitated back toward technology-related work. I have since come to understand that this is what burnout looks like. It’s a nice reminder that just because I hate my job right now does not mean that I want to throw in the towel forever.
The trail provides! Its wild how much of the peace and calm I brought back from the trail into my regular life. I live in a major US city and traffic used to drive me insane. Now, I could care less. I just chill and roll with it. I'm more tolerant of people. I'm never in rush to get anything done or get anywhere quickly. I appreciate all that I have and all that I don't need. It really was just what I needed.
Like you said, definitely privileged to be able to do this, but I also found that a lot of people hiked both the PCT and AT on a shoestring budget and made things work. Maybe not stopping in every town or not going out to eat as often. If a person is dedicated and there to get the experience, money only made it easier, but the experiences were all very much the same I found.
> a lot of people hiked both the PCT and AT on a shoestring budget
Agreed. I was one of those people. My budget for the entire trip, including a fair amount of equipment (I already owned boots), was $2000. This was in 2003. I worked a fairly low paid job (it was the best I could get) after college for two years in order to save up. I ended needing to dip into my credit card, which caused me a lot of stress, seeing as I did not have a job lined up after my return. The damage? $400. At the time that seemed like an unfathomable amount of debt, because I was living on very little.
My partner and I took advantage of a lot of charity on the trail (trail angels, kind strangers, etc). I would love to do something like that again (I’ve always dreamed of hiking the PCT) without such severe financial constraints. Still, there was not a lot of worrying on the trail. The stress did eventually come back after living for a couple years in Boston though!
Oh wow! Yea you did pull off a tight budget. I never expected to rely on strangers for hitches or trail angels for couch surfing, but those are some of my best memories. Its an incredible journey.
You absolutely have to go for the PCT! They don't call it the "Goldilocks Trail" for nothing. It has it all! I live on the east coast, like you, so the AT was like hiking in my own backyard, but the PCT. Absolutely stunning! I can't recommend it enough. I hope to hike the CDT either in sections or as a thru-hike once I hit retirement age (but I have a good amount of time before that day arrives lol my memories will have to carry me until then).
reply