> "Table 1 shows the number of results that were available and the R-Index for chapters that mentioned empirical results. The chapters vary dramatically in terms of the number of studies that are presented (Table 1). The number of results ranges from 2 for chapters 14 and 16 to 55 for Chapter 5. For small sets of studies, the R-Index may not be very reliable, but it is all we have unless we do a careful analysis of each effect and replication studies.
> Chapter 4 is the priming chapter that we carefully analyzed (Schimmack, Heene, & Kesavan, 2017). Table 1 shows that Chapter 4 is the worst chapter with an R-Index of 19. An R-Index below 50 implies that there is a less than 50% chance that a result will replicate. Tversky and Kahneman (1971) themselves warned against studies that provide so little evidence for a hypothesis. A 50% probability of answering multiple choice questions correctly is also used to fail students. So, we decided to give chapters with an R-Index below 50 a failing grade. Other chapters with failing grades are Chapter 3, 6, 7, 11, 14, 16. Chapter 24 has the highest highest score (80, which is an A- in the Canadian grading scheme), but there are only 8 results.
Which is to say in other words, most of the book probably replicates, particularly the parts based on Kahneman’s own work, and for the parts that don’t you can just skip the chapters or take them with a grain of salt.
Kahneman to me always struck me as the one eyed king of the replication crisis. Yes he fucked up but he fucked up notably less than his contemporaries and most of his work is still readable.
Without a copy of the book, I don't remember which parts were based on Kahneman’s own work, and I don't see that we can/should just skip the other chapters.
R-index guys said [0]: "Table 1 shows the number of results that were available and the R-Index for chapters that mentioned empirical results."
Chapters where estimated R-index < 50: Ch 3,4,6,7,11,14,16
Chapters where estimated R-index > 50: Ch 5,8,9,12,17,24
Chapters that don't cite empirical results (by Kahneman, or who?): 1,2,10,13,15,18-23, all of 25-38
As to the chapters that had empirical results, and had an estimated R-index > 50: scores of 55, 57, 60, 62 are really scraping by; saying that means they "probably replicate" is setting the bar really low, even quoting Tversky and Kahneman (1971) back at themselves. (The R-index guys say "Even some of the studies with a high R-Index seem questionable with the hindsight of 2020.")
As to whether he was the one-eyed king of the replication crisis, he certainly started speaking out in 2012 [3] after the social priming scandal broke; did insiders have suspicions about non-replicability before that and should people have pushed back more, earlier? The fallout from the Francesca Gino and Ariely scandals continues.
https://slate.com/technology/2016/12/kahneman-and-tversky-re...
https://replicationindex.com/category/thinking-fast-and-slow...
> "Table 1 shows the number of results that were available and the R-Index for chapters that mentioned empirical results. The chapters vary dramatically in terms of the number of studies that are presented (Table 1). The number of results ranges from 2 for chapters 14 and 16 to 55 for Chapter 5. For small sets of studies, the R-Index may not be very reliable, but it is all we have unless we do a careful analysis of each effect and replication studies.
> Chapter 4 is the priming chapter that we carefully analyzed (Schimmack, Heene, & Kesavan, 2017). Table 1 shows that Chapter 4 is the worst chapter with an R-Index of 19. An R-Index below 50 implies that there is a less than 50% chance that a result will replicate. Tversky and Kahneman (1971) themselves warned against studies that provide so little evidence for a hypothesis. A 50% probability of answering multiple choice questions correctly is also used to fail students. So, we decided to give chapters with an R-Index below 50 a failing grade. Other chapters with failing grades are Chapter 3, 6, 7, 11, 14, 16. Chapter 24 has the highest highest score (80, which is an A- in the Canadian grading scheme), but there are only 8 results.