Stuart Ritchie

The title summarizes the book's topic well. Chapter 2 recounts some egregious cases of fraud, like Stapel and Macchiarini, who made up data and lied about patients dying and suffering, respectively. But perhaps more worrying is the likely more widespread low-level p-hacking, selective reporting of findings and various small adjustments of the analysis and data to obtain publishable results that he turns to next. I know from personal experience that these are big issues in social science, but show more learnt here that they exist also in the harder sciences. The unfortunate result is that we cannot trust the scientific literature. I believe Ritchie is spot on when he identifies the perverse incentives faced by researchers as a major factor in these problems. Good publications are the key to tenure, promotions, salary increases, funding, prestige and the competition to obtain these publications have tempted many to tweak their analysis, underplay uncertainty and oversell their findings. What can be done? There is room for more honest behavior from individuals, but it is too much to hope for that that will be all that is needed. In the final chapter, Fixing science, Ritchie highlights a 2016 meta-analysis on antidepressant drug trials that illustrates many of the preceding problems and details how publication bias, p-hacking/outcome switching, spin, and citation bias may distort the scientific literature [The cumulative effect of reporting and citation biases on the apparent efficacy of treatments: the case of depression by de Vries and others.]. Instead of focusing on a specific solution, Ritchie lists a number of possible improvements. Some of these are ok, though not attacking the more structural problems or seem to labor intensive to do so, like naming and shaming, independent investigation of misconduct, algorithms to detect fake data/images/etc., hiring on merit rather than bean counting and a review service for pre-prints. Suggestions that seem to me to have more potential are journals being more acceptive of replications and null results (but needs to be accompanied by credit given by others), more comprehensive analysis like multiverse approaches and pre-registration, ideally combined with reviews in a registered report format. However, the problems with the trustworthiness of the scientific literature are so large that we should try multiple approaches. Highly recommended. show less

Intriguing if worrying account. Science is the road to demonstrable testable truth but pressure, dishonesty, mistakes, venality, ambition for fame, desire for tenure and promotion disturb the process. Unsurprising perhaps that, of all subjects , psychology scores the worst.

Really enjoyed this description of some of the big problems in science today. This is not in any way an anti-science book, Ritchie makes clear that he wants to improve science, not to dispense with it. Along with describing problems he also describes much of the process of science which I enjoyed.

He spends a lot of time on the reproducibility crises, p-hacking and other statistical cheating, and many other issues that one hears about when science problems get in the news.

This book has been show more well reviewed in general publications, but I was curious how science journals would review it. The only review I found in a professional science periodical (in the ultra-prestigious Nature) was basically positive with a few criticisms. show less

Well written plea for greater scientific rigor.