This book is a detailed critique of the way science is currently practiced, and the negative effects this has on the robustness and correctness of the scientific discoveries that are being made. It covers some of the same ground as Bad Science, but while that book put a lot of its focus on more deliberate manipulations and pseudoscience, Science Fictions makes the case that there are several sources of scientific bad practice: fraud, bias, negligence and hype, and it covers each in detail. Medicine and psychology seem to be the worst affected sciences, and there is a particlar highlight on the replication crisis in psychology, but all the sciences are affected to some extent.
The book’s main argument is that the incentive structure of science has resulted in people optimising the metrics to further their own careers, or to allow their institutions or publications to thrive, at the expense of the Mertonian Norms of science. I hadn’t seen these stated explicitly before, but they aligned with my own ideas about what science should be about. As stated in this book, they are:
- Universalism: Basically, the personal background of scientists shouldn’t affect how their work is received, as long as their methods are sound.
- Disinterestedness: Scientists and institutions should act on behalf of shared scientific enterprise rather than for their own personal gain.
- Communality: The products of scientific enquiry are owned by everyone, without secrecy.
- Organised Scepticism: Scientific claims have to be scrutinised collectively before being accepted.
Although it could be claimed that scientists claiming patents on certain methods goes against the ideal of Communality, it seems to me that the biggest problem with these norms is Disinterestedness. We live in a capitalist, competitive world in which scientists, academic institutions, publishers, and even nation states, compete with each other for funding and profit. Any individuals that genuinely exhibit disinterestedness in the successful outcome of their work will be outcompeted by those who do. This even extends to nation states, which might otherwise be able to pursure science disinterestedly, but also compete with each other for prestige.
One of the dodgy behaviours discussed is so-called p-hacking. “p” is a measure of the statistical significance of a result, often quoted in scientific papers. A value below 0.05 (the lower the better) is considered significant enough to publish. So there are various behaviours used to nudge a result into this range of statistical significance. These range from outright fraud by manipulating the underlying data to get the result you want, to more subtle statistical manipulation or interpretation. In some cases it’s simply just a lack of skill or a mistake, together with the unwilllingness to further examine a positive expected result. Another aspect of p-hacking is the reprocessing of data that has been collected for some other enquiry, in order to find some hypothesis for which p happens to be below 0.05, and the book covers why this isn’t a good way to do science.
While some of the poor science described in this book could be attributed to genuine mistakes that were conveniently overlooked in order to get significant results, the book also covers some outright fraudsters: (so-called) scientists whose entire careers were based on fraudulent results, until they were found out and fired in disgrace or forced to resign. The author asks the question of how many fraudsters, who were better at hiding their manipulations, might still be operating undetected.
The book concludes with some suggestions for how these problems can be fixed, including some recent positive developments, such as open access journals, and a move towards making scientific data more publicly available.
This book is rather bracing and uncomfortable reading for those of us who believe in science as a force for good in the world. The problems described in this book are widespread, affecting almost all areas of science, not easily fixed, and can result in incorrect decisions being made based on bad science that affects people’s lives in real ways. The book is written very clearly and rigorously. Richie manages to lighten the sometime depressing subject with humour throughout. Despite the subject matter being somewhat unpleasant reading for science lovers, I think this is a really important book that I enjoyed reading.