Crisis in Psychology?
April 5, 2022 at 11:53 am
(This post was last modified: April 5, 2022 at 1:46 pm by vulcanlogician.)
I know not everyone wants to watch a 10 minute video, so I'll give the tl;dr below. But it is worth watching if you have a little time to spare.
***
In 2014 and 2015, studies were published which estimated that only between 46-85% of psychological studies were replicable. This is a serious problem.
The video presents this thought experiment: you are given a box of 100 thermometers but you know that a significant proportion of them is inaccurate. If only 2% of them are inaccurate, this is hardly a problem. But, if this number is 50% the odds of you getting an accurate reading is no better than a coin flip. That is a serious issue, since we rely in psychological science to furnish us with information that impacts, not only mental health treatment, but also public spending to address societal issues.
Some suppose that the issue stems from publication bias. In fact, this is the most oft cited culprit in discussions of the replicability issue. From wikipedia: "In published academic research, publication bias occurs when the outcome of an experiment or research study biases the decision to publish or otherwise distribute it. Publishing only results that show a significant finding disturbs the balance of findings in favor of positive results."
Of course, as human beings interested in expanding our knowledge, a bias towards new, significant discoveries is understandable. But, IMO, furthering our knowledge is not the only goal of science. There is a skeptical or "let's make sure" spirit that is also essential to all sciences. To me, the replication issue in science isn't just a problem with psychological science. To me, it demonstrates that our desire for new knowledge has over-extended itself... to the extent that (paradoxically) we eagerly publish falsities and turn down publication of more accurate studies on account of "you didn't discover anything new."
One solution to the problem of publication bias (which I personally like) is that we weight studies that disprove other studies as much as we weight those which genuinely discover new knowledge. The end result of such a valuation would be that any and all genuine discoveries that we make in a given field, will stand on more solid ground.
Any thoughts on this? And what are your thoughts on publication bias in science generally? (Not just in psychology.)
