RE: Crisis in Psychology?
April 26, 2022 at 12:30 am
(This post was last modified: April 26, 2022 at 12:31 am by vulcanlogician.)
(April 6, 2022 at 8:07 pm)John 6IX Breezy Wrote: Well here's a question I've wondered about. What is the right conclusion to make once a study doesn't replicate?
That the results of the study aren't replicable and shouldn't be regarded as genuine scientific knowledge. Or maybe the replication attempt was flawed... but it's an either/or scenario. Non-replicable results don't allow us to confidently conclude anything. So they are as good, knowledge-wise, as no results whatsoever.
I expect more from psychological science.
Quote:Most people, I think, conclude that the first study is flawed and throws everything out. But I can't help but feel there is a temporal or sequential bias there. Would we be inclined to make the same conclusion if the studies came out in reverse order, for example? Meaning, if the study that didn't find any effect came first, and the one that found an effect came after?
If neither study is replicable, then neither one says anything. I prefer those psychological studies that are replicable. Even pessimistic estimates put these at at least around half of psychological studies. The portion of studies that aren't replicable need to achieve the standard set by those that are.
Quote:So, it's not immediately clear to me what our response to replication ought to be. And I should add that I think the root of the replication problem is statistics.
I disagree. The problem is the inability of [a certain portion] of these studies to be replicated. The size of that proportion is not the problem. We should only allow a tiny fraction of non-reproducibility (that imposed by chance or dumb luck). We shouldn't make systemic concessions within the science. We need psychological science to be reliable. If the things cited in the OP are true, it isn't.