Crisis in Psychology?

[John 6IX Breezy]

We need psychological science to be reliable.

Hmmm I think we need a psychological science that's adaptive and individualistic. Yes, maximizing reliability is important. But keep in mind that the object of study isn't consistent the way atoms might be. A single individual is developing throughout the lifespan. And the species as a whole is randomly evolving from generation to generation.

Psychology, in other words, is the study of change and variability. A correlational study, for example, might attempt to explain variability in one distribution (stress) in terms of variability in another distribution (finances). Consider cross-cultural studies. If nothing else, these studies measure the degree to which a finding consistently fails to replicate from one population to the next.

Recently, I've been diving into Bayesian Statistics. I'm still grappling with it conceptually, but I think it's the correct approach here: You update what you believed about the old study with the evidence you uncovered with the new study. It is an adaptive approach. You can't throw out every study that doesn't replicate because they still hold information about the world no matter how noisy it is (assuming they are internally robust).

ps. And what does successful replication do for us if a given study is internally flawed? Replication doesn't protect or uncover deeper problems. It isn't a means of falsification. If your methodology is flawed, replication only perpetuates the flaw even if the results are consistent.