That’s quite a takedown. In particular the existence of an apparently better run attempt to duplicate the result, which returned not much, seems like quite a nail to put in that coffin.
As well as the issue that, to a first approximation, nobody cares.
The correct response to the replication crisis was for every affected science to immediately stop all funding and new experiments, if not pausing the ones in progress, review the situation with a series of intense conferences, and figure out how to allocate a lot more resources to replication. If that sounds extreme, well, the response to "vast quantities of the science you're doing is effectively worse than useless and you lack the ability to tell which is which" should be extreme!
But, hey, I can put my realist hat on too. I know that was never on the table. A lesser response was all we could ever hope for.
But what we got is effectively no response. Or if you prefer, the bare minimum that can be just barely called more than nothing. Nobody cares. The science goes on being cited, both in the field and in popular magazines, because that's just so much more fun. And an indeterminate, but assuredly large percentage, of science money is worse than wasted, but used to generate false science instead. And the prestige of "science" will continue wearing away until it is all spent.
Not really. Replication is where you do exactly the same experiment as someone else and see if you get the same result or not. Nobody has tried this with Calhoun's universe experiments (they would probably cost too much, be hard to get through IRB as it could be argued that it is too cruel to the rats, and wouldn't be really fashionable these days as they are seen as part of the out of favor "behaviorism" movement). Simply saying that nobody has seen this behavior in nature or among humans isn't at all the same thing as replication.
Gwern notes Calhoun hardly piblished anything formal (much less peer-reviewed) about his research - mostly interpretation-heavy summaries of what he wanted to emphasize. Not enough detail rigorously 'replicate'!
But Gwern does mention a few followups that tried similar setups - and didn't really see the same major takeaways. So "nobody has tried" isn't really accurate. Nobody has succeeded!
