Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I don’t think all the problems that cause lack of reproducibility come from small sample sizes. For example, P-hacking is a thing (intentionally or not) and larger sample sizes don’t solve that. Experiment registration can help so you can track negative results but that doesn’t help if there’s no reproduction attempt (ie you could just have gotten lucky). There’s also straight up fraud you have to deal with.

The point is, op is right that it’s expensive. The computer industry that we’re in claims to be data driven but I’ve observed numerous poor quality studies being done to drive decisions that I’m pretty jaded (no reproduction, poor sample sizes, skewed sample sizes where it’s employees, etc etc). And these are smart people where the decisions being made can impact the financial outcome.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: