The thing that terrifies me about the state of affairs is what happens when the software gives the wrong result because it's poorly written? If a scientists has input B and expects output X, and writes code that accepts B, and it happens to output X, chances are they'll write the paper and send it off for publication. Even if there's a bug in the code, and the real output should have been Y. I genuinely believe that this happens more often than most of us would find acceptable. Hell, there have probably been instances of the code outputting Y and the scientist scratches their head like, "Hmmm, that's not right, something's wrong," and will massage the code until it eventually outputs X, and then they'll say, "Fixed it!" and move on.
I know for a fact it's happened at least once, and led to a scientific controversy that lasted for decades. Unfortunately, I don't remember the specifics; if someone recognizes my vague description please step in with a citation. But one group of scientists published a paper saying a certain dynamic system behaved in a certain way, and a second group published a paper saying it behaved in a different way, and the two results were completely incompatible with each other. Significant public disagreement ensued. One group published their code, and the other group attacked it saying it was poorly written, etc. The second group did not release their code. Decades later, some other scientist at the second institution released the code, and after a code review, it was found that the data was incorrectly initialized. They needed to initialize the particles with "random" initial velocity vectors, but the scientist who wrote the code didn't know how to do it correctly, and wrote an ad-hoc algorithm that gave the initial velocity vectors significant bias along an axis. But the paper was already written, peer reviewed, published, and cited, so even though the paper was wrong, the result was still accepted by (half of) the scientific community. AFAIK the paper was never retracted.
I have seen and fixed code that was first written as a prototype, then used as is in a follow on project and so on, until it turned out to be used for deciding on whether or not to grand certain subsidies. Except whoops in one spot it interpreted kilometers as meters without dividing by 1000. Along with literally dozens of other outright bugs. But nobody cares, really.
I know for a fact it's happened at least once, and led to a scientific controversy that lasted for decades. Unfortunately, I don't remember the specifics; if someone recognizes my vague description please step in with a citation. But one group of scientists published a paper saying a certain dynamic system behaved in a certain way, and a second group published a paper saying it behaved in a different way, and the two results were completely incompatible with each other. Significant public disagreement ensued. One group published their code, and the other group attacked it saying it was poorly written, etc. The second group did not release their code. Decades later, some other scientist at the second institution released the code, and after a code review, it was found that the data was incorrectly initialized. They needed to initialize the particles with "random" initial velocity vectors, but the scientist who wrote the code didn't know how to do it correctly, and wrote an ad-hoc algorithm that gave the initial velocity vectors significant bias along an axis. But the paper was already written, peer reviewed, published, and cited, so even though the paper was wrong, the result was still accepted by (half of) the scientific community. AFAIK the paper was never retracted.