Russia uses a similar system, the Kashtan CIWS. This article (https://www.bbc.com/news/world-europe-61103927) indicates that the Moskva did have one installed, though I don’t know why it didn’t help in this case.
Drones are reported to have been used to harass the vessel and keep its air defences distracted before the missiles were fired from a hidden battery near Odessa. At least two of the missiles are then reported to have struck the ship – causing a massive explosion and inferno as they are believed to have detonated one of Moskova’s exposed deckside missile tubes.
I wish tech review sites would do a molecular dynamics benchmark for processors like these. It's pretty easy to set up just a simple MD simulation, and there'd be tens of us interested in seeing the results.
Interesting. I, for some reason, was unaware that the benchmarks were user sourced. It appears to mainly be a system for bash scripts to run the program and then automatically format the results, is that correct? There's a benchmark for LAMMPS, but it'd be interesting to get one up for Gromacs or CHARMM.
I think your first paragraph is a result of you viewing this from the perspective of someone knowledgeable about technology. The vast majority of people using Android phones get their apps exclusively from the Play Store and assume Android is exactly the same as iOS in that way.
Not really answering the question, which was "how do you know it works?"
I would contend that if you have proven to yourself that your code works and are therefore capable of proving it to other folks (via e.g. solid testing), you should not be ashamed of the spit and glue.
It is research code, we all know what that looks like. But if, on the other hands you haven't proven to yourself it works, then it's definitely something to be ashamed of - scientifically speaking.
It’s much the same reason why you wouldn’t go to work wearing a days old, stained shirt and ripped pants — yeah, you’re more than likely going to work just as hard and as well than if you were wearing clean clothes, but that’s not going to stop people from judging you based on your appearance.
Most academics write code to just work. Not work well, or to be generalized, or to be efficient, just work. And while that’s absolutely fine, as your results being reproducible from the code is all that really matters, a lot of people don’t see it that way and will only see code slapped together haphazardly and dismiss you because of it.
This article shows very clearly why that does not work. The Princeton team could not reproduce the Berkeley results, yet the inaccessibility of the code meant that the latter persisted as a road-block for almost a decade.
Imagine if this reproducibility excuse were applied to experimental results and technique: we don't have be careful or explain in detail what we are doing, as reproducibility will take care of any errors. One consequence would be that, as the current state of knowledge became less certain, it would become less clear what to do next.
I understand the embarrassment, I am sometimes embarrassed as well looking at quickly-put-together code
I do appreciate papers that give you either a reference implementation or at least enough details on how it was done. I spent too many hours on papers that didn't trying to reproduce their claimed results...
Usually testing. Just because code works flawlessly doesn't mean it isn't embarrassing; and just because you are proud of the code doesn't mean it's bug free.
My PhD adviser was a good friend of Chandler's, and a lot of my doctoral work was based on previous work done in his lab. I'm honestly really surprised one of his students/post-docs made a mistake of this magnitude -- generating a velocity distribution which does not follow the Boltzmann distribution is not something I'd expect to be a result of code coming from his lab. But we all make mistakes, and this really just lends more weight to the idea that all code used in producing published results should be posted along with the article itself, just like data sets often are (at least from studies with government funding).
Sharing code is not always the answer. Independent thinking implies independence, in code and more abstractly.
The real 'mistake' is allowing the drama to escalate to the point that it is toxic. People who are truly interested don't care about who is right and who is wrong.
When disagreements like this come up, having common and good test cases are probably the most important (and is indeed the way in which the problem, generation of non-Boltzman behavior, was found).
In the end ST2 itself is flawed, and Princeton admitted that the discussion of water was not significantly advanced through this drama. Is it worth it?
Understanding the argument and its importance should be the focus.
Sharing code is not the full answer, especially in case of Monte-Carlo simulations in physics, because
that kind of algorithm is hard to test: What is the testing oracle?
What is the specification?
But sharing code is part of the answer.
Setting up a culture where it is unacceptable to submit a paper
without open-sourcing the code and suitable testing (for simple
edge ases), and suitable scripts that make reproducing the software
simulations easy, is good scientific 'hygiene'. See for example [1,
2] for efforts towards reproducible software submissions in computer
science.
Reproduciblility is the very essence of the scientific method.
In this case, independent thinking did come up with independent results, but the refusal to share code significantly delayed the recognition of which line of inquiry had been more successful.
You might want to be careful about using that "was it worth it?" argument - someone might reasonably ask, was it worth funding either team? If science doesn't show that it does good work, then its funding can be more easily questioned.
Ordinary text can be confusing. That is not a reason for not publishing, and one of the purposes of a journal's editors is to keep the confused and confusing stuff at bay.
I agree with your latter two points, but I must take issue with your first point about mental health. I've posted some links below for your further reading which can explain the stressors better than I can, but graduate school is a breeding ground for mental health problems -- you have constant pressure to always be working (often up to 80-100 hours a week), constantly comparing yourself intellectually to other graduate students, and dealing with the overbearing reminder that you need to take a candidacy exam and defend your thesis. As a poster above said, 30% is absolutely not normal for society as a whole. Most articles point out the dearth of students who seek help, which I don't think is uncommon, but what is uncommon is the additional stressors.
I don't know if you've been through a PhD program, but as someone who has, it's really difficult to explain the pressures and stress that you encounter during it to someone who hasn't.
In my field (computational biophysics), it's pretty common practice to post published code on a git repository. Code used to produce published results absolutely should be accessible.
I'm not in academia so maybe I'm being flippant, but what is even the point of a paper whose results can't be reproduced? Are you really advancing knowledge in any meaningful sense if someone can't repeat what you did?
Exactly. In one recent case, someone published a paper about an interesting graph centrality metric that I was interested in trying out. Unfortunately, their description in the paper was far too vague to be useful - "implemented as a simple extension of Brandes' algorithm". In attempting to reproduce it, that meant I needed to go read up on the algorithm they extended, and then try to figure out how they actually extended it. In the end, I couldn't actually reproduce the work, and never heard back about the code that the authors used in their published work. That severely degrades the utility of the paper. Yes - the paper does contain some knowledge that they shared with the world, but it was difficult to build upon and replicate since they failed to describe what amounts to the experimental apparatus and setup that was used to obtain the results they published. Unfortunately, this is relatively common in CS (at least, the corners of CS where I work).
I know this is a big issue in AI/ML right now. Deepmind's papers are notoriously hard to reproduce, because they will lay out the general terms of the architecture but not specific implementation details - things like filter length, stride, number of layers, number of hidden units, feature selection, and all the little tricks of initialization or normalization or a zillion other subtleties.
The trouble being that those "specific implementation details" are typically non-obvious and absolutely crucial to getting the system described to work at all. For instance, as far as I know, nobody's managed to implement a WaveNet that sounds anything like as good as Google's samples. Neural Turing Machines - published three years ago - were so finicky that someone actually figuring out how to implement the damn thing and have it actually work as described was enough to warrant a paper of its own (Implementing Neural Turing Machines, https://arxiv.org/pdf/1807.08518.pdf). Not to mention how hard it is to iterate on failed replications when you aren't blessed with ten thousand Nvidia Teslas and custom tensor ASICs and have to wait eternities for models to train. At this point, I think most of the community just kind of looks at their papers, sighs in jealousy, and moves on.
You must make possible for other researchers to reproduce your work, but that does not mean you have to give it to them for free.
I work in an university, but in close collaboration with industrial companies. We use models that we explain with detail in our papers, so that other researchers can write their own implementations and maybe confirm or disprove our results. But we do not make the code available.
This is what I see, that does not mean I think it is the best approach. I would really like to release all the code I write. It would make research advance faster, and I do not think that it would harm the company who pays for my work in any way.
But they pay, and they have strict policies. At least they allow me to share most of my code with other researches in a personal basis and write papers about it. I am quite sure that, if to publish papers I had to always share the code, they would just directly hire me or someone else to do it and there would be no papers at all.
I think a lot of this depend on the field and context. For example in physics there are a lot of commercially available simulation tools that help with analysis of an experiment. You of course describe the approach and setup of the simulation but you can not publish/reference the source code since you are just a licensee.
