I think it's a critical mass of people experiencing the "unit tests pass but the code is broken" problem. When unit tests are used to test glue code you end up testing your mocks and nothing else. Mocks are often done using a framework which encourages one off implementations per unit test. This introduces a maintenance burden where all the mocks have to be kept in sync.
Unit tests are very useful but we somehow landed ourselves in a place where we have lots of line coverage in our unit tests but little confidence that the code actually works.
There are at least 3 different applications at work where they unit tests are green while the code is broken or red when it's not. The cause is almost always the mocks. They either presume too much, making then fragile or they are flat out incorrect. Despite having a lot of test coverage there is little confidence for the developer that their change is correct.
In a sense the writers of the tests were "doing it wrong". The burst of articles on unit tests and their failure modes are a reaction to the prevalence of this in our industry.
Similarly, the unit tests never catch bugs and you're always changing them because of refactors or changing requirements. At some point it's like 'hey this set of tests have been rewritten 3 times and they never caught anything, so ... like does that mean that we just wasted time for the first two times?'
I've seen unit tests as documentation cause problems a few times. Like, maybe if you've got some sort of DSL where it's actually obvious what behavior is expected. However, more often it's 10-50 lines of setup / mock code and then some number of asserts and you're left trying to decide what the point is (ops, it turns out something got misinterpreted and the test is actually nonsense).
Finally, it seems like using the type system to design code where illegal states are not representable is starting to make some headway. Additionally, we're seeing increasingly powerful type systems make it into industry acceptable languages.
Even with typed driven development you still want unit tests in the form of property based tests. Also a nice way to resolve the 10-50 lines issue is to follow the Arrange, Act, Assert pattern where you could look at the second block to see the actions. Move as much of the Arrange section into a setup and it should be a bit clearer :)
> people experiencing the "unit tests pass but the code is broken"
Arguing against unit testing is like arguing against type-safety (and, usually, anti-unit-testing people are anti-type-safety people, too). The presumption always seems to be that if it doesn't solve every problem, it's unnecessarily slowing things down.
In my experience it's the pro-unit-testing people who are more likely to be anti-type safety people. The argument against type safety typically goes something like this:
I already have to write unit tests so why would I bother with types they don't add any real value.
Both camps are wrong. Unit tests are an unambiguous good. Types are also an unambiguous good. Both have some rather common failure modes though and guarding against those failure modes is useful.
Unit tests of purely functional code where you only need to provide an input and validate an output provide tremendous value. The unit test can treat the code as a black box and as a result the unit test is robust and resilient to changes in the black box while verifying that the box still produces the correct answer. Unit tests of code with hidden dependencies that need to be mocked require a lot more care to construct properly. Mock scripting frameworks encourage a number of bad habits. Things like "How many times did this method get called". Or "Always provide the same answer when this method get's called." The result is a hundred reimplementations of the same interface that are at best correct for the current version of the code they are testing and at worst completely incorrect reimplementations of whatever they are mocking. They all need to be kept in sync and maintained over time.
A shared in memory Fake will in general provide more value and be less fragile over time while also ensuring that your tests are actually testing the code and not the particular script you defined for the mock.
> (and, usually, anti-unit-testing people are anti-type-safety people, too)
This is the opposite of my experience. Most of the anti-unit testing people I've talked to are very much pro-type people. I wonder if anyone has done any studies that shows what the actual numbers look like.
> Arguing against unit testing is like arguing against type-safety
I disagree with this. Types (at least when they've been built on top of an actual logic) have the benefit of real costs and benefits. You can show what programs you are unable to write and you can show (mathematically) that certain failures won't happen.
Unit tests on the other hand are much more hand wavy. You can show that some refactors seem easier, but you can't prove it without a lot of data that has to be collected on a project by project basis. I'm not saying that I don't want unit tests if I'm doing a non-trivial refactor, but I am saying that that desire is more of an intuition thing. It's not like I can make any proofs around it like I can do with a type system.
Unit tests are very useful but we somehow landed ourselves in a place where we have lots of line coverage in our unit tests but little confidence that the code actually works.
There are at least 3 different applications at work where they unit tests are green while the code is broken or red when it's not. The cause is almost always the mocks. They either presume too much, making then fragile or they are flat out incorrect. Despite having a lot of test coverage there is little confidence for the developer that their change is correct.
In a sense the writers of the tests were "doing it wrong". The burst of articles on unit tests and their failure modes are a reaction to the prevalence of this in our industry.