Fantastic. We had a similar problem with test code that always worked on a remote team's computers during their work day but failed when we got into the office and tried to run it in a different timezone.
Reminds me of an app I worked on a while ago where both the server it was running on and the datetimes in the database had to be in a specific time zone. Change either (including changing both to an identical but different TZ) and the app just failed to run.
I've seen something similar where some automated unit tests always passed and when I tried to run it locally would always fail. After digging around I realized it was making assumptions about the timezone it ran in, which was on the east coast, whereas I was on the west.
I've had to fix any number of test suite bugs as well as regressions that happened because the people who wrote the tests had ensured the test suite didn't (most of the time) test what they thought it would by relying on date arithmetic from current date and time...
Many people think that test suite authoring is easy mode. It really isn't. Making a good test suite that actually does a good job of tracking invariants is a royal pain.
As far as I'm concerned, writing tests is a different skillset and its often underdeveloped. It then snowballs into hating testing, because all you've experienced is a poor test suite that gets worse over time. It's hard to do well as you say, and it also gets glossed over in code review as long as the tests pass. Do the tests actually verify anything useful? Who knows, build is green!
That's how you get an application with 50,000 unit and integration tests that you can't run locally, and requires massive parallelism in CI to finish in any reasonable timeframe.
My favourite is the test suite where each test relies on data from every other test, so running a test in isolation fails, and adding a new test is almost impossible without breaking a bunch of other tests.
