The script was a permission checker; roughly, for each client check whether they had any object types they didn't have permission for, plus a bit more logic for things like "they have permission A which is a superset of B but don't have permission B", and output a report. There was a bug in the DB cache used by our security service (I think caused by a broken clone() implementation that shallow-copied when it should have deep-copied, but this was a while ago now) that meant that if permissions for two different clients were requested in quick succession, both would end up with the intersection of their permissions - which were then cached in the main application (I guess there must have been some logic like the API always triggered a refetch to ensure it returned the canonical value, whereas for the rest of the application performance was more important?). In theory it could've been happening already, but I guess we never had two clients loading the same webpage close enough to the same time (and they would have to both have had the local cache expire - plus almost all of our live clients had very similar permissions, whereas we had some "test" clients with no permissions at all), whereas of course my script just called as fast as possible.

It was fairly easy to track down the bug once we passed the initial disbelief stage, and I don't think many clients noticed. But it could've been a lot worse - I'd been so close to running this last-thing on a Friday before I left.