Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> "can you really trust accounting software not to make mistakes? Won’t there be edge-cases in mortgage underwriting that software might miss, but an experienced underwriter would catch?" The proof is in the pudding: the world runs on software now.

Indeed.

https://en.wikipedia.org/wiki/Post_office_scandal



I worked as a PM for a startup that makes software to help review appraisals, and there's absolutely no question that stuff gets missed by humans in appraisal review as well as the rest of the underwriting process.

Every organization is built around tradeoffs between time (which is very much money here) and quality. You route appraisals to people of different skill levels depending on the likelihood of issues and the likelihood that those issues matter. With appraisals, for example, if buyer's putting down 40% where only 20% is required, you check only the super critical stuff, because even if you're meaningfully off on the appraised value it doesn't actually matter that much.

There are a ton of these tradeoffs of whether you should review something as well as whether you should have your most senior people review it. On top of that, reviewers are judged on speed, so naturally they're making tradeoffs of accuracy vs. time themselves on each review.

Using AI means you're able to review things that wouldn't even have been looked at otherwise, because there's no longer a tradeoff of time/cost vs. accuracy. Even the world's best reviewer can't catch things they're not looking at.


See also the Australia Robodebt scheme, which also claimed many lives: https://en.wikipedia.org/wiki/Robodebt_scheme


Wow, those two articles are as Kafka-esque as it gets. What a nightmare it must have been for the people caught up in them.


Why did they defend Horizon though? The Wiki article offers a POV from someone who was functionally powerless:

> We have to be careful, that we are not creating a cottage industry that damages the brand and makes clients like the DWP and the DVLA think twice. The DWP would not have re-awarded the Post Office card account contract, which pays out £18 billion a year, in the last month if they thought for a minute that this computer system was not reliable

I know that's something that someone said, but is it true? So what if a lot of people say that? Nobody knows who or what leads to sales or not sales. If sales were all that mattered, they wouldn't do the IT upgrade at all.

People use shitty software all the time.

> The new Horizon project became the largest non-military IT contract in Europe.

Also... really doubt that is true.

The Horizon IT report's first volume "will focus on redress (compensation) and the human impact of the Horizon scandal." Okay. But why did people feel so strongly about the technology in the first place? Who gives a fuck about bugs?




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: