That's why you hold the humans that run the computer accountable.

That's sort of the whole point though

The computer allows the humans a lot of leeway. For one thing, the computer represents a large diffusion of responsibility. Do you hold the programmers responsible? The managers who told them to build the software? How about the hardware manufacturers or the IT people who built or installed the system? Maybe the Executives?

What if the program was built with all good intentions and just has a critical exploit that someone abused?

It's just not so straightforward to hold the humans accountable when there are so many humans that touch any piece of commercial software

That's fair but I was referering to the humans that delegate "business decisions" to computers, which is what I thought the context was...

For example, if American Airlines uses a computer to help decide who gets refunds, they can't then blame the computer when it discriminates against group X, or steals money, because it was their own "business decision" that is responsible for that action (with the assist from a stupid computer they chose to use).

This is different from when their landing gear doesn't go down because of a software flaw in some component. They didn't produce the component and they didn't they didn't choose to delegate their "business decisions" to it, so as long as they used an approved vendor etc they should be ok. Choosing the vendor, the maintainence schedules, etc, etc: those are the "business decisions" they're responsible for.

> For example, if American Airlines uses a computer to help decide who gets refunds, they can't then blame the computer when it discriminates against group X

If American Airlines uses a computer to automatically decline refunds, which human(s) do we hold accountable for these decisions?

The engineers who built the system?

The product people who designed the system, providing the business rules that the engineers followed?

The executives who oversaw the whole thing?

Sometimes there is one person who you can pin blame on, who was responsible for "going rogue" and building some discrimination into the system

Often time it is a failure of a large part of a business. Responsibility is diffused enough that no one is accountable, and essentially we do in fact "blame the computer"

> which human(s) do we hold accountable for these decisions?

Personally I'd be satisfied holding the company as a whole liable rather than a single person.

What does it mean to hold "a company" liable?

All that does is create a situation where decision makers at companies can make the company behave unethically or even illegally and suffer no repercussions for this. They might not even still be at the company when the consequences are finally felt

> What does it mean to hold "a company" liable?

It means that the company is sued and is responsible for damages.

> decision makers at companies can make the company behave unethically or even illegally and suffer no repercussions for this

But now you've just argued yourself back to the "which human(s) do we hold accountable for these decisions?" question you raised that I was trying to get you out of.