You're saying a human would also offer bullsh*t suggestions that seem right but are in fact wrong?

Humans who make lots of mistakes with confidence that they aren't mistakes usually get fired or steered into a position where they can do the least amount of damage.

It's not that AI needs more background info for this type of of thing. It needs the ability to iteratively check it's own work and make corrections. This is what humans do better.