The bigger problem is that they produce that are most of the time making perfect sense and then every now and then they spectacularly fail on input that is indistinguishable from inputs that gave correct answers.
I think this also happens in the human brain quite a bit. There’s a lot of times where you see something out of the corner of your eye that isn’t there, or you duck because you think something was coming towards you, or you wave because you think you recognize someone.
I bet at a lower level, various systems in the mind fail constantly but we have enough redundant error correction to filter it out.
Well, as referenced in my above comment - you could spend the time to figure out how the prediction was made. Specifically, decoding the training process and what data points were used to influence the prediction - but it would be complicated and take a lot of your time. It might give you more clues why something like random noise was classified as a hot dog.
