This is a particular alignment issue. People are used to people spouting bullshit all the time, as long as it's aligned to what we are used to. Take religion for example. People tend to be very confident around the unknowable there.

It is scary in the sense that people love following confident sounding authoritarians, so maybe AI will be our next world leader.