This was my goto argument when people without technical background were laughing about ChatGPTs bullshit generation (before it was called hallucinating).

Put a gun to Einsteins head and force him to answer any question you ask him without hesitating or denying an answer.

This is not an argument for stating LLMs like ChatGPT are intelligent (it is not, but not for any of the reasons i have read about so far anywhere). But the hallucinations definitely don't show anything about intelligence. Actually, if they did, they would actually indicate intelligence rather than not. So, quite the opposite which should be very obvious.