If you use a few-shot technique (i.e. your prompt contains a couple of example questions and answers) you can mitigate this behavior by adding a question with the answer "I don't know".

More generally, if you teach the model to reject nonsense questions and admit if it doesn't know something it's more likely to do that