Do we want to turn customer service over to "this might all be bullshit" generators? Imagine coming into the showroom, agreeing on a price for a car, doing all the paperwork, and having them tell you that wasn't legally binding because of some small print somewhere?
I think that's a very simplified view of all of it.
Customer service has to be different levels of help tools. And current AI tools must be tested first in order for us to be able to improve them.
You have limited resources for Customer Support, so it's good to have filtering systems in terms of Docs, Forms, Search, GPT in front of the actual Customer Support.
To many questions a person will find an answer much faster from the documentation/manual itself than calling support. To many other types of questions it's possible LLM will be able to respond much more quickly and efficiently.
It's just a matter of providing this optimal pathway.
You don't have to think of Customer Support LLM as the same thing as a final Sales Agent.
You can think of it as a tool, that should have specialized information fed into it using embeddings or training and will be able to spend infinite time with you, to answer any stupid questions that you might have. I find I have much better experience with Chatbots, as I can drill deep into the "why's" which might otherwise annoy a real person.
That's pretty much what happens anytime you buy a car though. There's always some other bullshit fees even if you get incredibly explicit and specify this is the final price with no other charges. They are going to try to force stuff on and unless you are incredibly vigilant and uncompromising. It sucks when you have to drive hours away just to leave in your old car.
And actually based on my experience, customer sales agents, whether it's real estate or cars are notoriously dishonest. They may not hallucinate perhaps, but they leave facts unsaid, they will word things in such a way as to get you to buy something rather than get you to do the best decision - sometimes the decision could be not to buy anything from them.
So a ChatBot that can't intentionally lie or hide things could actually be an improvement in such cases.