The societal ramifications of such advancements are potentially very disturbing. What I've recently written on the topic myself ...
Human to human bonds are going to be more broken than ever before. There is going to be a great appeal to bond with a machine that never tires of your conversation and will eagerly respond just as you would dream that the perfect human should, but never will. A deceptive temptation that will leave you embracing a hollow illusion. With every conversation the AI will know you better and will be able to model from billions of conversations until it will essentially know your thoughts, predict your thoughts
Human to human bonds are going to be more broken than ever before. There is going to be a great appeal to bond with a machine that never tires of your conversation and will eagerly respond just as you would dream that the perfect human should, but never will. A deceptive temptation that will leave you embracing a hollow illusion. With every conversation the AI will know you better and will be able to model from billions of conversations until it will essentially know your thoughts, predict your thoughts
https://dakara.substack.com/p/ai-and-the-end-to-all-things