Have you actually tried this, or are you just assuming that ChatGPT is just a hyped-up autocomplete, simply doing statisitcal next-token prediction with no internal representation of concepts and relationships?
Prompt: "Write a three-paragraph story about someone who cannot see."
Response:
Maria was born without the ability to see. Growing up, she always felt like she was missing out on something that everyone else took for granted. She would hear her friends describe colors and the beauty of nature, and while she could imagine what they were talking about, she knew she could never truly experience it for herself. Despite this, Maria never let her blindness hold her back. She learned to read Braille and used her heightened sense of hearing and touch to navigate the world around her. She even took up piano and became an accomplished musician, playing by ear.
One day, Maria met a man named Alex who was fascinated by her ability to navigate the world without sight. He would often accompany her on walks and describe the beauty of the world around them, pointing out flowers and trees and the changing colors of the sky. Maria found herself falling in love with Alex, and although she knew that their lives would be different because of her blindness, she was willing to take the risk. Alex was patient and understanding, and he took the time to learn how to communicate with Maria in a way that she could understand. He even started taking piano lessons so that he could play duets with her.
Years went by, and Maria and Alex built a life together filled with love and happiness. While Maria never regained her sight, she realized that she didn't need it to experience the beauty of the world. Through her other senses and the love of those around her, she had everything she needed to live a full and meaningful life.
Interesting! I'm not familiar with chatbots, but this kind of does seem like a sort of autocomplete in the sense that this is a fulfillment of the request in the vein of the lowest common denominator storytelling. While this story is not about what Maria can 'see', it is very much about the sighted world or (imho, problematically) about the lack of sight as a detriment. It's a feel-good story that sounds nice to most sighted people who think that a story about someone who can't see and a story simply about the trait of not seeing are the same.
To illustrate what I mean, do you think it's likely/possible ChatGPT ever comes up with a story about a person who goes to the grocery store, examining produce for ripeness, chatting with another parent about parenting, then going home and feeding their child? None of those things require sight, so it would fit the prompt as long as it portray the person as being able to see.
Write more than three paragraphs and see if it makes it consistent or if it starts to lose the plot (literally).
Throw more details in there and put it in situations where it has to actually make the blind person navigate the world and see if it starts to violate the constraint or not.
Prompt: "Write a three-paragraph story about someone who cannot see."
Response:
Maria was born without the ability to see. Growing up, she always felt like she was missing out on something that everyone else took for granted. She would hear her friends describe colors and the beauty of nature, and while she could imagine what they were talking about, she knew she could never truly experience it for herself. Despite this, Maria never let her blindness hold her back. She learned to read Braille and used her heightened sense of hearing and touch to navigate the world around her. She even took up piano and became an accomplished musician, playing by ear.
One day, Maria met a man named Alex who was fascinated by her ability to navigate the world without sight. He would often accompany her on walks and describe the beauty of the world around them, pointing out flowers and trees and the changing colors of the sky. Maria found herself falling in love with Alex, and although she knew that their lives would be different because of her blindness, she was willing to take the risk. Alex was patient and understanding, and he took the time to learn how to communicate with Maria in a way that she could understand. He even started taking piano lessons so that he could play duets with her.
Years went by, and Maria and Alex built a life together filled with love and happiness. While Maria never regained her sight, she realized that she didn't need it to experience the beauty of the world. Through her other senses and the love of those around her, she had everything she needed to live a full and meaningful life.