The problem with LLMs is that they do what you tell them to do.
If you ask the LLM to translate something without context, it will translate without context, and the result will quite likely be incorrect. If you provide some context, the result will likely be better. And if you tell the LLM what you are actually trying to say, the result will often be even better.
In this case, the issue was that "speed bump" did not actually mean a speed bump. It was used in a figurative sense, as a cultural idiom. If you tell that to the LLM, it should be able to suggest better ways to express the same idea in other languages.
I wish LLMs asked more clarifying questions for precisely this reason. And that they disagreed with me more often.
I'm not sure that when asking for a translation, even with context, they would have replied that my request doesn't make sense (or advised against using LLMs for translation).
There was a few month window where ChatGPT was capable of giving you the reasons and rationale of how it came to a completely wrong answer, and then proceeded to explain why it was wrong. It was quite interesting and in most cases understandable, giving you the opportunity to give the additional context needed to succeed in the future.
Then for whatever reason it was nerfed. Same as the original chain of thought output where for a few days it was accidentally exposing the hardcoded guardrails OpenAI had included. You would actually see it saying “Making sure my answer doesn’t support any stereotypes or include information that could fuel discrimination” and then re-writing factual statements in the output if it happened to touch upon something remotely controversial.
