Looks like the LLM equivalent of a stroke - it's hallucinating non-contextual responses because it's traversing paths that are skipping important contextual connections.
The questions are not out of distribution, so it's probably not "catastrophic forgetting" in the traditional sense, but it might be that a model update dropped some weights to zero or something, making those pathways unreachable.
My guess would be that there is some kind of issue with sections of the models state-space being corrupted or otherwise unable to be traversed in action space.
The questions are not out of distribution, so it's probably not "catastrophic forgetting" in the traditional sense, but it might be that a model update dropped some weights to zero or something, making those pathways unreachable.
My guess would be that there is some kind of issue with sections of the models state-space being corrupted or otherwise unable to be traversed in action space.