> Every LLM company so far has been extremely careful to avoid using private people's data for LLM training

No, they haven’t. (Now, if you said “people's private data” instead of “private people's data”, you’d be, at least, less wrong.)