Laws have to be enforceable. When a technology comes along that breaks enforceability, the law/society changes. See also prohibition vs expansion of homebrewing 20’s/30’s, censorship vs expansion of media production 60’s/70’s, encryption bans vs open source movement 90’s, music sampling markets vs music electronics 80’s/90’s…
This is a good point. In this case, it does seem pretty easy to enforce, though - just require anyone hosting an LLM for others to use to have full provenance of all of the data that they trained that LLM on. Wouldn't that solve the problem fairly easily? It's not like LLM training can be done in your garage (at which point this requirement would kill off hundreds/thousands of small LLM-training businesses that would hypothetically otherwise exist).