But the word is not about agency: it expresses a state. How that state was achieved, doesn't matter. Are we going to say "enprisoned people", "people who somehow and totally against their will caught COVID", "people who accidentally experience permanent, fully reduced hearing"? Do you still call people who once were, but not longer are, slaves, enslaved?

The assumption behind it all is that if someone can make a inference about your thoughts, no matter how far-fetched, those thoughts must contribute to an undesirable state, hence you are a bad person.