So if I insult and berate you at a party, and you choose to go home after seeing that the host won't intervene, you're the one who's against free speech?
Insulting and berating has a somewhat threatening connotation to me and so I think isn't even a remotely accurate analogy.
Say Bob and Sue are at the party talking about some topic you don't approve of. They aren't talking to you, just each other. You request that the host remove them. The host refuses. So you attempt to get other people to very vocally leave the place in order to pressure the host to kick Bob and Sue out.
> So you attempt to get other people to very vocally leave the place
So...exercising your free speech to convince other people to exercise their free speech? Sounds like lots of free speech to me.
It's up to the host whose presence they value more. They can always decide you're an irritating busybody and no one wants your negativity killing the vibe.