I'll concede that defining freedom is a thorny subject, and there are limitations to the definition a I proposed. But I think that any definition of freedom that doesn't accept that private individuals can (and constantly do) limit each other's freedom regardless of the state is just as flawed.
This isn’t a very useful argument when balancing the rights of multiple entities in society though.