It’s more a question of curbing their corrupting influence on politics. You might notice that the US has policies that result in mass incarceration, and that US police are far more likely than police in other countries to use lethal force. Part of this is because American law and policy is heavily influenced by police unions and prison guard unions, who favor their own interests over the public interest.
It’s also not purely a matter of private association. Often there are laws encouraging or mandating collective bargaining for certain public sector jobs. These laws are often what keeps unions in business, since those laws protect them and allow them to eg sign collective bargaining agreements that effectively require public employees to join the union (or at least pay dues to it) as a condition of keeping their jobs. It was the repeal of these laws that Gov. Walker faced political pushback for in 2011.
It’s also not purely a matter of private association. Often there are laws encouraging or mandating collective bargaining for certain public sector jobs. These laws are often what keeps unions in business, since those laws protect them and allow them to eg sign collective bargaining agreements that effectively require public employees to join the union (or at least pay dues to it) as a condition of keeping their jobs. It was the repeal of these laws that Gov. Walker faced political pushback for in 2011.