A) I understand many of those countries also have government-dispersed drugs, which dramatically changes the situation by making there be legal drug transactions legal too.

Unless someone is proposing government run sex work, it's not a fair comparison.

B) Drug dealing and sex work are also quite different in that the former in it's modern day incarnation can be highly anonymous with dead drops and whatnot, but that isn't possible for most sex work.

C) Finally, maybe the above does make drug dealing a nastier business, but few people are worried about the well being of drug dealers that aren't also for legalization.