It's an adverse selection problem.
People who are most subject to the tax are inclined to leave.
People who benefit the most from the trade are inclined to move in.