The 10th amendment. Nowhere in the constitution is the Federal government granted power to restrict non-citizens from movement or employment. The constitution only delegates to the Federal government the power to determine rules for naturalization.
My understanding of the Constitution is that it sets limits on government power, e.g. it can't pass a law that restricts free speech. It does not enumerate all the things the federal government may do.
One of those limits (10th amendment) is all powers not enumerated by the constitution are left to the states or the people.
It’s ironic that one of the original grievances in the Declaration of Independence was the king restricting immigration to the colonies.
Freedom of movement is a common law right going back to the Magna Carta: It shall be lawful to any person, for the future, to go out of our kingdom, and to return, safely and securely, by land or by water, saving his allegiance to us, unless it be in time of war, for some short space, for the common good of the kingdom: excepting prisoners and outlaws, according to the laws of the land, and of the people of the nation at war against us, and Merchants who shall be treated as it is said above.
When you say "unconstitutional" though, do you mean "something that is not in the constitution" or "something that directly contradicts the constitution"?
I don't think those two things are meaningfully different.
10th amendment: The powers not delegated to the United States by the Constitution, nor prohibited by it to the States, are reserved to the States respectively, or to the people.
No. A non-citizen moving from one place to another isn’t commerce among the states. A non-citizen working at a restaurant is also not commerce among the states.
The purpose of the interstate commerce clause was to prevent states from enacting tariffs or banning goods from other states or nations. There’s plenty of analysis of the origin and intent of the commerce clause online if you’re interested.