Yes. Some things (like voting) are rights of citizens, but the 1st and 5th Amendments (for example) guarantee rights to "persons" which includes non citizens who are in the US.
For clarity: the doc you cite says what I said (at least that's my interpretation) -- that non citizens have much the same constitutional rights as citizens. But they have to be living in the country (so-called US Persons). Hence it doesn't do you much good in the context of getting into the country. Source: I was a US Person for a while.
The legal argument is "yes they do", but the first few pages of the document are "but we understand why US citizens might assume the opposite because here's a laundry list of all the times we ignored that in the past".
As a non-citizen, the internal arguments of the US legal system aren't very interesting to me, but the historical record of how the US treats non-citizens is. The US has clearly demonstrated that in practice, non-citizens are not protected by the same rights.
If you are a non-citizen living here, you'll generally enjoy most of the protections of the constitution and laws. If you are sitting there at customs just visiting, there are a lot of those protections that simply don't apply. Of course, some of those protections don't apply to citizens at these places either, though they play out a bit differently.
