According to the UN declaration of human rights, Article 13 specifically, which the US signed onto, we actually shouldn't even be able to block people at the border.
"Everyone has the right to freedom of movement and residence within the borders of each state."
You are misunderstanding that. It's "within the borders" not "across the borders." The second clause of Article 13 grants rights to leave countries, and to return to "his" country, which implies a right to deny non-citizens entry.
That countries can (as a rule) deny non-citizens entry for whatever reason they so desire should be obvious.
That says the opposite of your claim. "within the borders of each state" Once you wish to cross into another state (nation or country) your freedom of movement is subject to limitations
"Everyone has the right to freedom of movement and residence within the borders of each state."