Can hiring only white people be one of my chosen "ends," which the government has no business interfering with? Can serving food to only white people be one of my chosen "ends"?
Yes. Freedom does not work if we are not also permitted to make irrational choices.
What if the state thinks it's stupid to not only serve white people? That was true in my home state within living memory.
What if the state thinks it's irrational to not be a Christian?
What if the state thinks horses are "good enough," and cars are too dangerous to be allowed? Today's America would never allow the automobile to be invented. And Big Candle would probably lobby against the creation of lightbulbs.
Places that practice the kind of racism you are talking about will be the subject of ridicule and will face economic stagnation. Whether on the level of a single establishment, or a country. Rightly so. The civilized world--if there is one--will move on, and will help the discriminated relocate to free societies. (It's a shame we are not doing this for the Uigyurs--we should.)