I wonder if the homogeneity has come from gentrification and high property prices. NYC might have been a crime-ridden dump in the 60s, but it was cheap enough that Andy Warhol could afford to rent a massive studio. And a modern day Leonard Cohen wouldn't be welcome in the Chelsea Hotel.
Now you have to be a lawyer or work in finance to hope to even get a modest sized apartment in NYC.
That feels like a big factor. I see far more engineers and techies in SF than artists and poets. Not sure how many artists could survive in the city when not only rent is so expensive, but almost all other goods are also more expensive (largely because the stores and their employees pay those high rents too).
Now you have to be a lawyer or work in finance to hope to even get a modest sized apartment in NYC.