I'm on the weird spectrum and have a similar experience with neighbors. I suppose the more of an outlier you are, the less likely it is to randomly land in a compatible community, and therefore the harder you need to work to make it happen. Of all the places I've lived, high density residential areas near high densities of colleges were the best at making this possible. The problem is, settling in such areas tends to be expensive (NYC/Boston/Bay Area/etc), so you look further out, and wherever you can afford tends to be majority townies/normies.