I'm saying that our music is largely diatonic, and it's better to base our notation and terminology on the diatonic rather than the chromatic scale.

The idea that you can number semitones 1-12 has some mathematical elegance to it, but it's a terrible system in practice. It turns out that mathematical elegance doesn't count for much, and domain knowledge is important here.