For many (most maybe?) calling someone a women is not based on a social identity but rather a physical property someone has. It's not like calling someone a skater, or nerd. But more like calling someone brown skinned or blue eyed.

This is the core of the disagreement argument of the two "camps". What does it mean to call someone a man or a woman.