Bathrooms and prisons, definitely. There is no magical barrier stopping a man from entering a woman's bathroom. What you are suggesting otherwise is buff trans men having to use the same bathroom as cis women, basically fulfilling your transphobic stereotype just not in a way you anticipated :)

As for sports, that's for the specific sport organizations to decide the rules based on latest research.