Or take more inputs. If there are differences between race and gender and thats not captured as an input we should expect the accuracy to be lower.

If an x-ray means different things based off the race or gender we should make sure the model knows the race and gender.