That's certainly how I hear it, too. I've never heard a woman expressing fear that she might have waited too long, such that her child will get Downs. Rather, fear that she might have simply waited too long, and now it is too late.
I'm over 40, and I certainly have! Downs can be detected in time for an abortion, but it means that you have to get pregnant all over again, and risk Downs and the various other bad mutations again, and risk miscarriage for whatever reasons all over again, etc.
So for non-fundamentalist women, there is fear of having a Downs child, but it gets rolled into the package of "worried about getting to have a successful pregnancy at all".