Your intuition is assuming that rates would increase because mutations enter the general population and then spread through that population at some possibly small rate.

However, the prevailing genetic explanation for increases in autism prevalence rate is that mutations appear de novo (out of the blue) in a parent, and then are passed down to the proband with the disease. The de novo mutations in each case are different, more or less.

The reason that prevalence is increasing under this model is because there is is something that is increasing the rate with which the de novo mutations are passed onto offspring.

This reason is age: de novo mutations occur increasingly with age in germ cells, and if germ cells become offspring at later ages, you will see an increase in prevalence.

Studies suggest that spontaneous mutations in sperm become more common as fathers age, because of age-related issues, and these sperm with mutations are more likely to lead to autism. Because fathers are having offspring at later ages, autism is increasing in prevalence.

Maternal age is less strongly related because egg cells are formed earlier in development.

FWIW, I agree there's probably lots of environmental influences, but I don't think that's the only thing.