The are mainly large to increase resolution and sensitivity. Atmosphere limits how large you can make the mirror before hitting diminishing returns, but that is mitigated by good site selection, adaptive optics and lucky imaging (taking many exposures and keeping the least blurry ones).