Yeah the problem is optics (the more important part of getting really good images) are expensive, and few companies make decent lenses that target tiny sensors.
So you're stuck with the little optics included with the tiny hobby cameras from Pi or ArduCam (et all). They're okay for some purposes but a lot worse than even a cheap SLR/Mirrorless lens.
The best results I've gotten are with the C-mount camera module, an adapter, and a wide angle Sony, Nikon, or Canon lens—all of which cost in the hundreds :)
Thrift/antique stores are a place to look for old lenses on the cheap. If you've got a C-mount adapter for your sensor you can mount the lens on it. Then you can manually focus it and epoxy it on the right point.