This is only true if you're right up on the Peninsula. The South Bay area where YC is located is gorgeous for a huge fraction of the year, and the East bay is at least far less cold than downtown SF.
To each their own. SoCal is too hot for many people. Your comment might seem to imply that SF weather is silcon valley weather. 30 minutes south the weather is consistently warmer though leave the coast a little ways and the tempature can also dip more at night.
My family is from Victoria/Vancoucer and likes it cooler to enjoy active outdoor activities like running, biking and hiking. For this SF and Oakland weather is wonderful.
I also work a few blocks from the ferry building and wear shorts near year round, though that isn't saying much spending all
my time in a temperature controlled office.