Depends on the job. If you want to get on the US Supreme Court, you have to go to Yale or Harvard.
But in general yes, it's a good idea to shop around. It also depends on the field you want. CSU Northridge is a pretty marginal school, but it's great at geology.
Law is a big outlier in general. There the degree doesn't matter at all, only the prestige. But I still wouldn't recommend paying 200k to go to Yale if Northwestern offered you a full ride. There is one SCOTUS job every 4-5 years.
For law, I'd say get a scholarship or aid at a great school or don't go at all.
In many ways grad school is an outlier in general. It's often just not worth it to get a PhD from a less respected institution. The difference is that most PhD programs pay you, not nice versa.
But in general yes, it's a good idea to shop around. It also depends on the field you want. CSU Northridge is a pretty marginal school, but it's great at geology.