It's not really much different from a physical experiment, where the expectation is that you'll have to rebuild the experimental apparatus yourself to reproduce the experiment.
An independent implementation of the experiment is neccessary for a full reproduction anyways. If you just run their code again, you'll end up with all their bugs again.
(But, don't get me wrong. I like when researchers release their code. It's still very useful.)
This can't be upvoted enough. Makes me thinking that using published source code and reproducing/validating results is completely orthogonal. Maybe it's a good thing the source code for science gets not published.
I like the concept of rewriting all the code by an unbiased third party to see if they can reproduce the results, but in practice what this leads to is:
1. People will not bother. It took a lot of minds to come up with the software used (in my friend's case, several PhD's amount of work). No one is going to invest that much effort to invent their own software libraries to get it to work.
2. Even when you do write your own version of the software, there are a lot of subtleties involved in, say, computational physics. Choices you make (inadvertently) affect the convergence and accuracy. My producing a software that gives different results could mean I had a bug. It could mean they did. It could mean we both did. Until both our codes are in the open, no one can know.
It is very unlikely that you'll have a case of one group's software giving one result and everyone else's giving another. More like everyone else giving different results.
An independent implementation of the experiment is neccessary for a full reproduction anyways. If you just run their code again, you'll end up with all their bugs again.
(But, don't get me wrong. I like when researchers release their code. It's still very useful.)