That could have a huge impact. For example if it affected random draws of a hyperparameter in a Bayesian model, leading to incorrect credible intervals ultimately in the posterior distribution. Or worse, if the RNG bug was affecting the variance of randon samples in some deep component of an MCMC algorithm like NUTS or even simple Metropolis. Depending on the exact nature of the bug, it could even cause the model to violate Detailed Balance, and entirely invalidate sampling based inferences or conclusions.
Well that is why you would always run statistical validity tests on the RNG and other intermediate values whenever using a Monte Carlo model in production. Ideally the tests should run as part of a Continuous Integration workflow.
If you (dubiously) write your own RNG, sure. But you should never do that. And for library RNGs, you should execute the unit tests of the library. Frankly, running it as part of CI is at best overkill and at worst adds complexity that costs you. If you pin versions of your dependency and isolate the artifact into an in-house artifact repository, so that the library code is literally never changing unless you explicitly modify the version, then you should test it up front, then again only occasionally if you actually have evidence of a bug. And as part of any code review of the code change that introduces a version change.
