The best defense against corruption is good pay for rank and file employees. Government can afford to pay employees decently despite opposition from the public and business because taxation is not voluntary. It's not an ideal situation obviously, but government is in the best position to buck financial incentives for corruption because it already has tax revenue to cover costs. Private companies don't have that luxury and are more liable to be corrupted.
This is somewhat naive, but even if we grant the premise, it must be balanced against the built-in job security and predictable promotion schedule which is typical of government bureaucracies.
Less motive to take bribes? Yes, somewhat, although there's no amount of money where someone might not want some more. But a system where doing a good job is strictly supererogatory doesn't tend to lead to satisfactory outcomes either.
