I find it a problem when people's response is basically "I'm sorry I got caught", and not "I am legitimately sorry".

The part of the apology is realizing that a mistake was made in the first place. Failure to do that does not make you look good.