I don't like the idea, but I've noticed that in my city, if you're heading into the city centre you're absolutely out of luck as far as traffic is concerned, but if you're heading the other way, it's extremely light. So you wouldn't so much be "doubling" congestion as just applying it to both in and out flow equally. I don't know if that would actually cause more difficulties or not.
Not everybody is going to the exact same spot downtown, and there's also some actual reverse commuting, so there will be some overlap between the to-work and from-work trips, especially where there's the most congestion (I e. Downtown).
