There is a difference between "allowing it" and "allowing it secretly without your knowledge". There are issues of confidentiality (and potentially NDAs and therefore IP issues) and possibly issues of export depending what you're working on.
There's also an issue of 'what you bought' - this isn't necessarily a contractual thing. I work for a consultancy, if we sell you a project based on the skills and knowledge of a few of our experts and then when the project starts we actually staff the project with a few grads, you'd be annoyed right? Very carefully our contracts don't name individuals to allow us flexibility (e.g. someone gets ill, goes on leave etc.) so we're fine legally but you'd be understandably upset and might want to end the contract. [this by the way, is exactly what Accenture does quite a lot]
There's also an issue of 'what you bought' - this isn't necessarily a contractual thing. I work for a consultancy, if we sell you a project based on the skills and knowledge of a few of our experts and then when the project starts we actually staff the project with a few grads, you'd be annoyed right? Very carefully our contracts don't name individuals to allow us flexibility (e.g. someone gets ill, goes on leave etc.) so we're fine legally but you'd be understandably upset and might want to end the contract. [this by the way, is exactly what Accenture does quite a lot]