I worry that this would give a false sense of security. Definitely it's better to have something that will highlight problems than to have the problems go unnoticed, but there's no way that something like a Penetrator Monkey could be comprehensive, and less experienced people might fall into the trap of believing that the monkey is as competent as an actual attacker.
Something similar happens with code coverage. It's pretty easy to write tests that provide 100% code coverage yet still miss basic bugs. 100% code coverage gives you little more guarantee than "this code doesn't crash for naive inputs" unless the tests providing the coverage were well written - code coverage gives you no introspection to how good the tests might actually be.
In short, I agree that this would be a cool + useful tool. But it should come with a disclaimer that it's not a substitute for more thorough testing + verification.
Something similar happens with code coverage. It's pretty easy to write tests that provide 100% code coverage yet still miss basic bugs. 100% code coverage gives you little more guarantee than "this code doesn't crash for naive inputs" unless the tests providing the coverage were well written - code coverage gives you no introspection to how good the tests might actually be.
In short, I agree that this would be a cool + useful tool. But it should come with a disclaimer that it's not a substitute for more thorough testing + verification.