Unless of course you are writing software that goes into a pacemaker, in which case I would want that tested like crazy for every possible scenario you can imagine including automated fuzz testing. Same goes for databases and quite a few other classes of software.
In short, use common sense.
Personally I love TDD. Not because of the tests, but because the resulting code IS testable, and generally testable code is maintainable and easy to read.
If I knew one pacemaker was developed with TDD and one without it I would probably go for the one without it. Most programs are fine on the Happy Path, and TDD seems like it's focused on expanding the Happy Path vs actually writing correct software.
This could just be internal bias, but without TDD programmers more focused on the possible error conditions. I would rather see:
If ((a/2-1) + (b/2-1)) > (largestInt/2-5))
... do something
vs.
A large try catch block.
Arguably the second is just as safe, but it's the test's you don't think to run that tend to end up as production bugs and good tests require a level of paranoia which is more important than methodology IMO.
In short, use common sense.
Personally I love TDD. Not because of the tests, but because the resulting code IS testable, and generally testable code is maintainable and easy to read.