Rapid feedback and short iteration cycles are often considered to be an important part of agile development. TDD suggests that you write a test first, and try to make that test the smallest increment that moves you forward. It feels to me that TDD is an application of the principles of rapid feedback and short iteration cycles on the level of code.
If you think using TDD improves the quality of your code, then it's also important for maintaining high code quality, since that enables some of the other agile practices, such as frequent releases.
I don't worry too much about whether a particular practice is agile or not. If you find TDD useful, then great! It's often included in conversations about agile development, but that doesn't mean you have to be doing agile development in order to use it.
Agile warns against building stuff you don't need. TDD means that test provide the justification for feature work. If your feature work isn't making a failing test past, you don't need that feature work. Obviously this assumes that your tests are justified (ideally there is a stack of tests leading all the way to the end user experiment), but that's much easier to get right than justifying untested feature work.
I've seen this claim multiple times and it's never made sense to me. I do TDD on side projects where there's not even a hint of "agile."