Number theory is cool because it works with items that everybody understands, namely numbers. The only prerequisites you need are basic concepts like squares, even and odd numbers, and similar items. Instead of building up some giant edifice of weird symbols and concepts, like one does in calculus, you're just solving puzzles.

It will also answer 90% of the "hard" problems on the GRE, if anybody cares.

Also, I second not jumping into calculus too quick. It can be frustrating and isn't terribly useful for problems outside of Physics.