Nuclear-thermal rockets still require a "fuel tank". They heat up a reaction mass (typically LH2) and then throw it out the back. So they can't burn continuously, they'd run out of reaction mass.
You're thinking of nuclear-electric, which uses an ion engine just like solar does. It can burn continuously, but it has incredibly low thrust.
edit: (Note that I'm basically saying the same thing as baq, just in a different way)
The word you're looking for is propellant. In chemical rockets, the fuel and propellant are the same thing (to be precise, the combustion byproducts of the former comprise the latter). In nuclear fusion and ion thrusters, they're different. In all cases, Newton's second law means that you are using up propellant to generate an equal and opposite reaction in the direction you want to go, and thus the amount of "going" (delta-v) you can do is limited.
"Propellant" - Thank you! I knew it wasn't technically fuel, but it's still being expelled (as you mention, if you don't expel mass with a different velocity relative to the rocket, then you won't be able to accelerate a rocket)
this requires Isp in the high 10-100k, likely sub-1M range (think particle accelerators) instead of sub-1k (think very high temperature nuclear gas heaters).
Some of the better ones that are actually in use get in the range of low thousands, like ~3k or so, but keep in mind that the thrust is so low that even just getting to the Moon takes months. They're only suitable for unmanned craft, and even then, only after you get into orbit and if you don't care about taking potentially many years to get somewhere outside the local SoI.
