So have both CUDA and non-CUDA versions bundled, along with the specific CUDA and cudnn version it needs. Still not that large. Maybe a couple gigs at most for everything, and a LOT of headache saved in return which I think is a fine deal.

I mean, very often the Tensorflow version a piece of software wants doesn't actually have a version available for my currently-installed CUDA version. I'll have CUDA 10.1, 11.0, and 12.6 already on my system for other pieces of software, and your software is hellbent dependent on Tensorflow 2.3 and CUDA 11.1 and I'll end up having to install 11.1 in addition anyway just to run your software.

Until your bundled version of CUDA isn't compatible with your installed graphics card.

If so you wouldn't be able to run it anyway and you'd have to go to CPU only, whether or not you bundle stuff.