I think of conda as jupyter notebooks. It's ok to experiment with and get something running quickly, but it's a bad idea to use for production code.

Personally I never use either of these tools.

Conda let's you install a specific cuda version directly in a virtual environment with one click though. It's really useful when you have to switch between multiple PyTorch versions and convinient in general imo.

The PyTorch wheel comes with its own CUDA DLLs, they can’t be shared with other libraries but they won’t be your problem to install that way.