All the time. I don't really understand how you don't. Requirements change constantly, and the software's structure has to change with them, or become a big ball of tangly interdependencies.
Refactoring is not a big deal. If you keep your code clean, each individual refactoring is really quick - moving a method, renaming a trait to better fit its new role, ... If you can do these kind of things with a minimum of friction thanks to your IDE, you will do them, and you will not run into the situation where you have accumulated loads of technical debt and need two weeks to get everything into a maintainable state again.
I'm not sure what to say beyond that i personally never felt the need for this. One of the IDEs i use does have several refactoring features, but the only one i ever used was the identifier rename and i probably used it around 2-3 times the last five years.
It is a useful feature when you need it, but personally it wouldn't affect my choice of tools at all. After all i could just load an IDE that has it in the rare case i need it and then continue using the other tool i prefer, it isn't like modern computers can only run a single program at a time or anything :-P.
Refactoring is not a big deal. If you keep your code clean, each individual refactoring is really quick - moving a method, renaming a trait to better fit its new role, ... If you can do these kind of things with a minimum of friction thanks to your IDE, you will do them, and you will not run into the situation where you have accumulated loads of technical debt and need two weeks to get everything into a maintainable state again.