There are areas of control theory where you can learn the dynamics ("adaptive control"). The advantage over RL is that in control theory, you generally assume the dynamics are described by differential equations (sometimes difference equations), not by Markov decision processes. MDPs are more general, but basically any physical mechanism you're going to control doesn't need that generality.

There is a surprising amount of structure imposed by the assumption that the dynamics are differential equations, even if you don't know what the differential equations look like. As a consequence, adaptive control laws generally converge a lot faster (like, orders of magnitude faster) than MDP-based RL approaches on the same system being controlled.

The other advantage is that you can prove stability and in some cases have an idea of your performance margin with control theory. THis is important if you eg want your system to receive any sort of accreditation or if you want to fit it into the systems engineering of a more complex system. There's a reason autopilots don't use RL, and it isn't that RL can't be made to work. It's that you can't rigorously prove how robust the RL policy is to changes in the airplane dynamics.