Not meant as a critique to this book, but the fact that RL-based approaches rarely work for optimal control problems (in, for example, robotics) came as a surprise to me, given the hype and focus on RL [1]. It turns out that model-based methods for optimal control (e.g. linear quadratic control) invented quite a long time ago dramatically outperform RL-based approaches in most tasks and require multiple orders of magnitude less computational resources. Maybe there's some hope for RL method if they "course correct" for simpler control methods. At the moment, it seems like RL for robotics and control lies to the side of "research" and not "engineering."
for linear systems modern control theory is already optimal (?) so given a linear system I'd assume an RL method would just approximate the optimal control method. I think the potential for RL methods is in capturing nonlinearities in actuators like artificial muscle.
[1]: https://www.alexirpan.com/2018/02/14/rl-hard.html