Ceres is great for sparse, structured non-linear-least-squares problems (that is, problems where levenberg-marquardt is applicable). I believe it was initially designed for bundle-adjustment problems arising from structure-from-motion applications.
Ceres also does work for dense problems and general convex optimization, but it doesn't provide as much of an advantage over other packages (that is, you could just call into any existing l-bfgs implementation and wouldn't benefit much).
For some other problems, Ceres doesn't scale or can get in the way. For example, if you have a single value to maximize and a large number of variables, then ceres' auto-diff system won't work (ceres::Jet is based on "forward-mode" automatic differentiation, storing partial derivatives on the stack. If you have a high-dimensional gradient, it can exhaust stack space). On the other hand, Ensmallen doesn't seem to come with any auto-diff functionality.
Ceres also does work for dense problems and general convex optimization, but it doesn't provide as much of an advantage over other packages (that is, you could just call into any existing l-bfgs implementation and wouldn't benefit much).
For some other problems, Ceres doesn't scale or can get in the way. For example, if you have a single value to maximize and a large number of variables, then ceres' auto-diff system won't work (ceres::Jet is based on "forward-mode" automatic differentiation, storing partial derivatives on the stack. If you have a high-dimensional gradient, it can exhaust stack space). On the other hand, Ensmallen doesn't seem to come with any auto-diff functionality.