Well, the limitations are the usual limitations in mathematical analysis. Despite being differentiable, if your function has a very complicated structure you will get stuck in local minima with high probability.

People are referring to limitations regarding expressiveness.