I haven't been following this field, but anyone know what happened to Neural Programmer Interpreters (2015)? It seemed like such a promising direction back then. It showed that a neural network can learn to use arbitrary commands to execute algorithms such as multidigit addition and bubble sort: http://www-personal.umich.edu/~reedscot/iclr_project.html

That seems like a much better demo of using blackbox tools as substeps in problem solving. Is there a reason why it shouldn't work when the blackbox is a more complex function like sympy's eval?