TensorFlow and Theano are quite popular, and they're all about expressing differentiable computations in a "dataflow"-based framework. It might be a simple case of needing to write some support code to make OP's desired use cases more straightforward when using these frwmeworks.