In case you were looking for a similar text-based integration of a layout engine with your tool there is a textual DSL for ELK called ELKT:
http://rtsys.informatik.uni-kiel.de/elklive/
One could certainly feed that into a binary running the kernel similarly to how a DOT file is fed into graphviz.
https://github.com/eclipse/sprotty
This integrates with XText, ELK and the VS Code LSP and allows you to generate diagrams for your own DSL!
In case you were looking for a similar text-based integration of a layout engine with your tool there is a textual DSL for ELK called ELKT:
http://rtsys.informatik.uni-kiel.de/elklive/
One could certainly feed that into a binary running the kernel similarly to how a DOT file is fed into graphviz.