If I understand correctly, Guix can do this. Of course the derivation is the final output, so you lose all of the nix modules and nix packages, which kind of defeats the purpose of using NixOS.
I don't think you nessairly need to give up interop.
Kinda like his JavaScript can call wasm modules. The compiler would need to know how to use nixpkgs, but the user could write their derivations any way they see fit.
You can depend on other derivations rather easily, but interoperating with functions written in the nix language is more challenging. (And without nix language interop, basically all of NixOS that isn't nixpkgs is useless).