> There exists a higher level problem of holistic system behavior verification
This is the key observation. Strict, code-level verification of every line, while valuable in the small, doesn't contribute very meaningfully to understanding or building confidence in higher level system properties.
I see a future of modules with well defined interfaces and associated contracts. Module level contracts can be formally verified or confidently believed via property based testing. Higher level system behavior can then be proven or tested by assuming the module contracts are upheld.
Module boundaries allow for varying the desired level of confidence module by module (testing, property based testing with variable run time allotments, formal verification).
LLMs can handle each step of that chain today, allowing humans to invest specifically where we want more confidence.
Modeling workflows as state machines is incredibly powerful.
In addition to everything the author mentioned, the constraints of state machines allow a workflow platform to provide a ton of additional guarantees and capabilities around consistency, state propagation, reliable timers, inter-instance messaging, etc.
We built our workflow execution platform [1] around state machines and we've seen great results. We find our workflow code is incredibly simple and easy to understand.
this is exactly how I go about re-architecting overgrown workflows. reduce the workflow to a state machine. It takes forever to untangle what they are wanting to do and what they are doing, but the result is always a more robust solution.
the other benefit of a statemachine is the ability to accurately determine what parts can be collapsed into subworkflows which allows for reuse, replacement, or general modification
Rust ADTs and pattern matching are so much better than other mainstream languages I find that once my code compiles it actually is almost always correct.
The next step is to encode your transition logic in the From impls between the enum structs and you've got yourself a first-rate state machine.
Those languages have other issues that make them weird to use. Rust has its fair share of weirdness as well, but most of that weirdness is just a direct side effect of the way that it is.
It's very similar to a DAG you'd make for a workflow but not necessarily acyclic and events can be sent from external sources as well as internal processes.
States can be grouped into parent states, which gives you some additional nice things like parallel states and also a totally reasonable way to zoom out and hide info in visualizations.
When you have a pure function or a single effect, that should definitely just be a function rather than its own machine. Sate machines are most useful for orchestrating those effectful functions and for situations where you want to accept external events. So you're still writing regular code but you're writing it in small functions that get invoked by the state machine as needed.
That is, state machines don't replace code. They're just a really nice way to organize it.
Great point! State machines are a really nice way to orchestrate multi-step LLM invocations.
They're also a great way for LLMs to produce code. It's human-readable so you can vet that it's doing what you want it to and it's high-level enough that the remaining bits to fill in tend to be small functions that AI can easily generate.
> There exists a higher level problem of holistic system behavior verification
This is the key observation. Strict, code-level verification of every line, while valuable in the small, doesn't contribute very meaningfully to understanding or building confidence in higher level system properties.
I see a future of modules with well defined interfaces and associated contracts. Module level contracts can be formally verified or confidently believed via property based testing. Higher level system behavior can then be proven or tested by assuming the module contracts are upheld.
Module boundaries allow for varying the desired level of confidence module by module (testing, property based testing with variable run time allotments, formal verification).
LLMs can handle each step of that chain today, allowing humans to invest specifically where we want more confidence.