Last year I wrote a simple system using Semantic Kernel, backed by functions inside Microsoft Orleans, which for the most part was a business logic DSL processor by LLM. Your business logic was just text, and you gave it the operation as text.

Nothing could be relied upon to be deterministic, it was so funny to see it try to do operations.

Recently I re-ran it with newer models and was drastically better, especially with temperature tweaks.