> It seems like this would be a very useful starting point for LLM quality engineering, at least for simple inference.

Interesting. Can you elaborate on this? You mean this test can function as a metric or is it just an evaluation for applications?