> a real-time router that quickly decides which model to use based on conversation type, complexity, tool needs, and explicit intent
This is sort of interesting to me. It strikes me that so far we've had more or less direct access to the underlying model (apart from the system prompt and guardrails), but I wonder if going forward there's going to be more and more infrastructure between us and the model.
Consider it a low level routing. Keeping in mind it allows the other non active parts to not be in memory. Mistral afaik came up with this concept, quite a while back.
The router seems to only apply to the ChatGPT version, not the API, so it’s not really anything new. Gemini already has functionally dynamic reasoning effort.
This is sort of interesting to me. It strikes me that so far we've had more or less direct access to the underlying model (apart from the system prompt and guardrails), but I wonder if going forward there's going to be more and more infrastructure between us and the model.