Well, I meant it more in terms of x86 to microcode JIT compilers, but in software. So even existing code can potentially be run in exactly the same way they do know, but instead of cumbersome hardware pipelines, these could be done entirely in software where the complexity ceiling is perhaps somewhat higher. This JIT compiler could do the same “magic” what current CPUs do, reorder, branch predict, etc and even more, while in case of a bug those can be fixed without buying a new processor.
Ah, so a Transmeta style approach? That's certainly feasible, in the sense that their technology worked, but I think it would be tricky at best to match the performance of the more standard do-it-in-hardware approach.