But the problem is to display something for a moving angle, not display something for everything. As best I can tell, it is a different problem.
I want to restate what I'm hearing so you can correct me. You are saying something like "Gee, if we could show any retinally-matched conic the user's pointing their eyeball at? To do that we'd have to have the entire scene rendered anyway"
I'm saying you can't do it now. It hurts when I do that. So stop doing that. Instead of trying to render stereo scenes quicker, try to render a moving conic real-time quicker. When the codec runs, it's optimizing areas of a frame changing over time. If instead it optimized possible vision movement paths, you'd end up solving the framerate and resolution problems. Then you could concentrate on optimizing the codec in a different way than people are currently trying to optimize it. It couldn't hurt, and there may be opportunities for consolidation if you look at the problem as visual-movement-path-rendering instead of frame rendering. I don't know. I just know it's a problem now. Set your constraints differently and optimize along a different line. Sometimes that works.
Does that explain the differences here? Or do you want me to start spec'ing out what I mean by retinal-path codec? At some point this will a bit over the top.
ADD: I'll say it a little differently. Our constraint now is "how fast can you render this frame". Codecs are built for rendering frames on screens where people watch movies and play games. But that's not the world we're trying to solve a problem in. They may be configured the wrong way. Instead, require that anything the eyeball is looking at should render consistently in, say 5ms.
This sounds like the same thing, until you realize that the eyeball can't look at everything at the same time. Different parts of the image are temporally separated. So if I update the image in back of your head once every 200ms? You're not going to know. As far you're concerned, it's all instant.
I want to predict the temporal cost of focal path movement, then optimize the stream based on those temporal "funnels" I guess you'd call them. This is in opposition to looking at segments of the screen all equally and optimizing across frame changes. I don't care about frame changes. All I care about is how fast the eyeball can get from one spot to another -- and that's finite. If I can create the image I want instantly wherever the eyeball is looking, framerate and resolution issues are a non-sequiter. (This would also probably scale to more dense displays easier, but I'm just guessing)
