One of the ways I suggest people think about AR and VR is that -
One of the only significant differences is whether or not the skybox is filled in.
Broadly the technical requirements of the object tracking, 3d visuals and user interactions will be very very similar.
I suspect you'll end up writing applications with basically the same stack once we get there.
Useful applications across both VR and AR are more likely to be holographic in the sense that they are free standing in 3d space and designed to inhabit whatever 3d space WITH other applications.
The current VR "draw the whole world and totally control the ui" is this fields DOS era.
The desire for simplicity implicit in your statement is admirable. Unfortunately it is so simplistic that it ends up getting it wrong, especially if you don't specify which technology emulates which.
For example if you were to say: "To get idealized VR, take idealized AR and make the display opaque." that is a completely different technical claim than "To get to idealized AR, take idealized VR and make it see through." The reason being, the maximum limits of VR are lower than (or rather different from depending on your view) the maximum limits of AR.