So.. Fun fact, Osterhaut group(ODG) has something just like this and it was working as Apple demos showed and it was out 7 years ago. I tried it on and for me it was the biggest game changer I felt since the 2007 iphone. Microsoft bought the patents from odg and the company ran out of money trying its own strategy. Sad story for them but I hope apple really does this right as it will be an absolute game changer.

https://techcrunch.com/2017/01/03/odg-unveils-its-first-cons...