This is a cool idea. I did give it a spin and found that the fidelity between viewers was not all that true. For example, if I pause the video (as the room creator), the video gets paused for other viewers as expected, but not in the same place. In other words, not all viewers are in sync.
yea so there is a "drift threshold" of 2 seconds, and my thinking for this is that instead of having the frames jump around many times when we go out of sync it would be preferable to allow for some difference between peers to reduce the number of times we need to sync up.
During pause you can use that time to drag the forward people (including host) back to the oldest guest so people can "manually sync" if they ever choose to.