This is a good question. I guess what you're actually interested in is the latency between the physical display updating vs the local display updating, right? I just did a quick test under good circumstances using this video (machines on the same lan): https://www.youtube.com/watch?v=OLxY0HDakRk with the physical and remote display side by side. I noticed an approximately 3 frame lag between the physical display and the local / client display update.
Well what I really meant was the time between pressing a key and seeing a character on the screen. It’s easy to measure (except for deciding when the key is pressed. I go for hitting the key fast and timing from when it stops going down) by using some high speed camera (or medium speed, see https://apps.apple.com/gb/app/is-it-snappy/id1219667593 ) and it’s the latency that actually affects how usable something is for typing. The problem is that it’s only really good for comparing things as lots of devices in the middle add latency, eg keyboards can have 15-50ms of latency[1], typical deferred rendering can give you something like a frame of latency, monitors may add latency or just have slower switching times (it takes longer to switch a pixel to an accurate colour), and some pixels at the bottom of the screen will make it out of the computer 16ms (or whatever a frame is) later than those at the top.
For comparison with my (not particularly optimized) set up going over a fast internet connection, I get something like 150ms between pressing a key and the character starting to show up in emacs (it takes a few ms for the pixels to finish switching). 10 frames feels like a lot to me. My best guess is that without changing anything drastic or eg reducing resolution, I might be able to get that down to 100ms which is still looks like a pretty big number (eg a round trip between London and New York is something like 70ms).
Anyway, thank you for investigating this. A few frames from the network seems pretty good, especially with a reasonably large delta between frames in that video)