You need some sort of "terminal server" to maintain the session and station state, and I think screen's server does that thing quite well - I don't see a good reason to integrate the "terminal server" with the OS. You could split "UI session management" and "process management" somewhat (e.g. have Docker manage the process swarm and pass stdio to screen), but that's even more splitting.
The uglier part is that screen needs to communicate the UI state to a terminal emulator. Theoretically, you could have screen expose an API and have a client-side GUI client directly display a UI, and that is even probably a good idea (I'll like a Qt tmux client that can talk to a remote tmux with JSON over SSH), but nobody bothered to implement it yet, and a "VT100 over serial" client can be used without running new software at the "user endpoint".
PTYs are "terminal servers"—they're just sucky ones from a time when a terminal had fewer expected capabilities. But, indeed, you can connect a TTY directly to e.g. a serial modem, and then the person dialling your modem will get a TTY and login(8) will accept(2) on that descriptor and serve them a login prompt.
Telnet was just a protocol for serving whatever you'd get by connecting to the serial port, but multiplexed and over IP. Users got PTYs for their "session" and then login(8) was [usually] invoked.
Then, we got IPSec, to layer your telnet PTY tunnel inside an encrypted tunnel.
But SSH kind of turned things inside-out: it encapsulated a PTY inside an auth'ed login session which it managed itself, inside a carrier (e.g. TCP) which it also managed itself. It threw out the whole PTY-as-session concept (with PTY-managed-by-a-shell job control with Ctrl+Z, multiple PTY's-stdout writers to watch background jobs, etc.), creating its own sessions in the form of SSH "channels", and allowing the user to invoke arbitrary programs on the remote (with its own sshd_config specifying what is allowed) rather than handing off to the centralized PTY "terminal server" of login(8) to decide what to do with this session based on its origin device.
Until SSH, screen(1) and tmux(1)—and script(1) and nohup(1) and watch(1) and... were all just "enhancements" you'd run on top of a PTY as basically different shells that manage the PTY's jobs differently. Like different desktop environments manage windows differently. They might use additional PTYs to do this, but it was an implementation detail, and even then mostly only a hack—added later, not in the original designs—to ensure libcurses programs would work inside these.
But with SSH, since you lost the single encapsulating PTY session-group concept, we now saw things like tmux(1) and screen(1)'s PTYs as multiple root sessions. Now it was more like SSH was the terminal server, and screen/tmux was virtual desktops. Completely different paradigm, losing the thread of what PTYs were "for" in the process.
> PTYs are "terminal servers"—they're just sucky ones from a time when a terminal had fewer expected capabilities. But, indeed, you can connect a TTY directly to e.g. a serial modem, and then the person dialling your modem will get a TTY and login(8) will accept(2) on that descriptor and serve them a login prompt.
In a way, but ptys both don't do the "UI state" thing well, and don't do the "server" thing well - you can't really "take over" the controlling side of a pty easily.
> Now it was more like SSH was the terminal server, and screen/tmux was virtual desktops.
In the normal use of SSH+screen (`ssh -t foobaz screen -r`), SSH just tunnels the VT100-over-serial over the network, and provides the needed authentication and routing.
No, I was talking about two separate time periods. Sysadmins were defining IPSec transports to protect port 23 traffic between mainframes and their terminals for years before SSH appeared. Allowing Telnet sessions to traverse the public Internet was one of the original motivations for IPSec.
The original point of SSH was that it didn't expect the client and the server to be part of the same administrative domain—i.e. it didn't expect a sysadmin to have been involved in advance to pair the devices and define a tunnel. This is why SSH's auth didn't, for the longest time, support any kind of centralized certificate-based PKI: it was for the cases where there wasn't a central authority in common, but rather just two peers that happened to want to talk securely—perhaps without the consent of their respective sysadmins; perhaps even despite the policies of their respective sysadmins.
SSH's "on the spot, look at my keys" auth has become so ubiquitous that sysadmins seem to have forgotten that, most of the time, they do control the administrative domain both devices are under, and can in fact just pair them using e.g. client certificates issued by a private X.509 CA. Fingerprint-based auth should be seen as a last resort, not a first resort; and, likewise, Telnet shouldn't be seen as a last resort, but rather a core tool serving a core purpose in a layered design: enhancing a "local-equivalent" (LAN or VPN) IP link into being a PTY session, upon which any number of things can be run. (For example, "dial-in" MUDs, of which a number still exist.)
The uglier part is that screen needs to communicate the UI state to a terminal emulator. Theoretically, you could have screen expose an API and have a client-side GUI client directly display a UI, and that is even probably a good idea (I'll like a Qt tmux client that can talk to a remote tmux with JSON over SSH), but nobody bothered to implement it yet, and a "VT100 over serial" client can be used without running new software at the "user endpoint".