Matrix shifts much of the burden to the server so that clients can be simpler (except for e2ee, which has to be done clientside).

Which if true is still a burden that prevents simple self-hosting. You can (& I did briefly) install an XMPP server on a 10-year-old smart phone with postmarketOS on your network & have a reasonable experience.