A pain point you mention is that everyone must run the tailscale client. Have you considered exposing everything on the public internet using something like Cloudflare Tunnels? You can have cloudflare handle auth on their edge network, which mitigates the worry about having to deal with 0-days on a self hosted auth solution. You have a pretty sophisticated directory setup tho so I'm not sure how well this would fit in with the existing infra.
It is something I considered. Ultimately I didn't want to depend on Clouflare (or any other provider) for something as core to my setup as my remote access
But it's a totally valid option, just not one that fit with my preferences
