Depending on what it's serving, and how up to date it is, and who else is on that network and can access the server, and who else can come into that same room when you're not there, and from where you get the software that you install on that server... it might be less trustworthy than you think.

But if that's your standard then the laptop you're connecting from is not trusted either, and then you're not even allowed to use your own keys.

You're allowed to draw sensible boundaries.

With all these recent exploits, I wouldn't even be 100% sure of that.

But if I can't trust even that host, I also can't trust the host I'm working on and which doesn't need agent forwarding to access my SSH agent.

Trusting one host is safer than trusting two hosts.

This is where certs are nice, sign one every morning with a 8/12 hour TTL

Interesting idea. Does need some automation though to make it practical irl.