A compromised relay server can't access the data because it's encrypted.

A meaningful vulnerability would have to be in either the software itself or in the coordination server. That attack surface is the same whether or not you have relays.

You can reduce the attack surface to just the software if there's a way for users to verify keys manually. But again, same attack surface whether or not you have relays.