You can use Signal without the Play Store. Download the apk from Signal's website and it will use a background connection to receive calls and notifications. The downside is that it's heavier on the battery.
They seem to want everyone to use the official app. That's supposed to be fine because of the reproducible builds.
In any case, if we don't trust the official client/app/build, then I'm not sure if a F-Droid release would help that much. If we think there's something funny with the official apk, then we can't trust any part of Signal.
You can use Signal without the Play Store. Download the apk from Signal's website and it will use a background connection to receive calls and notifications. The downside is that it's heavier on the battery.