You've just moved the problem from storing the secret to storing the key in that case though. For AWS KMS, you're now paying [0] for the key storage, and if you use a GPG key, you still need to get it into the application somehow.
If you're using a GPG key, you don't necessarily need it to leave your PC. You just need to "re-deploy" the secrets manually when they change from a place that has access to the key
If you manage your own key, you can store it in a password manager or use a USB hardware key to store it
You could also use object/blob storage or your local filesystem to store a config file and optionally apply encryption to that