Private github repositories are private the same way that facebok messages are private - private from your roommate, not from the people who own the platform or determined attackers.
Nothing stops one from putting encrypted artifacts into a git repo, encryption could be done via hooks. Except this would negate the delta storage, each version would be completely different, and non diffable.
One can just encrypt the .git folder and wrap the git client to handle the encryption/decryption on use. It's always a question where and how well do you keep the keys.
Use a gpg smartcard (yubikey or similar). This is how I store Ansible Vault secrets.
You’re absolutely right about the deltas. Initially I had one secrets file per environment, but as my projects grew I ended up breaking them out to a file per environment-project. Both for storage reasons and because it’s difficult to modify one encrypted file from multiple branches without writing plaintext secrets to disk.