I could run your site locally with a customized host file so the referers all come from your domain. I don’t think it’s that much of a risk but I wouldn’t want to use a key associated with something that can bill me.

You could use Google actions to build your pages site injecting the api key at build time. It’s stored a repo secret rather than in code. Of course since you deploy the site publicly, the key will still be visible.