This is calculation is pretty pointless and the title is flat out wrong. It also gets lost in finer details while totally missing the bigger picture. After all, the original paper written by people either working for Google or at Google. So you can safely assume they used Google resources. That means they wouldn't have used H100s, but Google TPUs. Since they design and own these TPUs, you can also safely assume that they don't pay whatever they charge end users for them. At the scale of Google, this basically amounts to the cost of houseing/electricity, and even that could be a tax write-off. You also can't directly assume that the on paper performance of something like an H100 will be the actual utilization you can achieve, so basing any estimate in terms of $/GPU-hour will be off by default.
That means Google payed way less than this amount and if you wanted to reproduce the paper yourself, you would potentially pay a lot more, depending on how many engineers you have in your team to squeeze every bit of performance per hour out of your cluster.
Reproducibility is a key element of the scientific process
How is anyone else going to reproduce the experiment if it's going to cost them $10 million because they don't work at Google and would have to rent the infrastructure?
But what's the solution here? Not doing the (possibly) interesting research because it's hard to reproduce? That doesn't sound like a better situation.
That being said, yes, this is hard to reproduce for your average Joe, but there are also a lot of companies (like OpenAI, Facebook, ...) that are able to throw this amount of hardware at the problem. And in a few years you'll probably be able to do it on commodity hardware.
This specific paper looks plausible, but a lot of published AI papers are simply fake because it is one of the sectors where it is possible to make non-reproducible claims. "We don't give source-code or dataset", but actually they didn't find or do anything of interest.
It works and helps to get a salary raise or a better job, so they continue.
A bit like when someone goes to a job interview, didn't do anything, and claims "My work is under NDA".
They have changed both the title and the article since it was posted... almost certainly due to comments like these which used to be at the top. Though editing titles should be impossible imo. Editing comments is fine, but if you screw up titles you should be forced to resubmit and not be able to rug-pull an entire discussion.
That means Google payed way less than this amount and if you wanted to reproduce the paper yourself, you would potentially pay a lot more, depending on how many engineers you have in your team to squeeze every bit of performance per hour out of your cluster.