The trouble isn't renting those GPUs but it was finding that number of GPUs closely connected to each other for the duration of the training process.

The founder said that this is not quite as possible with most public clouds and that it is easier to buy the GPUs.