The smaller models, yes. I'd bet dollars to donuts that gpt-neo and EleutherAI models outperform most, if not all, of Facebook's.

Check out huggingface, you'll be able to run a 2.7b model or smaller.

https://huggingface.co/EleutherAI/gpt-neo-2.7B/tree/main

As the model weights (even quantized) would be several hundred GBs, it’s unlikely, unless special inference code is written that loads and processes only a small subset of weights and calculations at a time. But running it that way would be painfully slow.

The code is already there: DeepSpeed