To train small gpt-like models, there's also aitextgen: https://github.com/minim...

minimaxir · on Jan 11, 2023

As the creator of aitextgen, I'm mixed on continuing support since there doesn't seem to be as much demand as expected for small GPT models given the success and cost-effectiveness of GPT-3/ChatGPT, unfortunately.

I still have a few ideas there (including another secret approach at better text generation) but it's hard to determine ROI.

mboof · on Jan 11, 2023

I think what you have created still has great demand. It give devs who do not have the budget or need for the gigantic models, something to train and use for their own specific language tasks.

Not everyone is trying to replicate CHATGPT results for certain tasks.