I posted a screencast on Reddit earlier in the development process with audio demonstrating the text-to-speech feature. The UI has changed a bit since then, but you can hear what the voices sound like:
It's a self-hosted UI for ChatGPT right now, but my primary goal is to build a good open source chat interface that can be adapted to open source chat models as they become available.
Integrating with Alpaca, Llama, ChatGLM, OpenChatBox and whatever comes next should be straightforward once people figure out reliable and fast methods to run the models locally.
I think if you integrated with Llama, this repo would be wildly popular. I downloaded the weights over the weekend and decided I didn't want to spend my free time working on an acceptable UI.
OpenAI hasn't published any information about the size or hardware requirements for running ChatGPT. Reading between the lines, the default ChatGPT Turbo model seems to be significantly smaller than GPT-3 (it's a distilled model), but probably still heavier than the Alpaca and Llama 7B models people are running (very slowly) on their single GPU computers this week. You'd probably need multiple A100s to get comparable performance to the ChatGPT API.
Does the llama code that dropped leverage the GPU at all? On an M1 it appears to just run on as many CPU cores as you want to throw at it. The 65B heats up 8 cores real nicely, and it's slow, but I imagine it would be a lot faster on the GPU.
8 starts and runs a bit faster for me if plugged in and before the fan kicks on and the CPU starts throttling. Once that happens it's probably better to stick with 4.
Totally! I found ChatGPT to be more helpful in this use case since it outputted full code snippets instead of generating it line by line. The context was also extremely valuable in making iterations (like "make it work for mac")
