I'm in the exact same situation as you rn. Freshman year of college and really interested in LLMs. Did you decide how you're gonna go about it? Maybe we could share resources
I'm focused on finals at the moment but I'd be more than happy to share what I've been looking at so far (I plan to look into this a lot more in the summer).
When it comes to the theory, 3blue1brown released some really nice videos that solidified a lot of my current understanding, especially with the attention mechanism. I think I am also going to do a mixture of reading papers/watching youtube videos on things that are interesting (ex. Qlora for fine-tuning or diffusion models for image generation) and trying to build out a simple implementation myself and see where that takes me. Maybe I'll start out with the Karpathy nano-gpt videos but try and do it with a different data set.
But for people like myself who lack the math background, data, and compute to be able to train very strong LLMs, I think it is also a good idea to try and build some projects/apps that use a fine-tuned LLM, or just call the OpenAI API.
I'm still a bit lost myself, but in 2 weeks when I'm done with exams, I'm more than happy to keep exchanging resources with you.
Perfect. I have my own finals in a couple weeks haha, planning to look more into this and possibly build a project over the summer. 3B1B's videos sound like a good resource, and the nano-GPT videos look really useful too. I'll share what I come across as well!
