GPT4 is expensive to run, even more expensive to finetune, and for all practical purposes can’t be run offline (because the model is too big to run outside of a huge data center). Evaluation latency is also an issue for many usecases, and you have to share your query with openai, so you can’t run sensitive queries. The output is also controlled/censored by OpenAI.
Here’s a few usecases that I wouldn’t want to use OpenAI/GPT for
- Advanced autocomplete for texting and private communications
- Querying sensitive document databases like emails
- Traveling in low connectivity areas
- Politically incorrect usecases (generating erotic content for example)
IMO, the main reasons are (but are definitely not limited to):
- You can fine tune these models for very specific tasks, which GPT-4 might not be as good at.
- Open source models are free. You can use them as much as you want without worrying about a $xx,xxx bill at the end of the month which makes tinkering with them easier.
- Smaller models like this can run on consumer hardware, even phones, and can run offline.
- Privacy and not having to abide by a third parties terms. You don't have to deal with "As a large language model...", especially with uncensored models.
1) people can run a 1.6B model for free on consumer hardware
2) any model that's run on computational resources you are owning or leasing will have more privacy than an explicit cloud offering. running completely on your own local hardware will be private. this means you don't have to think twice about asking the LLM about the proprietary code or information you are working on.
3) smaller models gain the performance improvements from all the other improvements in interpreters and quantizing, allowing for even more consumer friendly offline use
4) oh yeah, offline use. could expand use cases to having LLM's baked into operating systems directly, including leading phones
5) showing what's possible, pushing towards the benchmarks of the best possible model while using less computational resources. this also makes the hosts of the best possible model realize that they could either A) be using less computational resources and increasing the bandwidth for their users B) further improve their own model because of competition. Basically if ChatGPT 4 was using similar improvements in technology across all areas of reasoning/whatever, there never would have been a rate limit on ChatGPT 4.
6) more demand for other computational resources. Nvidia is backordered till maybe Q2 2024 right now. If people realize AMD or even their ARM chips can offer same performance with the right combination of hardware and software, It alleviates pressure on other ventures that want computation power.
Imagine being on Mars and running on a small PV panel and needing to code a bugfix in your oxygen supply system through the wire with Microsoft Earth(tm) or something
moomooskycow, thanks for such a thoughtful response. I've read it and re-read it.
The vast majority the last decade has been solopreneuring. I've had a stint at larger companies, but usually as a consultant.
It's likely immature of me, but I gravitate strongly towards "meritocracy" and the simplest purest form of that - for me - has been to work almost entirely alone. And I am acutely aware of all the things I'm giving up in an idealized situation: working alongside great people who are smarter and more passionate than me, cool benefits, money, etc.
I deeply believe that I'd be incredibly valuable to the right company, for the right job, and that the salary and social benefits and such could change my life and lives of my family (and future family!).
But, I've failed as far as finding that just right job.
Here is immature Mike's train of thought regarding this, perhaps someone else can relate?:
"I don't want to go do the thing that so many of my peers have done: get a job at a largish tech company, work super hard for 6 months and then fade into a passive state of not really doing anything besides fielding slack notifications and collecting large pay checks. Idk it just feels so lame. It feels immoral. But those people seem to be killing it, they have the money and their girlfriends feel secure and stable, they have friends and get sent on cool business trips. Maybe I'm the one who has it all wrong, and I'm going to slowly age into irrelevance with nothing but my misplaced pride...etc etc etc"
-----
All that to say, perhaps it's fear of rejection that is keeping me from reaching out and finding that "just-right" job that leverages my "economically valuable" skills.
Your words have struck a chord, and I will make the aforementioned serious effort towards finding a compromise between earnings and outlets.
fully agree with pretty much everything you're saying. the wrong job can be pretty hellish and should be avoided with extreme prejudice. but i think there are options between total avoidance and complete immersion that achieve reasonable compromise between earnings and fulfillment.
part-time consulting work seems like a pretty solid way to make good money and maintain lots of free time, but you've done it and it didn't stick. what drawbacks outweighed the pay/flexibility benefits in your experience?
in my experience working for smaller companies and startups is also way better than working for big companies. i understand why people work for big companies and all the perks they provide (sometimes i just browse openai job postings and read all the compensation sections), but you're more flexible and have more impact at small companies and that's huge. especially for entrepreneurs: more flexibility means it's easier to build stuff in your free time. and more impact means it's easier to creatively express yourself on a larger scale by leveraging company resources.
the other point i want to make here though is: if you've been solopreneuring for a decade and you've built multiple revenue-generating businesses on your own already, i would not bet against that track record. if you've got runway to last you a while, maybe just maximize your free time and your wellness. you'll probably start something new.
Ted, Thanks! Come to think of if, something along the lines of bartending or waiting tables does check both the [x] physical and [x] social boxes! Good idea!
First you'll have to do a little bit of research to find local construction companies that seem professionally run and are actively hiring entry-level construction crews. Use the phone book, google, yelp, or local conventions or trade groups. Also ask people you know if they have any connections to local construction companies and would be willing to intro you.
Second, just be straight up with them - you've been working a sedentary desk job plugged into a computer building software apps for all of your twenties, and now want to spend some time learning how to build real world things and doing some actual physical activity. You're not sure how long you plan to do it, but at least long enough to learn the basics of the craft of building houses and similar sized buildings, maybe six months or so. You're inexperienced at construction, but are smart, have an engineering and problem solving mentality and experience, and can learn fast.
With luck you'll find one willing to take you on and assign you a crew that will mentor you. They'll start you on easy tasks, teach basic things like professional hammering technique, how to use power tools safely, etc. All things you can get a jump on just by searching on Youtube.
That's the gist of it. There will probably be more than one firm willing to take someone smart and reliable and train them up, even if you're up front about not knowing how long you intend to do it.
PS - it also occurs to me another interesting kind of company to work for would be a local machining and manufacturing company, if any are in your area. There are all sorts of cool videos on Youtube about precision machining and manufacturing that might appeal to a programmer. Same process as above.
I work in structural steel fabrication and construction, qualified boiler-maker here in Australia (Tasmania), 20 years in the game and 5 years in IT (up to remote-hands for the likes of Akamai and Google), and now driving a 4kW fiber laser cutter.
SkyMarshal's advice is sound. If you turn up, wearing steel cap books, an something resembling the appropriate clothing, and say something like "I can start now", I think you're in with a chance.
Here in Australia, particularly Tasmania, it's been so hard to find people willing to turn most days of the week I'd say you could get a job here tomorrow (okay, maybe not Saturday) for any number of construction companies.
My contact details are in my profile. I could potentially flick you some drawings for you to study so you'd be able to walk in and say "I can read a fabrication drawing and a marking-plan".
Go to a local job center. You will see tons of postings for plumber assistant or factory worker or call center worker. Talk to a councillor and they will place you.
Another option is temp companies.
Get your forklift license or trucking license if you want more of a career.
Adding to what Spice said, the starter equipment is:
1. Steel-toe boots
2. Safety glasses
3. Professional grade hammer *
4. Professional toolbelt
Show up with those four things and they'll take you seriously.
* practice with the hammer a bit first. There's actually an art to holding the hammer at its Point of Balance (PoB), flipping it with your wrist not your arm/elbow, and letting its weight and momentum do most of the work instead of your arm muscles. I can't find a video on this on YT atm, but will post one if I find it.
