"...Anthropic, an AI safety and research company, has selected Google Cloud as its cloud provider. The partnership is designed so that the companies can co-develop AI computing systems; Anthropic will leverage Google Cloud's cutting-edge GPU and TPU clusters to train, scale, and deploy its AI systems."
"AWS will become Anthropic’s primary cloud provider for mission critical workloads, providing our team with access to leading compute infrastructure in the form of AWS Trainium and Inferentia chips, which will be used in addition to existing solutions for model training and deployment. Together, we’ll combine our respective expertise to collaborate on the development of future Trainium and Inferentia technology."
“Azure will become Anthropic’s primary cloud provider for mission critical workloads, providing our team with access to leading compute infrastructure and Microsoft’s expertise in AI cloud services”.
How it works, is Amazon just paid a bunch of money just to be able to say that. This press release was written into the contract they signed, so us hearing about it isn't just happenstance. What happens under the hood is secondary, really.
Possible but would be pretty misleading stat if so. It sounded to me like statement meant it was for generative AI needs. I know midjourney uses TPUs but for training but don't really know who the other generative AI unicorns are besides OpenAI.
Depends on what you mean by works out. They are planning on doing huge training runs the coming years so they sure can make use of compute from both Google Cloud and AWS
Amazon as a corporate investor - Of course a lot of this is a futures contract on cloud compute. This indicates how much the leadership here thinks the compute will be problem. Money comes way cheaper outside of the big cloud providers (they also know the importance of compute and pull their leverage). This is not a sure bet. While true AGI is probably sitting behind a huge amount of compute, the “products” that are catching on right now are very much on lower end of the spectrum for required model performance. Small models are cheaper and can run on commodity compute. It’s not entirely clear to me that this is a financially sound bet..
Timing - This is an interesting time to do so. That indicates that the company feels that it’s shown some of its best work and right now is the time to bank and on that (so as to make the leap to the next big breakthrough). Openai did so on the heels of ChatGPT. This is somewhat discouraging, because outside of the context length hackery, Anthropic doesn’t have much to show as a differentiator. At best they’re a me-too startup set on the path to be acqui-hired by Amazon when the VC money subsidizing the compute drains up.
Structure/Size - there was a lot of information about the structure of the openai deal. We’re not so clear on what’s happening here. One of the big questions is valuation. Making a similar promise to openai (ie 50% of profit until 100b) would put the valuation of the company in to 10s of billions. Note that this is a very different proposition than a year ago. In navigating the “product maze” we’ve realized that there aren’t that many killer products. Most enterprises are throwing spend in this direction because the board requires you to have an “ai strategy”. At best, we’re talking about capturing all the VC money that’s going into companies with a new angle on knowledge management/search. As I mentioned above, that’s something that’s getting severely commoditized at the bottom of the market. The prospects here are pretty grim .
There really isn’t a product in text or other modalities that shows me this is a $1T dollar market in 10 yrs (as the valuations would imply).
To play devils advocate, even if there were some killer products, value capture seems particularly tricky. There just aren’t any business models that could sustain a market mass of this size.
Lots of cool demos and productivity software (mind you with a very narrow definition of what productivity is).
Unfortunately, the upper echelons of society run on bullshit. And so that's exactly why generative AI is revolutionary. The killer app is in separating dumb billionaires from their billions, just like crypto's killer app was. And just like crypto eventually filtered down to separating dumb retail investors from their thousands, generative AI may eventually filter down to generating scams & spam that separate dumb retail consumers from their thousands. That's why the dumb billionaires are interested in it.
A more logical solution might be to not run society on bullshit, but that seems to be beyond the capabilities of most humans.
From personal experience, I’ve saved hundreds of hours with ChatGPT already. It can piece together large neural nets in PyTorch that would take me an hour or more to get right, or annoying pieces of code which I don’t feel like correlating the disparate pieces of online info for. Of course I will review it and unit test it to make sure it works, but I honestly make more mistakes integrating it in my larger project than ChatGPT does in the code itself. And this is on the free version.
There are limitations of course but I’ve been impressed more often than not.
It means that there will be higher productivity expectations based on using this. If you can't use it very well you won't be as rewarded, you probably won't be compensated as well.
We have a long human and also programming history that as we gain capabilities, the expectations of that being a standard thing grows.
>A more logical solution might be to not run society on bullshit
And this is why you're poor. In fact, depending on your definition of bullshit, if you did not run society on bullshit there wouldn't be billionaires at all. Once you reach the level of technology that can provide for everyone, what need is there to make more?
>The reasonable man adapts himself to the world; the unreasonable one persists to adapt the world to himself. Therefore all progress depends on the unreasonable man.
> Once you reach the level of technology that can provide for everyone, what need is there to make more?
Because you can never reach that outcome no matter what your technology is.
Your premise is so vague as to be useless. Provide what for everyone? How much for everyone? Who decides how much for everyone? Based on what? Elected by whom?
If you actually dig into any of that what you'll find is that it's impossible. Unless you use a hyper violent authoritarian model that dictates every minute of every person's life. Otherwise, one person will always have different needs than another person, without exception (and that's so varied we could be here all day listing the differing ways, from freak health situations to genetics to activity choices).
And that's just on matters of basic needs. Then we get into luxuries, any and all things people want beyond basic needs. And then you have to debate the extent to which people require some manner of luxuries beyond basic needs in order to actually continue to exist in a healthy manner. And then you have to get into who decides on that, who dictates which people get which luxuries, if indeed it's decided people require some level of luxury in their life.
And that's just scratching the surface at a minimum for why the premise: once technology has provided for everyone - entirely implodes upon any inspection at all. It never can, it never will. Human desire is infinite up to the point of expiration. That includes the desire for new experiences (can people live without new experiences? doubtful, certainly they'll live horrific miserable lives without such), which is a gigantic envelope of things unto itself, beyond the basic needs category. Who decides what new experiences everyone has access to? How much allocation to that? And on and on and on it goes.
What are you even talking about, of course it can. If humanoid robots become a viable means of production, a lot of human labor could be replaced. You continue to use the current economic system to allocate.
>Human desire is infinite up to the point of expiration.
No, the only thing infinite is human greed, the difference between greed and desire is desire can make concessions for the needs of others, greed will burn down the world to get exactly what it wants.
> And then you have to get into who decides on that
As technology concentrates power, it will be the authoritarian leader. Putin is a great example of this. Greed driving him and Russia to ruin.
Do they have a moat though? When MS Office integrates a more powerful model than is even available to Jasper right into the word processor, what happens?
I think there are some killer products, but the surface area here is not that big. I can definitely see how we can improve on all of the above, but at the rate things are developing right now, you're looking at smaller and specialized models as the underlying infrastructure. Keeping the conversation focused on Anthropic, unless there is promise of some specific functions outside this range that require big powerful models, they're in trouble.
There is a world in which the dust settles and the current "era" of AI doesn't actually result in a significant amount of productivity/value creation and capture thereof. Everything rests on the current assumption that emergent behaviors and some semblance of consciousness can extrapolate infinitely. You have to believe that in order to justify the investments we're currently seeing in some of the big players.
If you can train a machine using video, and get it to repeat what you have taught it...and then manufacture millions of them - you have an (almost)-zero-marginal-cost revolution just like you had for software development.
Plus, it's replacing $$$ humans. 4X the productivity while paying for itself in a few months.
>If you can train a machine using video, and get it to repeat what you have taught it
I might be wrong, but from what I know about Transformer models and neural nets in general, what you are describing sounds a lot closer to something like an AGI.
When you say “train a machine” here it sounds a lot closer to “teach a machine” rather than “finding weights and baises for an existing function”.
I dont think AGI is required. But multimodality for sure and some piece to plan/execute actions. Prompting techniques now, would land you clear tasks, validations and such, given some objective "replace table leg". A unit to process this would be required.
ChatGPT is the killer product for text. It has rough edges, but in a lot of cases it is far better than Google. Maybe it won’t kill Google but it will hurt. If you think it hallucinates too much, well I have bad news for you about the future of content on the open internet too.
I’m pretty bearish on all of the other ChatGPT API call apps: I don’t see how limiting ChatGPT to a specific use case is adding value for me.
If you were to place all “available money” on a continuum, at one end of the spectrum you’d have the “dumb opportunistic money” and at the other end you have “strategic/partnership capital”. The money is very cheap if it comes from people that want to jump on the bandwagon of fomo. It’s more expensive when it comes from strategic partners that know they have something to offer (and when I say cheaper I mean literally how much $ you pay per preferred share).
I would consider most professional investors in AI somewhere in the middle of this continuum. The big clouds have the rare and coveted GPUs which are the lifeline to AI companies. That gives them way better terms than what a VC firm would get.
I've been using the claude model over the past months and I have to say I usually prefer it over ChatGPT.
Some things that impressed me particular are:
* It can often give you working urls of images related to your query. (e.g. give me some images to use with this blog paragraph).
* It can list relevant publications corresponding to paragraphs, chatgpt often halucinates new papers but claude consistenly gives highly cited and relevant suggestions.
You didn't say which version of GPT you prefer it over. My experience of Claude hasn't been good at all - I get obvious hallucinations all the time. Whereas I get pretty consistent accurate results with GPT-4.
I just tried again now and asked it about HTMX and Django since I had just been reading an article about it. Claude invented a package called `htmx.django` and a decorator that could be imported called `@hx_request`. This is typical of my experience with Claude.
Claude is bad at code and great at everything else, I thought that was pretty common knowledge by now. I second OP where I prefer using it over OpenAI models these days
I’ve been using Claude for months now, as has my dev team and a number of other people I know, and we don’t have this problem at all. In my experience is hallucinates less. This is compared to ChatGPT with GPT4. Anecdotal evidence is worthless it seem.
Out of 1000 people flipping coins over and over again, on average people get a mix of hits and misses - there will always be someone who just get 10 heads in a row and someone else with 10 tails in a row, this corresponds to some users being wowed by the superintelligence nailing the answer every time and other users getting made up citation after made up citation.
It won't hallucinate almost anything if you give it the entire documentation for the libraries you want to use. It can ingest many hundreds of pages of documentation without problem. Used like this it is far more powerful than GPT-4 or any existing model.
i know some people say its extended context is more hackery than not, but I've found better success with Claude in roughing out some paragraphs before editing when i provide it a lot of context
As a counter point, I've been actively assessing feasibility of migrating my experiments to Claude with not much success. Trying to migrate prompts from GPT 4 to Claude often yields half-baked results. Claude seems to require a lot more handholding to get the desired result.
In my information extraction prompts Claude is about as good as GPT-3.5 but slower. Its only advantage over OpenAI is context length. GPT-4 is better, 3.5 is faster.
But I am using it often to chat with PDFs and it works great for that. I would not want to give this ability up.
When using stuff like this you let the cream of top surface then use it. It will surface when enough people/publication give praise, otherwise you waste time evaluating too many models, it’s not a fun task as they can give you illusions of being great at some but end up with overall worse. Right now gpt4 is king and haven’t been any consistent chatter saying otherwise
FWIW I can't build OpenAI into our product without huge headaches around privacy policies and getting additional customer consent. I can build Amazon services into our products because Amazon is a trusted vendor and we use an AWS stack already.
Because Anthropic / Claude is available via Amazon Bedrock, they have a significant advantage in any company that's already using AWS. If you're on Azure, OpenAI has that advantage.
> FWIW I can't build OpenAI into our product without huge headaches around privacy policies and getting additional customer consent
Doesn't this imply that a new competitor beginning with a lenient customer privacy policies as well as a customer base that doesn't care as much about privacy will rise up to challenge you? And possibly iterate faster?
Because of its investment, FTX owns shares in Anthropic. In bankruptcy, those shares will be sold in some form to new investors in exchange for cash, at whatever the current value is. Just like any other asset.
In theory, I suppose that new investor could be Anthropic itself doing a stock buyback, but that would be extremely unusual. Buybacks are for mature companies with excess cash, not growing startups.
Let's say for sake of argument the investment by FTX was made at a valuation of $1B and Anthropic is now worth $10B.
Could Anthropic now say, "Oh no, that money was stolen, the investment was never valid, we're returning every last cent." and essentially do a huge share buyback at a steep discount to current valuation?
I guess that's not how corporate liquidation work.
A liquidator will evaluate the assets current market value and try to liquidate them in the most advantageous way for creditors.
Which means: they'll try to get at least the valuation that Anthropic got from the AWS deal. The problem is: they need to find someone willing to buy these shares at this price. If they do so, FTX creditors will be happy.
As a very crude analogy, if someone got access to my brokerage account and used to money in that to buy a Ferrari and got caught, do I get my money back or get the Ferrari (and presumably have to sell it)?
Bankruptcy court can do clawbacks, I'm curious how that could apply for an investment scenario though.. it's not exactly paying a creditor or buying a Porsche just before a bankruptcy...
Regardless of what's legally possible, it doesn't seem like an option that new FTX management would want to exercise, since they're trying to maximize the value of remaining assets for creditors.
Clawbacks would make sense for a charitable donation or shares that have gone down in value. For an investment that's increasing in value, it makes more sense to sell it at market.
Funny that FTX customers might end up having a non-trivial amount of cents on the dollar added to their claims by a lucky investment out of the many shotgunned by SBF and crew.
Could be made whole even? Hasn’t majority of the 8b lost by FTX been recovered by now - maybe this is enough to fill the remaining gap (on some timeframe)
Probably a little ambitious. Claims trading is currently priced at ~35c on the dollar[0], which isn't great but certainly better than what claims were trading at to begin with (~10c on the dollar).
Google invested 1B to Lyft, but Lyft used AWS for cloud compute. (I left Lyft in 2021, could be different now)
Investment department and cloud compute departments are very different entities and if the investment to Antropic was not a part of the Google cloud expansion strategy moving to a new cloud is such a pain, that noone will do it.
Opposite example is Kaggle. Google bought it as a marketing tool for Google Cloud => all the years since acquisition Kaggle team worked on adding differnt Google Cloud features to the platform
>Question: why isn’t Anthropic using Google Cloud, given who their past investors include?
They also have a fiduciary duty to their other investors, and going all-in on Google cloud would be a pretty big risk (relative to other cloud providers) given the uncertainty about whether it'll still exist in 5-10 years.
To you and me, that's a lot of money, for Google's size that's really not a lot of money. The fact that it just became profitable certainly helps the beancounters decide to keep it around though.
Being in Silicon Valley has warped my sense of money so much.
When I read $300m, what I read is "when you factor in office costs & health care as well as salary/stock grants, an average FAANG software engineer must be ~$1m. That's only 300 employees!"
It's a crazy way to think. Suddenly all prices that end with `m` and not `b` seem pointless.
Google includes the income from paid Gmail accounts in their cloud revenue. From all accounts, that is all of the profit and in fact it generates more profit that offsets the cloud losses.
I think that's true and Google cloud as such is unlikely to completely vanish (it may or may not morph significantly and any given service or price isn't necessarily safe)
Still, for proper context, google reader may be small but google stadia was big by any scale, with significant datacentre investment, dedicated consumer hardware, exclusive publishing deals, marketing campaigns, etc. And 10 years isn't as long as it used to be in the IT world :)
Stadia existed for a total of 3 years. And downsized significantly after only two.
Google cloud (it's earliest form anyway) launched in 2008. It's older than Drive and Photos. In a year or two, there will be Google employees younger than Cloud is.
I've seen 18 years old full times (some that never went to college). I wouldn't be surprised to see a 17 year old intern or full time if they graduated college early.
For context, there are 5 people and 1 bot betting, and it's not real cash or even something with a cash equivalent, and the bet amounts are less than what you get for free on sign up -- not exactly what I'd call a "market". And the bet is that they'll close Google Cloud in 2 years, not 5-10 years as is being discussed.
It's plausible that FTX customers will be made whole from the Anthropic investment profits in addition to the $7bn or so recovered so far...that'll be wild!
(Just speculating) They'll probably out SBF and Ellison through legal financial means at one point. Looks like a heavy burden to carry from this point on.
If they don't, Eric Schmidt - SBF family connection will start to look suspicious.
It doesn't look like SBF owns the stake, rather FTX does. Creditors seemed to have gotten lucky that that the bankruptcy handlers aborted the sale of their stake a few months ago[0].
Google VC doesn't push Google products in an effort of remaining neutral and getting access to the best deals. It's a strategic mis-step for Google the company, but a win for Google the VC arm. Also shows Google's internal dis-organization.
Not good for a company to receive funds from investors that we now know stole and grifted from their customers.
Wonder if this company is all just smoke and mirrors at this point. Never heard of this company prior to the Amazon investment.
Kind of reminds me of the Amazon and Rivian partnership. This nobody EV company gets an investment from Amazon, interest/hype boosts potential revenue/preorders. Valuations spike to billions. IPO debuts at ~$120/share. Insiders sell shares. Hype wears off. Now it’s trading at ~$21/share.
I’m no fan of the people you’re talking about myself.
But once we start going down that road we run smack on into companies with like the sovereign wealth fund in Riyadh or (to my embarrassment) Thiel or any number of people who make SBF look like the clueless, entitled, amoral but ultimately strictly small-change knucklehead he is.
There are some truly scary people invested in important companies. Do we hold all the founders to account for that or none of them? It’s not fair to pick and choose.
Yeah, we should totally put our trust in founders who would gladly accept money from immoral sources.
We need to understand that society only values who moves first. The early bird gets the worm. If they want to play the game, they need to raise funds from people who truly understand power and money, regardless of the human cost.
>Yeah, we should totally put our trust in founders who would gladly accept money from immoral sources.
You shouldn't put your trust in founders period. Or companies. Or famous people. Or anyone that you don't actually know. They're people or run by people with goals that greatly differ from your personal goals.
What you're asking is that they put on a veneer of morality and then once you're really convinced they're on your side they'll stab you in the back. Like with the old poster child Google.
And if I had told anyone five years ago that there would be eyeball scanners building a privately-held biometrics database at a level recently considered extreme for violent criminals? What letter anon would that have sounded like? I would have been like easy there mom, no one is doing that.
Are we allowed to talk about Loopt and Greenpoint and OpenAI and Autodesk and Siebel taking a year in Thailand while Socialcam flew into the side of a mountain on HN?
The broader point is that I think a lot of successful tech figures get a pretty unfair rap on HN. Few if any of these folks are stirring candidates for the Nobel Peace Prize, but Bezos or Mark or Elon or even Gates being uniquely bad?
Altman, with the full weight and backing of YC in its prime and a clear first-mover advantage got bailed out of the Loopt debacle [1] in a deal with Green Dot that maybe scraped above the “fire sale” line and is trivially the most celebrated person in the program to not merit an “I made something people want” shirt. pg helped, Conway helped, they got it sort of cashed out in a way that didn’t exactly fuck anyone sideways (other than Green Dot). He proceeded to fail upwards through preferential access to pre-IPO AirBnB shares and shit, became the leader of YC, and is now busily turning the world into Blade Runner 2023.
Seibel was running Socialcam at a time when anyone working at or near FB knew it was a ranking glitch, pawned it off on Autodesk for 60MM [2] saying with a straight face it was the next YouTube, and promptly started inconsistently replying to emails titled “Where the fuck are you?”. Also ran YC.
Are these guys uniquely bad? Debatable, this is how the game is played apparently. But it would be nice to enjoy your ill-gotten gains without a relentlessly resourceful [3] PR campaign about how we’re dealing with some real humanitarian visionaries.
All I’m proposing is that if we trash Elon, then Altman doesn’t get a free ride.
n.b. A lot of that is me claiming things (that I heard from people in the room) that are merely alluded to in primary and secondary sources, so it’ll be up to you to judge if it has what they call “the ring of truth”.
As someone who had been rabidly following developments in the space until earlier this year: I’ve fallen a bit behind.
I’be been walking around with the (seemingly misguided) view that the bigger LLaMA 2 models were sort of neck and neck with like 3.5-turbo, Claude, and the rest of the first string modulo GPT-4 that had a pretty steady if eroding lead.
What’s the lay of the land in 9/23? Or better yet, what do I need to bookmark to get the inside baseball?
This should not have stopped any of FAANGs from competing (especially Google), and yet they are still far behind OpenAI - more than a year after GPT-4 was trained.
That's not how it works. The investment is an asset on the balance sheet, it is very unlikely to reduce taxable profits. I don't know the US corporation tax situation much, but if it's anything like the UK then you couldn't use that to offset profits for the year. You could if the investment failed, but then you've lost a lot more money than the tax saving (as you've lost the entire $4bn investment).
You might be right that Anthropic could then spend that money back with AWS, but that would again be subject to tax on any profits it generates for AWS. Also, if they spent (say) $2bn of the money with AWS but had nothing to show for it, when Anthropic raises again you'd expect the valuation to decrease substantially.
I very much doubt it is a tax saving mechanism, nor do I think it is a particularly good way to juice sales (compared to spending say $4bn on marketing, or discounts, or whatever).
While AWS is probably providing sweetheart compute rates to it's portfolio, their stake in the company will also rise in value as the company grows out their revenue and goes for another round, or even an IPO.
This is the whole point of CVC - put strategic investments in rising companies playing in segments or sub-segments that you as a company don't want to enter directly yet while building out an ecosystem and also ensuring you don't become the next Kodak.
> AWS gets to reduce its profits by making this investment, which means they pay less taxes.
This has been Amazon strategy for a long long time now, almost two decades.
But Bezos views this in a different way, not as an accounting trick to look good to investors, but as a way to kill the competition, since you have lower costs by not paying taxes, so you can redirect a part of what would have been taxes to attract customers (lower prices, excellent customer support, ...)
I think you're both right. It's a bit inflated as AWS services are overpriced in general. On the other hand, we're speaking specifically about significant use of Nvidia GPUs which are in demand right now so it's not like Amazon can just use their spare capacity.
AWS has its own line of AI accelerators called Inferentia that offers reduced cost versus Nvidia, but it's often a pain to make stuff work because everything is written for CUDA. Inferentia team spends a lot of time porting OS models to their hardware for customers to use (e.g. through SageMaker).
Anthropic’s main mission is ethical. Profit is just fuel for the ethical mission (and they would rather have hockey stick AI safety than hockey stick growth given the choice)
Amazon, well you know them :-)
Flogging their model on Bedrock etc. must be part of the plan for Anthropic but AWS investment must surely create tension.
This sounds nice, and the information on https://www.anthropic.com/company seems to back up your claim, but do you know of anything on Anthropic's corporate governance level that will help ensure this mission is followed (forever)?
The profit motive is _very_ resilient. Right now I will take it on face value that it's fuel for the ethical mission, but in the future that might change. It's far (far) more likely that the ethical mission will change than the profit motive will change.
More funding doesn't change it, it just raises the stakes.
I share your concerns about the corrupting influence of money (and the importance of this space), but wanted to point out that Anthropic announced two relevant things last week regarding corporate governance:
First, that it has arranged to be controlled by a Long-Term Benefit Trust: "an independent body of five financially disinterested members with an authority to select and remove a portion of our Board that will grow over time (ultimately, a majority of our Board)". (https://www.anthropic.com/index/the-long-term-benefit-trust)
Hey, even if it does revert to being finance driven, we still get at least a few sincere "don't be evil" years like we did with Google. I think it's still worthwhile.
Anyone who unironically says that they are going to improve/police/redefine ethics should be considered immediately suspect. It's a massive red flag if someone is thinking that their ethics are better than other's. Bonus red flag points if that comes from a corporation.
It is also very telling that a corporation called literally "human-first" is developing a tool to displace said humans completely. Now it is possible that it won't be able to do this, but the problem is that almost everyone including the creators believe that NNs will do it.
Ai will overtake all human capabilities on the right timescale. (We just don't know whether that's an order of years, decades, or centuries). It... doesn't make sense that an entire human couldn't be perfectly artificially replicated eventually (and then, obviously, surpassed immediately), unless you believe in a non-physical soul which powers some aspect of our conscious thought.
Even if you believe in such a soul, does it govern the non-conscious systems that do almost all of the work of thinking and feeling? If not - once again, artificial systems will certainly surpass natural ones.
In other words, it's not a question of whether humans will always be the masters, but a question of whether the "gods" we create will love us or hate us. Anthropic is aiming for the former.
My point was not about potential future AI (and not about current NNs). My point was that Anthropic MARKETING team CLAIMS that it is aiming for the AI which would love humans, and it is a commendable effort. But I think, that just like any other corporation in the history, Anthropic cares only about the unrestricted growth and quarterly revenue/profit values. And any "ethics" bs they throw around is a marketing gimmick, designed to improve their image despite them creating a human job replacements right now, right here. That's both due to the direct job replacements, and also by crapping all around older industries, ignoring copyright laws and stealing private IP just because law don't specify NNs directly.
> but a question of whether the "gods" we create will love us or hate us.
You (and pretty much the entire debate around AI safety) have smuggled in the notion that these AIs even have the capacity to "love" and "hate," and have the agency to perhaps act vindictively. How we get from predicting the most plausible next token to a god-like entity with agency is... not clear at all.
Do people think that we're actually sleepwalking towards skynet? Or are they just saying that so that Sam Altman can get the government to put in rules that bind everyone else while allowing OpenAI to proceed without competition and capture the extremely mundane real market of writing ad copy and such.
> You (and pretty much the entire debate around AI safety) have smuggled in the notion that these AIs even have the capacity to "love" and "hate," and have the agency to perhaps act vindictively.
It's a metaphor, actually. I guess I needed heavier use of quotes.
The one thing we can be sure of is that if goal-oriented AI exists, it will follows goals.
> How we get from predicting the most plausible next token to a god-like entity with agency is... not clear at all.
It doesn't have to be clear. If it was clear, we'd be working on it. That's... how invention works.
Dario Amodei (ceo of anthropic) has spoken about rising model training costs, and how he expected them to climb into the hundreds of millions of dollars. You could tell he didn’t like it, because it means anthropic has to do deals like these just to keep up.
So now the gpu farms of meta, google, X, microsoft and amazon are pretty much reserved for large foundation model training of specific models. It was probably a good thing for Anthropic to reserve their spot at the pool, even if as you point out it creates tension with their ethical mission. What options remain for other players who want to compete at this level I wonder?
By the sounds, it's actually a $4B investment (taking minority stake) whilst also potentially providing free access to some AWS services? The degree or agreement of access to AWS services isn't totally clear though...
Yep. Censored like a god-fearing puritan just off the boat and butt-covering non-responses to everything else like a lawyer. So "aligned" that they can't even pass a turing test.
The only good LLM is the one you're running locally.
In practice, AI safety means the output is censored. But these companies (including OpenAI) seem to attract people who think they are protagonists in Terminator 2, Judgement Day:
"One Anthropic worker told me he routinely had trouble falling asleep because he was so worried about A.I. Another predicted, between bites of his lunch, that there was a 20 percent chance that a rogue A.I. would destroy humanity within the next decade. (Bon appétit!)"
I think it might be because they want to diversify their infrastructure and reduce the risk associated with relying solely on one provider. It's interesting to see Anthropic switching cloud providers from Google to Amazon. I think it might be because they want to diversify their infrastructure and reduce the risk associated with relying solely on one provider. Additionally, AWS Trainium and Inferentia chips could offer specialized hardware support that aligns better with their AI workloads. This move doesn't necessarily reflect poorly on Google Cloud's future, but it does indicate that Anthropic wants to explore multiple cloud options for their AI projects.
Alexa and google home have been excellent for discovering new music. You ask them for a song you know and they play either a very weird cover version or something entirely different.
Siri has been helpful for increasing my 3-year-old's vocabulary: he'll ask it to play a song he knows, and Siri will pick out an explicit R&B piece full of all sorts of words he's never been exposed to.</s>
(Seriously -- there's a song with the title "The ABC Song", which as one might expect, is about the ABCs. But it's impossible to get it to play that via voice control; it only wants to play "abcdefu", which is not the same song at all, and has a very clear F-bomb 5 seconds in.)
Haha, the speed at which you need to be able to get it to stop is a key parenting skill. Saying that 4 seconds into a Tom Cardy song as my son says "but I love this one".
Maybe this is why some people are so good at the intro round at pub quizzes, you've got a few seconds to identify the exact song that's playing and mentally run through the lyrics before making a call.
I asked Siri today about the current week number (in German), and it failed. Just when I thought I've lowered my expectations enough, Siri can do one "better".
siri is honestly only good for basic question/answers. like "how is the weather going to be later" or "play X song", or "how old is X". i wish they integrated LLMs into personal assistants to truly have a conversational type experience
It might be a European vs. American cultural thing because week number is much less used in America, and Siri was developed by Americans. Maybe this is more of a localization failure than a generic functionality failure.
If it gave you an answer that seemed wrong, well, there are also several slightly different definitions of week number, so it could be that it chose one that wasn't the one you expected (maybe also a localization failure).
It regularly picks up an entirely bizarre song, however in doing so it's also found some bangers. My kid loves diggy-diggy hole and it played the wind rose version (https://www.youtube.com/watch?v=34CZjsEI1yU) which is awesome.
I asked for I wanna be like you for the kids and got the expected one a few times. Then one time it returned the triple J version by the Psychedelic Porn Crumpets (https://www.youtube.com/watch?v=bD0VpmaSBjs) which became the new version for running around the room pretending to be a rollercoaster.
These are covers at least, but it will also just play the weirdest mishearings of something.
So semi-serious as I'm mocking it getting things wildly wrong, but also legitimately we discover new music this way.
If you like this kind of thing, also checkout the radio 1 live lounge stuff. Been going for a long time so I think there's playlists with a couple of hundred songs and they'll be a small amount of the full sets. Often tends towards more acoustic but you can find some great covers.
It's a joke. Google Assistant is notoriously awful and at home we often kid that if we ask it to play 90's rock on Spotify it will play some animal documentary on Netflix. It's not too far from the truth.
Oh I love that, you ask it for a song, it gets it wrong and thinks you want some video. Then it tries to play the video, can't and acts like you were daft for asking it to do that.
Somehow people have polluted and/or the assistants are too stupid to play well known songs from voice prompts. You get weird covers with clearly synthesized backing or random crap. It’s honestly weird. The covers shouldn’t even be on system half the time.
This is how I feel about pretty much all the most data-hungry services. It's never been about an improved use experience, because the quality is not there.
I can't count how many times I yelled/cried about Spotify trying to learn every mood I've ever felt (and what song would have been perfect for the moment!) only to play the same 4 songs over and over because I listened to one of them on purpose a week ago. (I no longer use Spotify. We'll see if Tidal falls into the same trap.)
google assistant is probably still not Bard powered, so it just means someone didn't coded some rule: user asked "closest one" -> locate user location -> locate home depot closest to that location
I wish Siri did that consistently. Right now my Apple Watch Siri is unusable even to set timers. I don't know if the latest updates on my old watch struggles to run or what else the issue could be.
There were a few weeks recently when she couldn't even set timers (the timer wasn't displayed on the screen leading to constant "Alexa, show me the timer" requests).
Voice computing is in the trough of despair. Tech companies over-invested in them. They aren't the next iphone (nothing will be). But like millions of homes we have two that we use for a few things. Expect improvements at a disappointing pace, leading to improved utility.
Many dunked on the apple watch, but it keeps getting better. Meta's next headset will be disappointing to some but have an audience and keep improving. etc.
That's their in-house LLM though. Amazon prefers investing $4B into Anthropic rather than using their own stuff. If you ask me, long ago they should've taken the top ML and engineering talent from Alexa and let them focus on building an in-house Openai/Anthropic. Then the engineers who left in Alexa could just integrate that stuff, as of the rest of the company (AWS, retail, etc).
A bunch of folks in the thread seem confused as to why they would do this after their Google investments. It's important to note that Amazon has been working closely with Anthropic for a long time, and already vends their models in AWS:
It’s likely a combination of 1) AWS credits, 2) hard cash, 3) tranches. If I had to guess probably $200-500M cash, and rest in credits and probably 2-4 tranches. Hence why it is “up to”.
They have an excellent LLM called Claude which is quite competitive with OpenAI offerings and has long context windows.
Other than OpenAI’s models they have been able to deliver the best results in projects I worked on and sometimes they outperformed OpenAI models too.
I’m happy to see this as this gives them funding to further train expensive models and compete - their product is really nice at the moment but could really benefit from bigger size and even bigger contexts.
They have Claude 2 which is up there with GPT-4 and PaLM 2 as one of the top three publicly accessible large language models. And presumably those billions will train Claude 3.
I'm sorry but I'll have to disagree with you - Claude 2 is worse than GPT-4 in almost all tasks I've tried it with, the two things that it's better at than GPT-4 are the context window and the dataset being from early 2023 instead of late 2021 (for 3.5 Turbo/4).
Being worse than GPT-4 doesn't dispute GP's claim that it's in the top 3 with GPT-4. To disagree with them you need to name models other than GPT-4 and PaLM 2 which are also better than Claude 2.
I’m still trying to get my mind around a world where Brain/DeepMind aren’t way ahead on all this. They were so far ahead so recently.
Is there a comeback story here? IIRC they’ve still got Dean and Hassabis and a bunch of other bruisers, along with arbitrary financing, the best training sets, and novel compute.
Is it time to put Hassabis in charge and start making Ilya and Karpathy offers they can’t refuse?
Sure, they're all competing labs, but Deepmind isn't a competitor for a product offering. Anthropic is quite analogous to OpenAI because they have an API you can pay for an a chatbot product you can use.
So what will this mean in relation to their recent Alexa announcements? are they using Anthropic in this announcement? Is it so difficult for a big company to build their own?
Alexa has been building their own LLM stuff for years now. Their models are also available on AWS. This Anthropic investment sounds much more of an AWS investment. If the Alexa LLMs were popular and capable like the best models, Amazon wouldn't invest in Anthropic. Currently if you want to use OpenAI models with better rate limits, you go to Azure. Since openai models are the most popular rn, that creates a problem for AWS. So they invested in the second most popular, which is still significantly more popular and capable than the Alexa models.
Crazy. I don’t know what they gain by super slow rollouts like this. It puts them at a serious disadvantage while ChatGPT gets entrenched in people’s daily workflows and the perception grows that it’s the best.
Interestingly, AWS (along with YC.org, Infosys, Musk, and others) was one of the founding donors for OpenAI, the non-profit. A few years and Satya Nadella successfully Mark Zuckerberged them.
At this point Apple is the only BigTech without any major investment in LLMs. Which isn't too surprising, they have historically never jumped on the shiny new thing until the dust clears off.
Apple makes shiny new things once in a while. Maybe they are not so good at catch up so plan to keep excelling at non AI shiny things (there is a lot of money in the not AI niche!)
Apple hasn’t released a dedicated chatbot but they use models heavily and in very targeted ways
all the iPhones have had a transformer co-processor for several generations, the neutral engine is primed for use with this current crop with just a little more dedicated ram to it
this is arguably better than the pick me chatbot thing
but Apple is also primed to disrupt all of that, they include a fine tuned model chatbot style client side on all their OS’ and that substantially shifts the battlefield, a smart client side siri that only optionally connects to a large cloud model? even better
an SDK for all the AI startups to leverage that instead of token price arbitraging with OpenAI api or other cloud providers like replicate? all hail the Apple supremacy
If Amazon has $4B to invest into what may end up being the Theranos of AI research, why don't they have $4B to invest into their workforce and, you know, actually pay people instead of all the extra-legal shenanigans they're pulling to avoid cutting paychecks?
Edit: For those not understand what I'm referencing, Amazon is currently trying to get out of owning their Seattle offices, and are using "get back to the office" tactics to harass workers into leaving so they can dump the real estate at loss: https://www.seattletimes.com/opinion/unpacking-amazons-steal...
It isn't about their global payroll spend dwarfing the $4B. Its about having a sense of ethics and a grasp of math.
Whenever I see such comments I ask myself if you have no imagination what Amazons payroll looks like? They have 1.5M employees, total payroll is well beyond $100B per year.
Now over what timespan do they invest this money? 5 years? 10 years?
Do you mean investing 4B internally instead? That would not be an investment into AI or LLM that Amazon considers strategic? Would they be able to invest 4B internally specifically for this? Maybe yes, but I’d argue they already probably tried it and it did not work as well as they hoped. Or they do know it may not be as effective.
Announcement "Anthropic Partners with Google Cloud" Feb 3, 2023 - https://www.anthropic.com/index/anthropic-partners-with-goog...
"...Anthropic, an AI safety and research company, has selected Google Cloud as its cloud provider. The partnership is designed so that the companies can co-develop AI computing systems; Anthropic will leverage Google Cloud's cutting-edge GPU and TPU clusters to train, scale, and deploy its AI systems."
Announcement "Expanding access to safer AI with Amazon" Sep 25, 2023 - https://www.anthropic.com/index/anthropic-amazon
"AWS will become Anthropic’s primary cloud provider for mission critical workloads, providing our team with access to leading compute infrastructure in the form of AWS Trainium and Inferentia chips, which will be used in addition to existing solutions for model training and deployment. Together, we’ll combine our respective expertise to collaborate on the development of future Trainium and Inferentia technology."