IMO this really feels like the Facebook / Twitter integration from early iOS. That only lasted a few years.
Apple clearly thinks it needs a dedicated LLM service atm. But still thinks it is only supplemental as they handle a bunch of the core stuff without it. And require explicit user consent to use OpenAI. And Apple clearly views it as a partial commodity since they even said they plan to add others.
Tough to bet against OpenAI right now...but this deal does not feel like a 10 year deal...
Ditto. They'll use it now while they stand to benefit and in 3 years they'll be lambasting OpenAI publicly for not being private enough with data and pretend that they never had anything to do with them.
The partnership is structured so that Apple can legally defend including language in their marketing that says things like "users’ IP addresses are obscured." These corporations have proven time and time again that we need to read these statements with the worst possible interpretation.
For example, when they say "requests are not stored by OpenAI," I have to wonder how they define "requests," and whether a request not having been stored by OpenAI means that the request data is not accessible or even outright owned by OpenAI. If Apple writes request data to an S3 bucket owned by OpenAI, it's still defensible to say that OpenAI didn't store the request. I'm not saying that's the case; my point is that I don't trust these parties and I don't see a reason to give them the benefit of the doubt.
The freakiest thing about it is that I probably have no way to prevent this AI integration from being installed on my devices. How could that be the case if there was no profit being extracted from my data? Why would they spend untold amounts on this deal and forcibly install expensive software on my personal devices at no cost to me? The obvious answer is that there is a cost to me, it's just not an immediate debit from my bank account.
> The partnership is structured so that Apple can legally defend including language in their marketing that says things like "users’ IP addresses are obscured." These corporations have proven time and time again that we need to read these statements with the worst possible interpretation.
What's the worst possible interpretation of Apple and CloudFlare's iCloud Private Relay?
I’m not sure I understand the paranoia that Apple is secretly storing your data. Sure they could secretly do so but it doesn’t make any sense. Their whole schtick is privacy. What would Apple benefit from violating what is essentially their core value prop? They’d be one whistleblower away from permanent and irreparable loss of image.
I'm reasonably sure you just described the SEC and the (paraphrasing Matt Levine) "everything is securities fraud"-doctrine. Yes Apple has some wiggle room if they rely on rule-lawyering, but.. I really don't think they can wide-spread ignore the intention of the statements made today.
Some people here somehow thinking they will simultaneously outsmart:
* The CEO of a three trillion dollar company that employs 100,000+ of the best talent you could find around the world, with the best lawyers in the world one phone call away. Also, one of the best performing CEOs in modern times.
AND
* The CEO of the AI company (ok ... non-profit) that pretty much brought up the current wave of AI to existence and who has also spent the best part of its life building and growing 1,000s of startups in SF.
You make it sound like it's merit or competence that landed Cook in that position, and that he somehow has earned the prestige of the position?
I could buy that argument about Jobs. Cook is just a guy with a title. He follows rules and doesn't get fired, but otherwise does everything he can with all the resources at his disposal to make as much money as possible. Given those same constraints and resources, most people with an IQ above 120 would do as well. Apple is an institution unto itself, and you'd have to repeatedly, rapidly, and diabolically corrupt many, many layers of corporate protections to hurt the company intentionally. Instead, what we see is simple complacency and bureaucracy chipping away at any innovative edge that Apple might once have had.
Maintenance and steady piloting is a far different skillset than innovation and creation.
Make no mistake, Cook won the lottery. He knew the right people, worked the right jobs, never screwed up anything big, and was at the right place at the right time to land where he is. Good for him, but let's not pretend he got where he is through preternatural skill or competence.
I know it's a silicon valley trope and all, but the c-class mythos is so patently absurd. Most of the best leaders just do their best to not screw up. Ones that actually bring an unusual amount of value or intellect to the table are rare. Cook is a dime a dozen.
I was with you until your last sentence. By all accounts Cook was one of the world's most effective managers of production and logistics -- a rare talent. He famously streamlined Apple's stock-keeping practices when he was a new hire at Apple. How much he exercises that talent in his day-to-day as CEO is not perfectly clear; it may perhaps have atrophied.
In any case, "dime a dozen" doesn't do him justice -- he was very accomplished, in ways you can't fake, before becoming CEO.
I look at it from a perspective of interchangeability - if you swapped Steve Ballmer in for Cook, nothing much would have changed. Same if you swapped Nadella in for Pichai, or Pichai for Cook. Very few of these men are exceptional; they are ordinary men with exceptional resources at hand. What they can do, what they should do, and what they can get away with, unseen, govern their impact. Leaders that actually impact their institutions are incredibly rare. Our current crop of ship steadying industry captains, with few exceptions, are not towering figures of incredible prowess and paragons of leadership. They're regular guys in extraordinary circumstances. Joe Schmo with an MBA, 120 IQ, and the same level of institutional knowledge and 2 decades of experience at Apple could have done the same as Cook; Apple wouldn't have looked much different than it does now.
There's a tendency to exaggerate the qualities of men in positions like this. There's nothing inherent to their positions requiring greatness or incredible merit. The extraordinary events already happened; their job is to simply not screw it up, and our system is such that you'd have to try really, really hard to have any noticeable impact, let alone actually hurt a company before the institution itself cuts you out. Those lawyers are a significant part of the organism of a modern mega corporation; they're the substrate upon which the algorithm that is a corporation is running. One of the defenses modern corporations employ is to limit the impact any individual in the organization can have, positive or otherwise, and to employ intense scrutiny and certainty of action commensurate with the power of a position.
Throw Cook into an start-up arena against Musk, Gates, Altman, Jobs, Buffet, etc, and he'd get eaten alive. Cook isn't the scrappy, agile, innovative, ruthless start-up CEO. He's the complacent, steady, predictable institutional CEO coasting on the laurels of his betters, shielded from the trials they faced through the sheer inertia of the organization he currently helms.
They're different types of leaders for different phases of the megacorp organism, and it's OK that Cook isn't Jobs 2.0 - that level of wildness and unpredictability that makes those types of leaders their fortunes can also result in the downfall of their companies. Musk acts with more freedom; the variance in behavior results in a variance of fortunes. Apple is more stable because of Cook, but it's not because he's particularly special. Simply steady and sane.
> They're different types of leaders for different phases of the megacorp organism, and it's OK that Cook isn't Jobs 2.0 - that level of wildness and unpredictability that makes those types of leaders their fortunes can also result in the downfall of their companies.
This is absolutely true. But that doesn’t imply that Tim Cook is so unexceptional that anyone with a 120 IQ could do the same job he does. The fact that Steve Jobs himself trusted Cook as his right hand man and successor when Apple probably has literally thousands of employees with at least a 120 IQ should be a sign of that.
Partly because little of this is really a question of intelligence. If you want to talk about it in psychometric terms, based on what I’ve read about the man he also seems to have extraordinarily high trait conscientiousness and extraordinarily low trait neuroticism. The latter of the two actually seems extremely common among corporate executive types—one gets the sense from their weirdly flat and level affect that they are preternaturally unflappable. (Mitt Romney also comes across this way.) I don’t recall where I read this, but I remember reading Jobs being quoted once that Cook was a better negotiator that he was because unlike Jobs, Cook never lost his cool. This isn’t the sign of an unexceptional person, just a person who is exceptional in a much different way than someone like Steve Jobs. And, contrary to what you claim at the top of your comment, someone like Tim Cook is pretty distinguishable from someone like Steve Ballmer in the sense that Ballmer didn’t actually do a good job running Microsoft. I don’t know if that was related to his more exuberant personality—being a weirdly unflappable corporate terminator isn’t the only path to success—but it is a point against these guys being fungible.
> I look at it from a perspective of interchangeability - if you swapped Steve Ballmer in for Cook, nothing much would have changed.
This is quite ridiculous. "Developers x3" Ballmer would have face-planted at Apple. He only coasted so far at Microsoft because Gates had already won the platform war.
Actually, just in three to five years, lots of "AI boxes" and those magical sparkling icons next to input fields summoning AI would be silently removed.
LLMs are not accurate, they aren't subject matter experts that'll be maybe within 5% error margin.
People will gradually learn and discover anf the cost of keeping a model updated and running won't drastically reduce so we'll most likely see dust settling down.
I truly hope the reckless enthusiasm for LLMs will cool down, but it seems plausible that discretized, compressed versions of today's cutting-edge models will eventually be able to run entirely locally, even on mobile devices; there are no guarantees that they'll get better, but many promising opportunities to get the same unreliable results faster and with less power consumption. Once the models run on-device, there's less of a financial motivation to pull the plug, so we could be stuck with them in one form or another for the long haul.
I don't believe this scenario to be very likely because a lot of the 'magic' in current LLMs (emphasis on 'large') is derived from the size of the training datasets and amount of compute they can throw at training and inference.
Llama 3 8B captures that 'magic' fairly well and runs on a modest gaming PC. You can even run it on an iPhone 15 if you're willing to sacrifice floating point precision. Three years from now I full expect GPT4 quality models running locally on an iPhone.
Three years is more than twice the time since GPT-4 was released to now. Almost twice the time ChatGPT existed. At this rate, even if we'll end up with GPT-4 equivalents runnable on consumer hardware, the top models made available by big players via API will make local LLMs feel useless. For the time being, the incentive to use a service will continue.
It's like a graphics designer being limited to chose between local MS Paint, and Adobe Creative Cloud. Okay, so Llama 3 8B, if it's really as good as you say, graduates to local Paint.NET. Not useless per se, but still not even in the same class.
No one knows how it will all shake out. I'm personally skeptical scaling laws will hold beyond GPT4 sized models. GPT4 is likely severely undertrained given how much data facebook is using to train their 8B parameter models. Unless OpenAI has a dramatic new algorithmic discovery or a vast trove of previously unused data, I think GPT5 and beyond will be modest improvements.
Alternatively synthetic data might drive the next generation of models, but that's largely untested at this point.
I know this isn’t really the point, but Adobe CC hasn’t really improved all that much from Adobe CS, which was purely local and perfectly capable. A better analogy might be found in comparing Encyclopedia Brittanica to Wikipedia. The latter is far from perfect, but an astounding expansion of accessible human knowledge that represents a full, worldwide paradigm shift in how such information is maintained, distributed, and accessed.
On the same token, those of us who are sufficiently motivated can maintain and utilize a local copy of Wikipedia…frequently for training LLMs at this point, so I guess the snake has come around, and we’ve settled into a full-on ouroboros of digital media hype. ;-)
They're extremely pessimistic, 3 years is 200% of how long it took ChatGPT 3.5.
Llama 8B is ChatGPT 3.5 (18 months before L3), running on all new iPhones released since October 2022, (19 months before L3). That includes multimodal variants (built outside Facebook).
The University of Washington is studying an AI application where a pair of headphones will isolate a single voice in a crowd when one simply looks at them. Amazing stuff…until you try it anywhere near your car, and then it starts playing the voice over your car stereo (presumably).
> People will gradually learn and discover anf the cost of keeping a model updated and running won't drastically reduce so we'll most likely see dust settling down.
As mentioned elsewhere, 3 to 5 years is some 3x to 5x as long as GPT-4 exists; some 2-3x as long as ChatGPT exists and LLMs suddenly graduated from being obscure research projects to general-purpose tools. Do you really believe the capability limit has already been hit?
Not to mention, there's lots of money and reputation invested in searching for alternatives to current transformer architecture. Are you certain that within the next year or two, one or more of the alternatives won't pan out, bringing e.g. linear scaling in place of quadratic, without loss of capabilities?
I'm pretty sure that statistical foundations of AI where a thing just been shy of 0.004 of the threshold value out of a million dimensional space can get miscategrized as something else will not deliver AGI or any useable and reliable AI for that matter other than that sequence of sequence mapping (voice to text,
text to voice etc.) applications.
As for money and reputation, that's a lot behind gold making too in medieval times and look where that lead too.
Scientific optimism is a thinking distortion and a fallacy too.
Tool seems like a strong term for whatever ChatGPT is right now. Absurdly overhyped curiosity? Insanely overengineered autocorrect? Dystopian MadLibs? Wall Street Wank Sock?
I’m not trying to downplay its potential, but I don’t know of anyone who trusts it enough for what I’d consider “tooling”.
LLMs are not accurate, they aren't subject matter experts that'll be maybe within 5% error margin.
You're asserting that the AI features will be removed in 3 to 5 years because they're not accurate enough today, but you actually need them to remain inaccurate in 3 years time for your prediction to be correct.
That seems unlikely. I agree that people will start to realize the cost, but the accuracy will improve, so people might be willing to pay.
The same argument can be used for Tesla full self driving: basically it has to be (nearly) perfect, and after years of development, it's not there yet. What's different about LLMs?
Death actually can be the price of being wrong. Just wait for someone to do the wrong thing with an AI tool they weren't supposed to use for what they were doing, and the AI to spit out the worse possible "hallucination" (in terms of outcome).
What you say is true, however with self-driving cars death, personal injury, and property damage are much more immediate, much more visible, and many of the errors are of a kind where most people are qualified to immediately understand what the machine did wrong.
An LLM that gives you a detailed plan for removing a stubborn stain in your toilet that involves mixing the wrong combination of drain cleaners and accidentally releasing chlorine, is going to happen if it hasn't already, but a lot of people will read about this and go "oh, I didn't know you could gas yourself like that" and then continue to ask the same model for recipes or Norwegian wedding poetry because "what could possibly go wrong?"
And if you wonder how anyone can possibly read about such a story and react that way, remember that Yann LeCun says this kind of thing despite (a) working for Facebook and (b) Facebook's algorithm gets flack not only for the current teen depression epidemic, but also from the UN for not doing enough to stop the (ongoing) genocide in Myanmar.
It's a cognitive blind spot of some kind. Plenty smart, still can't recognise the connection.
There's hundreds+ of companies making LLMs we can choose from, and the switching cost is low. There's only one company that can make self-driving software for Tesla. Basically, competition should lead to improvements.
Tesla aren't the only people trying to make self-driving cars, famously Uber tried and Waymo looks like they're slowly succeeding. Competition can be useful, but it's not a panacea.
Mercedes seems to be eating Tesla’s breakfast on FSD, in particular where safety and real-world implementation is concerned. Their self-driving vehicles are equipped with aqua-colored lights to alert other drivers that it is being controlled via computer, and Mercedes has chosen to honor its liability for incidents/accidents.
In Europe yes, especially with the Level 3, it means that Mercedes is taking the liability.
In the US it's different, because the US' FSD has nothing to do with the capabilities of the FSD in Europe (which is some sort of glorified driver assist), and it can clear navigate in many streets.
GPT-4 is 1 year old; 3.5 is 1 and a half. Before 3.5, this wasn't really a useful technology. 7 years ago it was a research project that Google saw no value in pursuing.
Anyone claiming that accuracy of AI models WILL improve is either unaware of how they really work or is a snake oil salesman.
Forget about a model that knows EVERYTHING. Let's just train a model that only is expert in not all the law of United states just one state and not even that, just understands FULLY the tax law of just one state to the extent that whatever documents you throw at it, it beats a tax consultancy firm every single time.
If even that were possible, OpenAI et.el would be playing this game differently.
Those use cases are never sold as "Mobile apps", but rather as "enterprise solutions", that cost the equivalent of several employees.
An employee can be held accountable, and fired easily. An AI? You'll have to talk to the Account Manager, and sit through their attempts to 'retain' you.
This is one of those "perfect is the enemy of good" situations. Sure, for things where you have a legal responsibility to get things perfectly right using an LLM as the full solution is probably a bad idea (although lots of accountants are using them to speed up processes already, they just check outputs). That isn't the case for 99% of task though. Something that's mostly accurate is good. People are happy with that, and they will buy it.
My experience suggests that LLMs become not less accurate, but less helpful.
Two years ago they output a solution for my query [1] right away, now they try to engage user to implement that thing. This is across the board, as far as I can see.
These LLMs are not about helping anyone, their goals are engagement and mining data for that engagement.
[1] The query is "implement blocked clause decomposition in haskell." There are papers (circa 2010-2012), there are implementations, but not in Haskell. BCD, itself, is easy, and can be expressed in a dozen-two lines of Haskell code.
> These LLMs are not about helping anyone, their goals are engagement and mining data for that engagement.
Wow, this is a really interesting idea! A sneaky play for LLM providers is to be helpful enough to still be used, but also sufficiently unhelpful that your users give you additional training data.
This is obvious in retrospect - instead of making LLMs work better, LLM's handlers invented various techniques to make LLMs to look like they work better, one such example is summarization. Next gen LLMs then get trained on that data.
Now instead of having some answer right away, the user has to engage in discussion, which increases the cost that is sunk into the work with LLMs.
I don't think that's really what Apple is going to do with it though, it's not going to be for factual question and answer stuff. It will be used more like a personal assistant, what's on my calendar this week, who is the last person who called me etc. I think it will more likely be an LLM in the background that uses tools to query iCloud and such, ie, making Siri actually useful.
How do you define a percent error margin on the typical output of something like ChatGPT? IIRC the image generation folks have started using metrics like subjective users ratings because this stuff is really difficult to quantify objectively.
IMHO the terribly overlooked issue with generative AI is that the end users' views of the response generated by the LLM often differs greatly from the opinion of the person actually interacting with the model
this is particularly evident with image generation, but I think it's true across the board. for example, you may think something I created on midjourney "looks amazing", whereas I may dislike it because it's so far from what I had in mind and was actually trying to accomplish when I was sending in my prompt
True, but generally what art I produce IRL is objectively terrible, whereas I can come up with some pretty nice looking images on Midjourney.... which are still terrible to me when I wanted them to look like something else, but others may find them appealing because they don't know how I've failed at my objective
In other words, there are two different objectives in a "drawing": (1) portraying that which I meant to portray and (2) making it aesthetically appealing
People who only see the finished product may be impressed by #2 and never consider how bad I was at #1
Right now they're basically a improved search engine, but they aren't solving the hard problem of making money.
Had Google become a utility and frozen it's search engine half a decade or more in the past, we would actually have something you could add AI on top of and come out with an improved product.
As it stands, capitalism isn't going to fix GIGO with AI
> LLMs are not accurate, they aren't subject matter experts that'll be maybe within 5% error margin.
The Gell Mann amnesia effect suggests people will have a very hard time noticing the difference. Even if the models never improve, they're more accurate than a lot of newspaper reporting.
> People will gradually learn and discover anf the cost of keeping a model updated and running won't drastically reduce so we'll most likely see dust settling down.
So, you're betting on no significant cost reduction of compute hardware? Seems implausible to me.
> …they’re more accurate than a lot of newspaper reporting.
Is that when they’re cribbing straight out of the newspaper pages, or is this just a cynical snipe at the poor state of media that, not for nothing, tech companies have had a fair hand in kneecapping?
The criticism of the performance of newspapers goes back well before Lovelace and Babbage:
"""I will add, that the man who never looks into a newspaper is better informed than he who reads them; inasmuch as he who knows nothing is nearer to truth than he whose mind is filled with falsehoods & errors. He who reads nothing will still learn the great facts, and the details are all false."""
"...our study reveals an exponential need for training data which implies that the key to "zero-shot" generalization capabilities under large-scale training paradigms remains to be found."
There’s a lot I don’t like about Sam Altman. There’s a lot I don’t like about OpenAI.
But goddamn they absolutely leapfrogged Google and Apple and it’s completely amazing to see these trillion dollar companies play catch-up with a start-up.
I want to see more of this. Big Tech has been holding back innovation for too long.
They "leapfrogged" Google on providing a natural language interface to the world knowledge we'd gotten used to retrieving throug web search. But Apple's never done more than toyed in that space.
Apple's focus has long been on a lifestyle product experience across their portfolio of hardware, and Apple Intelligence appears to be focused exactly on that in a way that has little overlap with OpenAI's offerings. The partnership agreement announced today is just outsourcing an accessory tool to a popular and suitably scaled vendor, the same as they did for web search and social network integration in the past. Nobody's leapfrogging anybody between these two because they're on totally different paths.
Siri is a toy, but I don't think that was Apple's intent. It's been a long-standing complaint that using Siri to search the web sucks compared to other companies offerings.
Apple's product focus is on getting Siri to bridge your first-party and third-party apps, your 500GB of on-device data, and your terabyte of iCloud data with a nice interface, all of which they're trying to deliver using their own technology.
Having Siri answer your trivia question about whale songs, or suggest a Pad Thai recipe modification when you ran out of soy sauce, is just not where they see the value. Poor web search has been an easy critique to weigh against Siri for the last many years, and the ChatGPT integration (and Apple's own local prompt prep) should fare far better than that, but it doesn't have any relevance to "leapfrogging" because the two companies just aren't trying to do the same thing.
That's the complaint! They play in the same space, they just don't seem to be trying. Siri happily returns links to Pad Thai recipes, it's not like they didn't expect this to be a use-case. They just haven't made a UX that competes with others.
And it's not just web search! Siri's context is abysmal. My dad routinely has to correct the spelling of his own name. It's a common name, there are multiple spellings, but it's his phone!
My favorite thing with names is I have some people in my contacts who have names that are phonetically similar to English words. When I type those words in a text or email, Siri will change those words to people’s names.
Apple bought Siri 14 years ago, derailed the progress and promise it had by neglect, and ended up needing a bail out from Sam once he kicked their ass in assistants.
Isn’t MS heavily invested in them and also letting them use Azure pretty extensively? Rather, I think this is more like an interesting model of a big tech company actually managing to figure out exactly how hands off they need to be, in order to not suffocate any ember of innovation. (In this mixed analogy people often put out fires with their bare hands I guess, don’t think too hard about it).
Change is inevitable in the AI space, and the changes come in fits and starts. In a decade OpenAI too may become a hapless fiefdom lorded over by the previous generation's AI talent.
Disagree. This feels more like the Google partnership with Apple' Safari that has lasted for long time. Except in this case, I think is OpenAI who will get the big checks.
This integration is way more limited and frictioned. Whereas with search Apple's fully outsourced and queries go straight to your 3rd-party default, Siri escalates to GPT only for certain queries and with one-off permissions. They seem to be calculating that their cross-app context, custom silicon, and privacy branding give them a still-worthwhile shot at winning the Assistant War. I think this is reasonable, especially if open source AI continues to keep pace with the frontier.
If Apple wasn't selling privacy, I'd assume the other way around. Or if anything, OpenAI would give the service out for free. There's a reason why ChatGPT became free to the public, GPT-4o moreover. It's obvious that OpenAI needs whatever data it can get its hands on to train GPT-5.
ChatGPT was free to the public because it was a toy for a conference. They didn't expect it to be popular because it was basically already available in Playground for months.
I think 4o is free because GPT3.5 was so relatively bad it means people are constantly claiming LLMs can't do things that 4 does just fine.
It's a win for OpenAI and AI. I remember someone on Hacker News commented that OpenAI is a company searching for a market. This move might prove that AI, and OpenAI, has a legitimate way to be used and profitable. We'll see.
Looking at their stock performance and the amount of work they’ve put into features that aren’t Dropbox file sync, he appears to have been right. iCloud doc syncing is what DB offered at that time.
I think he was right - now you've got OneDrive automatically bundled into Windows, iCloud in MacOS, Google Cloud in the Google ecosystem and Dropbox down 25% from IPO with no growth. I get nagging emails from them every month or so asking me to upgrade to a paid plan because I'll definitely not regret it.
Doubt that Apple can ever come up with a better LLM than OpenAI's, they stopped trying to make Siri as good as Google Assistant after 10+ years now. I don't think they are that good at Cloud or ML compared to other big techs
yeah somehow it reminded me of the fb integration too. we‘ll see how well it works in practice. i was hoping for them to show the sky demo with the new voice mode that openai recently demoed
Didn’t Apple say they’re using their own hardware for serving some of the AI workloads? They dubbed it ‘Private Cloud Compute’. Not sure how much of a vote of confidence it is for Nvidia.
Right, but are those going to run on Apple-owned hardware at all? It seems like Apple will first prioritize their models running on-device, then their models running on Apple Silicon servers, and then bail out to ChatGPT API calls specifically for Siri requests that they think can be better answered by ChatGPT.
I'm sure OpenAI will need to beef up their hardware to handle these requests - even as filtered down as they are - coming from all of the Apple users that will now be prompting calls to ChatGPT.
not necessarily so, in terms of tflops per $ (of apple’s cost of gpus, nit consumer), and tflops per watt their apple silicon is comparable if not better
flops/$ is simply not all (or even most) that matters when it comes to training LLMs.... Apple releases LLM research - all of their models are trained on nvidia.
Which is only a subset of requests Apple devices will serve and only with explicit user permission. That’s going to shrink over time as Apple continue to advance their own models and silicon.
Plus even if Apple is using their own chips for inferencing, they're still driving more demand for training, which Nvidia still has locked down pretty tight.
Interesting, I thought Apple Silicon mainly excelled at inferencing. Though I suppose the economics of it are unique for Apple themselves since they can fill racks full of barebones Apple Silicon boards without having to pay their own retail markup for complete assembled systems like everyone else does.
They say user data remains in the Secure Enclave at all times, which Nvidia GPUs would not be able to access. I am quite certain that their private cloud inference runs only Apple silicon chips. (The pre-WWDC rumors were that they built custom clusters using M2 Ultras.)
> They say user data remains in the Secure Enclave at all times
No they don't. They say that the Secure Enclave participates in the secure boot chain, and in generating non-exportable keys used for secured transport. It reads to me as though user devices will encrypt requests to the keys held in the Secure Enclave of a subset of PCC nodes. A PCC node that receives the encrypted request will use the Secure Enclave to decrypt the payload. At that point, the general-purpose Application Processor in the PCC node has a cleartext copy of the user request for doing the needful inference, which _could_ be done on an NVidia GPU, but appears to be done on general-purpose Apple Silicon.
There is no suggestion that the user request is processed entirely within the Secure Enclave. The Secure Enclave is a cryptographic coprocessor. It almost certainly doesn't have the grunt to do inference.
Not that it matters anyways, since Apple refuses to sign Nvidia GPU drivers for MacOS in the first place. So if they own any Nvidia hardware themselves, then they also own more third-party hardware to support it.
Maybe this is way too science fiction, but what are the chances Apple's GPU/AI engine designs on Apple Silicon were a testbed for full sized, dedicated GPU dies that could compete with Nvidia's power in their own data centers?
Very low? I guess anything is possible, but the M1 through M4 GPUs weren't really anything to write home about. It more closely resembles AMD's raster-focused GPU compute in my opinion, which is certainly not a bad thing for mobile hardware.
Nvidia's GPUs are complex. They have a lot of dedicated, multipurpose acceleration hardware inside of them, and then they use CUDA to tie all those pieces together. Apple's GPUs are kinda the opposite way; they're extremely simple and optimized for low-power raster compute. Which isn't bad at all, for mobile! It just gimps them design-wise when they go up against purpose-built accelerators.
If we see Apple do custom Apple Silicon for the datacenter, it will be a pretty radically new design. The first thing they need is good networking; a full-size Nvidia cluster will use Mellanox Infiniband to connect dozens of servers at Tb/s speeds. So Apple would need a similar connectivity solution, at least to compete. The GPU would need to be bigger and probably higher-wattage, and the CPU should really emphasize core count over single-threaded performance. If they play their cards right there, they would have an Apple Silicon competitor to the Grace superchip and GB200 GPU.
Yes, but GP was talking about the AI workloads Apple will be running on their own servers (which are indeed distinct from those explicitly labeled as ChatGPT).
They are the first ones to ship on-device inference at scale on non-nvidia hardware. Apple also has the means to build data center training hardware using apple silicon if they want to do so.
If they are serious about the OAI partnership they could also start to supply them with cloud inference hardware and strongarm them into only using apple servers to serve iOS requests.
> They are the first ones to ship on-device inference at scale on non-nvidia hardware
Which is neat, but it's not CUDA. It's an application-specific accelerator good at a small subset of operations, controlled by a high-level library the industry is unfamiliar with and too underpowered to run LLMs or image generators. The NPU is a novelty, and today's presentation more-or-less confirmed how useless it is for rich local-only operations.
> Apple also has the means to build data center training hardware using apple silicon if they want to do so.
They could, but that's not a competitor against an NVL72 with hundreds of terabytes of unified GPU memory. And then they would need a CUDA competitor, which could either mean reviving OpenCL's rotting corpse, adopting Tensorflow/Pytorch like a sane and well-reasoned company, or reinventing the wheel with an extra library/Accelerate Framework/MPS solution that nobody knows about and has to convert models to use.
So they can make servers, but Xserve showed us pretty clearly that you can lead a sysadmin to MacOS but you can't make them use it.
> they could also start to supply them with cloud inference hardware and strongarm them into only using apple servers to serve iOS requests.
I wonder how much money they would lose doing that, over just using the industry-standard Nvidia servers. Once you factor in the margins they would have made selling those chips as consumer systems, it's probably in the tens-of-millions.
> reinventing the wheel with an extra library/Accelerate Framework/MPS solution that nobody knows about and has to convert models to use.
This is Apple's favorite thing in the world. They already have an Apple-Silicon-only ML framework as of a few months ago, called MLX. Does anyone know about it? No. Do you need to convert models to use it? Yes.
You're approaching this from a developers point of view.
Users absolutely don't care if their prompt response has been generated by a CUDA kernel or some poorly documented apple specific silicon a poor team at cupertino almost lost their sanity to while porting the model.
And haven't they already spent quite a bit on money on their pytorch-like MLX framework?
> Users absolutely don't care if their prompt response has been generated by a CUDA kernel or some poorly documented apple specific silicon
They most certainly will. If you run GPT-4o on an iPhone with MLX, it will suck. Users will tell you it sucks, and they won't do so in developer-specific terms.
The entire point of this thread is that Apple can't make users happy with their Neural Engine. They require a stopgap cloud solution to make up for the lack of local power on iPhone.
> And haven't they already spent quite a bit on money on their pytorch-like MLX framework?
As well as Accelerate Framework, Metal Performance Shaders and previously, OpenCL. Apple can't decide where to focus their efforts, least of which in a way that threatens CUDA as a platform.
But the point stands, these systems occupy a niche that Apple Silicon is poorly suited to filling. They run normal Linux, they support common APIs, and network to dozens of other machines using Infiniband.
> Apple also has the means to build data center training hardware using apple silicon if they want to do so.
> If they are serious about the OAI partnership they could also start to supply them with cloud inference hardware and strongarm them into only using apple servers to serve iOS requests
Apple addressed both these points in today’s preso.
1. They will send requests that require larger contexts to their own Apple Silicon-based servers that will provide Apple devices a new product platform called Private Cloud Compute.
2. Apple’s OS generative AI request APIs won’t even talk to cloud compute resources that do not attest to infrastructure that has a publicly available privacy audit.
> Apple also has the means to build data center training hardware using apple silicon if they want to do so.
i'm seeing people all over this thread saying stuff like that, it reads like fantasyland to me. Apple doesn't have the talent or the chips or suppliers or really any of the capabilities to do this, where are people getting it from?
Apple is already one of the largest
(if not the largest) customers of TSMC and they have plenty of experience designing some of the best chips on the most modern nodes.
Their ability to design a chip and networking fabric which is fast/efficient at training a narrow set of model architecture is not far fetched by any means.
It's worth noting that one of Apple's largest competitor at TSMC is, in fact, Nvidia. And when you line the benchmarks up, Nvidia is one of the few companies that consistently beats Apple on performance-per-watt even when they aren't on the same TSMC node: https://browser.geekbench.com/opencl-benchmarks
Yes, also covered explicitly in the keynote that Apple user's requests to openAI are not tracked. (Plus you have the explicit opt-in to even access chatGPT via siri in the first place.)
There is a wide gap between complying with law enforcement requests and judicial orders and intentionally lying. Yes, if Apple can (trivially) read your data, then one must assume that at least the US government can access your data! Though if that's in your threat model I have a couple of other bad news items for you. Apple actively reduces that surface by moving ~everything to ee2e storage with keys held on customer devices. This is pretty transparently the attempt to be able to say "sorry can't do that without changing OS code and for _that_ discussion we have won in court. Really sorry that we can't help you". And yes, that's probably just to decrease the compliance costs. Still same result
Apple's put ChatGPT integration on the very edge of Apple Intelligence. It's a win for OpenAI to have secured that opportunity, and Nvidia wins by extension (as long as OpenAI continues to rely on them themselves), but the vast majority of what Apple announced today appears to run entirely on Apple Silicon.
You seem to be positioning this as a Ford vs Chevy duel, when (to me at least) the comparison should be to Ford vs Exxon.
Nvidia is an infrastructure company. And a darned good one. Apple is a user facing company and has outsourced infrastructure for decades (AWS & Azure being two of the well known ones).
Apple outsourced chips to IBM (PowerPC) for a long time and floundered all the while. They went into the game themselves w/ the PA Semi acquisition and now they have Apple Silicon to show for it.
But Apple is vertically integrating. Thats like Ford buying Bridgestone.
The only way it hurts Nvidia is if Apple becomes the runaway leader of the pc market. Even then, Apple hasn’t shown any intent of selling GPUs or AI processors to the likes of AWS, or Azure or Oracle, etc.
Nvidia has a much bigger threat from with Intel/AMD or the cloud providers backward integrating and then not buying Nvidia chips. Again, no signs that Apple wants to do this.
I think Apple is going to make rapid and substantial advancements in on-device AI-specific hardware. I also think nVIDIA is going to continue to dominate the cloud infrastructure space for training foundational models for the foreseeable future, and serving user-facing LLM workloads for a long time as well.
Nvidia obviously has an enormous, enormous moat but I do think this is one of the areas in which Apple may actually GAF. The rollout of Apple Intelligence is going to make them the biggest provider of "edge" inference on day one. They're not going to be able to ride on optimism in services growth forever.
It took almost a decade but the PA Semi acquisition showed that Apple was able to get out of the shadow of its PowerPC era.
Nvidia will remain a leader in this space for a long time. But things are going to play out wonky and Apple, when determined, are actually pretty good at executing on longer-term roadmaps.
Apple could have moved on Nvidia but instead they seem to have thrown in the towel and handed cash back to investors. The OpenAI deal seems like further admission by Apple that they missed the AI boat.
Exactly. Apple really needs new growth drivers and Nvidia has a 3bn market cap Apple wants to take a bite out of. One of the few huge tech growth areas that Apple can expand into.
I am of course wrong frequently, but I cannot see how that would happen.
If they create cpu/gpus that are faster/better than what Nvidia sells,
but they only sell them as part of a Mac desktop or laptop systems
it wont really compete.
For that they would have to develop servers that has a mass amount of
whatever it is or sell the chips in the same manner Nvidia does today.
I dont see that future for Apple.
Microsoft / Google / or other major cloud companies would do extremely well
if they could develop it and just keep it as a major win for their cloud
products.
Azure is running OpenAI as far as I have heard.
Imagine if M$ made a crazy fast GPU/whatever.
It would be a huge competitive advantage.
Well, good luck to Apple then. Hopefully this attempt at killing Nvidia goes better than the first time they tried, or when they tried and gave-up on making OpenCL.
I just don't understand how they can compete on their own merits without purpose-built silicon; the M2 Ultra doesn't shine a candle to a single GB200. Once you consider how Nvidia's offerings are networked with Mellanox and CUDA universal memory, it feels like the only advantage Apple has in the space is setting their own prices. If they want to be competitive, I don't think they're going to be training Apple models on Apple Silicon.
It's ripe for attack. But Nvidia is still in its growing phase, not some incumbent behemoth. The way Nvidia ruthlessly handled AMD tell us that they are ready for competition.
Let's check in with OpenCL and see how far it got disrupting CUDA.
You see, I want to live in a world where GPU manufacturers aren't perpetually hostile against each other. Even Nvidia would, judging by their decorum with Khronos. Unfortunately, some manufacturers would rather watch the world burn than work together for the common good. Even if a perfect CUDA replacement existed like it did with DXVK and DirectX, Apple will ignore and deny it while marketing something else to their customers. We've watched this happen for years, and it's why MacOS perennially cannot run many games or reliably support Open Source software. It is because Apple is an unreasonably fickle OEM, and their users constantly pay the price for Apple's arbitrary and unnecessary isolationism.
Apple thinks they can disrupt AI? It's going to be like watching Stalin try to disrupt Wal-Mart.
> Let's check in with OpenCL and see how far it got disrupting CUDA.
That's entirely the fault of AMD and Intel fumbling the ball in front of the other team's goal.
For ages the only accelerated backend supported by PyTorch and TF was CUDA. Whose fault was that? Then there was buggy support for a subset of operations for a while. Then everyone stopped caring.
Why I think it will go different this time: nVidia's competitors seem to have finally woken up and realized they need to support high level ML frameworks. "Apple Silicon" is essentially fully supported by PyTorch these days (via the "mps" backend). I've heard OpenCL works well now too, but have no hardware to test it on.
Eh, it seems from the keynote that ChatGPT will be very selectively used, while most features will be powered by on-device processing and Apple's own private cloud running apple silicon.
So all in all, not sure if it's that great for Nvidia.
If OpenAI is furiously buying GPUs to train larger models and Apple is handing OpenAI cash, then this seems like a win for Nvidia. You can argue about how big of a win, but it seems like a positive development.
What would not have been positive for Nvidia is Apple saying they've adapted their HW to server chips and would be partnering with OpenAI to leverage them, but that didn't happen. Apple is busy handing cash back to investors and not seriously pursuing anything but inference.
GPT4o access is a handy feature, but, what I was hoping to hear about is an improvement in Siri's language "understanding."
In today's WWDC presentation, there were a few small examples of Siri improvements, such as an ability to maintain context, e.g., 'Add her flight arrival time to my calendar,' wherein Siri knows who "her" refers to.
In my day-to-day experience with Siri, it's clear Siri doesn't have the kind of ability to understand language that LLMs provide. It still feels like clever son-of-Eliza hacks with stock phrases. If your utterance doesn't match with a pre-programmed stock phrase, it doesn't work. The other day I said something like "Play the song you played before the one I asked you to skip," and Siri didn't seem to know what I wanted. OTOH, GPT4o can easily handle statements like that.
Does anyone know to what extent Siri's underlying language models are being upgraded?
I agree, this is the biggest annoyance with voice assistants today. The good news is that, as you noted, the technology to interpret complex/unclear requests is definitely already here today with ChatGPT.
I think that Apple demoed this today where the presenter changed her mind mid-sentence during a weather query.
I'm hopeful that means they've added a LLM to interpret the intent of user requests.
That's something that I keep wondering about. The existing voice assistants are all garbage across the board. Whatever you say about Siri, Google's assistant is even worse. Meanwhile, for the past couple months, I was able to fire up ChatGPT app and speak to it casually, in noisy environments, and it would both correctly convert my speech to text (with less than 5% errors) and correctly understand what I'm actually saying (even in presence of transcription errors).
All it takes to make a qualitatively better voice assistant would be to give GPT-4 a spec of functions representing things it can do on your phone, and integrating that with the OS. So why none of the companies bothered to do it? For that matter, I wonder why OpenAI didn't extend the ChatGPT app in this direction?
> In today's WWDC presentation, there were a few small examples of Siri improvements, such as an ability to maintain context, e.g., 'Add her flight arrival time to my calendar,' wherein Siri knows who "her" refers to.
Didn't Cortana do this? Pretty underwhelming in 2024.
I thought from the Apple keynote that Siri is getting a big update to be based on Apple Intelligence, not that this context stuff was getting hacked into the existing Siri model. They talked about new voice transcription features, the ability to correct yourself while talking, deep knowledge of your personal context, etc.
It sounds like a bigger update, where they’re applying gen AI models more broadly across tons of things (including I things like photo categorization), but I guess we’ll see.
This sounds like exactly what I wanted. There have been a number of times I've been in the car and wanting to ask Siri something it couldn't handle has been a lot e.g. "What state am I in, and how far am I to the border to the state I'm going to cross next, and can I pump my own gas on each state I'm driving through?"
Though a bit of that is premised on whether it could extract information from google maps.
I think most of what you're talking about is going through Apple Intelligence, not chatGPT. That "Apple Intelligence" stuff is supposed to be more local and personal to you, accounting for where you are, your events, things like that. There's an API for apps to provide "intents," which Siri can use to chain everything together. (Like "cost of gas at the nearest gas station" or something like that.) None of that is OpenAI, according to the keynote.
Carplay Siri functionality is currently neutered. A lot of times it won't answer more complex questions that would otherwise be answered without Carplay.
I haven't found this to be the case. Does Siri explicitly refuse to answer questions, or does it misunderstand you? Maybe the microphone in your car makes hearing difficult?
> "What state am I in, and how far am I to the border to the state I'm going to cross next, and can I pump my own gas on each state I'm driving through?"
What kind of trip was this where these were pertinent questions? Couldn't you have just rephrased most of them?
"What is my current location?"
"Show maps."
"Which states don't allow you to pump your own gas?"
>... and can I pump my own gas on each state I'm driving through?
Huh? Seems like an odd thing to feel the need to ask, as up until last year, the answer was always, "Only if you're driving through Oregon or New Jersey".
It’s an interesting vote of confidence in OpenAI’s maturity (from a scale and tech perspective) to integrate it as a system wide, third-party dependency available to all users for free.
OpenAI is such a controversial company and good competitors like Anthropic, who arguably align better with their brand, exist. That makes the deal so weird to me.
Anthropic could be 10X better, and it still wouldn't matter to customers and public market investors as much as hearing the name 'ChatGPT.' Your mom has never heard of Anthropic.
People think Google won search because they had the best search engine. Yes, they did for a brief period before others implemented similar methods to pagerank (which itself was ripped off from the Baidu founder). But the reason they won the market is because of endless media coverage around them building the brand into a household verb. After that, it was impossible for anyone to compete.
"Just ask ChatGPT" will forever be the "Just Google it" of AI, and any media drama surrounding OpenAI only serves to cement that status.
It's also weird because Anthropic models are just better for these tasks. Claude responses are almost always better than GPT4.
I stopped using GPT4 because it would just yap on and on about things I don't want in the response. Claude 3 responses feel way more human like because it response with similar information a human would and not with a bunch of unneeded gibber.
By the time this roles out at the end of the year who knows what models would be the best. Why bet on one company's models? We have seen how fast open source models have caught up to GPT4. Why put all your chips into one basket?
OpenAI has nothing of particularly high value. They're giving away the store right now just to claim the onboarding. This unsustainable game will end badly and soon.
It's actually a beneficial feature that two people can look at a market and come to two completely different conclusions about it. Yes, I suspect that OpenAI has nothing of lasting competitive value, they're currently overvalued by entities who want their money back, and you can view their recent actions and partnerships through this lens without complication.
I was surprised how little they are leaning on OpenAI. Most of the impressive integrations that actually look useful are on-device or in their private cloud. OpenAIs ChatGPT was relegated to a corner of Siri for answering "google queries", if you grant it permission. This seems like an L for OpenAI, not being a bigger part of the architecture (and I'm glad).
Agreed. The rumors beforehand made it sound Apple and OpenAI would practically be merging. This felt like a fig leaf so Apple could say you can access SOTA models from you iPhone. But for me personally, the deep integration with the ecosystem + semantic index are way way more interesting.
I still don't know a single person who wants this crap. I don't want "AI" in my web browser, I don't want it in my email client, I don't want it on my phone, I just don't want it. And it feels like everyone I speak to agrees! So who is this all for?
It did help me to translate nursery rhymes for my kid from one language to another while they still rhyme and mean approximately the same thing. It sucked in gpt-3 but 4o (or whatever is the latest one) is actually really great for that.
It excels in "transfering style from one thing to another thing" basically.
However every time I asked it a factual thing that I couldn't find on Google, it was always hilariously wrong
I actually want a virtual assistant that can reliably process my simple requests. But so far all these companies look like they are still in the figuring out phase, basically throwing everything at the wall to see what sticks. Hopefully after 2 or 3 years things will settle down and we will get a great virtual assistant.
I highly agree. And everything it has generated so far has been incredibly mid. Yeah, there may be some legitimate use cases but as it usually goes everyone is overdoing it head first without really thinking enough about it beforehand.
Me! I’m dumping text I write into an LLM all-day to help with editing. And I often start brainstorming / research by opening ChatGPT in voice mode, talk to it and keep a browser open at the same time to fact-check the output.
Now you know a few. I love the idea of being able to ask my phone for things like "the guy who emailed last week about the interview, what was his name?" without having to dig through emails trying not to lose the context in my head.
I don't really care for AI in google search results or email. It's often wrong and not what I'm looking for. I would like a much better Siri, so hopefully that's part of what we get.
LLMs are very useful and very helpful, certainly more helpful than ony searching the web. Watching people apply the crypto lens to it is unfortunate for them, it's not a waste of electricity like most crypto, and it isn't useless output.
I may be wrong, but the first GPT response says that kanji means "spirit" "soul" or "ghost" but a quick Google search says it means "drops of rain"... do you trust GPT on this matter?
Yes, the top radical is for drops of rain but the i inclusion of the bottom part has a meaning that clearly aligns with spirit, especially when you see the rare kanji that use it as a component. I only was curious as it was part of another kanji (孁) that I was investigating.
Everyone wanted something like an iphone and when it came it took over the market. We had a product that got shifted into a third world product overnight.
This is one of those things that seems like a good idea but is really an existential threat to OpenAI.
Having a single extremely large customer gives that customer a disproportionate amount of power over your business. Apple can decide one day to simply stop paying you because, hey, they can afford the years of litigation to resolve it. Can you weather than storm?
Famously, Benjamin Moore (the paint company) maintains its own stores. They have not (and probably will not) sell their products through Home Depot or Lowe's. Why? This exact reason. A large customer can dictate terms and hold you over a barrel if they so choose.
AI/ML is something Apple cares about. They've designed their own chips around speeding up ML processing on the device. A partnerhship with OpenAI is clearly a stopgap measure. They will absolutely gut OpenAI if they have the opportunity and they will absolutely replace OpenAI when they can.
Apple just doesn't like relying on partners for core functionality. It's why Apple ditched Google Maps for the (still inferior) Apple Maps. The only reason they can't replace Google Search is because Google pays them a boatload of money and they've simply been unable to.
This may seem like a good move for OpenAI but all they've done is let the foxes run the hen house.
Is there a single citation for anything you just said?
> Apple can decide one day to simply stop paying you because, hey, they can afford the years of litigation to resolve it.
OpenAI and Microsoft also can do the same. Microsoft would be ecstatic to hurt Apple in any way. Also Apple has also no history of doing this with any of the providers they use.
> Benjamin Moore (the paint company) maintains its own stores. They have not (and probably will not) sell their products through Home Depot or Lowe's. Why?
Because Home Depot has their own brand, Behr. Each Behr color explicitly says what Benjamin Moore color it's copying, and they take 100% of the revenue as a direct alternative. Do you have any sources on this being a Benjamin Moore decision?
> It's why Apple ditched Google Maps for the (still inferior) Apple Maps.
How do you define "still inferior"? How many times a day do you use Apple Maps? Do you have any benchmarks that compare the two?
OpenAI already had a “single extremely large customer”: Microsoft. In fact the Apple deal is the first sign they’re not just a de facto Microsoft subsidiary.
> Privacy protections are built in when accessing ChatGPT within Siri and Writing Tools—requests are not stored by OpenAI, and users’ IP addresses are obscured. Users can also choose to connect their ChatGPT account, which means their data preferences will apply under ChatGPT’s policies.
So does this mean that by default, a random Apple user won't have their ChatGPT requests used for OpenAI training, but a paying ChatGPT Plus customer will?
Does this also mean that if I connect my ChatGPT Plus account that my data will be used for training?
It just seems strange to have a lower bar for privacy for paying customers vs users acquired via a partnership.
(yes I'm aware that the "Temporary Chat" feature or turning off memory will prevent data being used for training)
You can permanently disable OpenAI from training with your chat data for your account:
“To disable model training, navigate to your profile icon on the bottom-left of the page and select Settings > Data Controls, and disable “Improve the model for everyone." While this is disabled, new conversations won’t be used to train our models”
Great to know! Looks like they only made this change at the beginning of May. Prior to that you had to turn off chat history which wasn't worth it to me.
April 25, 2024: "To disable chat history and model training, navigate to ChatGPT > Settings > Data Controls and disable Chat history & training. While history is disabled, new conversations won’t be used to train and improve our models, and won’t appear in the history sidebar. To monitor for abuse, we will retain all conversations for 30 days before permanently deleting."
https://web.archive.org/web/20240425194703/https://help.open...
May 02, 2024: "To disable model training, navigate to your profile icon on the bottom-left of the page and select Settings > Data Controls, and disable “Improve the model for everyone.“ While this is disabled, new conversations won’t be used to train our models."
https://web.archive.org/web/20240502203525/https://help.open...
Companies really don’t like being sued for hundreds of millions in punitive damages just for the benefit of training on the small percentage of people that opt out.
That must be some really detailed 100+ pages contract.
I bet Microsoft is mentioned multiple times with things to the effect of: "Under no condition is Microsoft allowed to access any of the data coming from iPhones."
And also, that would still be more useful than the current situation where Siri would just answer that it can not give you the weather forecast because there is no city named "Appointment at 10".
~~This is not the direction I was hoping Apple would go with AI.
With all the neural this and that bits baked into apple silicon, it has seemed [0] for a while that Apple wanted to run all these workloads locally, but this partnership seems like a significant privacy backslide.
Another comment in this thread said something about they’re using b Apple silicon for these workloads, but didn’t give an indication of whether that silicon lives in Apple datacenters or OpenAI ones.
edit: I should have mentioned that I didn’t have a chance to watch the video yet; a reply to my comment mentioned that it’s addressed in the video so I’ll go watch that later
I don't think this is a fair take. It sounds like the vast majority of the new AI features (including the local personal context for Siri, the various text/image editing features, better photo categorization, and the list goes on) are all local, on-device models, which can, if needed, use Apple's private cloud. That requires public researcher verification of server software for iOS to even talk to it. (Allegedly :))
The OpenAI partnership is seemingly only if Siri decides it doesn't have the full context needed to answer. (E.g. if you ask something very creative/generative.) At that point, Siri says "hey, chatGPT might be better at answering this, do you consent to me sending your prompt to them?" and then you get to choose. Apple's partnership also seems to include the various settings that prevent OpenAI from tracking/training on the prompts sent in.
Honestly, that more creative side of genAI is not as interesting in the full context of Apple Intelligence. The real power is coming from the local, personal context, where Siri can deduce context based on your emails, messages, calendar events, maps, photos, etc. to really deeply integrate with apps. (Allegedly!) And that part is not OpenAI.
Agreed. Apple pretty clearly focused on building an action-tuned model. Also, notice how in the videos you barely see any "Siri speech". I wonder what they used for pre-training, but probably they did it with much more legit datasources-- They're launching with English only.
Apple is in the position where it caters primarily to the tech ignorant, so coming out an explaining that Apple LLM is a bit worse (read: far worse) than the cool LLM's on the internet because they are privacy conscious is a non-starter.
Local LLM's on regular local hardware (i.e. no $500+ dedicated GPUs) is way far behind SoTA models right now.
Apple is not gonna be in a position where you can practically real-time intelligently chat with Android phones while iPhones are churning out 3 tokens/second of near useless babbling.
I completely agree about the market positioning and not keeping up with other platforms’ abilities being a non-starter. I just hope it will be clear how to keep my external brain (phone) from being scanned by OpenAI.
(I don’t want it to seem like I’m just a hater of either Apple or OpenAI; I’m a more-recent adopter of Apple tech and I’m not looking back, and I have an OpenAI subscription and find it invaluable for some uses.)
Another thing I’m going to be looking for in the video is how this initiative jibes with the greenness Apple has been passing really hard. If they’re bringing this kind of generative AI from niches to every iphone, it seems that would add a fair amount of power consumption.
> I just hope it will be clear how to keep my external brain (phone) from being scanned by OpenAI.
It's very clear, the keynote demonstrates that Siri passing a prompt to chatGPT is completely opt-in and only happens when Siri thinks the prompt needs the more generative/creative model that OpenAI provides.
I should have put a disclaimer saying that I hadn’t had a chance to watch the video yet. Thanks for mentioning that it’s addressed, I’ll take a look later.
I think the headlines are REALLY muddying things. From watching the Keynote, most of Apple Intelligence is their own stuff, mostly on-device.
Siri explicitly asks you if you want to use chatGPT to answer a query. It does so when it thinks chatGPT will have a better answer. It sounds like that will be for very creative/generative types of things like "please create a 4 course meal with xyz foods," at which point Siri asks you if you want to use chatGPT. It will be very clear, according to Apple.
That said, the Apple Intelligence vs. OpenAI distinction seems much clearer than the Apple cloud vs. local distinction, which I find somewhat concerning.
Sure, the Apple cloud is ultra-secure and private and all, but I'd still like to know what happens where without having to test it myself by enabling airplane mode and seeing what still works.
When you ask Siri a question, it will prompt you to ask whether it can send your query/data to ChatGPT.
All other AI features within the OS are powered by Apple's Private Compute Cloud, which is Apple's code running on Apple's chips at Apple's Data Center.
> All other AI features within the OS are powered by Apple's Private Compute Cloud
Clarification: All other AI features within the OS are powered by on device models which can reach out to the private cloud for larger workflows & models.
I'm convinced that GOOG has the necessary engineering chops to pull the same thing off (or to put it less charitably, copy Apple), but hitherto they were hindered by bad product manager decisions leading them to engineer the wrong thing.
And why wouldn't it be? The strain on Microsoft servers and the free use of their resources by iOS users with very little, if any, in return is a win for Apple. Not so much for OpenAI or Microsoft.
Looking at the very pretty marketing page over at Apple's I can honestly say: I've not a single use case for this. I'm sure there's someone who has, but I have to jump through several mental hoops to even imagine how any of this might be barely helpful in very rare edge cases for me.
From watching it, it seems like it’s just a kit type integration as it’s super clear that it’s going to a partner and they said they may allow other partners.
What would you prefer? Less capable products with fewer features? Or a Google product designed in collaboration with their advertising data hoovering team?
My biggest disappointment was that Apple said nothing about leveraging GPT-4 to improve voice recognition in iMessage. Voice recognition of ChatGPT is incredibly accurate when compared to iOS. ChatGPT almost never gets anything wrong, while iMessage/iOS voice recognition is extremely frustrating.
So much so that I sometimes dictate to ChatGPT then cut & paste into iMessage.
You can set up a shortcut that will record you, hit the Whisper API, then copy to your clipboard. It's not as smooth as native transcription or the SOTA on Google phones but it's pretty good.
On-device models will not be big enough in the near future. What makes ChatGPT so awesome at recognition is that their model is huge, and so no matter how obscure the topic of the dictation, ChatGPT knows what you're talking about.
Apple also talked about their private compute cloud, which allows larger models and workflows to integrate with local AI models. It sounds like they will figure out which features require bigger models and which don't. So I think there is a lot of room for what you're mentioning in the future of this AI platform.
Plus, they talk about live phone call transcriptions, voice transcription in notes, the ability to correct words as you speak, contextual conversations in siri, etc. It 100% sounds like better voice recognition is coming
Pretty sure transcription is done locally on Pixel phones and it's pretty good. Not as good as ChatGPT, but most of the way there. If current iOS is like a 50, Pixel is like a 90 and OpenAI is like 98.
Honestly I was surprised at how limited the ChatGPT integration seems to be. It felt like they 80/20'd AI with the onboard models + semantic index, but also wanted to cover that last 20% with some kind of SOTA cloud model. But they didn't necessarily NEED to.
They need to in order to not look second-class in terms of chat capabilities. On the other hand, they want to make it clear when you are using ChatGPT, probably not just for privacy reasons, but also so that people blame ChatGPT and not Apple when it gets things wrong.
This may just be me because I'm a heavy ChatGPT user as-is, but I've had my fill of chat capabilities. What I really want is the context awareness, which is what they seemingly delivered on without OpenAI's help!
Note that this is announced as coming in beta this fall, which means they are currently well pre-beta. I would curb my expectations about how well it will work.
Um, wow. The major question in my mind: did Apple pay, or did OpenAI pay? (A-la google for search).
Apple is not going to lose control of the customer, ever, so on balance I would guess this is either not a forever partnership or that OpenAI won’t ultimately get what they want out of it. I’m very curious to see how much will be done on device and how much will go to gpt4o out of the gate.
I’m also curious if they’re using Apple intelligence for function calling and routing in device; I assume so. Extremely solid offerings from Apple today in general.
I don't believe that. Apple is in the driver's seat in this negotiation. I believe OpenAI wanted Apple as a jewel in their crown and bent over backwards to get them to sign. I don't see how OpenAI makes any money off of this, but I do see them losing a lot of money as iOS users slam their service for free as they eat the costs.
> Privacy protections are built in when accessing ChatGPT within Siri and Writing Tools—requests are not stored by OpenAI, and users’ IP addresses are obscured.
Does anybody believe Apple will not be able to know who sent a given request, and that OpenAI won't be able to use the data in the request for more or less anything they want? I read statements like this and just flat-out don't believe them anymore.
Apple clearly thinks it needs a dedicated LLM service atm. But still thinks it is only supplemental as they handle a bunch of the core stuff without it. And require explicit user consent to use OpenAI. And Apple clearly views it as a partial commodity since they even said they plan to add others.
Tough to bet against OpenAI right now...but this deal does not feel like a 10 year deal...