MLX doesn't use the neural engine still right? I still wish they would abandon that unit and just center everything around metal and tensor units on the GPU.
MLX is a training/research framework, and the work product is usually a CoreML model. A CoreML model will use any and all resources that are available to it, at least if the resource fits for the need.
The ANE is for very low power, very specific inference tasks. There is no universe where Apple abandons it, and it's super weird how much anti-ANE rhetoric there is on this site, as if there can only be one tool for an infinite selection of needs. The ANE is how your iPhone extracts every bit of text from images and subject matter information from photos with little fanfare or heat, or without destroying your battery, among many other uses. It is extremely useful for what it does.
>tensor units on the GPU
The M5 / A19 Pro are the first chips with so-called tensor units. e.g. matmul on the GPU. The ANE used to be the only tensor-like thing on the system, albeit as mentioned designed to be super efficient and for very specific purposes. That doesn't mean Apple is going to abandon the ANE, and instead they made it faster and more capable again.
> ...and it's super weird how much anti-ANE rhetoric there is on this site, as if there can only be one tool for an infinite selection of needs
That seems like a strange comment. I've remarked in this thread (and other threads on this site) about what's known re: low-level ANE capabilities, and it seems to have significant potential overall, even for some part of LLM processing. I'm not expecting it to be best-in-class at everything, though. Just like most other NPUs that are also showing up on recent laptop hardware.
An enormous number of people and products. I'm actually not sure if your comment is serious, because it seems to be of the "I don't, therefore no one does" variety.
I'm super not interested in arguing with the peanut gallery (meaning people who don't know the platform but feel that they have absolute knowledge of it), but enough people have apps with CoreML models in them, running across a billion or so devices. Some of those models were developed or migrated with MLX.
You don't have to believe this. I could not care less if you don't.
I don't believe it. MLX is a proprietary model format and usually the last to get supported on Huggingface. Given that most iOS users aren't selecting their own models, I genuinely don't think your conjecture adds up. The majority of people are likely using safetensors and GGUF, not MLX.
If you had a source to cite then it would remove all doubt pretty quickly here. But your assumptions don't seem to align with how iOS users actually use their phone.
Cite a source? That CoreML models are prolific on Apple platforms? That Apple devices are prolific? Search for it yourself.
You seem set on MLX and apparently on your narrow view of what models are. This discussion was about ANE vs "tensor" units on the GPU, and someone happened to mention MLX in that context. I clarified the role of MLX, but that from an inference perspective most deployments are CoreML, which will automatically use ANE if the model or some subset fits (which is actually fairly rare as it's a very limited -- albeit speedy and power efficient -- bit of hardware). These are basic facts.
>how iOS users actually use their phone.
What does this even mean? Do you think I mean people are running Qwen3-Embedding-4B in pytorch on their device or something? Loads of apps, including mobile games, have models in them now. This is not rare, and most users are blissfully unaware.
The neural engine not having a native programming model makes it effectively a dead end for external model development. It seems like a legacy unit that was designed for cnns with limited receptive fields, and just isn't programmable enough to be useful for the total set of models and their operators available today.
That's sadly true, over in x86 land things don't look much better in my opinion. The corresponding accelerators on modern Intel and AMD CPUs (the "Copilot PCs") are very difficult to program as well. I would love to read a blog post on someone trying though!
Wrt. language models/transformers, the neural engine/NPU is still potentially useful for the pre-processing step, which is generally compute-limited. For token generation you need memory bandwidth so GPU compute with neural/tensor accelerators is preferable.
I think the bigger deal for founders is that Nvidia can decide at a whim to deny you supply or, better yet give 100 billion dollars to your competitor.
The em4 does not cost 10000 dollars. It is going to sell for well under 1000$. Hesai's low end ATX is supposed going to be less than 300$, and the AT128 already sells for 400-500$.
You make it sound simple. Bazel's docs are minimal and what exists is often wrong. For a C or C++ toolchain you might turn to a contributing project such as toolchains_llvm, but even those experts are still figuring it out, and they give you little guidance on how to assemble your sysroot, which is the actual issue in the article. And, the upstream distributions of LLVM are built under certain assumptions, so unless you know exactly what you are doing, you probably also want to build LLVM instead of using their archives. If you figure out how to build LLVM in a completely consistent and correct way with respect to the STL and support libraries, you will be perhaps the 3rd or 4th person in history to achieve it.
You include the target glibc as part of your toolchain
And, I always prefer to actually pick my glibc independently from the host os and ship it with the binary (generally in an OCI image). This way you can patch/update the host without breaking the service.
no, you just need a compatible starlit/glibc. you pick an old version (e.g. RHEL8) or otherwise compile with multiple tool chains if you need newer glibc features
Why do all of you think prices haven't come down? I can buy an AT128 from hesai for a few hundred dollars in volume. It's higher performance than any spinning lidar I could buy in 2017.
You may have interpreted that and that is not what I said.
Once a product starts to sell after initial design, time is take to reduce the development cost. Try to reuse parts or replace part A with B. A machine from early 2018 can be little different than ones going out the door late 2018. _Kaizen_ was coined for this.
My point of view of when mass reduction in cost will be when self-driving is cost effect secondary feature on all Toyota vehicles. I see that as the litmus test for knowing that self-driving has reached true utility.
Also well designed vehicles would need a multi-sensor system to operate in self-driving mode. A human operating a car is using multi-sensor intake. Lack of multi-sensor in humans prevent them from operating a vehicle. Blind people need a secondary sensory input like walking stick. Vehicles need a multi-sensor system to prevent harming, mutilating, and killing the passengers and pedestrians.
In what universe is 'a few hundred dollars is way too much' for implementing full self-driving on an autonomous vehicle that moves like, and at the speeds of and in the spaces of, an automobile?
A two to four ton vehicle that can accelerate like a Ferrari and go over 100 mph, fully self-driving, and 'a few hundred dollars is way too much'.
Disagree. Even as they are dialing back the claims, which may or may not affect how people use the vehicles. These things respond too quickly for flaky senses based on human sensoriums.
Lidar is being manufactured in china in the volume of millions a year by robosense, Huawei, and hesai. Bom cost is on the order of a few hundred dollars - slightly more than automotive radar. The situation is a lot different in 2025 than in 2017.
It actually requires substantially less maintenance time if you look at the equivalent force it replaces, for example the associated awacs capability needed for a sortie for previous gen fighters.
Honestly, it's just absurd to think that any jet fighter is somehow low maintenance. The issue here isn't the f35, it's the host country becoming a unreliable/hostile partner.
And what if you look at the equivalent force it's competing with on the market? It's a bit pricey once you factor in CAS and supersonic interceptors to fill the gaps.
> The issue here isn't the f35, it's the host country becoming a unreliable/hostile partner.
Here? The issue is the F-35. What happened to Pakistan's F-16s when America became an unreliable/hostile partner to them? They kept flying them for decades, that's what happened. Same with Ukraine's Su-27s, Iran's F-14s, North Korea's MiG-29s... plenty of countries keep other nation's keepsakes in the air. The jet abides.
The F-35 has to be bought as a subscription package, you can't "own" features like sensor fusion without the US' consent. All but one nation has been denied the right to modify the airframe, everyone else is basically just renting the jet with permission to go eat an R-77T when the time comes.
The us provides has provided upkeep for pakistans f16s under strict supervision and according to some AI summary maintenance has been a huge issue for them that limited their effectiveness. So not sure the story you are painting is quite as rosy as the reality. Fighter jets are not easy to maintain, without a large domestic fighter jet industry.
Consider for example when one of the radar elements in the f35 burns out, among the thousands that are there. Where in Pakistan does on obtain custom GaN radar ASICs that integrate with the f35?
