If you're going to have your own platform where you control all the parts and none of them are replaceable, why would you religiously stick to the standards when you don't need to? Maybe they saved a slight amount of power consumption by not supporting 64K queues, like the nvme standard supports. A standard will try to be all things to all people, but it will never be the best thing for any given application. There is always room to specialize.
The non-standard NVMe queue format is because they support offloading encryption/decryption to the NVMe controller, so they had to increase the queue entry size to be able to fit an encryption key in there.
The "queue" format itself is incredibly similar to the normal NVMe queue.
The normal queue is (more-or-less) a ringbuffer with N slots in memory and a head/tail pointer. You append the command to the next slot and increase the tail by writing to a doorbell. Once the controller is done it increases the head the same way.
Apple's "queue" instead is just a memory region without those head/tail pointers. Command submission now works by again putting the request into a free slot followed by just writing the ID of that slot to a MMIO register. Once a command is done the CPU again gets an interrupt and can just take the command out of the buffer again.
This probably makes the driver a little bit easier to implement.
On top of that a similar structure (which identifies the DMA buffers that need to be allowed) also needs to be put into their NVMe-IOMMU with a reference to the command buffer entry. The slightly weird thing about the encryption is that you put the key/iv into this buffer instead of the normal queue. My best guess is that this IOMMU design pushed them to also simplify the command queue to make the matching easier.
Hiding the encryption part inside the IOMMU also makes sense for them because the whole IOMMU management is hidden inside a highly protected area of their kernel with more privileges while the NVMe driver itself is just a regular kernel module which possibly doesn't have access to keys.
Ah, I was thinking of what they did for the T2 Macs (change the queue entry size), not the new changes for M1. But yeah, once they're doing proprietary variants of the NVMe spec, they can do whatever they want and probably had a good reason to make this change too.
(For those following: sven has been working on the NVMe stuff for Asahi Linux recently, he knows about this more than me)
There's probably an interesting groupthink/psychology to Apple, but I think in this case the proprietary-ness of parts of M1 is them focusing on getting it out of the door: Apple have a lot of experience making ARM mobile chips but this was their first desktop part.
I say this based on nothing but gossip, of course.
>USB4... but Apple made it their own, as usual
I am wondering about these two points. Why ?