I think you're implying significantly more magic behind Option than there is.
> Having an array of Option<...>s is a non-trivial operation for the compiler
This is just not true. Arrays are always a number of values laid out in memory, exactly like in C. There is no magic for an array of options. It works the same as an array of any other value: you put N of them in memory, one after the other.
I should clarify. It is not that once the compiler computes the size of an Option that there is further confusion. My point is that something like Option<byte> is not necessarily something a programmer can just look at and know how large it is, because people who get very used to reference semantics like in Python, or smart compilers just doing things for them without having to think about the memory layout, can find it easy to forget that there's nowhere in a "byte" to stick a "None" value.
Sum types in general usually have compiler magic associated with them, because the common case of options or sums between various integers can get good results with the compiler being smart enough to pack the "None" option somewhere clever. As the size of the thing being used as a sum type goes up that amortizes to being less important. The naive way of using some integer as a tag and then having a chunk of memory large enough to store the largest value in the sum type can get very inefficient for small "largest values", especially if you have to round to a full 64-bit machine word for some reason.
Also, to be abundantly clear, I think this is all a good thing. It is intrinsically part of the value of a sum type in a language that you're not forced to think about the modestly complicated memory layout such things entail. It is good that compilers have some special cases for when you're summing on small values.
> Also, to be abundantly clear, I think this is all a good thing. It is intrinsically part of the value of a sum type in a language that you're not forced to think about the modestly complicated memory layout such things entail. It is good that compilers have some special cases for when you're summing on small values.
Seems more likely you'd limit your enums to 256 entries and always use a byte than require a word by default, no?
niche-value optimisation is a much more complicated affair so it's unlikely as a baseline indeed.
> Having an array of Option<...>s is a non-trivial operation for the compiler
This is just not true. Arrays are always a number of values laid out in memory, exactly like in C. There is no magic for an array of options. It works the same as an array of any other value: you put N of them in memory, one after the other.