Also curious about latency. In the past I've worked around latency using video sensors for high-bandwidth high-latency features, then literally glued a contact mic to my interface to get low latency tap detection. How does the Board hide latency?
Unpopular opinion: hundreds of years from now, loss of mammalian species will seem like sentimental naval gazing when our descendants consider the millions of strains of fungi, bacteria, archaea, and viruses we could have saved, were it not for our micro-blindness.
It looks like almost every AI researcher and lab who existed pre-2017 is now focused on transformers somehow. I agree the total number of researchers has increased, but I suspect the ratio has moved faster, so there are now fewer total non-transformer researchers.
Well, we also still use wheels despite them being invented thousands of years ago. We have added tons of improvements on top though, just as transformers have. The fact that wheels perform poorly in mud doesn’t mean you throw out the concept of wheels. You add treads to grip the ground better.
If you check the DeepSeek OCR paper it shows text based tokenization may be suboptimal. Also all of the MoE stuff, reasoning, and RLHF. The 2017 paper is pretty primitive compared to what we have now.
