I can find descriptions at one level or another (eg RNN vs CNN) but is there a deeper kingdom/phylum/class type taxonomy of neural network architectures that can help a layman understand how they differ and how they align, ideally with specific references to contemporary ones in use or being researched?
I don't know why I'm interested because I'm not planning to actually do any work in the space, but I always struggle to understand when some new architecture is announced if it's a fundamental shift or if it's an optimization.
You might find "The neural network zoo" helpful; it's a chart showing the different types of neural networks, along with a brief discussion of each type: https://www.asimovinstitute.org/neural-network-zoo/
I don't know why I'm interested because I'm not planning to actually do any work in the space, but I always struggle to understand when some new architecture is announced if it's a fundamental shift or if it's an optimization.