They are a chip architecture that communicates similarly to how neurons do - i.e. spiking behaviours. Makes it easy to approximate some neural systems (such as a neuromorphic retina).

neuromorphic chips try to mimic human brain's neuronal structure for computing, entirely new paradigm. checkout IBM truNorth architecture.

I think the point is that even GPUs are suboptimal for neural network computation and we need something more specialized.