Not quite. The Lottery Ticket Hypothesis paper showed that models could shrink to around 10% of their original size without a loss of accuracy [0]. So around a million neurons instead of 10 million.

[0] https://arxiv.org/abs/1903.01611v1

Read the original papers on the perceptron for similar claims.

I mean yeah... but the performance will be accordingly.