Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Isn’t ReLU an element of neuroscience that was incorporated into machine learning to great success?


Not really, no. That's motivated by not getting impractically small gradients on the plateaus and spoiling the optimization properties when used for deep ANNs. The sigmoids it replaced had a bit more neuroscience inspiration, but so oversimplified it's just barely.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: