According to fast.ai there's this (very simple) technique to automate optimizing the learning rate parameter for deep neural networks, that went unnoticed for a couple years, because it was discovered by a no-name researcher, while all the big wig researchers were spending tons of manual hours finding the optimal learning rate.