It's pretty reassuring to see that constantly fiddling with the model and trying to adjust learning rates on the fly is also normal at leading research labs. Although on the other hand it only makes the replication crisis even worse.

After a quick look through, I really hope releasing raw notes like this becomes more of a trend!