It was kinda hard for me to understand the basics of deep learning. A lot of formal courses makes a lot of assumptions.
I ended up watching the deep learning series by 3blue1brown, reading the book it was based on. Watching lots of linear algebra and calculus videos. Then taking the deep learning specialization program on Coursera. This process took 2 months, it taught me the basics.
Then I built my own project and realized there were lots wholes in my knowledge base. I found an old post on hn that recommended deeplizard on YouTube, that was an excellent recommendation and intro to pytorch.
One thing to keep in mind is that as you are learning, new methods are being invented.
Almost all modern AI is a search problem; We're just trying to find a clever way to search the problem space.
Most supervised learning methods are a search problem for best programs (models) that explain the data; We're just trying to find a clever way to express the model and efficient algorithms to search the problem space.
Many popular ML algorithms (NN, SVM, etc.) are an optimization problem; you just want to find parameters to maximize some performance metrics. But you don't want to search all the parameters exhaustively. You want something that can be improved incrementally over time.
NN is popular because it's flexible (can have thousands of inputs/outputs) and can be built incrementally. The secret sauce is its performance metrics (loss function) being differentiable. This way, you can incrementally improve the model via gradient descent.
NN is popular also because in theory you can stack as many layers as you want, giving it more power to learn complex relationships. It's applicable to many tasks.
CNN/RNN/LTSM/Transformer are just variations of these. They have different structures, but all operating on the same principle: differentiable loss function and iterative training methods.
I went through multiple resources on the net until it clicked for me. I guess each book / video / blog post gives you the bits you are ready for, and when enough of these bits accumulate you understand it. Looking back, neural networks are not that difficult to grasp, but there are many important details that one should understand.
What I learned is that it is very important to test intermediate results thoroughly. Nets will gladly learn from garbage if that is what you (unknowingly) feed them, and they might even produce some results - not at acceptable level though.
I didn't continue working in this field, but I'm happy I learned it (and used it in a project too) and would recommend that to any developer. ML really moves the frontier of what we can achieve in software much further.
Sorry, speaking of web-based books about Deep Learning, I remember one in which the figures allowed changing parameters and seeing the results - I cannot remember the source or author, can anyone?
I confused it with the one from Michael Nielsen...
If you want just one book to learn all things deep learning from scratch with coding including the latest techniques, it should be Dive into deep learning [0]. Really underrated and not well-known.
I'm so glad to see this posted! It's a great resource. I wonder if it's less well known because people prefer video content and/or it's rather formal and not very gentle.
>>most of the time Packt is a reliable signal for low quality
In my opinion, Packt is the first one with a book on a new technology or framework or use case. When I was getting started with Clojure , I ran into a bunch of problems in applying it. Packt has more clojure data science books than any other source and they were pretty helpful to me. They were not as fundamental to my learning as "Clojure for the Brave.." but ideas from those books helped me on.
That was a good book. I looked at it recently and it’s now in the 3rd edition! Congrats.
Any other suggestions for good Packt books?
I agree, Packt’s quality is much lower than other publishers. As a rule of thumb I stay away but occasionally there’s a gem.
I’ve been looking at “Machine Learning for Algorithmic Trading”. It feels like a dump of wikipedia and a bunch of jupyter notebooks with sloppy code. I cannot decide if it’s worth the pain if slogging through that mess.
I ended up watching the deep learning series by 3blue1brown, reading the book it was based on. Watching lots of linear algebra and calculus videos. Then taking the deep learning specialization program on Coursera. This process took 2 months, it taught me the basics.
Then I built my own project and realized there were lots wholes in my knowledge base. I found an old post on hn that recommended deeplizard on YouTube, that was an excellent recommendation and intro to pytorch.
One thing to keep in mind is that as you are learning, new methods are being invented.
Edit: typos