The reference
- The deep learning book
If you want to get into deep learning by studying, the academic way, then you should read this book. It does not include of course the latest recent developments (such as transformers), but it’s still pretty state-of-the-art.
Video lectures & courses
At least, these video courses by Nando De Freitas have my preferences because they are quite generic about Machine Learning (beyond deep learning) and are of very high quality. This link is just to the first video of the serie, you have to look for all the others there.
- Great lectures by Stanford Winter 2017
Then, the above lectures are great as well, and they are more specialized into deep learning for Natural Language Processing.
- Video lectures by A. Karpathy
Then you have these lectures above by A. Karpathy that complete the Standford lectures from the image processing point of view.
Blogs
To get an easy-to-read, but still quality overview of the progress and ideas behind deep learning, I recommend to read good blogs made by researchers. Here are a few of them, all high quality. Given th quality of these blogs, I do think they are worth scientific review papers.
Major breakthroughs in deep learning, year by year. This link points to 2018 recent progress (BERT, video processing…)
Sebastian Ruder’s blog is a must-read with extremely high quality readings in deep learning, with an applicative focus on NLP, but there is really very general machine learning stuff as well. For example, the link points to a blog about optimization; but you should actually explore all of his posts, tackling many deep learning topics !
Lilian’s blog is very very good in summarizing within a few pages only all the important material and concepts of deep learning.
Hands on
There are many tutorials about deep learning on the internet, with enlightening practical programming exercices. I would recommend the following ones, but they’re just my choice:
- A gentle do-it-yourself free online book, on basics about deep neural networks.
- The pytorch tutorials
- Keras tutorials
Special mention to Jeremy Fix’s Keras tutorials, must read ! :-)
Other resources
- A special topic: machine learning for security, with lots of resources
- There are many other video courses, papers, datasets available at Academic Torrents . Go there, look for the keywords that are of interest for you, and enjoy !