Theoretical and advanced machine learning with TensorFlow

Before starting on the learning materials below, be sure to:

  1. Complete our curriculum Basics of machine learning with TensorFlow, or have equivalent knowledge

  2. Have software development experience, particularly in Python

This curriculum is a starting point for people who would like to:

  1. Improve their understanding of ML

  2. Begin understanding and implementing papers with TensorFlow

You should already have background knowledge of how ML works or completed the learning materials in the beginner curriculum Basics of machine learning with TensorFlow before continuing. The below content is intended to guide learners to more theoretical and advanced machine learning content. You will see that many of the resources use TensorFlow, however, the knowledge is transferable to other ML frameworks.

To further your understanding of ML, you should have Python programming experience as well as a background in calculus, linear algebra, probability, and statistics. To help you deepen your ML knowledge, we have listed a number of recommended resources and courses from universities, as well as a couple of textbooks.

Step 1: Refresh your understanding of math concepts

ML is a math heavy discipline. If you plan to modify ML models, or build new ones from scratch, familiarity with the underlying math concepts is important. You don't have to learn all the math upfront, but instead you can look up concepts you are unfamiliar with as you come across them. If it's been a while since you've taken a math course, try watching the Essence of linear algebra and the Essence of calculus playlists from 3blue1brown for a refresher. We recommend that you continue by taking a class from a university, or watching open access lectures from MIT, such as Linear Algebra or Single Variable Calculus.

Essence of Linear Algebra
by 3Blue1Brown

A series of short, visual videos from 3blue1brown that explain the geometric understanding of matrices, determinants, eigen-stuffs and more.

Essence of Calculus
by 3Blue1Brown

A series of short, visual videos from 3blue1brown that explain the fundamentals of calculus in a way that give you a strong understanding of the fundamental theorems, and not just how the equations work.

MIT 18.06: Linear Algebra

This introductory course from MIT covers matrix theory and linear algebra. Emphasis is given to topics that will be useful in other disciplines, including systems of equations, vector spaces, determinants, eigenvalues, similarity, and positive definite matrices.

MIT 18.01: Single Variable Calculus

This introductory calculus course from MIT covers differentiation and integration of functions of one variable, with applications.

Step 2: Deepen your understanding of deep learning with these courses and books

There is no single course that will teach you everything you need to know about deep learning. One approach that may be helpful is to take a few courses at the same time. Although there will be overlap in the material, having multiple instructors explain concepts in different ways can be helpful, especially for complex topics. Below are several courses we recommend to help get you started. You can explore each of them together, or just choose the ones that feel the most relevant to you.

Remember, the more you learn, and reinforce these concepts through practice, the more adept you will be at building and evaluating your own ML models.

Take these courses:

MIT course 6.S191: Introduction to Deep Learning is an introductory course for Deep Learning with TensorFlow from MIT and also a wonderful resource.

Andrew Ng's Deep Learning Specialization at Coursera also teaches the foundations of deep learning, including convolutional networks, RNNS, LSTMs, and more. This specialization is designed to help you apply deep learning in your work, and to build a career in AI.

MIT 6.S191: Introduction to Deep Learning

In this course from MIT, you will gain foundational knowledge of deep learning algorithms and get practical experience in building neural networks in TensorFlow.

Deep Learning Specialization

In five courses, you will learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects and build a career in AI. You will master not only the theory, but also see how it is applied in industry.

⬆ And ⬇ Read these books:

To complement what you learn in the courses listed above, we recommend that you dive deeper by reading the books below. Each book is available online, and offers supplementary materials to help you practice.

You can start by reading Deep Learning: An MIT Press Book by Ian Goodfellow, Yoshua Bengio, and Aaron Courville. The Deep Learning textbook is an advanced resource intended to help students deepen their understanding. The book is accompanied by a website, which provides a variety of supplementary materials, including exercises, lecture slides, corrections of mistakes, and other resources to give you hands on practice with the concepts.

You can also explore Michael Nielsen's online book Neural Networks and Deep Learning. This book provides a theoretical background on neural networks. It does not use TensorFlow, but is a great reference for students interested in learning more.

Deep Learning
by Ian Goodfellow, Yoshua Bengio, and Aaron Courville

This Deep Learning textbook is a resource intended to help students and practitioners enter the field of machine learning in general, and deep learning in particular.

Neural Networks and Deep Learning
by Michael Nielsen

This book provides a theoretical background on neural networks. It does not use TensorFlow, but is a great reference for students interested in learning more.

Step 3: Read and implement papers with TensorFlow

At this point, we recommend reading papers and trying the advanced tutorials on our website, which contain implementations of a few well known publications. The best way to learn an advanced application, machine translation, or image captioning, is to read the paper linked from the tutorial. As you work through it, find the relevant sections of the code, and use them to help solidify your understanding.