Pre-requisites | Evaluation | Logistics | Schedule | Quizzes/Assignments | What Next?
Note: The video lectures for this course are now available on youtube

Pre-requisites


There are no official pre-requisites for this course but it would help if you have done the following courses (preferably in the order mentioned below) : If you can solve most of this assignment then you are ready for this course!

Logistics


Evaluation


70% Assignments | 30% Exams

Reference Textbooks


  1. Ian Goodfellow and Yoshua Bengio and Aaron Courville. Deep Learning. An MIT Press book. 2016.
  2. Charu C. Aggarwal. Neural Networks and Deep Learning: A Textbook. Springer. 2019.
  3. Dive into Deep Learning

Schedule


*M = Module (each lecture is broken down into smaller modules)
Lecture# Contents Lecture Slides Lecture Videos Extra Reading Material
Lecture 0 Syllabus, Logistics Slides - -
Lecture 1 (Partial) History of Deep Learning, Deep Learning Success Stories pdf M1 | M2| M3 | M4 | M5| M6 | M7 | M8| M9 Juergen Schmidhuber, Deep Learning in Neural Networks: An Overview. 2014
Lecture 2 McCulloch Pitts Neuron, Thresholding Logic, Perceptrons, Perceptron Learning Algorithm and Convergence, Multilayer Perceptrons (MLPs), Representation Power of MLPs pdf M1 | M2| M3 | M4 | M5| M6 | M7 | M8 Chapters 1,2,3,4 from Neural Networks by Rojas
Lecture 3 Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks, Representation Power of Feedforward Neural Networks pdf M1 | M2| M3 | M4 | M5 http://neuralnetworksanddeeplearning.com/chap4.html
Lecture 4 Feedforward Neural Networks, Backpropagation pdf M1 | M2 | M3 | M4 | M5 | M6 | M7 | M8 | M9 | M10 See Lecture 2 [Training Neural Networks] by Hugo Larochelle
Lecture 5 Gradient Descent (GD), Momentum Based GD, Nesterov Accelerated GD, Stochastic GD, AdaGrad, RMSProp, Adam T | H M1 and M2 | M3 | M4 | M5 | M6 | M7 | M8 | M9 | M9 part 2
Lecture 6 Eigenvalues and eigenvectors, Eigenvalue Decomposition, Basis, Principal Component Analysis and its interpretations, Singular Value Decomposition T | H M1 | M2| M3 | M4 | M5| M6 | M6 part 2 | M7| M8
Lecture 7 Autoencoders and relation to PCA, Regularization in autoencoders, Denoising autoencoders, Sparse autoencoders, Contractive autoencoders T | H M1 | M2| M3 | M4 | M5| M6
Lecture 8 Bias Variance Tradeoff, L2 regularization, Early stopping, Dataset augmentation, Parameter sharing and tying, Injecting noise at input, Ensemble methods, Dropout T | H M1 | M2| M2 part 2 | M3 | M4| M5 | M6 | M7 | M8 | M9 | M10| M11
Lecture 9 Greedy Layerwise Pre-training, Better activation functions, Better weight initialization methods, Batch Normalization T | H M1 | M2| M3 | M4 | M5
Lecture 10 Learning Vectorial Representations Of Words T | H M1 | M2| M3 | M3 part 2 | M4| M5 | M5 part 2 | M6 | M7 | M8 | M9 | M10
Lecture 11 Convolutional Neural Networks, LeNet, AlexNet, ZF-Net, VGGNet, GoogLeNet, ResNet T | H M1 | M2| M3 | M3 part 2 | M4| M4 part 2 | M5
Lecture 12 Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks T | H M1 | M2| M3 | M4 | M5| M6 | M7 | M8 | M9 | M10
Lecture 13 Recurrent Neural Networks, Backpropagation Through Time (BPTT), Vanishing and Exploding Gradients, Truncated BPTT T | H M1 | M2| M3 | M4 | M5
Lecture 14 Gated Recurrent Units (GRUs), Long Short Term Memory (LSTM) Cells, Solving the vanidhing gradient problem with LSTMs T | H M1 | M2| M3 | M3 part 2
Lecture 15 Encoder Decoder Models, Attention Mechanism, Attention over images, Hierarchical Attention T | H M1 | M2| M3 | M3 part 2 | M4| M5
Lecture 16 Directed Graphical Models T | H Will be available soon
Lecture 17 Markov Networks T | H Will be available soon
Lecture 18 Using joint distributions for classification and sampling, Latent Variables, Restricted Boltzmann Machines, Unsupervised Learning, Motivation for Sampling T | H Will be available soon
Lecture 19 Markov Chains, Gibbs Sampling for training RBMs, Contrastive Divergence for training RBMs T | H Will be available soon
Lecture 20 Variational autoencoders T | H Will be available soon
Lecture 21 Autoregressive Models: NADE, MADE, PixelRNN T | H Will be available soon
Lecture 22 Generative Adversarial Networks (GANs) T | H Will be available soon

Quizzes/Assigments


Topics Resources Release Date Submission Date
Assignment 0 History of DL pdf 05-Feb-2021 15-May-2021
Assignment 1 Feedforward Neural Networks Link 20-Feb-2021 13-Mar-2021
Assignment 2 Convolutional Neural Networks Link 13-Mar-2021 03-Apr-2021
Assignment 3 Recurrent Neural Networks Link 03-Apr-2021 24-Apr-2021
Assignment 4 Deep Generative Models Link 28-Apr-2021 25-May-2021

Tutorials


Topics Resources Release Date Submission Date
Tutorial 1 Calculus pdf 13-Feb-2021
Tutorial 2 Linear Algebra pdf 13-Feb-2021
Tutorial 3 MP Neurons, Perceptrons pdf 20-Feb-2021
Tutorial 4 Sigmoid Neurons, Gradient Descent pdf 26-Feb-2021
Tutorial 5 Feedforward Neural Networks, Backpropagation pdf 26-Feb-2021

What Next?


You may find the following courses to be useful: