Pre-requisites | Evaluation | Logistics | Schedule | Quizzes/Assignments | What Next?
Note: The video lectures for this course are now available on youtube

Pre-requisites


There are no official pre-requisites for this course but it would help if you have done the following courses (preferably in the order mentioned below) : If you can solve most of this assignment then you are ready for this course!

Logistics


Evaluation


60% Assignments | 10% Quiz 1 | 30% Endsem

Schedule


*T = Teaching mode - this version of the slides contains animations (is good for first time viewing)
*H = Handout mode - this version of the slides will be available soon and does not contain animations (is good for revision before exams and for printing)
*M = Module (each lecture is broken down into smaller modules)
Lecture# Contents Lecture Slides Lecture Videos Extra Reading Material
Lecture 0 Jargon Busting Slides - -
Lecture 1 (Partial) History of Deep Learning, Deep Learning Success Stories T | H M1 | M2| M3 | M4 | M5| M6 | M7 | M8| M9 Juergen Schmidhuber, Deep Learning in Neural Networks: An Overview. 2014
Lecture 2 McCulloch Pitts Neuron, Thresholding Logic, Perceptrons, Perceptron Learning Algorithm and Convergence, Multilayer Perceptrons (MLPs), Representation Power of MLPs T | H M1 | M2| M3 | M4 | M5| M6 | M7 | M8 Chapters 1,2,3,4 from Neural Networks by Rojas
Lecture 3 Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks, Representation Power of Feedforward Neural Networks T | H M1 | M2| M3 | M4 | M5 http://neuralnetworksanddeeplearning.com/chap4.html
Lecture 4 Feedforward Neural Networks, Backpropagation T | H M1 | M2 | M3 | M4 | M5 | M6 | M7 | M8 | M9 | M10 See Lecture 2 [Training Neural Networks] by Hugo Larochelle
Lecture 5 Gradient Descent (GD), Momentum Based GD, Nesterov Accelerated GD, Stochastic GD, AdaGrad, RMSProp, Adam T | H M1 and M2 | M3 | M4 | M5 | M6 | M7 | M8 | M9 | M9 part 2
Lecture 6 Eigenvalues and eigenvectors, Eigenvalue Decomposition, Basis, Principal Component Analysis and its interpretations, Singular Value Decomposition T | H M1 | M2| M3 | M4 | M5| M6 | M6 part 2 | M7| M8
Lecture 7 Autoencoders and relation to PCA, Regularization in autoencoders, Denoising autoencoders, Sparse autoencoders, Contractive autoencoders T | H M1 | M2| M3 | M4 | M5| M6
Lecture 8 Bias Variance Tradeoff, L2 regularization, Early stopping, Dataset augmentation, Parameter sharing and tying, Injecting noise at input, Ensemble methods, Dropout T | H M1 | M2| M2 part 2 | M3 | M4| M5 | M6 | M7 | M8 | M9 | M10| M11
Lecture 9 Greedy Layerwise Pre-training, Better activation functions, Better weight initialization methods, Batch Normalization T | H M1 | M2| M3 | M4 | M5
Lecture 10 Learning Vectorial Representations Of Words T | H M1 | M2| M3 | M3 part 2 | M4| M5 | M5 part 2 | M6 | M7 | M8 | M9 | M10
Lecture 11 Convolutional Neural Networks, LeNet, AlexNet, ZF-Net, VGGNet, GoogLeNet, ResNet T | H M1 | M2| M3 | M3 part 2 | M4| M4 part 2 | M5
Lecture 12 Object Detection, RCNN, Fast RCNN, Faster RCNN, YOLO T | H Will be available soon
Lecture 13 Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks T | H M1 | M2| M3 | M4 | M5| M6 | M7 | M8 | M9 | M10
Lecture 14 Recurrent Neural Networks, Backpropagation Through Time (BPTT), Vanishing and Exploding Gradients, Truncated BPTT T | H M1 | M2| M3 | M4 | M5
Lecture 15 Gated Recurrent Units (GRUs), Long Short Term Memory (LSTM) Cells, Solving the vanidhing gradient problem with LSTMs T | H M1 | M2| M3 | M3 part 2
Lecture 16 Encoder Decoder Models, Attention Mechanism, Attention over images, Hierarchical Attention T | H M1 | M2| M3 | M3 part 2 | M4| M5
Lecture 17 Directed Graphical Models T | H Will be available soon
Lecture 18 Markov Networks T | H Will be available soon
Lecture 19 Using joint distributions for classification and sampling, Latent Variables, Restricted Boltzmann Machines, Unsupervised Learning, Motivation for Sampling T | H Will be available soon
Lecture 20 Markov Chains, Gibbs Sampling for training RBMs, Contrastive Divergence for training RBMs T | H Will be available soon
Lecture 21 Variational autoencoders T | H Will be available soon
Lecture 22 Autoregressive Models: NADE, MADE, PixelRNN T | H Will be available soon
Lecture 23 Generative Adversarial Networks (GANs) T | H Will be available soon

Quizzes/Assigments


Topics Resources Release Date Submission Date
Assignment 1 Derivatives + Probability [Theory] PDF | Source Code 21-Jan-2019 27-Jan-2019
Assignment 2 Linear Algebra [Theory] - 24-Jan-2019 31-Jan-2019
Assignment 3 Backpropagation [Programming] - 31-Jan-2019 12-Feb-2019
Quiz 1 Lectures 1-7 - 20-Feb-2019
Assignment 4 Convolutional Neural Networks [Programming] - 28-Feb-2019 15-Mar-2019
Quiz II Lectures 8-15 - 27-Apr-2019
Assignment 5 Recurrent Neural Networks [Programming] - 15-Mar-2019 10-Apr-2019
Assignment 6 Probability Refresher [Theory] - 10-Apr-2019 15-Apr-2019
Assignment 7 Variational Autoencoders [Programming] - 10-Apr-2019 26-Apr-2019
End Sem Lectures 1-23 - 01-May-2019

What Next?


You may find the following courses to be useful: