Calculus [Online course from MIT]
Linear Algebra [CS6015 or equivalent]  [Online course from MIT]
Probability Theory [CS6015 or equivalent]  [Online course from MIT]
Nonlinear Optimization [CS5020 or equivalent]  [First Course in Optimization by Prof. Soman (IITB) available on CDEEP]
Pattern Recognition and Machine Learning [CS5691 or equivalent]  [Andrew Ng's ML course]
Instructor: Mitesh M. Khapra
When: JanMay 2024
Lectures: Slot H
Where: CS25
Teaching Assistants:
Name  Lab  Office hours  Days  

Anushka Singh  AI4Bharat  cs22s015@smail.iitm.ac.in  34 pm  Tuesday, Thursday 
Oikantik Nath  AI4Bharat  cs22s013@smail.iitm.ac.in  121 pm  Tuesday, Thursday 
Bibhuti Majhi  AI4Bharat  cs22m031@smail.iitm.ac.in  24 pm  Wednesday 
Sarthak Naithani  AI4Bharat  cs22m078@smail.iitm.ac.in  34 pm  Tuesday, Wednesday 
Putta Sai Sree Ram  AI4Bharat  cs22m076@smail.iitm.ac.in  34 pm  Monday, Friday 
Ravi Prakash Singh  AI4Bharat  cs22m069@smail.iitm.ac.in  12 pm  Tuesday, Friday 
Guddeppagari Shathish Kumar Reddy  AI4Bharat  cs22m080@smail.iitm.ac.in  34 pm  Monday, Friday 
Poorbi Mukesh Dalal    cs22m064@smail.iitm.ac.in  13 pm  Wednesday 
Ashok R    cs22m023@smail.iitm.ac.in  24 pm  Wednesday 
Amit Kumar    cs22m010@smail.iitm.ac.in  24 pm  Wednesday 
Lecture#  Contents  Lecture pdf  Lecture Videos  Extra Reading Material 

Lecture 1  (Partial) History of Deep Learning, Deep Learning Success Stories  Slides  M1  M2 M3  M4  M5  Juergen Schmidhuber, Deep Learning in Neural Networks: An Overview. 2014 
Lecture 2  McCulloch Pitts Neuron, Thresholding Logic, Perceptrons, Perceptron Learning Algorithm and Convergence, Multilayer Perceptrons (MLPs), Representation Power of MLPs  Slides  M1  M2 M3  M4  M5 M6  M7  M8  Chapters 1,2,3,4 from Neural Networks by Rojas 
Lecture 3  Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks, Representation Power of Feedforward Neural Networks  Slides  M1  M2 M3  M4  M5 M6  M7 M8  http://neuralnetworksanddeeplearning.com/chap4.html 
Lecture 4  Feedforward Neural Networks, Backpropagation  Slides  M1  M2  M3  M4  M5  M6  M7  M8  See Lecture 2 [Training Neural Networks] by Hugo Larochelle 
Lecture 5  Gradient Descent (GD), Momentum Based GD, Nesterov Accelerated GD, Stochastic GD, AdaGrad, RMSProp, Adam  Slides1  Slides2  M1  M2  M3  M4  M5  M6  M7  M8  M9  M10  M11  M12  M13  M14 M15  
Lecture 6  Bias Variance Tradeoff, L2 regularization, Early stopping, Dataset augmentation, Parameter sharing and tying, Injecting noise at input, Ensemble methods, Dropout  Slides  M1 M2  M3  M4 M5  M6  M7  M8  M9  M10 M11M12  
Lecture 7  Greedy Layerwise Pretraining, Better activation functions, Better weight initialization methods, Batch Normalization  Slides  M1  M2 M3  M4  M5 M6  
Lecture 8  Convolutional Neural Networks, LeNet, AlexNet, ZFNet, VGGNet, GoogLeNet, ResNet  Slides  M1  M2 M3  M4  M5 M6  M7 M8  
Lecture 9  Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks  Slides  M1  M2 M3  M4  M5 M6  M7  M8  M9  M10 

Lecture 10  Learning Vectorial Representations Of Words  Slides  M1  M2 M3  M4  M5 M6  M7  M8  M9  M8  M9  M10 M11  
Lecture 11  Recurrent Neural Networks, Backpropagation Through Time (BPTT), Vanishing and Exploding Gradients, Truncated BPTT  Slides  M1  M2 M3  M4  M5  
Lecture 12  Gated Recurrent Units (GRUs), Long Short Term Memory (LSTM) Cells, Solving the vanidhing gradient problem with LSTMs  Slides  M1  M2 M3  M4  
Lecture 13  Encoder Decoder Models, Attention Mechanism, Attention over images, Hierarchical Attention  Slides  M1  M2 M3  
Lecture 14  Transformers: Multiheaded Self Attention, Cross Attention  Slides  M1  M2 M3  M4  M5 
Topics  Resources  Release Date  Submission Date  

Assignment 1 (Graded)  Feedforward Neural Networks  Link  19Feb2024  10Mar2024 
Assignment 2 ( Ungraded )  Convolutional Neural Networks  Link  03Mar2024  03Apr2024 
Assignment 3 ( Ungraded )  Recurrent Neural Networks  Link  03Mar2024  03Apr2024 
Assignment 4 ( Ungraded )  RBMs and GANs  Link  03Mar2024  03Mar2024 
Assignment 5 ( Graded )  Transformers  Link  22Mar2024  22Apr2024 
Endsem  11May2024 (Saturday)   
Topics  Resources  Release Date  Submission Date  

Tutorial 1  Calculus  
Tutorial 2  Linear Algebra  
Tutorial 3  MP Neurons, Perceptrons  
Tutorial 4  Sigmoid Neurons, Gradient Descent  
Tutorial 5  Feedforward Neural Networks, Backpropagation 
Deep Learning for Computer Vision [from Stanford]
Deep Learning for NLP [from Stanford]