Course Contents

Handout

• The Learning Problem
  • The learning problem: Risk functions, Well-posed and ill-posed problems; Supervised Learning Vs Function Approximation; Bias Variance Tradeoff; Curse of Dimensionality.


• Linear Models for Regression
  • Least Squares Vs Nearest Neighbors; Subset Selection methods; Shrinkage Methods: Ridge Regression, LASSO, Group LASSO, Least Angle Regression (LAR).


• Pattern Recognition and Machine Learning Basics
  • Bayes'; PCA and LDA; Perceptron Learning; Decision Boundaries and Discriminant Functions; Kernel density; Mixture models; Hierarchical clustering.


• Regularization
  • Tikhonov Regularization; Iterative Regularization via Early Stopping; Empirical and Structural Risk Minimization.


• Regularization Networks and SVM
  • RKHSs, Mercer's Theorem, Representer theorem, VC Dimension; Hard & Soft margin SVMs; Multiple Kernel Learning, Risk/regret bounds for SVMs, Kernel regression, Convex losses for classification.


• Advanced topics (Selectively among)
  • Sparse Representation Classifier: BOW & Dictionary Learning; Proximal Gradient; ADMM; Basis Pursuit (BP), M-BP, IrM-BP, M-FOCUSS; M-SBL; Auto-encoder & Deep Learning, Transfer Learning & Domain Adaptation, On-line Learning.


• Target Applications (for assignments)
  • Face Recognition and Verification, Video event representation, CBIR in Large Scale Dataset (e.g. ImageNet), Fraud and Rumor Detection in Social Media, DNA Sequencing.

References

Textbooks

V. N. Vapnik; Statistical Learning Theory. Wiley, 1998.

T. Hastie, R.Tibshirani, J. Friedman, "The Elements of Statistical Learning: Data Mining, Inference and Prediction", Springer Series in Statistics, 2009.

Kevin R Murphy, "Machine Learning - A Probabilistic Perspective", The MIT Press, 2012.

References

Michael J. Kearns and Umesh Vazirani; An Introduction to Computational Learning Theory; The MIT Press, 1994.

Journal of the Royal Statistical Society: Series B (Statistical Methodology).

Foundations and Trends in Machine Learning; Now Publishers Inc.

Journal of Machine Learning Research; JMLR, Inc. and Microtome Publishing (United States).

Bishop, Christopher M. "Pattern recognition and machine learning", Springer, 2006.

Conference Proceedings of ICML, NIPS.

R.O. Duda, P.E. Hart and D.G. Stork "Pattern Classification (2nd ed.)", John Wiley & Sons, Inc., 2003.

Lecture Slides

Introduction to Statistical Learning Theory
Basics of Linear algebra
Basic of Statistics and Probability
Least Square Regression, Bias-Variance Tradeoff
Overview of Supervised Learning (part 1)
Overview of Supervised Learning (part 2)
Linear Methods for Regression (part 1)
Linear Methods for Regression (part 2)
Pattern classification
Machine Learning Definition
Principal Component Analysis and SVD
Logistic Regression
Clustering Methods
Gaussian Mixture Model
Alternating Directions for Method of Multipliers (ADMM)
Transfer Learning and Domain Adaptation Methods
Ridge Regression vs. PCA

Assignment

Software Assignment 1
        Problem Statement :
        Important tutorials links :     Basic R Tutorial             R tutorials in terms of machine learning             Install R and R studio on ubuntu             Install R and R studio on windows    

Announcement

Marks Distribution

• End Sem : 40 marks
• Mid Sem : 15-20 marks
• Software Assignments : 30-35 marks
• Tutorials : 10 marks

Tentative Tutorial Dates

• Tutorial 1 :
• Tutorial 2 :
• Tutorial 3 :

Logistic Details

• TA: Sonam Gupta (cs18d005[at]smail[dot]iitm[dot]ac[dot]in)
  
• TA: Riya Verma (cs21d012[at]smail[dot]iitm[dot]ac[dot]in)
  

Course Handout

Important Dates

End Semester Exam:

Software Assignment 1:

Software Assignment 2:

Self-study

Self Study Topics

• Statistical Theorems
  Link to the pdf
• Cramer's Theorem
  
• Karhunen-Loeve Theorem
  
• Law of Total Variance
  
• Law of Total Covariance
  
• Neyman-Pearson Lemma
  
• Shannon-Hartley Theorem
  

Useful Links

Links

Statistical Help