CS5691: Pattern recognition and Machine learning

Course information

  • When: Jan-May 2020

  • Lectures: Slot J

  • Where: CS25

  • Teaching Assistants: TBA

Course Content

  • Review of probability

  • Bayes decision theory and Bayes classifer

  • Maximum likelihood and Bayesian parameter estimation

  • Statistical learning theory

    • PAC learning, empirical risk minimization, uniform convergence and VC-dimension

  • Linear models

    • Linear least-squares regression, logistic regression, regularized least squares, bias-variance tradeoff

  • Dimensionality reduction

    • Singular value decomposition, principal component analysis

  • Support vector machines and kernel methods

  • Online learning

    • Prediction with expert advice, Perceptron and Winnow algorithms

  • Mixture densities and EM algorithm

  • K-means clustering

  • Multilayer neural networks

    • Feedforward networks, backpropagation

  • The schedule of lectures from the 2019 run of this course is available here

Grading

  • Quiz-I and II: 15% each

  • Final exam: 30%

  • Mini-quizzes: 10% (Best 2 out of 3)

  • Programming Assignment(PA)-I and II: 10% each

  • Programming Contest: 10%

Important Dates

  • Mini-quizzes: Feb 3, Mar 9, Apr 13

  • Quiz-I: Feb 19, Quiz-II: Mar 18

  • Final: May 2

  • PA-I: Available on Feb 20, submit by Mar 8

  • PA-II: Available on Mar 20, submit by Apr 8

  • Contest: Apr 25

Textbooks

  • Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar, Foundations of Machine Learning, MIT Press, 2019

  • Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.

  • Richard O. Duda, Peter E. Hart and David G. Stork, Pattern Classification, John Wiley, 2001

Additional References

  • Shai Shalev-Shwartz and Shai Ben-David, Understanding Machine Learning, Cambridge Univ. Press, 2014

  • Simon Haykin. Neural networks and learning machines, Pearson Education, 2009.

  • Bernhard Scholkopf, Alexander J. Smola, Learning with Kernels, MIT press, 2002.

  • Michael Kearns and Umesh Vazirani, An Introduction to Computational Learning Theory, MIT press, 1994.

Lecture Notes