Course Contents

Handout

• Basics of Linear Algebra, Probability Theory and Optimization
  • Vectors, Inner product, Outer product, Inverse of a matrix, Eigenanalysis, Singular value decomposition, Probability distributions - Discrete distributions and Continuous distributions; Independence of events, Conditional probability distribution and Joint probability distribution, Bayes theorem, Unconstrained optimization, Constrained optimization - Lagrangian multiplier method.


• Methods for Function Approximation:
  • Linear models for regression, Parameter estimation methods - Maximum likelihood method and Maximum a posteriori method; Regularization, Ridge regression, Lasso, Bias-Variance decomposition, Bayesian linear regression.


• Probabilistic Models for Classification
  • Bayesian decision theory, Bayes classifier, Minimum error-rate classification, Normal (Gaussian) density - Discriminant functions, Decision surfaces, Maximum-Likelihood estimation, Maximum a posteriori estimation; Gaussian mixture models -- Expectation-Maximization method for parameter estimation; Naive Bayes classifier, Non-parametric techniques for density estimation -- Parzen-window method, K-nearest neighbors method, Hidden Markov models (HMMs) for sequential pattern classification -- Discrete HMMs and Continuous density HMMs.


• Discriminative Learning based Models for Classification
  • Logistic regression, Perceptron, Multilayer feedforward neural network - Gradient descent method, Error backpropagation method; Support vector machine.


• Dimensionality Reduction Techniques
  • Principal component analysis, Fisher discriminant analysis, Multiple discriminant analysis.


• Non-Metric Methods for Classification
  • Decision trees, CART.


• Ensemble Methods for Classification
  • Bagging, Boosting, Gradient boosting.


• Pattern Clustering
  • Criterion functions for clustering, Techniques for clustering -- K-means clustering, Hierarchical clustering, Density based clustering and Spectral clustering; Cluster validation.

References

Textbooks

Christopher M. Bishop, Pattern recognition and machine learning, Springer, 2006.

T. Hastie, R.Tibshirani, J. Friedman, The El- ements of Statistical Learning: Data Mining, Inference and Prediction, Springer Series in Statistics, 2009.

Lecture Slides

1	Introduction(Part-1)
2	Introduction(Part-2)
3	Overview of supervised learning (Part-1)
4	Overview of supervised learning (Part-2)
5	Linear least square regression
6	Linear Algebra basics
7	Statistics basics
8	Linear methods for regression (part-1)
9	Linear methods for regression (part-2)
10	Pattern Classification
11	Maximum Likelihood Estimation (MLE)
12	Maximum a Posteriori Estimation (MAP)
13	Gaussian Mixture Model (GMM)
14	Clustering Methods
15	Spectral Clustering
16	Logistic regression
17	CART
18	Ensemble Methods
19	Support Vector machines

Additional Resources

Basic Python/Numpy programming Resources

1. https://cs231n.github.io/python-numpy-tutorial/
2. https://www.tutorialspoint.com/numpy/index.htm
3. http://www.astro.up.pt/~sousasag/Python_For_Astronomers/Python_qr.pdf
4. https://developers.google.com/edu/python/

Coding resources

1. Google Colab tutorial
2. K-means clustering in python
3. Principal Component Analysis (PCA) from Scratch

Schedule

Tentative Grading Policy:

• MID-SEM (50 mins) : 15%
• End Sem Exam (120-150 mins): 40%
• Assignments : 35%
• Tutorials(best n-1) : 10%

Tentative Tutorial Dates

• Tutorial 1 : 23-02-2023
• Tutorial 2 : 30-03-2023
• Tutorial 3 : 20-04-2023

Tentative Assignments Dates

• Assignment 1 release: 06-02-2023
• Assignment 1 deadline : 04-03-2023
• Assignment 2 release : 27-03-2023
• Assignment 2 deadline : 28-04-2023

Important Dates

Mid Semester Exam: 09-03-2023

End Semester Exam: 02-05-2023