CS6464: Concepts In Statistical Learning Theory

January - May, 2023

Course Contents

Handout Download Slide

  • Objectives:
    • In the recent past, algorithms of solving many ill-posed problems in the field of multi-dimensional signal processing and big data analytics have gained importance. New methods of signal representation, modeling, optimization and leaning have been formulated, which spans over various areas of Machine Learning, Pattern Recognition, Vision and Natural Language Processing. This course will provide an overview of the theories and current practices, required by students and scholars who intend to specialize in this field, to solve complex problems in Machine Learning Applications for image, video, text and bioinformatics.
  • References

    V. N. Vapnik; Statistical Learning Theory. Wiley, 1998.
    T. Hastie, R.Tibshirani, J. Friedman, "The Elements of Statistical Learning: Data Mining, Inference and Prediction", Springer Series in Statistics, 2009.
    Kevin R Murphy, "Machine Learning - A Probabilistic Perspective", The MIT Press, 2012.

    Michael J. Kearns and Umesh Vazirani; An Introduction to Computational Learning Theory; The MIT Press, 1994.
    Journal of the Royal Statistical Society: Series B (Statistical Methodology).
    Foundations and Trends in Machine Learning; Now Publishers Inc.
    Journal of Machine Learning Research; JMLR, Inc. and Microtome Publishing (United States).
    Bishop, Christopher M. "Pattern recognition and machine learning", Springer, 2006.
    Conference Proceedings of ICML, NIPS.
    R.O. Duda, P.E. Hart and D.G. Stork "Pattern Classification (2nd ed.)", John Wiley & Sons, Inc., 2003.

    Lecture Slides

    Introduction to Statistical Learning Theory  Download Slide
    Basics of Linear algebra  Download Slide
    Basic of Statistics and Probability  Download Slide
    Least Square Regression, Bias-Variance Tradeoff  Download Slide
    Overview of Supervised Learning (part 1)  Download Slide
    Overview of Supervised Learning (part 2)  Download Slide
    Linear Methods for Regression (part 1)  Download Slide
    Linear Methods for Regression (part 2)  Download Slide
    Pattern classification  Download Slide
    Machine Learning Definition  Download Slide
    Principal Component Analysis and SVD  Download Slide
    Logistic Regression  Download Slide
    Clustering Methods  Download Slide
    Gaussian Mixture Model  Download Slide
    Alternating Directions for Method of Multipliers (ADMM)  Download Slide
    Transfer Learning and Domain Adaptation Methods   Download Slide
    Ridge Regression vs. PCA   Download Slide


    Software Assignment 1
            Problem Statement : Download Slide
            Important tutorials links :     Basic R Tutorial             R tutorials in terms of machine learning             Install R and R studio on ubuntu             Install R and R studio on windows    


    Marks Distribution

    Tentative Tutorial Dates

    Logistic Details

    Course Handout Download Slide

    Important Dates
    End Semester Exam:
    Software Assignment 1:
    Software Assignment 2:


    Self Study Topics

    Useful Links

    Statistical Help