Back to Courses

Introduction to Machine Learning

With the increased availability of data from varied sources there has been increasing attention paid to the various data driven disciplines such as analytics and machine learning. In this course we intend to introduce some of the basic concepts of machine learning from a mathematically well motivated perspective. We will cover the different learning paradigms and some of the more popular algorithms and architectures used in each of these paradigms.

The tables below enlists the courses materials for Week 1 to Week 12. Each topic has both YouTube link and VideoKen link.

Week 1 - Introduction to Machine Learning & Probability Theory

  • A brief introduction to machine learning
    		•
  • Supervised learning
    		•
  • Unsupervised learning
    		•
  • Reinforcement learning
    		•
  • Probability tutorial - 1
    		•
  • Probability tutorial - 2
    		•
  • Assignment 1
  • Solution 1
    •

    Week 2 - Linear Algebra, Statistical Decision Theory, Linear Regression & Dimensionality Reduction

  • Linear algebra tutorial - 1
    		•
  • Linear algebra tutorial - 2
    		•
  • Statistical decision theory - regression
    		•
  • Statistical decision theory - classification
    		•
  • Bias-variance
    		•
  • Linear regression
    		•
  • Multivariate regression
    		•
  • Subset selection - 1
    		•
  • Subset selection - 2
    		•
  • Shrinkage methods
    		•
  • Principal components regression
    		•
  • Partial least squares
    		•
  •    Assignment 2
  •    Solution 2
    •

    Week 3 - Classification: Linear Models

  • Linear classification
    		•
  • Logistic regression
    		•
  • Linear discriminant analysis - 1
    		•
  • Linear discriminant analysis - 2
    		•
  • Linear discriminant analysis - 3
    		•
  • Weka tutorial
    		•
  •    Assignment 3
  •    Assignment 3 data
  •    Solution 3
    •

    Week 4 - Optimisation & Classification: Separating Hyperplane Approaches

  • Optimisation tutorial
    		•
  • Perceptron learning
    		•
  • SVM - formulation
    		•
  • SVM - interpretation & analysis
    		•
  • SVMs for linearly non-separable data
    		•
  • SVM kernels
    		•
  • SVM hinge loss formulation
    		•
  •    Assignment 4
  •    Assignment 4 data
  •    Solution 4
    •

    Week 5 - Artificial Neural Networks & Parameter Estimation

  • Early artificial neural network models
    		•
  • Backpropogation - 1
    		•
  • Backpropogation - 2
    		•
  • Initialisation, training & validation
    		•
  • Maximum likelihood estimate
    		•
  • Priors & MAP estimate
    		•
  • Bayesian parameter estimation
    		•
  •    Assignment 5
  •    Solution 5
    •

    Week 6 - Decision Trees

  • Introduction to decision trees
    		•
  • Regression trees
    		•
  • Stopping criteria & pruning
    		•
  • Loss functions for classification
    		•
  • Categorical attributes
    		•
  • Multiway splits
    		•
  • Missing values, imputation & surrogate splits
    		•
  • Instability, smoothness & repeated subtrees
    		•
  • Decision trees tutorial
    		•
  •    Assignment 6
  •    Assignment 6 data
  •    Solution 6
    •

    Week 7 - Evaluation Measures & Hypothesis Testing

  • Evaluation measures
    		•
  • Bootstrapping & cross validation
    		•
  • 2 class evaluation measures
    		•
  • The ROC curve
    		•
  • Minimum description length & exploratory analysis
    		•
  • Introduction to hypothesis testing
    		•
  • Hypothesis testing: basic concepts
    		•
  • Sampling distribution & the Z test
    		•
  • Student's T test
    		•
  • The two sample & paired sample T tests
    		•
  • Confidence intervals
    		•
  •    Assignment 7
  •    Solution 7
    •

    Week 8 - Ensemble Methods

  • Bagging, committee machines & stacking
    		•
  • Boosting
    		•
  • Gradient boosting
    		•
  • Random forest
    		•
  •    Assignment 8
  •    Solution 8
    •

    Week 9 - Graphical Models

  • Naive Bayes
    		•
  • Bayesian networks
    		•
  • Undirected graphical models - introduction
    		•
  • Undirected graphical models - potential functions
    		•
  • Hidden Markov models
    		•
  • Variable elimination
    		•
  • Belief propogation
    		•
  •    Assignment 9
  •    Solution 9
    •

    Week 10 - Clustering

  • Partitional clustering
    		•
  • Hierarchical clustering
    		•
  • Threshold graphs
    		•
  • The BIRCH algorithm
    		•
  • The CURE algorithm
    		•
  • Density based clustering
    		•
  •    Assignment 10
  •    Solution 10
    •

    Week 11 - Gaussian Mixture Models, Spectral Clustering & Learning Theory

  • Gaussian mixture models
    		•
  • Expectation maximisation - 1
    		•
  • Expectation maximisation - 2
    		•
  • Spectral clustering
    		•
  • Learning theory
    		•
  •    Assignment 11
  •    Solution 11
    •

    Week 12 - Frequent Itemset Mining & Reinforcement Learning

  • Frequent itemset mining
    		•
  • The apriori property
    		•
  • Introduction to reinforcement learning
    		•
  • RL framework & TD learning
    		•
  • Solution methods & applications
    		•
  • Miscellaneous: Multi-class classification
    		•
  •    Assignment 12
  •    Solution 12
    •