Course Contents

Linear Algebra:

• Matrices
  • Matrix Multiplication, Transposes, Inverses, Gaussian Elimination, Matrix norms, factorization A=LU, rank.


• Vector spaces
  • Column and row spaces, Solving Ax=0 and Ax=b, Independence, basis, dimension, linear transformations.


• Orthogonality
  • Orthogonal vectors and subspaces, Normal matrix, projection and least squares, Gram-Schmidt orthogonalization, QR factorization, Cholesky decomposition, polar decomposition.


• Determinants
  • Determinant formula, cofactors, inverses and volume.


• Eigenvalues and Eigenvectors
  • Characteristic polynomial, Diagonalization, Hermitian and Unitary matrices, Toeplitz matrices, Spectral theorem, Change of basis.


• Positive definite matrices
  • Positive definite and semi-definite matrices, Pseudo-inverse, Singular Value Decomposition (SVD), RRQR factorization.

Random processes:

• Preliminaries
  • Events, probability, conditional probability, independence, product spaces.


• Random Variables
  • Distributions, law of averages, discrete and continuous r.v.s, random vectors, Monte Carlo simulation.


• Discrete Random Variables
  • Probability mass functions, independence, expectation, conditional expectation, sums of r.v.s.


• Continuous Random Variables
  • Probability density functions, independence, expectation, conditional expectation, functions of r.v.s, sum of r.v.s, multivariate normal distribution, sampling from a distribution.


• Convergence of Random Variables
  • Modes of convergence, Borel-Cantelli lemmas, laws of large numbers, central limit theorem, tail inequalities.



• Estimation of parameters from data
  • Kullback-Leibler Divergence, Principal Component Analysis, Chi-square test, Student's T-test, Maximum Likelihood Estimate, Expectation-Maximization.


• Advanced topics
  • Markov chains, minimum mean squared error estimation.

References

<

Textbooks

Linear Algebra and Its Applications - Gilbert Strang- Fourth Edition- Cengage Learning, 2006.

Probability, Random Variables and Stochastic Processes, Papoulis and Unnikrishnan, Fourth Edition, 2002.

Probability and Random Processes, Geoffrey Grimmett and David Stirzaker, Third Edition.

References

Probability and Statistics with Reliability, Queuing, and Computer Science Applications - Kishore S Trivedi, PHI.

Linear Algebra (Second edition)- Kenneth Hoffman and Ray Kunze, Prentice Hall India, 2013.

Linear Algebra (Second Edition) Cheney and Kincaid, Jones and Bartlett learning, 2014.

Probability and Random Processes: An introduction for application scientists and engineers, W B Davenport, 1970, McGraw Hill.

Lecture Slides

1	Introduction
2	Introduction to Matrices
3	Matrix Multiplication
4	Vectors and Matrix Norms
5	Gaussian Elimination
6	LU Factorization
7	Matrix Inverse and Transpose
8	Introduction to Vectors Spaces
9	Vectors Spaces: Solving Ax=0 and Ax=b
10	Vectors Spaces: Independence, Basis, Dimension
11	Vectors Spaces: Linear transformations
12	Vectors Spaces: Four Fundamental Subspaces
13	Orthogonality: 3.1 and 3.2 of Strang's book
14	Orthogonality: 3.3 and 3.4 of Strang's book
15	Determinants
16	EigenValues and EigenVectors
17	Positive Semidefinite Matrices
18	Basics of Probability
19	Permutations and Combinations
20	Random Variables and their Distributions
21	Discrete Random Variables
22	Discrete Distributions
23	Continuous Random Variables
24	MLE and KLD
25	Principal Component Analysis

Assignment

Software Assignment 1 (Deadline : 28/09/2018 11:55pm)
        Problem Description :         Input-Output Specifications and Coding Description :         Submission Instructions :

Software Assignment 2 (Deadline : 05/10/2018 11:55pm)
        Problem Description :         Input-Output Specifications and Coding Description :         Submission Instructions :

Software Assignment 3 (Deadline : 10/11/2018 11:55 pm)
        Problem Statement :         Input-Output Specifications :         Submission Instructions :         Allotted Input Data :         Click here to go the Data Repository page

Software Assignment 4 (Deadline : )
        Problem Statement :         Input-Output Specifications :         Submission Instructions :

    General Instructions

• All codes should be written in C/C++ programming languages only.
• The code should be written in a way that it conforms with the given input/output specifications (see the input/output specification document).
• Incremental penalty rule will be imposed for late submissions upto 6 days. Your submission will not be accepted a week past the announced deadline. Marks for these cases will be given as zero (0).
• Any sort of plagiarism/cheating will be dealt very strictly. All the submitted codes will be matched with that submitted by others registered for the course.
• Submissions will be accepted through Moodle (https://courses.iitm.ac.in). Navigate to the page for the course "CS6015: Linear Algebra and Random Processes".
• Submissions through e-mail will not be accepted.

Tutorial

Tutorial No.	Date	Time	Solution
1	14/08/2018	08:00-08:50
2	28/08/2018	08:00-08:50
3	11/09/2018	08:00-08:50
4	09/10/2018	08:00-08:50
5	23/10/2018	08:00-08:50
6	10/11/2018	14:00-15:30
7	13/11/2018	08:00-08:50

Announcement

Marks Distribution

• End Sem    : 30
• Mid Sem    : 30
• Tutorial     : 20
• Programming
  Assignment   : 20

Logistic Details

• TA: Prateep Bhattacharjee (prateepcs4@gmail.com)
• TA: Saptakatha Adak (saptakatha2@gmail.com)
• TA: Sandeep Narayan P (sandeepnarayan.p@gmail.com)
• TA: R Janani (janani.ramaswamy96@gmail.com)
• TA: Manas Ranjan Sahoo (cs17m024@smail.iitm.ac.in)
• TA: Mahesh Hebbar (cs17m023@smail.iitm.ac.in)

Important Dates (Tentative)
Tutorial Dates	14/08/2018, 28/08/2018, 11/09/2018, 09/10/2018, 23/10/2018, 10/11/2018
Extra Classes	18/08/2018, 06/10/2018, 27/10/2018
Mid Semester Exam	29/09/2018 (10:00 am - 12:00 pm, Venue: CSB15)
End Semester Exam	16/11/2018 (10:00 am - 12:00 pm , Venue: CSB15)

Self-study

  Will be Updated Soon