CS6015: Linear Algebra and Random Processes
Course information
When: Jul-Nov 2019
Lectures: Slot D
Where: CS36
Teaching Assistants: Nirav Bhavsar, Ajay Pandey, Nithia V, Monisha J
Office Hours: Wed 2:30pm to 3:30pm
Course Content: See Schedule below
Grading
Mid-term (Linear Algebra concepts): 30%
Final exam (Probability concepts): 30%
Quizzes: 20% (Best 4 out of 6)
Programming Assignments: 20%
Target Audience
Masters (M.Tech/M.S.) and Ph.D. students
Important Dates
Quizzes: Aug 13, Aug 29, Sep 19, Oct 14, Oct 30, Nov 11
Mid-term: 10am to 12pm, Sep 28
End-sem: 2pm to 5pm, Nov 20
Lecture Notes
Linear Algebra (incomplete)
Probability
Textbooks
Linear Algebra:
Linear algebra and applications by Gilbert Strang
Linear Algebra: Pure and Applied by Edgar Goodaire
Probability
Probability and random processes by Geoffrey Grimmett and David Stirzaker
ECE 313 UIUC course lecture notes by Bruce Hajek
Additional references:
Introduction to Probability by Bertsekas and Tsitsiklis
Introduction to probability models by Sheldon Ross
Schedule from 2017
Part I: Linear Algebra | ||
Lecture number | Topics Covered | Section reference (Strang's book) |
---|---|---|
Lecture 1 | Course Organization Motivation for studying linear algebra | |
Lecture 2 | Geometry of linear equations - row picture, col picture Vector space, subspace - definition, examples Linear combinations, linear independence | 1.2 |
Lecture 3 | Transpose - properties Inverse Gaussian elimination | 1.3 |
Lecture 4 | Computational cost of elimination Matrix multiplication - four view Gauss-Jordan method | 1.4 |
Lecture 5 | Factorization A=LU and A=LDU Row exchanges, permutation matrices Uniqueness of LU/LDU factorization for invertible matrices | 1.5 |
Lecture 6 | Column and null spaces Null space computation by solving Ax=0 Pivot and free variables, special solutions row reduced echelon form | 2.1, 2.2 |
Lecture 7 | Dimension = number of vectors in any basis Rank, row-rank = col-rank rank + nullity = number of columns | 2.3, 2.4 |
Lecture 8 | Orthogonal vectors and subspaces Row space orthogonal to null space | 3.1 |
Quiz 1 | ||
Lecture 9 | Projection onto a line Projection onto a subspace | 3.2 |
Lecture 10 | Least squares data fitting Orthonormal vectors | 3.3 |
Lecture 11 | Four fundamental subspaces (again) Least squares data fitting | 2.4, 3.3 |
Lecture 12 | Orthogonal bases Gram Schmidt algorithm Factorization A=QR | 3.4 |
Quiz 2 | ||
Lecture 13 | Linear transformations: definition, matrix representation | 2.6 |
Lecture 14 | Composition of linear transformations Change of basis | Change of basis: Section 46 of Halmos's text |
Lecture 15 | Similarity of transformations | Section 47 of Halmos's text |
Quiz 3 | ||
Lecture 16 | Determinants: Properties, Formula | 4.2, 4.3 |
Lecture 17 | Determinants: Cofactors, Applications in graphs | 4.3, 4.4 Graph applications: Section 3.3 of Goodaire's text |
Lecture 18 | Eigenvalues and Eigenvectors | 5.1 |
Lecture 19 | Similarity and diagonalization | 5.2 |
Lecture 20 | Spectral theorem for real symmetric matrices: case when eigenvalues are distinct | 5.5 |
Lecture 21 | Complex vector space, Hermitian and unitary matrices | 5.5 |
Quiz 4 | ||
Lecture 22 | Schur's theorem Spectral theorem as a corollary | 5.5 |
Lecture 23 | Singular value decomposition | 6.3 |
Mid-sem Exam |
Part II: Random Processes | ||
Lecture number | Topics Covered | Section reference (Grimmett's book) |
---|---|---|
Lecture 1 | Events as sets, probability spaces | 1.2 |
Lecture 2 | Cardinality, countability and infinite sums | Sections 4 and 5 of Manjunath Krishnapur's notes |
Lecture 3 | Properties of probability measure | 1.3 |
Lecture 4 | Conditional probability | 1.4 |
Quiz 5 | ||
Lecture 5 | Independence Gambler's ruin | 1.5 |
Lecture 6 | Random variables and distribution function | 2.1 |
Lecture 7 | Uncountable probability spaces, distribution functions | Section 14 of Manjunath Krishnapur's notes |
Lecture 8 | Law of averages - Bernstein's inequality | 2.2 |
Lecture 9 | Discrete r.v.s - definition and examples Independence | 3.1 |
Quiz 6 | ||
Lecture 10 | Discrete r.v.s - expectation, higher moments and examples | 3.3, 3.5 |
Lecture 11 | Discrete r.v.s - joint distribution function Covariance, correlation | 3.6 |
Lecture 12 | Discrete r.v.s - conditional distribution and expectation | 3.7 |
Lecture 13 | Discrete r.v.s - conditional expectation Sums of r.v.s | 3.7, 3.8 |
Quiz 7 | ||
Lecture 14 | Continuous r.v.s - p.d.f., independence, expectation | 4.1, 4.2, 4.3 |
Lecture 15 | Continuous r.v.s - examples | 4.4 |
Lecture 16 | Continuous r.v.s - dependence | 4.5 |
Quiz 8 | ||
Lecture 17 | Continuous r.v.s - conditional distributions and expectation | 4.6 |
Lecture 18 | Continuous r.v.s - functions of r.v.s | 4.7 |
Lecture 19 | Continuous r.v.s - change of variables | 4.7 |
Lecture 20 | Continuous r.v.s - multivariate normal distribution | 4.9 |