Prashanth L.A.
×
Home
Biography
Awards
Research
Books
Publications
By Year
Tutorials
Risk-RL
Sim-opt
Risk-bandits
Teaching
List
Course notes
RL
Bandits
Sto-opt
ML
Markov chains
LARP
Oops lab
PDS
OS
UGRC
Mentoring
MS/PhD
Projects
Others
Activities

Lectures on Reinforcement Learning Theory

Html-ized notes on theory of MDPs

Click here

Handwritten notes

  • Finite horizon MDPs

  • Stochastic shortest path problems

  • Discounted MDPs

  • RL foundations

  • Full state algorithms

  • Markov chains (review)

  • RL with function approximation

  • Policy gradient algorithms

Introductory Lectures on Bandits

Click here

Note: Bandits notes is highly incomplete.

Introduction to Stochastic Optimization

  • Introduction to zeroth-order optimization

  • Machine learning applications

  • Smoothness, convexity and strong convexity

  • Martingales

  • Introduction to stochastic approximation

  • Asymptotic convergence of stochastic approximation

  • Variants of zeroth-order gradient estimates

  • Bounds for gradient descent (noiseless case)

  • Non-asymptotics for stochastic gradient

  • Reinforcement learning

Introductory Lectures on Machine Learning

  • Probability review

  • Singular value decomposition, principal component analysis

  • Bayes classifier, maximum likelihood estimation

  • Linear models

  • PAC learning

  • Crash course on optimization

  • Logistic regression

  • Support vector machines, kernel methods

  • Online learning

  • Mixture models

Markov Chains

  • Poisson processes

  • DTMCs (transient behavior)

  • DTMCs (limiting behavior)

  • CTMCs

Last edited on May 15th 2025 12:50PM (Time Zone: IST).
Powered by jemdoc + new design.