Course Contents

Computer Vision Handout

Objectives:

• Computer Vision focuses on development of algorithms and techniques to analyze and interpret the visible world around us. This requires understanding of the fundamental concepts related to multi-dimensional signal processing, feature extraction, pattern analysis visual geometric modeling, stochastic optimization etc. Knowledge of these concepts is necessary in this field, to explore and contribute to research and further developments in the field of computer vision. Applications range from Biometrics, Medical diagnosis, document processing, mining of visual content, to surveillance, advanced rendering etc.

• Digital Image Formation and low-level processing
  • Overview and State-of-the-art, Fundamentals of Image Formation, Transformation: Orthogonal, Euclidean, Affine, Projective, etc; Fourier Transform, Convolution and Filtering, Image Enhancement, Restoration, Histogram Processing.


• Depth estimation and Multi-camera views
  • Perspective, Binocular Stereopsis: Camera and Epipolar Geometry; Homography, Rectification, DLT, RANSAC, 3-D reconstruction framework; Auto-calibration.


• Feature Extraction
  • Edges - Canny, LOG, DOG; Line detectors (Hough Transform), Corners - Harris and Hessian Affine, Orientation Histogram, SIFT, SURF, HOG, GLOH, Scale-Space Analysis- Image Pyramids and Gaussian derivative filters, Gabor Filters and DWT.


• Image Segmentation
  • Region Growing, Edge Based approaches to segmentation, Graph-Cut, Mean-Shift, MRFs, Texture Segmentation; Object detection.


• Pattern Analysis
  • Clustering: K-Means, K-Medoids, Mixture of Gaussians, Classification: Discriminant Function, Supervised, Un-supervised, Semi-supervised; Classifiers: Bayes, KNN, ANN models; Dimensionality Reduction: PCA, LDA, ICA; Non-parametric methods.


• Motion Analysis
  • Background Subtraction and Modeling, Optical Flow, KLT, Spatio-Temporal Analysis, Dynamic Stereo; Motion parameter estimation.


• Shape from X
  • Light at Surfaces; Phong Model; Reflectance Map; Albedo estimation; Photometric Stereo; Use of Surface Smoothness Constraint; Shape from Texture, color, motion and edges.

References

Textbooks

Richard Szeliski, Computer Vision: Algorithms and Applications, Springer-Verlag London Limited 2011.

Computer Vision: A Modern Approach, D. A. Forsyth, J. Ponce, Pearson Education, 2003.

References

Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, Second Edition, Cambridge University Press, March 2004.

Christopher M. Bishop; Pattern Recognition and Machine Learning, Springer, 2006

R.C. Gonzalez and R.E. Woods, Digital Image Processing, Addison- Wesley, 1992.

K. Fukunaga; Introduction to Statistical Pattern Recognition, Second Edition, Academic Press, Morgan Kaufmann, 1990.

Journals

IEEE-T-PAMI (IEEE Transactions on Pattern Analysis and Machine Intelligence).

IJCV (International Journal of Computer Vision) - Springer.

Lecture Slides

Topic-Wise References

1	Introduction
2	Neighborhood and Connectivity of pixels
3	Fourier Theory and Filtering in spatial and spectral domains
4	(i)Enhancement & (ii)Restoration
5	Histogram based image processing
6	3D transformations and projection
7	Concepts in Edge Detection
8	Hough Transform
9	Image segmentation
10	Pattern Recognition
11	Motion Detection and Tracking
12	Shape from Shading
13	Texture analysis using Gabor filters
14	SCALE-SPACE - Theory and Applications
15	Local Feature Detectors and Descriptors
13	Motion
17	Wavelet transform
18	Morphology
19	Image Restoration

Additional Resources

Video Lectures Links:

• CNN
  
  
  1. Convolutional Neural Networks (Course 4 of the Deep Learning Specialization By Andrew Ng)
  2. MIT 6.S191: Convolutional Neural Networks
  3. Week 3 Lecture: Convolutional neural networks


• RNN AND LSTM
  
  
  1. NYU Week 6 Lecture: CNN applications, RNN, and attention
  2. NYU Week 6 Practicum: RNN and LSTM architectures
  3. MIT 6.S191: Recurrent Neural Networks and Transformers
  4. MIT 6.S191 (2020): Recurrent Neural Networks
  5. C5W1 Recurrent Neural Networks
  6. Deep Learning Chapter 10 Sequence Modeling: Recurrent and Recursive Nets presented by Ian Goodfellow


• Auto Encoders and Variational Autoencoders
  
  
  1. NYU Week 7 Practicum: Under- and over-complete autoencoders
  2. NYU Week 8 Practicum: Variational autoencoders
  3. Autoencoders Explained Easily
  4. Variational Autoencoders - EXPLAINED!


• Generative Adversarial Networks:
  
  
  1. Introduction to GANs, NIPS 2016 | Ian Goodfellow, OpenAI
  2. NYU Week 9 Lecture: Group sparsity, world model, and generative adversarial networks (GANs)
  3. NYU Week 9 Practicum: (Energy-based) Generative adversarial networks
  4. MIT 6.S191: Deep Generative Modeling


• Transformers:
  
  
  1. NYU Week 12 Practicum: Attention and the Transformer
  2. NYU Week 12 Lecture: Deep Learning for Natural Language Processing (NLP)
  3. Transfer learning and Transformer models (ML Tech Talks)


• Google Colab Tutorial:
  https://youtu.be/RLYoEyIHL6A

Github Links:

• CNN, RNN, GAN:
  https://github.com/omerbsezer/Fast-Pytorch


• LSTM, RNN:
  https://github.com/omerbsezer/LSTM_RNN_Tutorials_with_Demo


• Auto Encoders:
  https://github.com/nathanhubens/Autoencoders


• Variational Auto encoders:
  https://github.com/AntixK/PyTorch-VAE


• Transformers:
  
  
  1. https://github.com/NielsRogge/Transformers-Tutorials
  2. https://github.com/huggingface/transformers

Interesting links:

• Papers with Code


• Object Detection


• Image Classification


• Video Prediction

Term Project Assignment

Term Project Allotment List: To be Decided

Term Project Assignment List
SL No.	Problem Statement	Performance based marks (indicative)			Group IDs Alloted	Details
		Satisfactory	Good	Excellent
1	Domain Adaptation for Object Detection in target domain using weakly-supervised / self-supervised or semi-supervised settings	17	-	41	--
2	Face Recognition using Face Images obtained from the Internet	16	-	39	--
3	YOLOv5++ for overlap object detection from cluttered indoor shots, invariant to sensor, lighting and affine transformation	17	-	41	--
4	Video Analytics: Prediction, affordances; or any new/novel ideas/tasks yet unheard of in CV/DL paradigm *	15	-	43	--
5	Tiny object Detection/Segmentation in Cluttered Images	17	-	41	--
6	Scene Segmentation of indoor Panorama	16	-	42	--
7	Joint Image Deblurring/Super-Resolution and Low-light Image Enhancement	17	-	38	--
8	Learn image auto-correction (brush restoration) and auto-enhancement from few training samples #	17	-	41	--
9	Image to Image transformation (few samples) using VAE, GANs etc	15	-	40	--
10	Real (True) depth estimation from indoor scenes, given a model (DL tool) for virtual depth estimation	16	-	42	--
11	3D Topologically-Aware Semantic Scene reconstruction and depth map / wireframe / Point-Cloud from single RGB panorama scene (or Two views)	16	-	43	--
12	Synthesis(Generation) of adversarial (image) datasets to prove null hypothesis on DL systems (pick any SOA: OR, FR, Segmentation)	17	-	41	--
13	Auto-estimation of homography over a planar patch, from a single view #	15	-	39	--
14	Face Recognition in the Wild	16	-	40	--
15	Student Distillation for high-end compute-intensive DL process to run on low-end GPU	17	-	41	--
16	DL based Depth estimation of landmarks/salient points and Scene Reconstruction, from arbitrary pair of stereo views	16	-	42	--
17	Human Cell Detection Using Very Few Samples	15	-	38	--	TBU

* Get permission from course instructor for the novel plan of work.
# Can be solved using shallow techniques of computer vision

Tutorial

  Note: These dates are tentative.

Tutorial No.	Date	Time
1	10/08/2023	10:00-10:50
2	24/08/2023	10:00-10:50
3	07/09/2023	10:00-10:50
4	05/10/2023	10:00-10:50
5	19/10/2023	10:00-10:50
6	02/11/2023	10:00-10:50

Schedule

Marks Distribution

• End/Mid Sem : 35-40
• Quiz : 15-20
• Tutorial : 10-15
• TPA : 30-35

Logistic Details

• TA: S Meena Padnekar(cs21d015@smail.iitm.ac.in)
  
• TA: Oikantik Nath(cs22s013@smail.iitm.ac.in)
  
• TA: Pooja Kumari(cs20d006@smail.iitm.ac.in)
  

Class Schedule (Slot G)
Monday	12:00-12:50pm
Wednesday	5:00-5:50pm
Thursday	10:00-10:50pm
Friday	9:00-9:50am

Announcements

Optional TPA : 18/10/2023

Extra Class : To be announced

Final TPA Review: Slot 1: 21/11/2023, Slot 2: 27/11/2023

Self-study

Slides for Self-Study Content

Sl. no.	Content (pdf filename)	Portion	Link
1	Neighborhood and Connectivity of pixels	Entire pdf
2	Fourier Theory and Filtering in spatial and spectral domains	Slides 1-27, 37-44
3	Enhancement	Slides 21-27
4	Histogram based image processing	First 23 slides
5	3D transformations and projection	Slides 1-10
6	Concepts in Edge Detection	Slides 57-67, 70-93(Adv. studies)
7	Hough Transform	Slides 35-40
8	Pattern Recognition	Slides 44-63
9	Local Feature Detectors and Descriptors	First 25 slides

    Note:

• Preparation based on self-study contents are expected to be done as and when the topic comes up for discussion in the class.
• Students may expect the course instructor to inform them 1-2 days in advance.
• Self-study contents are part of the syllabus and will be asked in the exam.