Course Contents

Computer Vision Handout

Objectives:

• Computer Vision focuses on development of algorithms and techniques to analyze and interpret the visible world around us. This requires understanding of the fundamental concepts related to multi-dimensional signal processing, feature extraction, pattern analysis visual geometric modeling, stochastic optimization etc. Knowledge of these concepts is necessary in this field, to explore and contribute to research and further developments in the field of computer vision. Applications range from Biometrics, Medical diagnosis, document processing, mining of visual content, to surveillance, advanced rendering etc.

• Digital Image Formation and low-level processing
  • Overview and State-of-the-art, Fundamentals of Image Formation, Transformation: Orthogonal, Euclidean, Affine, Projective, etc; Fourier Transform, Convolution and Filtering, Image Enhancement, Restoration, Histogram Processing.


• Depth estimation and Multi-camera views
  • Perspective, Binocular Stereopsis: Camera and Epipolar Geometry; Homography, Rectification, DLT, RANSAC, 3-D reconstruction framework; Auto-calibration.


• Feature Extraction
  • Edges - Canny, LOG, DOG; Line detectors (Hough Transform), Corners - Harris and Hessian Affine, Orientation Histogram, SIFT, SURF, HOG, GLOH, Scale-Space Analysis- Image Pyramids and Gaussian derivative filters, Gabor Filters and DWT.


• Image Segmentation
  • Region Growing, Edge Based approaches to segmentation, Graph-Cut, Mean-Shift, MRFs, Texture Segmentation; Object detection.


• Pattern Analysis
  • Clustering: K-Means, K-Medoids, Mixture of Gaussians, Classification: Discriminant Function, Supervised, Un-supervised, Semi-supervised; Classifiers: Bayes, KNN, ANN models; Dimensionality Reduction: PCA, LDA, ICA; Non-parametric methods.


• Motion Analysis
  • Background Subtraction and Modeling, Optical Flow, KLT, Spatio-Temporal Analysis, Dynamic Stereo; Motion parameter estimation.


• Shape from X
  • Light at Surfaces; Phong Model; Reflectance Map; Albedo estimation; Photometric Stereo; Use of Surface Smoothness Constraint; Shape from Texture, color, motion and edges.

References

Textbooks

Richard Szeliski, Computer Vision: Algorithms and Applications, Springer-Verlag London Limited 2011.

Computer Vision: A Modern Approach, D. A. Forsyth, J. Ponce, Pearson Education, 2003.

References

Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, Second Edition, Cambridge University Press, March 2004.

Christopher M. Bishop; Pattern Recognition and Machine Learning, Springer, 2006

R.C. Gonzalez and R.E. Woods, Digital Image Processing, Addison- Wesley, 1992.

K. Fukunaga; Introduction to Statistical Pattern Recognition, Second Edition, Academic Press, Morgan Kaufmann, 1990.

Journals

IEEE-T-PAMI (IEEE Transactions on Pattern Analysis and Machine Intelligence).

IJCV (International Journal of Computer Vision) - Springer.

Lecture Slides

Topic-Wise References

1	Introduction
2	Neighborhood and Connectivity of pixels
3	Fourier Theory and Filtering in spatial and spectral domains
4	(i)Enhancement & (ii)Restoration
5	Histogram based image processing
6	3D transformations and projection
7	Concepts in Edge Detection
8	Hough Transform
9	Image segmentation
10	Pattern Recognition
11	Motion Detection and Tracking
12	Shape from Shading
13	Texture analysis using Gabor filters
14	SCALE-SPACE - Theory and Applications
15	Local Feature Detectors and Descriptors
13	Motion
17	Wavelet transform
18	Morphology
19	Image Restoration

Additional Resources

Video Lectures Links:

• CNN
  
  
  1. Convolutional Neural Networks (Course 4 of the Deep Learning Specialization By Andrew Ng)
  2. MIT 6.S191: Convolutional Neural Networks
  3. Week 3 Lecture: Convolutional neural networks


• RNN AND LSTM
  
  
  1. NYU Week 6 Lecture: CNN applications, RNN, and attention
  2. NYU Week 6 Practicum: RNN and LSTM architectures
  3. MIT 6.S191: Recurrent Neural Networks and Transformers
  4. MIT 6.S191 (2020): Recurrent Neural Networks
  5. C5W1 Recurrent Neural Networks
  6. Deep Learning Chapter 10 Sequence Modeling: Recurrent and Recursive Nets presented by Ian Goodfellow


• Auto Encoders and Variational Autoencoders
  
  
  1. NYU Week 7 Practicum: Under- and over-complete autoencoders
  2. NYU Week 8 Practicum: Variational autoencoders
  3. Autoencoders Explained Easily
  4. Variational Autoencoders - EXPLAINED!


• Generative Adversarial Networks:
  
  
  1. Introduction to GANs, NIPS 2016 | Ian Goodfellow, OpenAI
  2. NYU Week 9 Lecture: Group sparsity, world model, and generative adversarial networks (GANs)
  3. NYU Week 9 Practicum: (Energy-based) Generative adversarial networks
  4. MIT 6.S191: Deep Generative Modeling


• Transformers:
  
  
  1. NYU Week 12 Practicum: Attention and the Transformer
  2. NYU Week 12 Lecture: Deep Learning for Natural Language Processing (NLP)
  3. Transfer learning and Transformer models (ML Tech Talks)


• Google Colab Tutorial:
  https://youtu.be/RLYoEyIHL6A

Github Links:

• CNN, RNN, GAN:
  https://github.com/omerbsezer/Fast-Pytorch


• LSTM, RNN:
  https://github.com/omerbsezer/LSTM_RNN_Tutorials_with_Demo


• Auto Encoders:
  https://github.com/nathanhubens/Autoencoders


• Variational Auto encoders:
  https://github.com/AntixK/PyTorch-VAE


• Transformers:
  
  
  1. https://github.com/NielsRogge/Transformers-Tutorials
  2. https://github.com/huggingface/transformers

Interesting links:

• Papers with Code


• Object Detection


• Image Classification


• Video Prediction

Term Project Assignment

Term Project Allotment List: To be Decided

Term Project Assignment List
SL No.	Problem Statement	Performance based marks (indicative)			Group IDs Alloted	Details
		Satisfactory	Good	Excellent
1	Domain Adaptation for Object Detection in target domain using weakly-supervised / self-supervised or semi-supervised settings	17	30	41	--
2	Comparison of the performance of few SOTA stereo depth estimation techniques (both shallow and deep) for a foreground object, to achieve high accuracy	15	28	40	--
3	YOLOv5++ for overlap object detection from cluttered indoor shots, invariant to sensor, lighting and affine transformation	17	30	41	--
4	Video Analytics: Prediction, affordances; or any new/novel ideas/tasks yet unheard of in CV/DL paradigm *	15	31	43	--
5	Tiny object Detection/Segmentation in Cluttered Images	17	30	41	--
6	Scene Segmentation of indoor Panorama	16	28	42	--
7	Joint Image Deblurring/Super-Resolution and Low-light Image Enhancement	17	27	38	--
8	Learn image auto-correction (brush restoration) and auto-enhancement from few training samples #	17	30	41	--
9	Exploration of efficient shallow learning methods for object recognition, face recognition, image classification, etc. as alternative to deep learning (training time, dataset size) #	19	32	42	--
10	Real (True) depth estimation from indoor scenes, given a model (DL tool) for virtual depth estimation	16	30	42	--
11	3D Topologically-Aware Semantic Scene reconstruction and depth map / wireframe / Point-Cloud from single RGB panorama scene (or Two views)	16	28	43	--
12	Synthesis(Generation) of adversarial (image) datasets to prove null hypothesis on DL systems (pick any SOA: OR, FR, Segmentation)	17	28	41	--
13	Compute change in pose (3D) of a small object (coin, key, mobile phone, purse, clock, plate, checkerboard etc.) with very high accuracy from two successive views (arbitrary) of the same object (objects can have a near planar surface) #	20	33	43	--
14	Face Recognition in the Wild	17	30	41	--
15	Student Distillation for high-end compute-intensive DL process to run on low-end GPU	17	28	41	--
16	DL based Depth estimation of landmarks/salient points and Scene Reconstruction, from arbitrary pair of stereo views	16	30	42	--
17	Human Cell Detection Using Very Few Samples	15	26	38	--	TBU
18	Augmented Reality based apparel fitting	-	-	-	--

* Get permission from course instructor for the novel plan of work.
# Can be solved using shallow techniques of computer vision

Tutorial

  Note: These dates are tentative.

Tutorial No.	Date	Time
1	16/08/2024	9:00-9:50
2	05/09/2024	9:00-9:50
3	03/10/2024	9:00-9:50
4	17/10/2024	9:00-9:50
5	30/10/2024	9:00-9:50

Schedule

Marks Distribution

• End/Mid Sem : 35-40
• Quiz : 15-20
• Tutorial : 10-15
• TPA : 30-35

Logistic Details

• TA: Pooja Kumari(cs20d006@smail.iitm.ac.in)
  
• TA: Apu Sarkar(cs23d003@smail.iitm.ac.in)
  
• TA: S Meena Padnekar(cs21d015@smail.iitm.ac.in)
  

Class Schedule (Slot G)
Monday	12:00-12:50pm
Wednesday	5:00-5:50pm
Thursday	10:00-10:50pm
Friday	9:00-9:50am

Announcements

Self-study

Slides for Self-Study Content

Sl. no.	Content (pdf filename)	Portion	Link
1	Neighborhood and Connectivity of pixels	Entire pdf
2	Fourier Theory and Filtering in spatial and spectral domains	Slides 1-27, 37-44
3	Enhancement	Slides 21-27
4	Histogram based image processing	First 23 slides
5	3D transformations and projection	Slides 1-10
6	Concepts in Edge Detection	Slides 57-67, 70-93(Adv. studies)
7	Hough Transform	Slides 35-40
8	Pattern Recognition	Slides 44-63
9	Local Feature Detectors and Descriptors	First 25 slides

    Note:

• Preparation based on self-study contents are expected to be done as and when the topic comes up for discussion in the class.
• Students may expect the course instructor to inform them 1-2 days in advance.
• Self-study contents are part of the syllabus and will be asked in the exam.