Course Page for CS550 (Machine Learning) to be taught at IIT Bhilai, India in the Monsoon Semester of 2023.
Course Instructor: Dr. Gagan Raj Gupta

Machine Learning is concerned with computer programs that automatically improve their performance through experience (e.g., programs that learn to recognize objects, Analyze sentiments, recommend music and movies, and drive autonomous robots). This course covers the theory and practical algorithms for machine learning from a variety of perspectives.

Topics include:

• Supervised Learning (Regression/Classification), Linear models: Linear Regression, Logistic Regression, Generalized Linear Models, Support Vector Machines, Nonlinearity and Kernel Methods, Multi-class/Structured Outputs, Ranking/Grading
• Evaluating Machine Learning algorithms and Model Selection, Ensemble Methods (Boosting, Bagging, Random Forests), Sparse Modeling and Estimation,
• Unsupervised Learning, Clustering: K-means/Kernel K-means, Dimensionality Reduction: PCA and kernel PCA, Matrix Factorization and Matrix Completion,
• Generative Models (mixture models and latent factor models), Diffusion models, GAN, (Variational) Autoencoders
• Assorted Topics: learning theory (bias/variance tradeoffs, practical advice)
• Deep Learning and Feature Representation Learning: CNN, RNN, GNN

The course will also discuss recent machine-learning applications, such as computer vision, medical imaging, time-series mining, bioinformatics, web and industrial data processing.

Programming assignments include hands-on experiments with various learning algorithms. This course is designed to give a graduate-level student a thorough grounding in the methodologies, technologies, mathematics, and algorithms currently needed by people researching machine learning.

• Implement and analyze existing learning algorithms, including well-studied methods for classification, regression, structured prediction, clustering, and representation learning
• Integrate multiple facets of practical machine learning in a single system: data preprocessing, learning, regularization, and model selection
• Describe the formal properties of models and algorithms for learning and explain the practical implications of those results
• Compare and contrast different paradigms for learning (supervised, unsupervised, etc.)
• Design experiments to evaluate and compare different machine learning techniques on real-world problems
• Employ probability, statistics, calculus, linear algebra, and optimization in order to develop new predictive models or learning methods
• Given a description of an ML technique, analyze it to identify
  1. the expressive power of the formalism;
  2. the inductive bias implicit in the algorithm;
  3. the size and complexity of the search space;
  4. the computational properties of the algorithm:
  5. any guarantees (or lack thereof) regarding termination, convergence, correctness, accuracy or generalization power.

Lectures: Wednesday and Friday 11:30 am to 12:20 p.m., Thursday 8:30 am to 9:20 am Lab: Tuesday 3:30-5:30 p.m.

The grading policy has been designed to give as much hands-on practice to students while making sure the fundamentals are strong.

• Mid-sem Exam: 25%
• Theory/Conceptual Homework: 10%
• Programming Assignments: 30%
• Major Project: 25%
• Surprise Quiz(es): 10% (5-6 in-class/lab assessments, typically after every segment). No prior intimation will be given.

Asg 1: Regression + Classification Concepts: Feature engineering, Train/Val/Test Splits, Hyper-parameter tuning, Understanding of performance metrics, Reporting

Asg 2: Ensemble Models Concepts: Random Forests, Boosting, Mixing Models,

Asg 3: Image Classification and Segmentation Concepts: CNN architectures, Transfer Learning, Model customization, Regularization, Trade-off between model size and accuracy

Asg 4: Generative Models Concepts: Auto-encoders, GAN, Diffusion Models

• Students are encouraged to discuss homework and assignments with each other, but the submissions have to be original. If we find plagiarism, your grade will be reduced to D or F.
• Assignments will be individual effort only
• Major Project can be done in groups of 2 or 3 with appropriate justification

• Every homework and/or assignment will have plenty of time to complete. There will be no late days allowed for any homework/assignment/project deadline.
• Students are encouraged to make regular submissions to Canvas portal and not wait for the last minute.
• In exceptional circumstances, the student can seek the instructor's permission for skipping/late submission of an assignment. This will be done MAX one time for any student in the semester.

• Students are expected to attend each class and lab session. There will be surprise quizzes and attendance may also be taken.
• If a student has attendance less than 75% in the lecture component, they will not be allowed to appear for the exam.

Students entering the class are expected to have a pre-existing working knowledge of probability, linear algebra, statistics and algorithms, though the class has been designed to allow students with a strong numerate background to catch up and fully participate.

• Programming experience in a general programming language. Specifically, you need to have written from scratch programs consisting of few hundred lines of code. Note: For each programming assignment, you will be required to use Python. You will be expected to know, or be able to quickly pick up, that programming language.

• Basic familiarity with probability and statistics: (Conditional probability, Bayes Rule, Random variable, independence, conditional independence, Expectation, Variance, Concentration Inequalities, Distributions, Gaussian, Multi-variate)

• Linear Algebra: Vectors and matrices, inner product, projection, Basis, (complete, orthonormal), Orthogonality, linear (in)dependence eigenvalues and eigenvectors; singular values and vectors; SVD

• Discrete mathematics: (Proofs, Induction, Logic, Combinatorics, Graphs)

• You must strictly adhere to these pre-requisites! Even if IIT Bhilai's registration system does not prevent you from registering for this course, it is still your responsibility to make sure you have all of these prerequisites before you register.

• [HML] Hands on Machine Learning Aurélien Géron, 3rd edition.
• [CIML] A Course in Machine Learning, Hal Daumé III
• [MML] Mathematics for Machine Learning, Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong

• [ML] Machine Learning, Tom Mitchell.
• [PRML] Pattern recognition and machine learning, Chistopher Bishop
• [PML] Probabilistic Machine Learning, Kevin Murphy (2nd edition)

• Google Drive Link:
• Canvas Link:
• CheatSheet: https://stanford.edu/~shervine/teaching/cs-229/cheatsheet-supervised-learning
• Example Projects: https://cs229.stanford.edu/proj2021spr/

• Stanford CS229: https://cs229.stanford.edu/syllabus-fall2021.html
• Harvard CS181: https://harvard-ml-courses.github.io/cs181-web/schedule
• MIT 6.036 : https://canvas.mit.edu/courses/7509
• IIT Delhi COL341: https://www.cse.iitd.ernet.in/~rahulgarg/Teaching/COL341.htm
• UW CSE446: https://courses.cs.washington.edu/courses/cse446/22wi/
• UCB CS189: https://people.eecs.berkeley.edu/~jrs/189/

Legend:

• Linear Algebra Notes from Stanford - short
• 3blue1brown -beautiful animated explanations
• Linear Algebra Notes
• inear Algebra Review
• The Matrix Cookbook - It won't teach you linear algebra, but this free desktop reference on matrices may come in handy.
• Probability Notes from Stanford - short
• Review of probability from a course by David Blei at Princeton
• Andrew Moore's Probability tutorial slides (somewhat incomplete)
• Another probability review, from UCI
• https://www.cs.utoronto.ca/~fidler/teaching/2015/slides/CSC411/
• https://www.cs.cmu.edu/~epxing/Class/10701/lecture.html
• https://web.cs.ucla.edu/~sriram/courses/cs188.winter-2017/html/index.html
• https://people.eecs.berkeley.edu/~jrs/189/
• https://alex.smola.org/teaching/cmu2013-10-701/
• https://sli.ics.uci.edu/Classes/2015W-273a

We will benefit from other people’s efforts:

• Google Dataset Search
• Amazon’s AWS datasets
• Kaggle datasets
• Wikipedia’s list
• UC Irvine Machine Learning Repository
• Quora.com
• Reddit
• Dataportals.org
• Opendatamonitor.eu
• Quandl.com