MTH594 Advanced data mining: theory and applications

Lecture 1

Basic notations

Linear regression

Gradient descent

Normal equations

Lecture 2

Locally weighted regression

Linear regression: probabilistic interpretation

Logistic regression

Perceptron

Lecture 3

Newton's method

Exponential family

Generalized Linear Models (GLM)

Generative learning algorithms

Lecture 4

Gaussians

Gaussian discriminant analysis

Generative vs Discriminant comparison

Naive Bayes

Laplace Smoothing

Lecture 5

Event models

Neural networks

Lecture 6

Support vector machines: intuition

Primal/dual optimization problem and KKT

SVM dual

Kernels

Soft margin algorithm

SMO algorithm

Lecture 7

Generalized additive models (GAM)

Tree-based methods

Boosting

Boosting trees

Lecture 8

Bias / variance

Empirical risk minimization (ERM)

Union bound / Hoeffding inequality

Uniform convergence

VC dimension

Lecture 9

Model selection

Feature selection

Bayesian approach and regularization

Online learning

Advices for apply ML algorithms

Lecture 10

Clustering (k-means)

Mixture of Gaussians

Jensen's inequality

General EM algorithm

EM algorithm for the mixture of Gaussians

EM algorithm for the mixture of Naive Bayes

Lecture 11

Factor analysis

Principal component analysis

Latent semantic indexing

Independent component analysis