A collection of Machine Learning algorithms implemented in Python from scratch.

Transforming an algorithm from math equations to code is a sometimes difficult but always fruitful journey every Data Scientist should take at least once. The goal of this project is to go down to the last detail of each algorithm, fill the gap between math and code, demystify ML models complexity and revisit their mathematical background.

• Supervised Learning

  • Multi-layer Neural Networks [demo] [code]
  • Classification Trees [demo] [code]
  • Adaptive Boosting [demo] [code]
  • Support Vector Machine [demo] [code]
  • Linear Regression [demo] [code]

• Unsupervised Learning

  • K-means [demo] [code]
  • DBSCAN [demo] [code]
  • Principal Component Analysis (PCA) [demo] [code]

• Reinforcement Learning

  • N-armed Bandits [demo] [code]
  • Dynamic Programming [demo] [code]
  • SARSA [demo] [code]
  • Q-Learning [demo] [code]

• numpy: Used in all implementations for vector/matrix operations and vectorized calculations
• cvxopt: Used in SVM for solving the quadratic programming problem
• scipy: Borrowed its KDTree implementation for fast nearest neighbours calculation

Each algorithm is accompanied with a notebook with mathematical background, application of the methodology on toy datasets and visualizations.

Fead-forward Neural Network
---------------------------
4 layers: [10, 10, 10, 1]

       Inputs
       \ | /
 o o o o o o o o o o 
       \ | /
 o o o o o o o o o o 
       \ | /
 o o o o o o o o o o 
          | 
          o 
       Outputs

{'expr': 'feature_2 <= 2.45',
 'no': {'expr': 'feature_3 <= 1.75',
        'no': {'expr': 'feature_2 <= 4.85',
               'no': {'label': 2.0},
               'yes': {'label': 2.0}},
        'yes': {'expr': 'feature_2 <= 4.95',
                'no': {'label': 2.0},
                'yes': {'label': 1.0}}},
 'yes': {'expr': 'feature_0 <= 4.35',
         'no': {'expr': 'feature_0 <= 4.45',
                'no': {'label': 0.0},
                'yes': {'label': 0.0}},
         'yes': {'label': 0.0}}}