cbrs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Case-based Reasoning System\n",
    "\n",
    "<h5>Yefferson A. Marín Cantero</h5>\n",
    "Artificial intelligence course <br>\n",
    "Systems Engineering Program <br>\n",
    "Universidad Tecnológica de Bolívar <br>\n",
    "Cartagena de Indias, D.T. y C - Bolívar <br>\n",
    "1p-2020\n",
    "\n",
    "## Definitions\n",
    "A **Case-based reasoning** (CBR) is a paradigm of artificial intelligence and cognitive science that models the reasoning process as primarily memory based. Case-based reasoning systems solve new problems by retrieving stored ‘cases’ describing similar prior problem-solving episodes and adapting their solutions to fit new needs. \n",
    "\n",
    "Case-based reasoning has been [formalized](https://www.idi.ntnu.no/emner/tdt4171/papers/AamodtPlaza94.pdf) for purposes of computer reasoning as a four-step process:\n",
    "\n",
    "- **1. Retrieve** Given a target problem, retrieve from memory cases relevant to solving it. A case consists of a problem, its solution, and, typically, annotations about how the solution was derived. \n",
    "- **2. Reuse:** Map the solution from the previous case to the target problem. This may involve adapting the solution as needed to fit the new situation. \n",
    "- **3. Revise:** Having mapped the previous solution to the target situation, test the new solution in the real world (or a simulation) and, if necessary, revise. \n",
    "- **4. Retain:** After the solution has been successfully adapted to the target problem, store the resulting experience as a new case in memory. \n",
    "\n",
    "## Introduction\n",
    "This algorithm is done using the following requirements:\n",
    "\n",
    "* [Python 3.*](https://python.org)\n",
    "* [Pandas](https://pandas.pydata.org/)\n",
    "* [Numpy](https://numpy.org)\n",
    "* [Scipy spatial distance](https://docs.scipy.org/doc/scipy/reference/spatial.distance.html#module-scipy.spatial.distance)\n",
    "* [Matplotlib](https://matplotlib.org)\n",
    "* [Seaborn](https://docs.scipy.org/doc/scipy/reference/spatial.distance.html#module-scipy.spatial.distance)\n",
    "\n",
    "And its taking a **library cases** stored in [input/library.csv](input/library.csv) to get the Case-based reasoning from test **problem cases** stored in [input/cases.csv](input/cases.csv).\n",
    "\n",
    "The purpose is designing a system that fits the test cases into the base library cases in order to find the most appropriate solution. \n",
    "\n",
    "## Steps\n",
    "### 1. Retrieve\n",
    "#### Library and problem cases\n",
    "First it's required to retrieve the base cases or the library of relevant cases with their data (used to compute) and its solutions. Also we need to retrieve our test problem cases.\n",
    "\n",
    "### 2. Reuse\n",
    "#### One-hot encoding\n",
    "Machine learning algorithms cannot work with categorical data directly, this must be converted to numbers. This technique is called one-hot encoding and is a representation of categorical variables as binary vectors.\n",
    "\n",
    "We must transform our library and problem cases by firstly requiring that the categorical values be mapped to integer values, and then, representing each integer value as a binary vector that is all zero values except the index of the integer, which is marked with a 1.\n",
    "\n",
    "#### Mahalanobis distance\n",
    "Having mapped both (library and problem cases), we should define a similarity comparison method, the best match is the Mahalanobis distance which is an effective multivariate distance metric that measures the distance between a point (P) and a distribution (D). It is an extremely useful metric having, excellent applications in multivariate anomaly detection, classification on highly imbalanced datasets and one-class classification. \n",
    "\n",
    "Mahalanobis distance is widely used in cluster analysis and classification techniques, as a multi-dimensional generalization of the idea of measuring how many standard deviations away P is from the mean of D.\n",
    "\n",
    "$$ D(\\overrightarrow{u}, \\overrightarrow{v}) = \\sqrt{(\\overrightarrow{u}-\\overrightarrow{u})V^{-1}(\\overrightarrow{u}-\\overrightarrow{v})^T} $$\n",
    "\n",
    "Where $\\overrightarrow{u}$ and $\\overrightarrow{v}$ are arrays, and $V^{-1}$ The inverse of the covariance matrix.\n",
    "\n",
    "#### Covariance matrix\n",
    "A covariance matrix (also known as auto-covariance matrix, dispersion matrix, variance matrix, or variance–covariance matrix) is a square matrix giving the covariance between each pair of elements of a given random vector. In the matrix diagonal there are variances, i.e., the covariance of each element with itself. Intuitively, the covariance matrix generalizes the notion of variance to multiple dimensions.\n",
    "\n",
    "### 3. Revise\n",
    "After compare the shortest distances, we get the solution based on proximity calculated (similarity) with library cases.\n",
    "\n",
    "### 4. Retain \n",
    "We store them as a new case on the base cases library.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implementation\n",
    "Initially we import the needed libraries, as it follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from scipy.spatial import distance\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Library and problem cases \n",
    "The we get our **library cases** stored in [input/library.csv](input/library.csv) and the test **problem cases** stored in [input/cases.csv](input/cases.csv)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the input .csv library and problem cases\n",
    "# {pandas.DataFrame}\n",
    "library, cases = pd.read_csv('input/library.csv'), pd.read_csv('input/cases.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Outlook</th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Humidity</th>\n",
       "      <th>Windy</th>\n",
       "      <th>Play</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Hot</td>\n",
       "      <td>High</td>\n",
       "      <td>False</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Hot</td>\n",
       "      <td>High</td>\n",
       "      <td>True</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Overcast</td>\n",
       "      <td>Hot</td>\n",
       "      <td>High</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Mild</td>\n",
       "      <td>High</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Cool</td>\n",
       "      <td>Normal</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Cool</td>\n",
       "      <td>Normal</td>\n",
       "      <td>True</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Overcast</td>\n",
       "      <td>Cool</td>\n",
       "      <td>Normal</td>\n",
       "      <td>True</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Mild</td>\n",
       "      <td>High</td>\n",
       "      <td>False</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Cool</td>\n",
       "      <td>Normal</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Mild</td>\n",
       "      <td>Normal</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Mild</td>\n",
       "      <td>Normal</td>\n",
       "      <td>True</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Overcast</td>\n",
       "      <td>Mild</td>\n",
       "      <td>High</td>\n",
       "      <td>True</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Overcast</td>\n",
       "      <td>Hot</td>\n",
       "      <td>Normal</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Mild</td>\n",
       "      <td>High</td>\n",
       "      <td>True</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Outlook  Temperature  Humidity   Windy  Play\n",
       "0      Sunny          Hot      High   False    No\n",
       "1      Sunny          Hot      High    True    No\n",
       "2   Overcast          Hot      High   False   Yes\n",
       "3      Rainy         Mild      High   False   Yes\n",
       "4      Rainy         Cool    Normal   False   Yes\n",
       "5      Rainy         Cool    Normal    True    No\n",
       "6   Overcast         Cool    Normal    True   Yes\n",
       "7      Sunny         Mild      High   False    No\n",
       "8      Sunny         Cool    Normal   False   Yes\n",
       "9      Rainy         Mild    Normal   False   Yes\n",
       "10     Sunny         Mild    Normal    True   Yes\n",
       "11  Overcast         Mild      High    True   Yes\n",
       "12  Overcast          Hot    Normal   False   Yes\n",
       "13     Rainy         Mild      High    True    No"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Outlook</th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Humidity</th>\n",
       "      <th>Windy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Mild</td>\n",
       "      <td>Normal</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Cool</td>\n",
       "      <td>Normal</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Overcast</td>\n",
       "      <td>Cool</td>\n",
       "      <td>High</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Cool</td>\n",
       "      <td>High</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Hot</td>\n",
       "      <td>High</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Cool</td>\n",
       "      <td>High</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Outlook  Temperature  Humidity   Windy\n",
       "0     Sunny         Mild    Normal   False\n",
       "1     Rainy         Cool    Normal   False\n",
       "2  Overcast         Cool      High   False\n",
       "3     Sunny         Cool      High    True\n",
       "4     Rainy          Hot      High    True\n",
       "5     Rainy         Cool      High    True"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, we can verify which kind of data its represented:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Outlook         object\n",
       " Temperature    object\n",
       " Humidity       object\n",
       " Windy          object\n",
       " Play           object\n",
       "dtype: object"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "library.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Outlook         object\n",
       " Temperature    object\n",
       " Humidity       object\n",
       " Windy          object\n",
       "dtype: object"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cases.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Base & Initial one-hot encoding\n",
    "As we verified, our data is categorical, so we are going to convert them using one-hot encoding method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Select columns from library to use as base cases, except solutions\n",
    "base = library.iloc[:, range(library.shape[1] - 1)]      # Exclude last column\n",
    "\n",
    "# Initial One-hot encoding\n",
    "base = pd.get_dummies(base)\n",
    "problems = pd.get_dummies(cases)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our initial library cases (base) and case/problems to evaluate, with this technique will lock as it follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Outlook_Overcast</th>\n",
       "      <th>Outlook_Rainy</th>\n",
       "      <th>Outlook_Sunny</th>\n",
       "      <th>Temperature_ Cool</th>\n",
       "      <th>Temperature_ Hot</th>\n",
       "      <th>Temperature_ Mild</th>\n",
       "      <th>Humidity_ High</th>\n",
       "      <th>Humidity_ Normal</th>\n",
       "      <th>Windy_ False</th>\n",
       "      <th>Windy_ True</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Outlook_Overcast  Outlook_Rainy  Outlook_Sunny   Temperature_ Cool  \\\n",
       "0                  0              0              1                   0   \n",
       "1                  0              0              1                   0   \n",
       "2                  1              0              0                   0   \n",
       "3                  0              1              0                   0   \n",
       "4                  0              1              0                   1   \n",
       "5                  0              1              0                   1   \n",
       "6                  1              0              0                   1   \n",
       "7                  0              0              1                   0   \n",
       "8                  0              0              1                   1   \n",
       "9                  0              1              0                   0   \n",
       "10                 0              0              1                   0   \n",
       "11                 1              0              0                   0   \n",
       "12                 1              0              0                   0   \n",
       "13                 0              1              0                   0   \n",
       "\n",
       "     Temperature_ Hot   Temperature_ Mild   Humidity_ High   Humidity_ Normal  \\\n",
       "0                   1                   0                1                  0   \n",
       "1                   1                   0                1                  0   \n",
       "2                   1                   0                1                  0   \n",
       "3                   0                   1                1                  0   \n",
       "4                   0                   0                0                  1   \n",
       "5                   0                   0                0                  1   \n",
       "6                   0                   0                0                  1   \n",
       "7                   0                   1                1                  0   \n",
       "8                   0                   0                0                  1   \n",
       "9                   0                   1                0                  1   \n",
       "10                  0                   1                0                  1   \n",
       "11                  0                   1                1                  0   \n",
       "12                  1                   0                0                  1   \n",
       "13                  0                   1                1                  0   \n",
       "\n",
       "     Windy_ False   Windy_ True  \n",
       "0               1             0  \n",
       "1               0             1  \n",
       "2               1             0  \n",
       "3               1             0  \n",
       "4               1             0  \n",
       "5               0             1  \n",
       "6               0             1  \n",
       "7               1             0  \n",
       "8               1             0  \n",
       "9               1             0  \n",
       "10              0             1  \n",
       "11              0             1  \n",
       "12              1             0  \n",
       "13              0             1  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "base"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Outlook_Overcast</th>\n",
       "      <th>Outlook_Rainy</th>\n",
       "      <th>Outlook_Sunny</th>\n",
       "      <th>Temperature_ Cool</th>\n",
       "      <th>Temperature_ Hot</th>\n",
       "      <th>Temperature_ Mild</th>\n",
       "      <th>Humidity_ High</th>\n",
       "      <th>Humidity_ Normal</th>\n",
       "      <th>Windy_ False</th>\n",
       "      <th>Windy_ True</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Outlook_Overcast  Outlook_Rainy  Outlook_Sunny   Temperature_ Cool  \\\n",
       "0                 0              0              1                   0   \n",
       "1                 0              1              0                   1   \n",
       "2                 1              0              0                   1   \n",
       "3                 0              0              1                   1   \n",
       "4                 0              1              0                   0   \n",
       "5                 0              1              0                   1   \n",
       "\n",
       "    Temperature_ Hot   Temperature_ Mild   Humidity_ High   Humidity_ Normal  \\\n",
       "0                  0                   1                0                  1   \n",
       "1                  0                   0                0                  1   \n",
       "2                  0                   0                1                  0   \n",
       "3                  0                   0                1                  0   \n",
       "4                  1                   0                1                  0   \n",
       "5                  0                   0                1                  0   \n",
       "\n",
       "    Windy_ False   Windy_ True  \n",
       "0              1             0  \n",
       "1              1             0  \n",
       "2              1             0  \n",
       "3              0             1  \n",
       "4              0             1  \n",
       "5              0             1  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "problems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate\n",
    "Our main code can be divided in the following steps:\n",
    "\n",
    "1. Calculate *inverse covariance matrix* for the *base* cases.\n",
    "2. Get the *case* to evaluate.\n",
    "3. Calculate *mahalanobis distance* using *case*, *base* and *inverse covariance matrix*.\n",
    "4. *Minimum distances calculated* will be stored.\n",
    "5. *Minimum distance calculated index* will be used to solve the problem, by using the *index* solution in *base* cases.\n",
    "6. Append solution to the *library*, to use it in future cases, and store other relevant data (eg. *covariance heat maps*).\n",
    "7. If there are more cases, it evaluates, getting the new *base* (one-hot) encoded. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> For case/problem 0: ['Sunny' ' Mild' ' Normal' ' False'], solution is  Yes\n",
      "> For case/problem 1: ['Rainy' ' Cool' ' Normal' ' False'], solution is  Yes\n",
      "> For case/problem 2: ['Overcast' ' Cool' ' High' ' False'], solution is  Yes\n",
      "> For case/problem 3: ['Sunny' ' Cool' ' High' ' True'], solution is  Yes\n",
      "> For case/problem 4: ['Rainy' ' Hot' ' High' ' True'], solution is  No\n",
      "> For case/problem 5: ['Rainy' ' Cool' ' High' ' True'], solution is  No\n"
     ]
    }
   ],
   "source": [
    "# Move through all problem cases\n",
    "for i in range(problems.shape[0]):\n",
    "    # Get inverse covariance matrix for the base cases\n",
    "    covariance_matrix = base.cov()                                      # Covariance\n",
    "    inverse_covariance_matrix = np.linalg.pinv(covariance_matrix)       # Inverse\n",
    "\n",
    "    # Get case row to evaluate\n",
    "    case_row = problems.loc[i, :]\n",
    "\n",
    "    # Empty distances array to store mahalanobis distances obtained comparing each library cases\n",
    "    distances = np.zeros(base.shape[0])\n",
    "\n",
    "    # For each base cases rows\n",
    "    for j in range(base.shape[0]):\n",
    "        # Get base case row\n",
    "        base_row = base.loc[j, :]\n",
    "\n",
    "        # Calculate mahalanobis distance between case row and base cases, and store it\n",
    "        distances[j] = distance.mahalanobis(case_row, base_row, inverse_covariance_matrix)\n",
    "\n",
    "    # Returns the index (row) of the minimum value in distances calculated\n",
    "    min_distance_row = np.argmin(distances)\n",
    "\n",
    "    # Get solution based on index of found minimum distance, and append it to main library\n",
    "    # From cases, append library 'similar' solution\n",
    "    case = np.append(cases.iloc[i, :], library.iloc[min_distance_row, -1])\n",
    "\n",
    "    # Print\n",
    "    print(f'> For case/problem {i}: {cases.iloc[i, :].to_numpy()}, solution is {case[-1]}')\n",
    "\n",
    "    # Store\n",
    "    # Get as operable pandas Series\n",
    "    case = pd.Series(case, index = library.columns)         # Case with Solution\n",
    "    library = library.append(case, ignore_index = True)     # Append to library\n",
    "\n",
    "    # Save 'covariance heat map (biased)' output as file\n",
    "    sn.heatmap(np.cov(base, bias = True), annot = True, fmt = 'g')\n",
    "    plt.gcf().set_size_inches(12, 6)\n",
    "    plt.title(f'Covariance Heat map #{i} \\n Library cases stored {j} - Base to solve problem {i}')\n",
    "    plt.savefig(f'output/covariance_heat_map_{i}.png', bbox_inches='tight')\n",
    "    plt.close()\n",
    " \n",
    "    # Reuse\n",
    "    base = library.iloc[:, range(library.shape[1] - 1)]     # Exclude last column (solution)\n",
    "    base = pd.get_dummies(base)                             # Get new one-hot encoded base\n",
    "\n",
    "# Save 'library' output as file\n",
    "library.to_csv('output/library.csv', index = False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Results\n",
    "Finally, we can output our library plus cases/problems solved."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Outlook</th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Humidity</th>\n",
       "      <th>Windy</th>\n",
       "      <th>Play</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Hot</td>\n",
       "      <td>High</td>\n",
       "      <td>False</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Hot</td>\n",
       "      <td>High</td>\n",
       "      <td>True</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Overcast</td>\n",
       "      <td>Hot</td>\n",
       "      <td>High</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Mild</td>\n",
       "      <td>High</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Cool</td>\n",
       "      <td>Normal</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Cool</td>\n",
       "      <td>Normal</td>\n",
       "      <td>True</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Overcast</td>\n",
       "      <td>Cool</td>\n",
       "      <td>Normal</td>\n",
       "      <td>True</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Mild</td>\n",
       "      <td>High</td>\n",
       "      <td>False</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Cool</td>\n",
       "      <td>Normal</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Mild</td>\n",
       "      <td>Normal</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Mild</td>\n",
       "      <td>Normal</td>\n",
       "      <td>True</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Overcast</td>\n",
       "      <td>Mild</td>\n",
       "      <td>High</td>\n",
       "      <td>True</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Overcast</td>\n",
       "      <td>Hot</td>\n",
       "      <td>Normal</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Mild</td>\n",
       "      <td>High</td>\n",
       "      <td>True</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Mild</td>\n",
       "      <td>Normal</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Cool</td>\n",
       "      <td>Normal</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Overcast</td>\n",
       "      <td>Cool</td>\n",
       "      <td>High</td>\n",
       "      <td>False</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Sunny</td>\n",
       "      <td>Cool</td>\n",
       "      <td>High</td>\n",
       "      <td>True</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Hot</td>\n",
       "      <td>High</td>\n",
       "      <td>True</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Rainy</td>\n",
       "      <td>Cool</td>\n",
       "      <td>High</td>\n",
       "      <td>True</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Outlook  Temperature  Humidity   Windy  Play\n",
       "0      Sunny          Hot      High   False    No\n",
       "1      Sunny          Hot      High    True    No\n",
       "2   Overcast          Hot      High   False   Yes\n",
       "3      Rainy         Mild      High   False   Yes\n",
       "4      Rainy         Cool    Normal   False   Yes\n",
       "5      Rainy         Cool    Normal    True    No\n",
       "6   Overcast         Cool    Normal    True   Yes\n",
       "7      Sunny         Mild      High   False    No\n",
       "8      Sunny         Cool    Normal   False   Yes\n",
       "9      Rainy         Mild    Normal   False   Yes\n",
       "10     Sunny         Mild    Normal    True   Yes\n",
       "11  Overcast         Mild      High    True   Yes\n",
       "12  Overcast          Hot    Normal   False   Yes\n",
       "13     Rainy         Mild      High    True    No\n",
       "14     Sunny         Mild    Normal   False   Yes\n",
       "15     Rainy         Cool    Normal   False   Yes\n",
       "16  Overcast         Cool      High   False   Yes\n",
       "17     Sunny         Cool      High    True   Yes\n",
       "18     Rainy          Hot      High    True    No\n",
       "19     Rainy         Cool      High    True    No"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "library"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generated files  \n",
    "#### Library\n",
    "Initial library plus cases/problems solved, will be available at [output/library.csv](output/library.csv). \n",
    "\n",
    "#### Heat maps\n",
    "An image for each Covariance Heat map, using library cases stored (base) at each iteration to solve a specific problem, will be available at [output/heatmap_x.png](output)\n",
    "\n",
    "![heatmap_0](output/covariance_heat_map_0.png)\n",
    "![heatmap_1](output/covariance_heat_map_1.png)\n",
    "![heatmap_2](output/covariance_heat_map_2.png)\n",
    "![heatmap_3](output/covariance_heat_map_3.png)\n",
    "![heatmap_4](output/covariance_heat_map_4.png)\n",
    "![heatmap_5](output/covariance_heat_map_5.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Source code\n",
    "#### Repository\n",
    "All code has been deployed at *Github* and its available at [yammadev/cbrs](https://github.com/yammadev/cbrs). \n",
    "\n",
    "#### Testing\n",
    "Executable using `py cbrs.py` will output (and also generate files):\n",
    "\n",
    "```sh\n",
    "> Initial Library\n",
    "\n",
    "     Outlook  Temperature  Humidity   Windy  Play\n",
    "0      Sunny          Hot      High   False    No\n",
    "1      Sunny          Hot      High    True    No\n",
    "2   Overcast          Hot      High   False   Yes\n",
    "3      Rainy         Mild      High   False   Yes\n",
    "4      Rainy         Cool    Normal   False   Yes\n",
    "5      Rainy         Cool    Normal    True    No\n",
    "6   Overcast         Cool    Normal    True   Yes\n",
    "7      Sunny         Mild      High   False    No\n",
    "8      Sunny         Cool    Normal   False   Yes\n",
    "9      Rainy         Mild    Normal   False   Yes\n",
    "10     Sunny         Mild    Normal    True   Yes\n",
    "11  Overcast         Mild      High    True   Yes\n",
    "12  Overcast          Hot    Normal   False   Yes\n",
    "13     Rainy         Mild      High    True    No\n",
    "\n",
    "> Calculating\n",
    "\n",
    "> For case/problem 0: ['Sunny' ' Mild' ' Normal' ' False'], solution is  Yes\n",
    "> For case/problem 1: ['Rainy' ' Cool' ' Normal' ' False'], solution is  Yes\n",
    "> For case/problem 2: ['Overcast' ' Cool' ' High' ' False'], solution is  Yes \n",
    "> For case/problem 3: ['Sunny' ' Cool' ' High' ' True'], solution is  Yes\n",
    "> For case/problem 4: ['Rainy' ' Hot' ' High' ' True'], solution is  No\n",
    "> For case/problem 5: ['Rainy' ' Cool' ' High' ' True'], solution is  No\n",
    "\n",
    "> Output library\n",
    "\n",
    "     Outlook  Temperature  Humidity   Windy  Play\n",
    "0      Sunny          Hot      High   False    No\n",
    "1      Sunny          Hot      High    True    No\n",
    "2   Overcast          Hot      High   False   Yes\n",
    "3      Rainy         Mild      High   False   Yes\n",
    "4      Rainy         Cool    Normal   False   Yes\n",
    "5      Rainy         Cool    Normal    True    No\n",
    "6   Overcast         Cool    Normal    True   Yes\n",
    "7      Sunny         Mild      High   False    No\n",
    "8      Sunny         Cool    Normal   False   Yes\n",
    "9      Rainy         Mild    Normal   False   Yes\n",
    "10     Sunny         Mild    Normal    True   Yes\n",
    "11  Overcast         Mild      High    True   Yes\n",
    "12  Overcast          Hot    Normal   False   Yes\n",
    "13     Rainy         Mild      High    True    No\n",
    "14     Sunny         Mild    Normal   False   Yes\n",
    "15     Rainy         Cool    Normal   False   Yes\n",
    "16  Overcast         Cool      High   False   Yes\n",
    "17     Sunny         Cool      High    True   Yes\n",
    "18     Rainy          Hot      High    True    No\n",
    "19     Rainy         Cool      High    True    No\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}