Netflix RS - Hilary (3 of 3).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "02990446",
   "metadata": {},
   "source": [
    "# Capstone (3/3) - Netflix Recommender (TensorFlow)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4158d8a",
   "metadata": {},
   "source": [
    "## 1. Import Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "67c53e24",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np \n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from keras.models import Model\n",
    "from keras.layers import Input, Reshape, Dot, Concatenate, Dense, Dropout\n",
    "from keras.layers.embeddings import Embedding\n",
    "from keras.optimizers import Adam\n",
    "from keras.regularizers import l2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53668baf",
   "metadata": {},
   "source": [
    "## 2. Load File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "71545ece",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 5116704 entries, 0 to 5116703\n",
      "Data columns (total 6 columns):\n",
      " #   Column        Dtype \n",
      "---  ------        ----- \n",
      " 0   movie_id      int64 \n",
      " 1   user_id       int64 \n",
      " 2   rating        int64 \n",
      " 3   date          object\n",
      " 4   release_year  int64 \n",
      " 5   movie_title   object\n",
      "dtypes: int64(4), object(2)\n",
      "memory usage: 234.2+ MB\n"
     ]
    }
   ],
   "source": [
    "# load file\n",
    "df =  pd.read_csv('2005-d4.csv')\n",
    "movie_df = pd.read_csv('2005-d41kMT.csv')\n",
    "\n",
    "# Using info to check the shape, column names and type of df\n",
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0cb64220",
   "metadata": {},
   "source": [
    "## 3. Simple EDA and Configurations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "367d92e2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movie_id</th>\n",
       "      <th>user_id</th>\n",
       "      <th>rating</th>\n",
       "      <th>date</th>\n",
       "      <th>release_year</th>\n",
       "      <th>movie_title</th>\n",
       "      <th>user</th>\n",
       "      <th>movie</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>6105</td>\n",
       "      <td>2606550</td>\n",
       "      <td>2</td>\n",
       "      <td>2005-08-21</td>\n",
       "      <td>2000</td>\n",
       "      <td>Paranoid</td>\n",
       "      <td>342296</td>\n",
       "      <td>6006</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1865</td>\n",
       "      <td>482356</td>\n",
       "      <td>4</td>\n",
       "      <td>2005-11-09</td>\n",
       "      <td>2004</td>\n",
       "      <td>Eternal Sunshine of the Spotless Mind</td>\n",
       "      <td>63573</td>\n",
       "      <td>1833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4996</td>\n",
       "      <td>49106</td>\n",
       "      <td>4</td>\n",
       "      <td>2005-06-18</td>\n",
       "      <td>2000</td>\n",
       "      <td>Gone in 60 Seconds</td>\n",
       "      <td>6417</td>\n",
       "      <td>4913</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1307</td>\n",
       "      <td>1090820</td>\n",
       "      <td>3</td>\n",
       "      <td>2005-07-17</td>\n",
       "      <td>2003</td>\n",
       "      <td>S.W.A.T.</td>\n",
       "      <td>143469</td>\n",
       "      <td>1285</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>9628</td>\n",
       "      <td>502622</td>\n",
       "      <td>3</td>\n",
       "      <td>2005-09-02</td>\n",
       "      <td>1983</td>\n",
       "      <td>Star Wars: Episode VI: Return of the Jedi</td>\n",
       "      <td>66280</td>\n",
       "      <td>9471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5116699</th>\n",
       "      <td>1865</td>\n",
       "      <td>1500751</td>\n",
       "      <td>5</td>\n",
       "      <td>2005-09-25</td>\n",
       "      <td>2004</td>\n",
       "      <td>Eternal Sunshine of the Spotless Mind</td>\n",
       "      <td>197136</td>\n",
       "      <td>1833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5116700</th>\n",
       "      <td>7193</td>\n",
       "      <td>1646303</td>\n",
       "      <td>5</td>\n",
       "      <td>2005-10-14</td>\n",
       "      <td>1987</td>\n",
       "      <td>The Princess Bride</td>\n",
       "      <td>216403</td>\n",
       "      <td>7073</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5116701</th>\n",
       "      <td>1073</td>\n",
       "      <td>129941</td>\n",
       "      <td>4</td>\n",
       "      <td>2005-06-27</td>\n",
       "      <td>2005</td>\n",
       "      <td>Coach Carter</td>\n",
       "      <td>17002</td>\n",
       "      <td>1053</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5116702</th>\n",
       "      <td>2782</td>\n",
       "      <td>104920</td>\n",
       "      <td>4</td>\n",
       "      <td>2005-11-17</td>\n",
       "      <td>1995</td>\n",
       "      <td>Braveheart</td>\n",
       "      <td>13663</td>\n",
       "      <td>2738</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5116703</th>\n",
       "      <td>17508</td>\n",
       "      <td>2121864</td>\n",
       "      <td>5</td>\n",
       "      <td>2005-08-15</td>\n",
       "      <td>2000</td>\n",
       "      <td>Requiem for a Dream</td>\n",
       "      <td>278577</td>\n",
       "      <td>17253</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5116704 rows × 8 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         movie_id  user_id  rating        date  release_year  \\\n",
       "0            6105  2606550       2  2005-08-21          2000   \n",
       "1            1865   482356       4  2005-11-09          2004   \n",
       "2            4996    49106       4  2005-06-18          2000   \n",
       "3            1307  1090820       3  2005-07-17          2003   \n",
       "4            9628   502622       3  2005-09-02          1983   \n",
       "...           ...      ...     ...         ...           ...   \n",
       "5116699      1865  1500751       5  2005-09-25          2004   \n",
       "5116700      7193  1646303       5  2005-10-14          1987   \n",
       "5116701      1073   129941       4  2005-06-27          2005   \n",
       "5116702      2782   104920       4  2005-11-17          1995   \n",
       "5116703     17508  2121864       5  2005-08-15          2000   \n",
       "\n",
       "                                       movie_title    user  movie  \n",
       "0                                         Paranoid  342296   6006  \n",
       "1            Eternal Sunshine of the Spotless Mind   63573   1833  \n",
       "2                               Gone in 60 Seconds    6417   4913  \n",
       "3                                         S.W.A.T.  143469   1285  \n",
       "4        Star Wars: Episode VI: Return of the Jedi   66280   9471  \n",
       "...                                            ...     ...    ...  \n",
       "5116699      Eternal Sunshine of the Spotless Mind  197136   1833  \n",
       "5116700                         The Princess Bride  216403   7073  \n",
       "5116701                               Coach Carter   17002   1053  \n",
       "5116702                                 Braveheart   13663   2738  \n",
       "5116703                        Requiem for a Dream  278577  17253  \n",
       "\n",
       "[5116704 rows x 8 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Copy df as a new df\n",
    "new_df = df.copy(deep=True)\n",
    "\n",
    "# use label encoder for movie id and user id, such that they will be in running sequence\n",
    "le = LabelEncoder()\n",
    "new_df['user'] = le.fit_transform(new_df['user_id'].values)\n",
    "new_df['movie'] = le.fit_transform(new_df['movie_id'].values)\n",
    "\n",
    "n_users = new_df['user'].nunique()\n",
    "n_movies = new_df['movie'].nunique()\n",
    "\n",
    "new_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70993f9d",
   "metadata": {},
   "source": [
    "## 4. Split Data into Train, Test and Validation Sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3e68219e",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Set the X and y variables\n",
    "X = new_df[['user', 'movie']].values\n",
    "y = new_df['rating'].values\n",
    "\n",
    "# Split train and test data (for test model performance at last)\n",
    "X_training, X_test, y_training, y_test = train_test_split(X, y, test_size=0.1)\n",
    "\n",
    "# Split train and validation data (to monitor model performance in training)\n",
    "X_train, X_val, y_train, y_val = train_test_split(X_training, y_training, test_size=0.1)\n",
    "\n",
    "# Set the embedding dimension d of Matrix factorization\n",
    "e_dimension = 50\n",
    "\n",
    "X_train_array = [X_train[:, 0], X_train[:, 1]]\n",
    "X_val_array = [X_val[:, 0], X_val[:, 1]]\n",
    "X_test_array = [X_test[:, 0], X_test[:, 1]]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a27e54c1",
   "metadata": {},
   "source": [
    "## 5. Create the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "7123dada",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"functional_1\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "input_1 (InputLayer)            [(None, 1)]          0                                            \n",
      "__________________________________________________________________________________________________\n",
      "input_2 (InputLayer)            [(None, 1)]          0                                            \n",
      "__________________________________________________________________________________________________\n",
      "embedding (Embedding)           (None, 1, 50)        17398900    input_1[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "embedding_1 (Embedding)         (None, 1, 50)        875600      input_2[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "reshape (Reshape)               (None, 50)           0           embedding[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "reshape_1 (Reshape)             (None, 50)           0           embedding_1[0][0]                \n",
      "__________________________________________________________________________________________________\n",
      "dot (Dot)                       (None, 1)            0           reshape[0][0]                    \n",
      "                                                                 reshape_1[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "dense (Dense)                   (None, 128)          256         dot[0][0]                        \n",
      "__________________________________________________________________________________________________\n",
      "dropout (Dropout)               (None, 128)          0           dense[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "dense_1 (Dense)                 (None, 1)            129         dropout[0][0]                    \n",
      "==================================================================================================\n",
      "Total params: 18,274,885\n",
      "Trainable params: 18,274,885\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n",
      "Epoch 1/20\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda\\envs\\tf\\lib\\site-packages\\tensorflow\\python\\framework\\indexed_slices.py:431: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "64759/64759 [==============================] - ETA: 0s - loss: 1.1540 - root_mean_squared_error: 1.0636WARNING:tensorflow:From C:\\Anaconda\\envs\\tf\\lib\\site-packages\\tensorflow\\python\\training\\tracking\\tracking.py:111: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "This property should not be used in TensorFlow 2.0, as updates are applied automatically.\n",
      "WARNING:tensorflow:From C:\\Anaconda\\envs\\tf\\lib\\site-packages\\tensorflow\\python\\training\\tracking\\tracking.py:111: Layer.updates (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "This property should not be used in TensorFlow 2.0, as updates are applied automatically.\n",
      "INFO:tensorflow:Assets written to: Model_1\\assets\n",
      "64759/64759 [==============================] - 12342s 191ms/step - loss: 1.1540 - root_mean_squared_error: 1.0636 - val_loss: 1.0671 - val_root_mean_squared_error: 1.0166\n",
      "Epoch 2/20\n",
      "64759/64759 [==============================] - ETA: 0s - loss: 1.0246 - root_mean_squared_error: 0.9925INFO:tensorflow:Assets written to: Model_1\\assets\n",
      "64759/64759 [==============================] - 11941s 184ms/step - loss: 1.0246 - root_mean_squared_error: 0.9925 - val_loss: 1.0628 - val_root_mean_squared_error: 1.0105\n",
      "Epoch 3/20\n",
      "64759/64759 [==============================] - 11084s 171ms/step - loss: 0.9919 - root_mean_squared_error: 0.9728 - val_loss: 1.0660 - val_root_mean_squared_error: 1.0098\n",
      "INFO:tensorflow:Assets written to: Model_1\\assets\n"
     ]
    }
   ],
   "source": [
    "# Build user embedding matrix\n",
    "user = Input(shape=(1,))\n",
    "u = Embedding(n_users, e_dimension, embeddings_initializer='he_normal',\n",
    "              embeddings_regularizer=l2(1e-6))(user)\n",
    "u = Reshape((e_dimension,))(u)\n",
    "\n",
    "# Build movie embedding matrix\n",
    "movie = Input(shape=(1,))\n",
    "m = Embedding(n_movies, e_dimension, embeddings_initializer='he_normal',\n",
    "              embeddings_regularizer=l2(1e-6))(movie)\n",
    "m = Reshape((e_dimension,))(m)\n",
    "\n",
    "\n",
    "x = Dot(axes=1)([u, m])\n",
    "\n",
    "\n",
    "# Build last deep learning layers \n",
    "x = Dense(128, activation='relu')(x)\n",
    "x = Dropout(0.2)(x)\n",
    "x = Dense(1)(x)\n",
    "\n",
    "model = Model(inputs=[user, movie], outputs=x)\n",
    "model.compile(loss='mean_squared_error', optimizer=Adam(lr=0.001), metrics=[tf.keras.metrics.RootMeanSquaredError()])\n",
    "\n",
    "# Set up for early stop if the validation loss stop improving for more than 1 epoch\n",
    "callbacks_list = [keras.callbacks.EarlyStopping(monitor='val_loss', patience=1,),\n",
    "                  # Saves the weights after every epoch\n",
    "                  keras.callbacks.ModelCheckpoint(filepath='Model_1', monitor='val_loss', save_best_only=True,)]\n",
    "\n",
    "# Print model info summary\n",
    "model.summary()  \n",
    "\n",
    "history = model.fit(x=X_train_array, y=y_train, batch_size=64, epochs=20,\n",
    "                    verbose=1, callbacks=callbacks_list, validation_data=(X_val_array, y_val))\n",
    "\n",
    "# Save the model\n",
    "model.save(\"Model_1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc747396",
   "metadata": {},
   "source": [
    "### (I). Visualise the Training and Validation Loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "46288473",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmWElEQVR4nO3de5QU9Z338fcHGETkoghGAyq4UYkX5DKISjRo8mxEjagxiSyPiCYixo1R1xgTk+Cadfc5G3aPy5OoS7wQEyL65MLRiPFO0KiJaAiC4gYN6BgMI4RbUAT8Pn9UDTZD9UzP0NU9zHxe5/Tpqt+vqvrbNTX97fr9qn+liMDMzKyxTtUOwMzM2iYnCDMzy+QEYWZmmZwgzMwskxOEmZllcoIwM7NMThBWEZIelHRBuZetJknLJX0yh+2GpI+k07dK+lYpy7bidSZIeri1cTax3TGS6sq9Xau8LtUOwNouSRsLZrsDm4Ft6fwlETGr1G1FxNg8lm3vImJKObYjaSDwJ6AmIram254FlPw3tI7HCcKKiogeDdOSlgNfjIhHGy8nqUvDh46ZtR9uYrIWa2hCkPQ1SW8Bd0raR9IvJdVL+ms6PaBgnXmSvphOT5L0lKRp6bJ/kjS2lcsOkjRf0gZJj0r6vqQfF4m7lBi/I+k36fYeltS3oP58SSskrZZ0XRP7Z5SktyR1Lig7W9KidPpYSc9IWitppaTvSepaZFszJf1LwfxX03X+LOmiRsueLun3ktZLekPS9QXV89PntZI2Sjq+Yd8WrH+CpOckrUufTyh13zRF0kfT9ddKWiLpzIK60yS9lG7zTUlXp+V907/PWklrJD0pyZ9XFeYdbq21P9AHOBiYTHIs3ZnOHwS8A3yvifVHAa8AfYF/B26XpFYs+xPgd8C+wPXA+U28Zikx/gNwIbAf0BVo+MA6Argl3f6H09cbQIaI+C3wN+CURtv9STq9DbgyfT/HA58AvtRE3KQxnJrG87+AQ4HG/R9/AyYCewOnA5dKOiutOyl93jsiekTEM4223Qd4AJievrf/BB6QtG+j97DTvmkm5hrgfuDhdL0vA7MkHZ4ucjtJc2VP4Cjg8bT8n4A6oB/wIeAbgMcFqjAnCGut94GpEbE5It6JiNUR8bOI2BQRG4AbgY83sf6KiPhBRGwDfggcQPJBUPKykg4CRgLfjoj3IuIp4L5iL1hijHdGxP9ExDvAvcDQtPxc4JcRMT8iNgPfSvdBMXcD4wEk9QROS8uIiOcj4tmI2BoRy4H/zogjy+fS+BZHxN9IEmLh+5sXES9GxPsRsSh9vVK2C0lC+WNE/CiN625gKfDpgmWK7ZumHAf0AP5P+jd6HPgl6b4BtgBHSOoVEX+NiBcKyg8ADo6ILRHxZHjguIpzgrDWqo+IdxtmJHWX9N9pE8x6kiaNvQubWRp5q2EiIjalkz1auOyHgTUFZQBvFAu4xBjfKpjeVBDThwu3nX5Ary72WiRnC+dI2gM4B3ghIlakcRyWNp+8lcbxryRnE83ZIQZgRaP3N0rSE2kT2jpgSonbbdj2ikZlK4D+BfPF9k2zMUdEYTIt3O5nSJLnCkm/lnR8Wv5dYBnwsKTXJF1b2tuwcnKCsNZq/G3un4DDgVER0YsPmjSKNRuVw0qgj6TuBWUHNrH8rsS4snDb6WvuW2zhiHiJ5INwLDs2L0HSVLUUODSN4xutiYGkmazQT0jOoA6MiN7ArQXbbe7b959Jmt4KHQS8WUJczW33wEb9B9u3GxHPRcQ4kuanOSRnJkTEhoj4p4g4BDgTuErSJ3YxFmshJwgrl54kbfpr0/bsqXm/YPqNfAFwvaSu6bfPTzexyq7E+FPgDEkfSzuUb6D5/5+fAF8hSUT/r1Ec64GNkgYDl5YYw73AJElHpAmqcfw9Sc6o3pV0LElialBP0iR2SJFtzwUOk/QPkrpI+jxwBElz0K74LcnZxjWSaiSNIfkbzU7/ZhMk9Y6ILST75H0ASWdI+kja17SOpN+mqSY9y4EThJXLTcCewNvAs8CvKvS6E0g6elcD/wLcQ/J7jSw30coYI2IJcBnJh/5K4K8knahNaegDeDwi3i4ov5rkw3sD8IM05lJieDB9D4+TNL883miRLwE3SNoAfJv023i67iaSPpffpFcGHddo26uBM0jOslYD1wBnNIq7xSLiPZKEMJZkv98MTIyIpeki5wPL06a2KSR/T0g64R8FNgLPADdHxBO7Eou1nNzvY+2JpHuApRGR+xmMWXvnMwjbrUkaKenvJHVKLwMdR9KWbWa7yL+ktt3d/sDPSTqM64BLI+L31Q3JrH1wE5OZmWVyE5OZmWXKrYlJ0h0kV0WsioijMuoHkwx7MBy4LiKmFdTtDdxG8tP7AC5qPDRAlr59+8bAgQPLEr+ZWUfw/PPPvx0R/bLq8uyDmEkyzs1dRerXAJcDZ2XU/Rfwq4g4N73mvHvGMjsZOHAgCxYsaHmkZmYdlKTGv6DfLrcmpoiYT5IEitWviojnSMZc2U5Sb5IfFt2eLvdeRKzNK04zM8vWFvsgBpH86vPOdOji2yTtVWxhSZMlLZC0oL6+vnJRmpm1c20xQXQh6Ze4JSKGkQxhXHSgroiYERG1EVHbr19mM5qZmbVCW/wdRB1Ql46pD8kYOB7J0awN2rJlC3V1dbz77rvNL2xV1a1bNwYMGEBNTU3J67S5BBERb6V3wzo8Il4huZnKS9WOy8x2VldXR8+ePRk4cCDF7/dk1RYRrF69mrq6OgYNGlTyerk1MUm6m2SQrcOV3J7yC5KmSJqS1u8vqQ64CvhmukyvdPWGu04tIrkpyb/mFSezZsHAgdCpU/I8y/dwNyvVu+++y7777uvk0MZJYt99923xmV5uZxARMb6Z+rcofsvGhUBtDmHtaNYsmDwZNqX3m1mxIpkHmDCh+Hpmtp2Tw+6hNX+ntthJXTnXXfdBcmiwaVNSbmbWwXXsBPH66y0rN7M2ZfXq1QwdOpShQ4ey//77079//+3z7733XpPrLliwgMsvv7zZ1zjhhBPKEuu8efM444wzyrKtSunYCeKgxndsbKbczHZNmfv89t13XxYuXMjChQuZMmUKV1555fb5rl27snXr1qLr1tbWMn369GZf4+mnn96lGHdnHTtB3HgjdG80ikf37km5mZVXQ5/fihUQ8UGfX5kvDJk0aRJTpkxh1KhRXHPNNfzud7/j+OOPZ9iwYZxwwgm88sorwI7f6K+//nouuugixowZwyGHHLJD4ujRo8f25ceMGcO5557L4MGDmTBhAg2jYc+dO5fBgwczYsQILr/88mbPFNasWcNZZ53FkCFDOO6441i0aBEAv/71r7efAQ0bNowNGzawcuVKTjrpJIYOHcpRRx3Fk08+Wdb91ZQ2d5lrRTV0RF93XdKsdNBBSXJwB7VZ+TXV51fm/7m6ujqefvppOnfuzPr163nyySfp0qULjz76KN/4xjf42c9+ttM6S5cu5YknnmDDhg0cfvjhXHrppTv9ZuD3v/89S5Ys4cMf/jCjR4/mN7/5DbW1tVxyySXMnz+fQYMGMX58k9fnADB16lSGDRvGnDlzePzxx5k4cSILFy5k2rRpfP/732f06NFs3LiRbt26MWPGDD71qU9x3XXXsW3bNjY13oc56tgJApID0wnBLH8V7PP77Gc/S+fOnQFYt24dF1xwAX/84x+RxJYtWzLXOf3009ljjz3YY4892G+//fjLX/7CgAE7Xmh57LHHbi8bOnQoy5cvp0ePHhxyyCHbf18wfvx4ZsyY0WR8Tz311PYkdcopp7B69WrWr1/P6NGjueqqq5gwYQLnnHMOAwYMYOTIkVx00UVs2bKFs846i6FDh+7KrmmRjt3EZGaVU8E+v732+mD4tm9961ucfPLJLF68mPvvv7/obwH22GOP7dOdO3fO7L8oZZldce2113LbbbfxzjvvMHr0aJYuXcpJJ53E/Pnz6d+/P5MmTeKuu4oNkF1+ThBmVhlV6vNbt24d/fv3B2DmzJll3/7hhx/Oa6+9xvLlywG45557ml3nxBNPZFba9zJv3jz69u1Lr169ePXVVzn66KP52te+xsiRI1m6dCkrVqzgQx/6EBdffDFf/OIXeeGFF8r+HopxgjCzypgwAWbMgIMPBil5njEj9ybea665hq9//esMGzas7N/4Afbcc09uvvlmTj31VEaMGEHPnj3p3bt3k+tcf/31PP/88wwZMoRrr72WH/7whwDcdNNNHHXUUQwZMoSamhrGjh3LvHnzOOaYYxg2bBj33HMPX/nKV8r+HoppV/ekrq2tDd8wyKxyXn75ZT760Y9WO4yq27hxIz169CAiuOyyyzj00EO58sorqx3WTrL+XpKej4jMkSt8BmFmtot+8IMfMHToUI488kjWrVvHJZdcUu2QysJXMZmZ7aIrr7yyTZ4x7CqfQZiZWSYnCDMzy+QEYWZmmfK8YdAdklZJWlykfrCkZyRtlnR1Rn1nSb+X9Mu8YjQzs+LyPIOYCZzaRP0a4HJgWpH6rwAvlzkmM2tHTj75ZB566KEdym666SYuvfTSouuMGTOGhsvhTzvtNNauXbvTMtdffz3TphX7aErMmTOHl1764G7I3/72t3n00UdbEH22tjQseG4JIiLmkySBYvWrIuI5YKeBUSQNAE4HbssrPjPb/Y0fP57Zs2fvUDZ79uySBsyDZBTWvffeu1Wv3ThB3HDDDXzyk59s1bbaqrbaB3ETcA3wfnMLSposaYGkBfX19bkHZmZtx7nnnssDDzyw/eZAy5cv589//jMnnngil156KbW1tRx55JFMnTo1c/2BAwfy9ttvA3DjjTdy2GGH8bGPfWz7kOCQ/MZh5MiRHHPMMXzmM59h06ZNPP3009x333189atfZejQobz66qtMmjSJn/70pwA89thjDBs2jKOPPpqLLrqIzZs3b3+9qVOnMnz4cI4++miWLl3a5Pur9rDgbe53EJLOAFZFxPOSxjS3fETMAGZA8kvqfKMzs2KuuAIWLizvNocOhZtuKl7fp08fjj32WB588EHGjRvH7Nmz+dznPockbrzxRvr06cO2bdv4xCc+waJFixgyZEjmdp5//nlmz57NwoUL2bp1K8OHD2fEiBEAnHPOOVx88cUAfPOb3+T222/ny1/+MmeeeSZnnHEG55577g7bevfdd5k0aRKPPfYYhx12GBMnTuSWW27hiiuuAKBv37688MIL3HzzzUybNo3bbiveUFLtYcHb4hnEaOBMScuB2cApkn5c3ZDMrK0qbGYqbF669957GT58OMOGDWPJkiU7NAc19uSTT3L22WfTvXt3evXqxZlnnrm9bvHixZx44okcffTRzJo1iyVLljQZzyuvvMKgQYM47LDDALjggguYP3/+9vpzzjkHgBEjRmwf4K+Yp556ivPPPx/IHhZ8+vTprF27li5dujBy5EjuvPNOrr/+el588UV69uzZ5LZL0ebOICLi68DXAdIziKsj4n9XMyYza15T3/TzNG7cOK688kpeeOEFNm3axIgRI/jTn/7EtGnTeO6559hnn32YNGlS0WG+mzNp0iTmzJnDMcccw8yZM5k3b94uxdswZPiuDBd+7bXXcvrppzN37lxGjx7NQw89tH1Y8AceeIBJkyZx1VVXMXHixF2KNc/LXO8GngEOl1Qn6QuSpkiaktbvL6kOuAr4ZrpMr7ziMbP2qUePHpx88slcdNFF288e1q9fz1577UXv3r35y1/+woMPPtjkNk466STmzJnDO++8w4YNG7j//vu3123YsIEDDjiALVu2bB+iG6Bnz55s2LBhp20dfvjhLF++nGXLlgHwox/9iI9//OOtem/VHhY8tzOIiGjyMoKIeAsY0Mwy84B55YvKzNqj8ePHc/bZZ29vamoYHnvw4MEceOCBjB49usn1hw8fzuc//3mOOeYY9ttvP0aOHLm97jvf+Q6jRo2iX79+jBo1antSOO+887j44ouZPn369s5pgG7dunHnnXfy2c9+lq1btzJy5EimTJnSqvfVcK/sIUOG0L179x2GBX/iiSfo1KkTRx55JGPHjmX27Nl897vfpaamhh49epTlxkIe7tvMWs3Dfe9ePNy3mZmVhROEmZllcoIws13Snpqp27PW/J2cIMys1bp168bq1audJNq4iGD16tV069atReu1ud9BmNnuY8CAAdTV1eFhbtq+bt26MWBAkxeO7sQJwsxaraamhkGDBlU7DMuJm5jMzCyTE4SZmWVygjAzs0xOEGZmlskJwszMMjlBmJlZJicIMzPL5ARhZmaZnCDMzCxTnneUu0PSKkmLi9QPlvSMpM2Sri4oP1DSE5JekrRE0lfyitHMzIrL8wxiJnBqE/VrgMuBaY3KtwL/FBFHAMcBl0k6IpcIzcysqNwSRETMJ0kCxepXRcRzwJZG5Ssj4oV0egPwMtA/rzjNzCxbm+6DkDQQGAb8tollJktaIGmBR5Q0MyufNpsgJPUAfgZcERHriy0XETMiojYiavv161e5AM3M2rk2mSAk1ZAkh1kR8fNqx2Nm1hG1uQQhScDtwMsR8Z/VjsfMrKPK7YZBku4GxgB9JdUBU4EagIi4VdL+wAKgF/C+pCuAI4AhwPnAi5IWppv7RkTMzStWMzPbWW4JIiLGN1P/FpB1/7unAOUSlJmZlazNNTGZmVnb4ARhZmaZnCDMzCyTE4SZmWVygjAzs0xOEGZmlskJwszMMjlBmJlZJicIMzPL5ARhZmaZnCDMzCyTE4SZmWVygjAzs0xOEGZmlskJwszMMuWWICTdIWmVpMVF6gdLekbSZklXN6o7VdIrkpZJujavGM3MrLg8zyBmAqc2Ub8GuByYVlgoqTPwfWAsyR3mxks6IqcYzcysiNwSRETMJ0kCxepXRcRzwJZGVccCyyLitYh4D5gNjMsrTjMzy9YW+yD6A28UzNelZZkkTZa0QNKC+vr63IMzM+so2mKCaJGImBERtRFR269fv2qHY2bWbrTFBPEmcGDB/IC0zMzMKqgtJojngEMlDZLUFTgPuK/KMZmZdThd8tqwpLuBMUBfSXXAVKAGICJulbQ/sADoBbwv6QrgiIhYL+kfgYeAzsAdEbEkrzjNzCxbbgkiIsY3U/8WSfNRVt1cYG4ecZmZWWnaYhOTmZm1AU4QZmaWyQnCzMwyOUGYmVkmJwgzM8vkBGFmZpmcIMzMLJMThJmZZXKCMDOzTE4QZmaWyQnCzMwyOUGYmVkmJwgzM8vkBGFmZpmcIMzMLFOuCULSHZJWSVpcpF6SpktaJmmRpOEFdf8uaYmkl9NllGesZma2o7zPIGYCpzZRPxY4NH1MBm4BkHQCMBoYAhwFjAQ+nmegZma2o1wTRETMB9Y0scg44K5IPAvsLekAIIBuQFdgD5Jblf4lz1jNzGxH1e6D6A+8UTBfB/SPiGeAJ4CV6eOhiHi5CvGZmXVYJSUISXtJ6pROHybpTEk1eQUl6SPAR0nuWd0fOEXSiUWWnSxpgaQF9fX1eYVkZtbhlHoGMR/oJqk/8DBwPkn/wq56EziwYH5AWnY28GxEbIyIjcCDwPFZG4iIGRFRGxG1/fr1K0NIZmYGpScIRcQm4Bzg5oj4LHBkGV7/PmBiejXTccC6iFgJvA58XFKX9Ezl44CbmMzMKqhLictJ0vHABOALaVnnEla6GxgD9JVUB0wl6XAmIm4F5gKnAcuATcCF6ao/BU4BXiTpsP5VRNxfYqxmZlYGpSaIK4CvA7+IiCWSDiHpRG5SRIxvpj6AyzLKtwGXlBibmZnloKQEERG/Bn4NkHZWvx0Rl+cZmJmZVVepVzH9RFIvSXsBi4GXJH0139DMzKyaSu2kPiIi1gNnkVxRNIjkSiYzM2unSk0QNenVRGcB90XEFpLOYzMza6dKTRD/DSwH9gLmSzoYWJ9XUGZmVn2ldlJPB6YXFK2QdHI+IZmZWVtQaid1b0n/2TCkhaT/IDmbMDOzdqrUJqY7gA3A59LHeuDOvIIyM7PqK/WHcn8XEZ8pmP9nSQtziMfMzNqIUhPEO5I+FhFPAUgaDbyTX1iV9W//BhLsuSd065Y8N35klXfrBjU1ybpmZu1NqQliCnCXpN7p/F+BC/IJqfJuuAHefbd163bqVFoyKWd5ly5OSmYdTQRs2QLvvQebN3/wvHlzUjd4cPlfs9SrmP4AHCOpVzq/XtIVwKLyh1R5f/tbspPfeSd5vPvuB9OFj9aU19dnL9vahATZSampsxwnJbPSRMC2bR988GZ9GLe2rBzbKGb//WHlyvLvj1LPIIAkMRTMXgXcVNZoqqTwA7dS3n+//EmpoazaSalcycpJqf1q+BAu54dnucqizD8B7toV9tgjeTRMZ5X17FnaclllPXqUN+YGLUoQjfhfdxfsbkmpuWXbalJqaXlNbvdJrLz336/+h22xsm3byvtea2pK+0Dt3r1lH7y7Wra791HuSoLwUBu7mbaalFp79pRHUurcOZ+mOqnyH8pbt5bv7wjJGV0pH4q9e2cvl9eHcteuu/eHcFvWZIKQtIHsRCCggh8ztruqdlIqZ39SXkmpUOfOpX0o9ugBffpU9ttwp1J/NWXtRpMJIiJ6tnbDku4AzgBWRcRRGfUC/ovkjnKbgEkR8UJadxBwG8n9qgM4LSKWtzYW61jaYlKC0j6UOzd7n0azytmVJqbmzAS+B9xVpH4scGj6GAXckj6TrnNjRDwiqQfwfo5xmu2yaiQls7zlliAiYr6kgU0sMg64K73t6LOS9pZ0ALAP0CUiHkm3szGvGM3MrLhqtir2B94omK9Lyw4D1kr6uaTfS/qupKIn3pImNwwiWF9fn3PIZmYdR1vsduoCnAhcDYwEDgEmFVs4ImZERG1E1Pbr168yEZqZdQDVTBBvknRCNxiQltUBCyPitYjYCswBhlc+PDOzjq2aCeI+YKISxwHrImIl8Bywt6SG04FTgJeqFaSZWUeVWye1pLuBMUBfSXXAVKAGICJuBeaSXOK6jOQy1wvTum2SrgYeSy+FfR74QV5xmplZtjyvYhrfTH0AlxWpewQYkkdcZmZWmrbYSW1mZm2AE4SZmWVygjAzs0xOEGZmlskJwszMMjlBmJlZJicIMzPL5ARhZmaZnCDMzCyTE4SZmWVygjAzs0xOEGZmlskJwszMMjlBmJlZJicIMzPLlGuCkHSHpFWSFhepl6TpkpZJWiRpeKP6XpLqJH0vzzjNzGxneZ9BzARObaJ+LHBo+pgM3NKo/jvA/FwiMzOzJuWaICJiPrCmiUXGAXdF4lmSe1EfACBpBPAh4OE8YzQzs2zV7oPoD7xRMF8H9JfUCfgP4OrmNiBpsqQFkhbU19fnFKaZWcdT7QRRzJeAuRFR19yCETEjImojorZfv34VCM3MrGPoUuXXfxM4sGB+QFp2PHCipC8BPYCukjZGxLVViNHMrEOqdoK4D/hHSbOBUcC6iFgJTGhYQNIkoNbJwcyssnJNEJLuBsYAfSXVAVOBGoCIuBWYC5wGLAM2ARfmGY+ZmZUu1wQREeObqQ/gsmaWmUlyuayZmVVQW+2kNjOzKnOCMDOzTE4QZmaWyQnCzMwyOUGYmVkmJwgzM8vkBGFmZpmcIMzMLJMThJmZZXKCMDOzTE4QZnmaNQsGDoROnZLnWbOqHZFZyao9mqtZ+zVrFkyeDJs2JfMrViTzABMmFF/PrI3wGYRZXq677oPk0GDTpqTcbDfgBGGWl9dfb1m5WRvjBGGWl4MOalm5WRvjBGGWlxtvhO7ddyzr3j0pN9sN5JYgJN0haZWkxUXqJWm6pGWSFkkanpYPlfSMpCVp+efzitEsVxMmwIwZcPDBICXPM2a4g9p2G3lexTQT+B5wV5H6scCh6WMUcEv6vAmYGBF/lPRh4HlJD0XE2hxjNcvHhAlOCLbbyi1BRMR8SQObWGQccFd629FnJe0t6YCI+J+CbfxZ0iqgH7A2r1jNzGxn1eyD6A+8UTBfl5ZtJ+lYoCvwarGNSJosaYGkBfX19bkEambWEbXZTmpJBwA/Ai6MiPeLLRcRMyKiNiJq+/XrV7kAzczauWomiDeBAwvmB6RlSOoFPABcFxHPViE2M7MOr5oJ4j5gYno103HAuohYKakr8AuS/omfVjE+M7MOLbdOakl3A2OAvpLqgKlADUBE3ArMBU4DlpFcuXRhuurngJOAfSVNSssmRcTCvGI1M7Od5XkV0/hm6gO4LKP8x8CP84rLzMxK02Y7qc3MrLqcIMzMLJMThJmZZXKCMDOzTE4QZmaWyQnCzMwyOUGYmVkmJwgzM8vkBGFmZpmcIMzMLJMThJmZZXKCMDOzTE4QZmaWyQnCzMwyOUGYmVmmXBOEpDskrZK0uEi9JE2XtEzSIknDC+oukPTH9HFBnnGamdnO8j6DmAmc2kT9WODQ9DEZuAVAUh+SO9CNAo4FpkraJ9dIzcxsB7kmiIiYD6xpYpFxJPeejoh4Fthb0gHAp4BHImJNRPwVeISmE42ZmZVZtfsg+gNvFMzXpWXFynciabKkBZIW1NfX5xaomVlHU+0EscsiYkZE1EZEbb9+/aodjplZu1HtBPEmcGDB/IC0rFi5mZlVSLUTxH3AxPRqpuOAdRGxEngI+HtJ+6Sd03+flpmZWYV0yXPjku4GxgB9JdWRXJlUAxARtwJzgdOAZcAm4MK0bo2k7wDPpZu6ISKa6uw2M7MyyzVBRMT4ZuoDuKxI3R3AHXnEZWZmzat2E5OZmbVRThBmZrurWbNg4EDo1Cl5njWrrJvPtYnJzMxyMmsWTJ4MmzYl8ytWJPMAEyaU5SV8BmFmtju67roPkkODTZuS8jJxgjAz2x29/nrLylvBCcLMbHd00EEtK28FJwgzs93RjTdC9+47lnXvnpSXiROEmdnuaMIEmDEDDj4YpOR5xoyydVCDr2IyM9t9TZhQ1oTQmM8gzMwskxOEmZllcoIwM7NMThBmZpbJCcLMzDIpGXG7fZBUD6xo5ep9gbfLGE65OK6WcVwt47hapj3GdXBEZN6vuV0liF0haUFE1FY7jsYcV8s4rpZxXC3T0eJyE5OZmWVygjAzs0xOEB+YUe0AinBcLeO4WsZxtUyHist9EGZmlslnEGZmlskJwszMMrX7BCHpDkmrJC0uUi9J0yUtk7RI0vCCugsk/TF9XFDhuCak8bwo6WlJxxTULU/LF0paUOG4xkhal772QknfLqg7VdIr6b68tsJxfbUgpsWStknqk9blub8OlPSEpJckLZH0lYxlKn6MlRhXxY+xEuOq+DFWYlwVP8YkdZP0O0l/SOP654xl9pB0T7pPfitpYEHd19PyVyR9qsUBRES7fgAnAcOBxUXqTwMeBAQcB/w2Le8DvJY+75NO71PBuE5oeD1gbENc6fxyoG+V9tcY4JcZ5Z2BV4FDgK7AH4AjKhVXo2U/DTxeof11ADA8ne4J/E/j912NY6zEuCp+jJUYV8WPsVLiqsYxlh4zPdLpGuC3wHGNlvkScGs6fR5wTzp9RLqP9gAGpfuuc0tev92fQUTEfGBNE4uMA+6KxLPA3pIOAD4FPBIRayLir8AjwKmViisink5fF+BZYEC5XntX4mrCscCyiHgtIt4DZpPs22rENR64u1yv3ZSIWBkRL6TTG4CXgf6NFqv4MVZKXNU4xkrcX8Xkdoy1Iq6KHGPpMbMxna1JH42vLBoH/DCd/inwCUlKy2dHxOaI+BOwjGQflqzdJ4gS9AfeKJivS8uKlVfDF0i+gTYI4GFJz0uaXIV4jk9PeR+UdGRa1ib2l6TuJB+yPysorsj+Sk/th5F8yytU1WOsibgKVfwYayauqh1jze2vSh9jkjpLWgisIvlCUfT4ioitwDpgX8qwv3xHuTZO0skk/7wfKyj+WES8KWk/4BFJS9Nv2JXwAsnYLRslnQbMAQ6t0GuX4tPAbyKi8Gwj9/0lqQfJB8YVEbG+nNveFaXEVY1jrJm4qnaMlfh3rOgxFhHbgKGS9gZ+IemoiMjsiys3n0HAm8CBBfMD0rJi5RUjaQhwGzAuIlY3lEfEm+nzKuAXtPC0cVdExPqGU96ImAvUSOpLG9hfqfNodOqf9/6SVEPyoTIrIn6esUhVjrES4qrKMdZcXNU6xkrZX6mKH2PpttcCT7BzM+T2/SKpC9AbWE059le5O1Xa4gMYSPFO19PZsQPxd2l5H+BPJJ2H+6TTfSoY10EkbYYnNCrfC+hZMP00cGoF49qfD35geSzwerrvupB0sg7igw7EIysVV1rfm6SfYq9K7a/0vd8F3NTEMhU/xkqMq+LHWIlxVfwYKyWuahxjQD9g73R6T+BJ4IxGy1zGjp3U96bTR7JjJ/VrtLCTut03MUm6m+SqiL6S6oCpJB09RMStwFySq0yWAZuAC9O6NZK+AzyXbuqG2PGUMu+4vk3Sjnhz0t/E1khGa/wQyWkmJP8wP4mIX1UwrnOBSyVtBd4BzovkaNwq6R+Bh0iuNrkjIpZUMC6As4GHI+JvBavmur+A0cD5wItpOzHAN0g+fKt5jJUSVzWOsVLiqsYxVkpcUPlj7ADgh5I6k7T43BsRv5R0A7AgIu4Dbgd+JGkZSfI6L415iaR7gZeArcBlkTRXlcxDbZiZWSb3QZiZWSYnCDMzy+QEYWZmmZwgzMwskxOEmZllcoIwa0Y6aufCgkc5RxEdqCIj1JpVW7v/HYRZGbwTEUOrHYRZpfkMwqyV0nsA/Ht6H4DfSfpIWj5Q0uNK7rXwmKSD0vIPSfpFOgjdHySdkG6qs6QfpOP9Pyxpz3T5y5Xcn2CRpNlVepvWgTlBmDVvz0ZNTJ8vqFsXEUcD3wNuSsv+L/DDiBgCzAKmp+XTgV9HxDEk97Zo+BXwocD3I+JIYC3wmbT8WmBYup0p+bw1s+L8S2qzZkjaGBE9MsqXA6dExGvpQG9vRcS+kt4GDoiILWn5yojoK6keGBARmwu2MZBkCOdD0/mvATUR8S+SfgVsJBnNdE58cF8As4rwGYTZroki0y2xuWB6Gx/0DZ4OfJ/kbOO5dKROs4pxgjDbNZ8veH4mnX6adMA0YALJCJwAjwGXwvabwPQutlFJnYADI+IJ4Gsko4judBZjlid/IzFr3p4FI3wC/CoiGi513UfSIpKzgPFp2ZeBOyV9FagnHb0V+AowQ9IXSM4ULgVWFnnNzsCP0yQiYHok9wMwqxj3QZi1UtoHURsRb1c7FrM8uInJzMwy+QzCzMwy+QzCzMwyOUGYmVkmJwgzM8vkBGFmZpmcIMzMLNP/B7p+qjXFIum3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualize the training and validation loss\n",
    "\n",
    "history_dict = history.history\n",
    "loss_values = history_dict['loss']\n",
    "val_loss_values = history_dict['val_loss']\n",
    "epochs = range(1, len(loss_values) + 1)\n",
    "\n",
    "plt.plot(epochs, loss_values, 'ro', label='Training loss')\n",
    "plt.plot(epochs, val_loss_values, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "851d2383",
   "metadata": {},
   "source": [
    "### (II). Compute RMSE Score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7e82504c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0086962"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m = tf.keras.metrics.RootMeanSquaredError()\n",
    "m.update_state(model.predict(X_test_array), y_test)\n",
    "m.result().numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb2c0857",
   "metadata": {},
   "source": [
    "## 6. Verdict for using Keras Recommender"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6690b00",
   "metadata": {},
   "source": [
    "From the above RMSE results (ranges from 97.28% to 106.36%) for 5 million worth of data, Keras is not one of the best 3 RMSE achieved as compared to the surprise library. \n",
    "\\\n",
    "Hence, recommendations will be provided for the best three RMSE attained are in << Netflix RS - Hilary (2 of 3) >>. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8eaf01e",
   "metadata": {},
   "source": [
    "## == END == "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}