lstm-for-sine-wave.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Libraries\n",
    "\n",
    "- numpy: package for scientific computing \n",
    "- matplotlib: 2D plotting library\n",
    "- tensorflow: open source software library for machine intelligence\n",
    "- **learn**: Simplified interface for TensorFlow (mimicking Scikit Learn) for Deep Learning\n",
    "- mse: \"mean squared error\" as evaluation metric\n",
    "- **lstm_predictor**: our lstm class "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "from tensorflow.contrib import learn\n",
    "from sklearn.metrics import mean_squared_error, mean_absolute_error\n",
    "from lstm_predictor import generate_data, lstm_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parameter definitions\n",
    "\n",
    "- LOG_DIR: log file\n",
    "- TIMESTEPS: RNN time steps\n",
    "- RNN_LAYERS: RNN layer information\n",
    "- DENSE_LAYERS: Size of DNN,  [10, 10]: Two dense layer with 10 hidden units\n",
    "- TRAINING_STEPS\n",
    "- BATCH_SIZE\n",
    "- PRINT_STEPS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "LOG_DIR = './ops_logs'\n",
    "TIMESTEPS = 5\n",
    "RNN_LAYERS = [{'steps': TIMESTEPS}]\n",
    "DENSE_LAYERS = [10, 10]\n",
    "TRAINING_STEPS = 100000\n",
    "BATCH_SIZE = 100\n",
    "PRINT_STEPS = TRAINING_STEPS / 100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate waveform\n",
    " - fct: function\n",
    " - x: observation\n",
    " - time_steps\n",
    " - seperate: check multimodal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X, y = generate_data(np.sin, np.linspace(0, 100, 10000), TIMESTEPS, seperate=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a regressor with TF Learn\n",
    "**Parameters**: \n",
    "- model_fn: regression model\n",
    "- n_classes: 0 for regression\n",
    "- verbose\n",
    "- steps: training steps\n",
    "- optimizer: (\"SGD\", \"Adam\", \"Adagrad\")\n",
    "- learning_rate\n",
    "- batch_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "regressor = learn.TensorFlowEstimator(model_fn=lstm_model(TIMESTEPS, RNN_LAYERS, DENSE_LAYERS), \n",
    "                                      n_classes=0,\n",
    "                                      verbose=1,  \n",
    "                                      steps=TRAINING_STEPS, \n",
    "                                      optimizer='Adagrad',\n",
    "                                      learning_rate=0.03, \n",
    "                                      batch_size=BATCH_SIZE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ValidationMonitor\n",
    " - x\n",
    " - y\n",
    " - every_n_steps\n",
    " - early_stopping_rounds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "validation_monitor = learn.monitors.ValidationMonitor(X['val'], y['val'],\n",
    "                                                      every_n_steps=PRINT_STEPS,\n",
    "                                                      early_stopping_rounds=1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train and validation\n",
    "\n",
    "- fit: fitting using training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Input iterator is exhausted: .\n",
      "WARNING:tensorflow:Input iterator is exhausted: .\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TensorFlowEstimator(steps=100000, optimizer=Adagrad, learning_rate=0.03, batch_size=100, n_classes=0, verbose=1, class_weight=None, continue_training=False, clip_gradients=5.0, params=None)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regressor.fit(X['train'], y['train'], monitors=[validation_monitor], logdir=LOG_DIR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate using test set\n",
    "\n",
    "Evaluate our hypothesis using test set. The mean squared error (MSE) is used for the evaluation metric.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error: 0.000294\n"
     ]
    }
   ],
   "source": [
    "predicted = regressor.predict(X['test'])\n",
    "mse = mean_squared_error(y['test'], predicted)\n",
    "print (\"Error: %f\" % mse)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting\n",
    "\n",
    "Then, plot both predicted values and original values from test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x10ff530f0>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEACAYAAABRQBpkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XV4lFf6//H3iZAAIRAsSAhOcHcp7mSCOy1SSm13f9tu\n9VuhsrRQ3ZaWtkCxQoEQIAluDYtT3K0UCRLcQjzn90dSStlgycyckft1XXMxM3ky5zMPz+SeR845\nSmuNEEII9+ZhOoAQQgjzpBgIIYSQYiCEEEKKgRBCCKQYCCGEQIqBEEIIrFAMlFI/KqXilFJ7H7DM\nV0qpo0qp3UqpOjltUwghhHVZY89gCtDpfj9USnUBKmitKwLPABOs0KYQQggrynEx0FqvA64+YBEL\nMC1z2S1AAaVUYE7bFUIIYT32OGdQEjh91+NYIMgO7QohhHhE9jqBrO55LGNgCCGEA/GyQxtngFJ3\nPQ7KfO4vlFJSIIQQIhu01vd+4X5s9tgziAKeBFBKNQauaa3jslpQay03rXn33XeNZ3CUm6usi6TU\nJP61/BX8PyhM4dDPqdfkFhMnamJjs1g2SbN8dQLNX/oa9UogNV79G6fO3XaZdSHbhXVv1pLjPQOl\n1M9AS6CwUuo08C7gDaC1/l5rvUQp1UUpdQyIB4bltE0hnMmp66cIm9mbc0eKU3L9Ab4cU4QOHe6/\nfK5c0KGNLx3avMjR04Po+t2zlBvTkLanWtsvtHA7OS4GWusBj7DMizltRwhntDduL+2mdCFh7d/5\nW71/8e5GRa5cj/77FUsFcPjD2bweMYFPvnyV3v8YzKxPGj7WawjxKOxxzkA8platWpmO4DCceV3s\nOr+LVpM6kr70S+a8NYDOnbP3OkopxvZ+Hp+r1xh7shvN+0exZnpj/Pysm9eZOPN24aiUNY855YRS\nSjtKFiFy6ujlozSc0BK17CvWjO9N7drWed3ow0voO3MYFTcvZ/282vj7W+d1hfNSSqGtcAJZioEQ\nVnYx/iI1vmpEwso32Pj1SKpVs+7rz90fzog5L1Nz6xbWRBbHx8e6r38vpXL8d0ZYSVZ/I6UYCOGA\nUtNTafxNRw6sbMj60R9Rt65t2hn9y/t8vWwJLX+PYd5sXzxseF1g5h8b2zUgHsn9/h+sVQxk1FIh\nrOjFhW+wb48nc0Z9aLNCAPBuq7dpXTeYTQX+zvvv264d4T6kGAhhJVEHljJly1xeLf8zoV09bdqW\nUoofu0/Cp8pqvl41n6VLbdqccANymEgIK7h8+zJlxtak5m8/sX5Ga+x1mH1L7BY6T7fgMWk7O2KC\nCA62fhtymMgxyGEiIZxAzynPoff1I/JL+xUCgEZBjfhnsxcpNPQZhg3XpKfbr21XVqZMGdasWQPA\nmDFjGDlypM3bjImJoVSpUg9f0EakGAiRQz/viGLjsd3MHjmGwoXt3/7rzV8nV5FYTvnP5ttv7d++\nK7r7Cqo333yTiRMnPvR3hg4dyttvv23LWDblUMVA9kSFs4lPjmfUwr/TMfVbunXyNZLB29ObSaET\nud7oJd75+DLHjxuJ4bBSU1NNR3AKDlUM5swxnUCIx/P87H+TcrwpM95vazRHo6BG9K/Zh4qj3uQf\n/zAaxW7KlCnDxx9/TLVq1ShYsCDDhw8nKSmJmJgYgoKCGDduHMWLF2fEiBForfn444+pUKEChQsX\npl+/fly9+uecXDNmzKB06dIULlyYMWPG/KWd0aNHM2TIkDuP169fT9OmTQkICCA4OJhp06YxceJE\nZs2axbhx48iXLx9hYWEAnD17ll69elG0aFHKlSvH119/fed1EhISGDp0KAULFqRatWr8+uuvNl5j\nD2F6xL27Rt7TxUqk6qtXtRBOYX/cQe31RmH9zfSzpqNorbW+cvuKLjquqC7dYI+OirLe62b8mXA8\npUuX1jVq1NCxsbH6ypUrulmzZvqtt97SMTEx2svLS7/++us6OTlZJyQk6C+//FI3adJEnzlzRicn\nJ+tRo0bpAQMGaK213r9/v/bz89Pr1q3TSUlJ+qWXXtJeXl569erVWmutR48erQcPHqy11vrEiRM6\nX758evbs2To1NVVfvnxZ79q1S2ut9dChQ/Xbb799J19aWpquW7eu/uCDD3RKSoo+fvy4LleunF6+\nfLnWWuvXXntNP/HEE/rq1av69OnTulq1arpUqVL3fb/3+3/IfD7nf4Ot8SJWCQK62d8m6tdfv++6\nEMKh1BkXqkv1+1Snp5tO8qfxW8br2l+01eXKp+vkZOu85sOKQcYB3pzfHleZMmX0999/f+fxkiVL\ndPny5XVMTIzOlSuXTkpKuvOzKlWq3PnjrrXWZ8+e1d7e3jo1NVW/9957dwqD1lrHx8frXLly3Vn+\n3XffvVMMxowZo3v27JllnqFDh+q33nrrzuPNmzfr4ODgvywzZswYPWzYMK21/kth0FrrH374QQcF\nBd33/dq6GDjUYaKjQe/w/ZSbnD1rOokQD7b62Dr2xu1l8qgX7Xr10MOMqj+KJO+z+NdbxOTJ9mnT\nWuUgO+6++iY4OJizmX88ihQpQq67hnY9ceIEPXr0ICAggICAAKpWrYqXlxdxcXGcO3eOoKA/Z+LN\nkycPhQoVyrK906dPU65cuUfKdvLkSc6ePXunzYCAAD766CMuXLgAZBxCuje/SQ5VDDpVak/IiLHS\no1I4NK01I2a/QvULH9K+tY0HBnpMXh5efNbhM642eIX3P0zl9m3TiWzr1KlTf7lfokQJ4H/HUwoO\nDmbZsmVcvXr1zu327duUKFGC4sWLc/r0n9O03759m8uXL2fZXnBwML/99luWP8uqzbJly/6lzRs3\nbrBo0SIAihcv/j/5TXKoYvB+q/c57D+B8MUXOXLEdBohsvbTjgjOnE9m2isPncrDiE4VOlG6cCAl\nO81k/HjTaWxHa823337LmTNnuHLlCv/+97/p379/lss+++yzvPnmm3f+4F68eJGoqCgAevfuzaJF\ni9iwYQPJycm88847pN+nw8bAgQNZtWoV4eHhpKamcvnyZXbv3g1AYGAgx++6lKthw4bky5ePcePG\nkZCQQFpaGvv27WPbtm0A9O3bl48++ohr164RGxv7l5PLJjhUMShdoDT9q/ej6jPjGD3adBoh/lda\nehovL36b5okfU7OGQ3187lBK8UHrD4ir/D6ffJ7CtWumE9mGUoqBAwfSoUMHypcvT8WKFXnrrbfQ\nWv/Pt/R//OMfWCwWOnTogL+/P02aNGHr1q0AVK1alW+++YaBAwdSokQJChYs+JfDN0qpO68XHBzM\nkiVL+OyzzyhUqBB16tRhz549AIwYMYIDBw4QEBBAz5498fDwYNGiRezatYty5cpRpEgRnnnmGW7c\nuAHAu+++S+nSpSlbtiydOnXiySefNDpCrMMNR3Hmxhmqf1sD9e0Btq4pRoUKppMJ8afpO+bw9MT/\nsP35DdSo4UAnC7LQfkZ7bm3uS2jJkbz5ZvZfx1GHoyhbtiyTJ0+mTZs2pqPYhdsNR1HSvyRP1X6S\nisM+ZuxY02mE+FO6Tuf1xR/SIOFthy8EAB+0/oCTZT7kP+OTXP7cgcg5hysGkNG9/nDuaYQvjeOu\n8zpCGDV3z0Iuxfny1d86mY7ySBoHNaZm8SqU6DyDH380nUY4Ooc7TPSH5xY9x86NhWgU/yH/+Y/B\nYEKQcbKy9Jh6FNk/mu2zLKbjPLKYEzE8FT4K/c0Bfjvqibf347+Gox4mcjdud5joD/9q+i+O+H/H\ntJ9vcuWK6TTC3S0+soQLF9P55OlQ01EeS8vSLSkWUID8DSOZNct0GuHIHLYYlC9YnvYV2lKx/0Qe\nYcBAIWzq/5Z8StGjr9G6teOfK7ibUorXmr1GSsOxfPGlznbnLuH6HLYYALza9FVOl/yC8ROSSUkx\nnUa4qx3ndnDk4jHe7tXboXobP6qwkDC0zzUu+61l/XrTaYSjcuhiUK9EPaoXDyFPw59ZsMB0GuGu\nRi//As/tf2fIwGwccHcAnh6evNLsFfJ1Hovhfk3CgTl0MYCMcwfJ9b7gy//I/q2wv9gbsaz4fTEv\nNB6Jr5npCqxiSM0hXPLewfJth+UKPZElhy8GHcp3wDvPbY6nbCCzF7cQdvPpf8ejdw/hn88VMB0l\nR3y8fHim3kjK9BvPhAmm0whH5PDFwEN58GLDFyka+jXff286jXAn8cnxTNoxifb+f6dYMdNpcu7Z\n+s9y0n8mP0y7QUKC6TTWcfdcxdk1depUWrRoYaVEzsvhiwHA0NpDOe29krlLz5A5rIcQNjdr7894\nnGnKy8PKm45iFUH+QXSo0I4i7acxb57pNNYhfSCsxymKgb+PP4NqDqRE2HdyrbSwm8//+x1+h56j\nVSvTSaznbw3/xo2Q8UyclPWonM5kyJAhnDp1itDQUPLly8enn37K5s2b70xJWbt2bdauXXtn+alT\np1K+fHn8/f0pV64cs2bN4tChQzz77LNs2rSJfPnyUbBgQYPvyDBrzJBjjRsPmero4MWDOuDfgbpm\n3USHmllKuKatsVu13/+V0R+PTTUdxarS09N1zW9r6QL1l+nDhx/tdx722TSpTJkyd2Yki42N1YUK\nFdJLly7VWmu9cuVKXahQIX3p0iV969Yt7e/vr48cOaK11vr8+fN6//79Wmutp06dqps3b27mDTyG\n+/0/YKWZzryMVqLHULlwZeoG1WB3/gi2bRtIgwamEwlX9tXG70jZMophMz1NR7EqpRR/a/QiY89O\nYNKkjowbZ4XXfM86nS/0uzk73PPTTz/RpUsXOnXKGDuqXbt21K9fn8WLF9O7d288PDzYu3cvQUFB\nBAYGEhgYmNGuHGYCcJ5iADCq3jO8duJbfvhBioGwnasJV4k4OJ+ORQ5TtKjpNNbXv3p/Xl72Cj/+\ndI4PPyzOXbNDZktO/4hby8mTJwkPDyc6OvrOc6mpqbRp04Y8efIwZ84cPv30U0aMGEGzZs347LPP\nCAkJMZjYsTjFOYM/WEIs3PDZz+yVR7l1y3Qa4apm7JmB7+lOvDjMBSsB4JfLj77Ve5OvxRQyZ2B0\nWndPBhMcHMyQIUP+Ms3kzZs3efXVVwHo0KEDK1as4Pz581SuXJmRI0f+z2u4M6cqBj5ePgyt8yRF\nO06WHsnCJrTWfLXxO9T253DlOVNG1htJfMgkfpjo3CeSAwMD78xJPGjQIKKjo1mxYgVpaWkkJiYS\nExPDmTNnuHDhApGRkcTHx+Pt7U3evHnx9PS88xqxsbGkuPmYN05VDABG1BnBlVLTmDrDvf/jhG1s\nPL2R6zc0g1q0wNO1Thf8RYMSDSgWkI8NZ9dw7pzpNNn3xhtv8OGHHxIQEEB4eDiRkZGMGTOGokWL\nEhwczGeffYbWmvT0dL744gtKlixJoUKFWLduHRMye9+1bduWatWqUaxYMYq64nHBR+Sw8xk8SLPJ\nLdj93UscWtCDoCAbBxNuZUTU00RNqUT0a6/SuLHpNLY1fut4Ppu3jhcD5/Dyy/dfTq7ldwxuO5/B\ng4yqP5KC7SYyc6bpJMKVxCfHE74vAt/DQ2jUyHQa2xtUYxAX/Zczde4l01GEA3DKYtC7am+u+W1m\n8rxTMj67sJqIgxEUSWzK4LDiTjlU9eMKyB1Aj6oWTheczr59ptMI05yyGOTxzsPAWv24VOInduww\nnUa4iqm7pnJj7TAGDDCdxH6G1xlGrobTmDHDdBJhmlMWA4Anaw3Bo850pk2XXQORc79f/Z2dZ/ZS\n+EooNWqYTmM/Lcu0xDPvVaYu3U26c19YJHLIaYtBk6Am5PVPZcbqbaSmmk4jnN203dModaM/g/r5\nuMUhoj94KA+G1RuMrjmDmBjTaYRJTlsMlFIMrzuEXA2mk8MRbIWbS9fpTNs1jdjoYfTvbzqN/Q2p\nOYTESjOZNkO+Vbkzpy0GAENqDSG+7Gxmzk42HUU4sbUn1uKR4k/5vHWoUMF0GvurUqQK5QuXImLn\nahITs15GKSU3wzdbc+piUC6gHFWLhjB/zzKSpR6IbPppz08Uin2SAf3d6PjQPYbXG0LeJtNZvvx/\nf2aNETHlZrXRnW3GqYsBwNMNnsSn4XRWrDCdRDijxNREFhxawJH5/enXz3Qac/pX78+N4ouYOU9m\nj3JXTl8M+lTtQ3yxlUwPv2o6inBCS48upaRnLWqXL0nJkqbTmFMkbxFalm7Fot8iXGZKTPF4clwM\nlFKdlFKHlFJHlVKvZfHzVkqp60qpnZm3t3La5t0CcgfQrmwHoo/PlY1YPLZZ+2bhc3SgW/UtuJ+R\nDZ7Ep9F0li0znUSYkKNioJTyBMYDnYCqwAClVJUsFl2rta6TefswJ21m5ekGg/GpP5OlS639ysKV\n3Ui6wYpjKziysBe9e5tOY17XSl1JDtjFtPlnTUcRBuR0z6AhcExrfUJrnQLMBsKyWM6mZ+Y6VehE\nSsA+pkTE2rIZ4WIWHFxABe9WtKhfkEKFTKcxz9fLl9BKYSyPDef2bdNphL3ltBiUBE7f9Tg287m7\naaCpUmq3UmqJUqpqDtv8Hz5ePlhCwlh5NlwmvRGP7Od9P8OegW7Zt+B+htbvj0/d2SxZYjqJsLec\nTnv5KNc67QBKaa1vK6U6AwuBSlktOHr06Dv3W7VqRatWrR45yNB6/Vla710WLfqnfLjFQ8XdimPT\n6c3oxfPp/o3pNI6jbdm2pBUYzJQFv9O7d1nTcUQWYmJiiLFBd/EczWeglGoMjNZad8p8/AaQrrUe\n+4Df+R2op7W+cs/zjzyfQVZS0lIoOKYETfdvZfls2YjFg43fOp6fYrZQ6tcZhIebTuNYnpo3ijnf\nleNy5GvkzWs6jXgYR5nPYBtQUSlVRimVC+gHRN29gFIqUGV2n1NKNSSjAF3535fKGW9Pb3pX7cXa\nS3O5IZdKi4eYtXcWCVvlKqKsDK3Xn1x15jj9/Mji8eSoGGitU4EXgeXAAWCO1vqgUmqUUmpU5mK9\ngb1KqV3Al4DNDuI8VS/jeGd0tK1aEK7gxLUTHL50lJO/tKNLF9NpHM8TpZ/Aw/8ckyMPm44i7CjH\n/Qy01ku11iFa6wpa648yn/tea/195v1vtNbVtda1tdZNtdabc9rm/bQIboGHfxyTF8pGLO5v3oF5\nhKT3oHuoN76+ptM4Hk8PT/rV6Mt/L8/h5k3TaYS9OH0P5LtlbMR92HB9DteumU4jHNW8A/O4ur6P\nXGjwAE/V64d3ndlERsp8Ie7CpYoBwFN1+5Or7mwWLpSNWPyvU9dPceTSMS5tb0XbtqbTOK7GQY3x\nyRfPpCiZD9NduFwx+GMjnrxor+kowgFFHIigfEoYvXt44+1tOo3j8lAeDKrVj003Z8tetptwuWKg\nlGJgrT5sjZ/HFatfsySc3byD87i8Tg4RPYpBdXqTq3YECxbIXrY7cLliADCgVi98akewcKHpJMKR\nxN6IZX/cIRIPtqF5c9NpHF+DEg3w8bvN5OgDpqMIO3DJYtAoqBFeftf4MeqQ6SjCgcw/OJ8ySaH0\n75MLT0/TaRyfUop+NXuyLT6Cy5dNpxG25pLFwEN50Kd6D7bFz+fSJdNphKMI3x/OxbVyiOhx9K/Z\nC986ESxYYDqJsDWXLAaQsRHnqScbschw9uZZ9pzfT67T7WjQwHQa59G0VFPwi2Nq1DHTUYSNuWwx\naFG6BWl+p5m68HfTUYQDWHBwAaUSujGgrw92mFvcZXh6eNK7eg+23Y7g4kXTaYQtuWwx8PLwome1\nMLYnzOfCBdNphGnhB8KJ+6W3HCLKhv41e5G3fgTz55tOImzJZYsBQL8avfBrIBuxu4u7FceOM7sp\ndK0DNWqYTuN8WpZuSYrfcaYtPGU6irAhly4Gbcq2ISnfIWYslGn83NmCQwsoHt+ZQf185RBRNnh7\netO9qoWdifM5f950GmErLl0McnnmIrRyV3YmLJCN2I0tPBTJhbU96NfPdBLn1bd6T/I1mk9EhOkk\nwlZcuhgA9K3eC//GcqjIXd1Musl/T6ynVHJHKmU5v554FO3Ltee2315mLJBvVa7K5YtBx/Iduem3\nnRnz5VIId7TitxUUTmhK/x7+pqM4NR8vH7qGdGZvykLOnDGdRtiCyxeD3N656VypI3uSIjkrpw7c\nTuThKK5vtdCrl+kkzq9vtV7kbxLBvHmmkwhbcPliANCnWi8CmsrxTneTmp5K1MHFFLseSkiI6TTO\nr3PFzlzLu4VZ86+ajiJswC2KQeeKnbnit55ZETJtkzvZdHoTPonBDOgSbDqKS8jjnYe25VtzIGUJ\np+QqU5fjFsXA38efZqWbsPf2cjne6UYWHookcY+F3r1NJ3EdPaqEUbh5pFyQ4YLcohhAxkYc2DJS\njne6Ca014XsiCYizUK2a6TSuo1ulblz0X0H4/CTTUYSVuU0xsIRYuJh/CfMWpJiOIuzg8OXDXL2V\nwMA2daSjmRUVzVuUmsWrs/vmGum742LcphgE+QdRsUhZdl5aT1yc6TTC1iIPReFx1EKf3lIJrK1H\n5TCKt4qUEYFdjNsUA4AeVbpTsm0kkZGmkwhbm70zitynLNSubTqJ6wmrHMblwlHMi0g3HUVYkVsV\ng7CQMK4GRjIvQuZ0dWUX4y9y8PJeBjZtLYeIbKBSoUoU9c/PltPbZAY0F+JWxaB60erkzaPYeGwP\nV+VSaZe16MhivE+3p39vH9NRXFb3KmEEtYskKsp0EmEtblUMlFJ0rxxGqfaRREebTiNsZea2KHL9\nbpEZzWyoe+XuxAdFSkdOF+JWxQAyjncmlpaN2FUlpiay/uxq+tTuIoeIbKhhyYakeF0mZs8xbtww\nnUZYg9sVg+bBzbnhcZLV205zUzoku5w1v6/B82ItBvcsbDqKS/NQHlgqhxLcPpJFi0ynEdbgdsXA\ny8OLrpW6ENw+iqVLTacR1jZ9axSexyw0bWo6iesLCwkjraLsZbsKtysGkLERU1k2YleTrtNZeiya\n0EpheLjllm1fbcu15ZzezYr1l7h923QakVNu+ZHpWKEjp9M3s+yX6yQkmE4jrGX72e2k3PLnya4V\nTUdxC75evrQv347gdotYtsx0GpFTblkM/HL58USZFpRsvZSVK02nEdYyZ1cUaQcttGxpOon7CAsJ\nw7uG7GW7ArcsBpCxEeeuLRuxKwnfE0XD/BZ8fU0ncR9dK3blt/TVLF5xmyQZu86puW0xsIRY+I1l\nRC9JJjnZdBqRUyeunSDu9jkGt2xsOopbKZSnEPVL1qNE81WsWmU6jcgJty0GxfyKUblICMUa/ZeY\nGNNpRE4tPBiNPtKVbl09TUdxO5ZKFvI3iJa9bCfntsUAMg4VBTSRQ0WuYOavUZS8ZaFkSdNJ3I8l\nxMJvXtFERqWTlmY6jcguty4GlhALJ3JFsWixRsvYdU7reuJ19lzZQt967U1HcUvlC5anqF9hCtbY\nyubNptOI7HLrYlC1SFV8fbzwKL6HvXtNpxHZtezYMnKdb0GPrn6mo7gtS4iFIs2jWLzYdBKRXW5d\nDJRSWCpZKNkmiiVLTKcR2fXzjig4LAPTmWQJsXA+v3yOnJlbFwPI2IivFJGN2FmlpKWw8sRSOpXr\nJr2ODWpYsiG3ucSJG78RG2s6jcgOt//4NA9uzqW039h+9IzMceCE1p9aj9fN8vTtJGeOTfJQHoRW\nCqVcZ/li5azcvhh4e3rTuWJnynVcJL2RnVDE/igSd4XRoYPpJMISYiGxtBQDZ+X2xQAyLjFNrxQp\nJ7+cjNaaeXsjqeVrIX9+02lE23JtOZ26nTWbrkhvZCckxQDoWL4jJ/U6lqy6RbrM8e009l/cT/zt\ndPq1qmE6igDyeOehTbnWBDZbyrp1ptOIxyXFAMjvm5+mwU3wqbqC7dtNpxGPKvJQFPqQhW7dZEoz\nRxEWEoZPbdnLdkZSDDJZQiwUaCTXSTuTObui8D9noVIl00nEH7pW7MpJzxUsWibHiZyNFINMoZVC\nic2zmMVLpD+9Mzh/6zxHrx6mV/0nZK5jBxLoF0iNYlW57LeWY8dMpxGPI8fFQCnVSSl1SCl1VCn1\n2n2W+Srz57uVUnVy2qYtlC5QmtIBJTl4axNxcabTiIdZdGQRec91xNI1l+ko4h6WEAuBT8i0ss4m\nR8VAKeUJjAc6AVWBAUqpKvcs0wWooLWuCDwDTMhJm7bUvXIYJdpEyqxNTmDe3ihu77TwxBOmk4h7\nWUIsXCqUMeaXcB453TNoCBzTWp/QWqcAs4Gwe5axANMAtNZbgAJKqcActmsTlhALN4rLeQNHdzvl\nNmtPxtAmuDM+PqbTiHtVKVwF/7w+rDu6i/h402nEo8ppMSgJnL7rcWzmcw9bJiiH7dpE3eJ1Ublu\nsWzbYVJSTKcR97Pq+Cr8b9WnZ+cA01FEFpRSdK9sIbBFFL/8YjqNeFReOfz9R90PvPcUX5a/N3r0\n6Dv3W7VqRatWrbIVKruUUnSvYiG6QRQbN74ic+k6qIUHo7ixzUKXl0wnEfdjCbEwp/xLLF78Lt26\nmU7jWmJiYoixwYxcSudgIH+lVGNgtNa6U+bjN4B0rfXYu5b5DojRWs/OfHwIaKm1jrvntXROsljL\n0qNLGTVzDP1vr2PcONNpxL3SdTqFPypOyWUb2fvf8qbjiPtITU+l8NhA8k7fTeyBILniy4aUUmit\nc7yGc3qYaBtQUSlVRimVC+gHRN2zTBTwJNwpHtfuLQSOpHXZ1lzx3kPUqoumo4gsbD2zFY/EIvRq\nI4XAkXl5eBEa0oXE0lEcOGA6jXgUOSoGWutU4EVgOXAAmKO1PqiUGqWUGpW5zBLguFLqGPA98HwO\nM9uUr5cvHSq042zeJZw8aTqNuFfkoUg4FEbXrqaTiIcJqxxG3roycJ2zyNFhImtylMNEANN3T+et\nnxbyRrn5PPec6TTibhW/rMaVH3/k4q5GMn+Bg7uZdJPAcSWoG3OG9av9TcdxWY5ymMgldanYhUv+\nq4lakmg6irjLsSvHiLtxGUv9BlIInEA+n3w0C27G9msruH7ddBrxMPKRykLhPIWpXawWa0+tISHB\ndBrxh+jD0fifDyW0m2y2zqJHVQsFm0TJXCFOQD5V99GzmoX8DaOwwRVcIpsWHIji8gYL7dubTiIe\nVWilUK4RVxgnAAAeGElEQVQXXUL04lTTUcRDSDG4D0uIhdtB0SxeIhMcOIIrCVfYdnY7TYu3JV8+\n02nEoyqVvxRlAkoTvWuDzBXi4KQY3EelQpUo6JePBZt34CDntd3a0qNLKXKrNWFd8piOIh5TnxoW\nqBzFzp2mk4gHkWLwAH1qhnGzZCSHDplOIiIPR3Fti0UuKXVClhBL5rSy8q3KkUkxeICwEAve1eU6\nadOS05JZemQ5gde7UV76mjmd2sVqkyt3MhFr5VuVI5Ni8ACNgxqTlvsc81adMB3Fra09sZb8KVXo\n3t4hB7sVD6GUolc1C4dVJBelY7/DkmLwAJ4enoSGdGVHfDQ3bphO476iDkehD1pkwDMn1qOqhbx1\nZcIbRybF4CF6VrOQr55cJ22K1poFB6O4td1C06am04jsalm6JQl+Bwhf4rDDkrk9KQYP0b58e+ID\ntjAvWrpQmrAnbg/JCd50aVgVr5wOuC6M8fHyoX25Dqw+vZikJNNpRFakGDyEXy4/mpRswaLDy0iV\nfjN2F3k4knxnQ+nWVcZAdnZ9alrIXTtSOnI6KCkGj6B/7TC8qkWycaPpJO5nwcFIzsWE0amT6SQi\np7pU7EJ80V+IiLptOorIghSDR9CtUjcSg5axIErmwrSn09dPc/zySeoWbk6hQqbTiJwqmLsgtYrW\nY/6u1dKR0wFJMXgEJfKVoHxABeZuXicbsR1FHY6i2M0uhHaVkwWuol9tCynloti1y3QScS8pBo+o\nX20LN4pHSW9kO4o8HMmVjWFySakLCQuxkFoumoVRMlCRo5Fi8IjCKlvwqBJFZKTsGtjD9cTrbDy1\nmTznOlK1quk0wlrKFyxPUb9CzFm31XQUcQ8pBo+oRtEa+OZOZ/aa/aajuIVlx5ZRIrUFPbr6yWTq\nLqZf7TBO5Y4iNtZ0EnE3KQaPSClFnxphHNJRxEm/GZuLPBxJ0u4wunc3nURYW/cqFnLVjCIqynQS\ncTcpBo8ho0t9JIsWmU7i2lLSUlhyZBk3t3ejeXPTaYS1NSzZEPJcYvby30xHEXeRYvAYnij9BIl+\nR5i75JzpKC5t7cm1FEiriKV1Cel17II8lAdhlbux5ZqM+eVIpBg8Bm9PbzqV78Ta84u4Lf1mbCby\nUCSex+QQkSvrXT2MvPUiWbHCdBLxBykGj6l3DQt+daNYtcp0EteUMTBdJOfXhtGhg+k0wlbalmtL\nQoHtzI2+YjqKyCTF4DF1qtCJW4XXMi8q3nQUl7Tr/C5SEnPRvnZV8sgMly4rj3cenijVmsVHlpKc\nbDqNACkGjy0gdwB1izUgcu8qGbjOBiIPR+J/Lowe3eV6UlfXt5YF31pRMnCdg5BikA1/DFy3fr3p\nJK5nwcFIzq6RXsfuoFulbiQUX0H4fNk1cARSDLIhtFIoSaUXER6RZjqKSzl57SQnrsRSv1hTGZjO\nDQT6BVK5cBXmbYshTT5KxkkxyIayAWUpmb8YczdsIV2GWLGaqMNRFL3WlZ5hcj2pu+hby4Jn1Sg2\nbTKdREgxyKY+NS3oSlFslSFWrGbhoUgu/DeMsDDTSYS9WEIspJaPImK+jPllmhSDbLKEWFCVo5g/\n33QS13Al4QpbTv9KWd2BMmVMpxH2UqVwFfzz5mJOzG4ZHt4wKQbZVL9EffC9xuwVR2UjtoLow9EE\n3m5L3+55TUcRdqSUold1C0llI9mxw3Qa9ybFIJs8lAc9qoVyq2Q0u3ebTuP85h2I4OrGnvTsaTqJ\nsLfulcPwri572aZJMciBsJCMgetkI86Zm0k3WXM8hqLXulG5suk0wt6aBTcjwecEc5bKmNYmSTHI\ngTZl23DFZydzoi+bjuLUlhxdQmBSM/qEFjAdRRjg5eFFaOUuXC4czcGDptO4LykGOZDbOzftK7Tl\nQv4lMh1mDsw/OJ+bv/akVy/TSYQpYSEW/BvIoSKTpBjkUFiIhUJN5VBRdiWkJLDkyDLynAqjVi3T\naYQpnSp04lLuDYRH3jQdxW1JMcih0JBQzudZybxIGdM6O1YeX0mhlDr07VpUprd0Y/l88tG8dFN+\n91zGiROm07gnKQY5VDhPYRoHN+SYWiIbcTZEHIwgeVcvuYpI0LtqLwo9MY8FC0wncU9SDKygb7U+\nFG4pG/HjSk5LJvJgNGn7e9Cokek0wrQeVXoQl28ZcxfIXrYJUgysoEflHlzIt4y5CxJMR3EqMSdi\n8E+pRO8OQXjIluj2CucpTONSDdmTsIRzMrOs3clH0AqK5C1Cw6D67Lm9VDbixxBxIAIO9qRHD9NJ\nhKPoV70PhVuGExlpOon7kWJgJX2r96ZIy3AWLjSdxDmkpacx/8BCbmzpScuWptMIR9Gjcg8u5V/G\nnPlyqMjepBhYSc8qPblYYKkcKnpE60+tJ1dKMbo/UQFvb9NphKPI2MtuwNYrS7lwwXQa9yLFwEqK\n5i1K/ZJ12XJ5GZelQ/JDzdk/B59jfeUqIvE/BtTsS+GW4UREmE7iXqQYWFH/Gn0o3GKeHO98iNT0\nVObtj+DiL/1o3950GuFoelTuwaUCy/h5nuxl25MUAyvqWaUnVwotYdbcRNNRHFrMiRhyp5aiS+MK\n5M5tOo1wNEXyFqFRqfpsvy4XZNiTFAMrCvQLpF7J2my6sJy4ONNpHNecfXPwOtSPAQNMJxGOql/1\nPhRpNZd580wncR/ZLgZKqYJKqZVKqSNKqRVKqSyHnFRKnVBK7VFK7VRKufwkkf1q9KFo63DCw00n\ncUzJaclEHFjApbV96dzZdBrhqHpUyThUNCtcDhXZS072DF4HVmqtKwGrMx9nRQOttNZ1tNYNc9Ce\nU+hZpScXAxYzc44cKsrKquOr8E+pRK+2pfHxMZ1GOKqieYvSIKge+xKXEivTHNhFToqBBZiWeX8a\n0P0By7rNEGTF/IpRv2Rt9iUtlbGKsjBn/xzS98ohIvFw/av3pUirubKXbSc5KQaBWus/jozHAYH3\nWU4Dq5RS25RSI3PQntMYXGsQRdrMZPZs00kcS2JqIgsPRJGwrQ+tW5tOIxxdr6q9iPNfyow5Mqy1\nPXg96IdKqZVAsSx+9H93P9Baa6XU/aaFb6a1PqeUKgKsVEod0lqvy2rB0aNH37nfqlUrWrVq9aB4\nDqtXlV78P7+X+Sn8Oq+/nt90HIex/NhyAlJqEtq1BF4P3PKEyBirqFXZFmz0XciBA0OoWtV0Isew\ncvVKnv76aQbXGIy3p/V6bD7wI6m1vu9V4EqpOKVUMa31eaVUcSDL/oJa63OZ/15USi0AGgIPLQbO\nLCB3AO0qtOG/+eezd+8watQwncgxzN43m6Rt/RnwhukkwlkMrjmIg09MY9q0IYwdazqNY0gMSqR0\nWGn+PezfALz33ntWed2cHCaKAp7KvP8U8D+j8iil8iil8mXezwt0APbmoE2nMbjmIPK3mMmUKaaT\nOIb45HgWHV6K7++9aNLEdBrhLCwhFi75bmbqvDhSU02ncQyz9s7C78QAEqx8oVVOisHHQHul1BGg\nTeZjlFIllFKLM5cpBqxTSu0CtgCLtNYrchLYWXSt2JUrvtuZNv8cycmm05i34NACCt5qxtMDZEYz\n8ejy5spLWJVQfOvNYdUq02nMu5F0g0WHl3Issi++vtZ97WwXA631Fa11O611Ja11B631tcznz2qt\nu2beP661rp15q661/shawR1dbu/c9KzanYDms1m0yHQa86bunMHlX4bw5JOmkwhnM6jGIFStmUyb\n9vBlXV3EgQgK3mjFiAGFrP6lSnog29DA6gNJrzaLH380ncSsczfPsenUVpoUDKNUKdNphLNpV64d\nt71PEL3xKNeumU5j1pQd07m0xjZfqqQY2FCbsm247R3LugNHOHvWdBpzZu2dRf6zPXj6KRmISDw+\nLw8v+tfoR3DXWW59ufap66fYcWYP7YK7Uby49V9fioENeXp4MqB6fyr0mMn06abTmDN52wziNw4h\nLMx0EuGsBtUYxI3SM/l2gkbf7yJ2Fzdzz0zynOjDcyNt03VfioGNDao5iLjAn5j8Y7pbbsR74/Zy\n5soVnmrV0uonvIT7aFiyIb6507maexubNplOY39aayZunYHH3iE2G/ZdioGN1StejwJ+uUktsY61\na02nsb+pO2eQunMQLzwvm5rIPqUUT9Z6ktKW6UyYYDqN/W0/t53L15J4PrQpnp62aUM+oTamlGJ4\nneEU6TCFiRNNp7GvtPQ0pmybSU09hJAQ02mEs3uq1lMc9PyZqCWJXLpkOo19/bh9BsnbBzN8uO2u\ny5ZiYAeDaw7miEcki1bc5MoV02nsZ83va0i+UozXhss4AiLnShcoTb0SdanVb6FbdeZMSUth5q7Z\nNPIdTFCQ7dqRYmAHRfMWpVWZllTpPZcZM0ynsZ9P10zGe/9wunUznUS4iuF1hpNU9UcmTIC0NNNp\n7GPRkUVwtSIvDqxo03akGNjJ8DrDia+UcajIHU4kX7p9iZjYZbz4xEAZlE5YTffK3fnt9g7yljzJ\nsmWm09jHVxsmkf7rSEJDbduOFAM76VyhMxfTjnHL5zCbN5tOY3tTtv8Eh0MZ9VSA6SjChfh6+dK/\nen8q9JrmFieST18/zebYTQyp28fmk0FJMbATb09vhtQcQtmeU13+RLLWmq83TKJa8tM2PcYp3NPw\nOsPZqaewbn06F7IcK9l1TNk5Fa9D/XlmWB6btyXFwI6G1RnGwVzTmL8wlRs3TKexnS1ntnDlehIv\ndH3CdBThguoUq0OB3Plp0CeGOXNMp7GddJ3Ot5snU+L809Subfv2pBjYUdUiVSlbsDRVwxYza5bp\nNLYzftMkUn99mj59ZHhSYX1/XK6dVmsyM2eaTmM7q4+vJvl6QV7oWdcu7UkxsLPn6z9PUs0JTJ1q\nOolt3Ey6yfyDEXQMfAp/f9NphKsaXHMwu+KXcDzuAkePmk5jGxO2TCJhw9MMHGif9qQY2Fmfan04\nlbqd49eOcfCg6TTW9/O+n/E934pnBmY1W6oQ1lEwd0F6VO5Bxb4/uuRe9oX4Cyw/toL2xQZSuLB9\n2pRiYGe+Xr4Mqz2MMr2/c7nx2bXWfL5+PHrLC3ToYDqNcHUvNHiB3wt9x4yZaS53ufbE7RPJe7I3\nzw4tYLc2pRgYMKreKI7lnca0mQkuNZXfulPruHQ1mSHN2+JtvXm6hchSvRL1KBUQSGKpJaxfbzqN\n9aSmp/L15u9Q216gY0f7tSvFwIDyBcvTOLgBuRvMZYULTQL69ZbxpG16keHD5MSxsI8XGr6AX+tv\n+P5700msJ/JQJB43yjAqrLbNBqXLihQDQ55v8Dy6/rcu0+fgzI0zLDuyigrxT9rlMjghAPpW68vl\nXDuIXHeMy5dNp7GOrzaP5+bqFxk+3L7tSjEwpHOFzujcF1hzeCunTplOk3M/bP+BQmcH8uJIuYRI\n2I+vly/D6wyjVI8JLnEObt+Ffew5e5jG+XtQpox925ZiYIinhyf/aPx3Art/7vS7uEmpSUz49Qeu\nrXyevn1NpxHu5rkGz3GmyFS+nXzD6U8kj986Hv+jo3hmRC67ty3FwKARdUdwId9Kfph7gqQk02my\n76c9P5EvvjbDulUlt0xzLOysTIEydA5pT3zlSU59Du5C/AVm7ZlD6qZn6dHD/u1LMTDI38efZ+qP\nIHfr/xAebjpN9qTrdMZu+IRLka/y/POm0wh39a+m/yK5zpeM+yzFdJRs+2brNxS73Jd/PB1oZKRf\nKQaG/b3R37kaPI2Pv7zmlLu4i44sIvG6H23Lt6KibYdbF+K+6peoT42S5dmZFM6uXabTPL745Hi+\n2TqBuAUv8/TTZjJIMTAsyD+I7lW7cqXsDyxZYjrN4xu7fhyJq1/l1VfkclJh1ivN/kXutp/wyafO\n963qx50/UvBWC4ZZKlGwoJkMUgwcwMtNXyax9n/48ONEp9o72Hh6I8fizlIprSeNG5tOI9xd54qd\n8Q9IJnrfak6cMJ3m0aWmp/LJhs+Ii3iVl14yl0OKgQOoXaw2TcvW5Zj/JNatM53m0X28/mN8drzM\nq/+SqcyEeR7KgzeeeI0Clg/49DPn+VY1Z98cPG8F071BI4KDzeWQYuAgRrd6l9TGH/PBR4mmozyS\nbWe3senEDnIfHCFzHAuHMbDGQLwLnmPa2l+cYuKbtPQ03l/7Adej3+Zf/zKbRYqBg6hfoj5Nytbm\n17TJbN9uOs3DjY4ZTdEjr/PKP33xkK1IOAgvDy9Gt34bv26j+eprx987+Hnfz6TfKkKjIu2oUcNs\nFvkYO5D3Wr+Lav4x/x7r2J0Otp7ZyrbY3Vxe+TSDB5tOI8RfDagxgNyF4/g6eg03b5pOc3+p6am8\nt/Y9Epe9x2uvmr8AQ4qBA2lQsgGNytRk5eVJHDpkOs39jY4ZTYnf3uDlf/ji62s6jRB/5eXhxfvt\n3iZXx3f5/nvH3TuYuWcmPkklCbzdmpYtTaeRYuBwPmr/ITzxIR9+4piTJK89sZZdZw5wdtEIXnjB\ndBohsjag+gD8il7mo3lLHLJ3f2JqIqPXjub24vd5522FMr9jIMXA0dQpXoeuldsTcX6cww1gl67T\neWnFS5Q8+BGvv+JDnjymEwmRNU8PT74O/YSklq8weYrjTRry1ZavKJRSi6IJTxAaajpNBikGDmhc\npw+hwQTe+CjWdJS/mLlnJkkJXpxZ0Z9Ro0ynEeLBulbsSpVSxXlj3kSHOndwMf4i4zaM48LMcYwZ\ng0PsFYAUA4cUnD+Y5xqMYv61txyma/3tlNv835r/w2PF57w3WsmAdMLhKaWY2Pszkpu8x3tjr5uO\nc8fomNHUYBAVC1aiTRvTaf4kxcBBjW73Oj5VVzL0nQ0O0St57PqxFE9vhPf5ZnafdEOI7KpdrDaW\nKl0Yv+89hzjsuiduD3P2zWXv1+/wxRem0/yVFAMH5e/jz3fdv+BQhVHMmpNsNMuhS4cY/+s3nPrh\nC776CrtOxSdETo0PG4tX7Zk8+dp2o1+s0nU6z0Q/Q+Wz/2ZI70LUrGkuS1akGDiwftX7UKdcMM9O\n+5xLl8xk0Frz7KJnqXnlHdo1DKJZMzM5hMiuInmL8GW3cWwOHMmccHMnk7/f9j0J8V78Fv40o0cb\ni3FfUgwcmFKKWYO+IbXhpzz1z2NGMkzaMYnzV25xaMYLfPmlkQhC5NiIek9SvXwAz0z+j5G5kk9f\nP807v7zD9Znf89V/PMif3/4ZHkaKgYMrG1CWD9q9zZoCg5k1x74Tdxy7cow3Vr/J7ZnT+O5bTwoV\nsmvzQliNUorZg74ntfHH9H1xv10PF6XrdJ5a+BTlL/2T5pWq0aeP/dp+HFIMnMBLzf9G7SoFGDnj\nQ7v1TE5NT2XI/CGUOv42netXIyzMPu0KYSsVClbg864fs6nEAMZ9br8BIT/f9DnnL6Rwdu5rjB9v\nt2Yfm9KOcKkKoJTSjpLFEZ27eY7KX9Yh/6rZ7Ituhb+/bdt7Y9UbLNiygzzzl7Jxg4cMOyFcgtaa\nrtP6EhNdjBX//JrmzW3b3pbYLXT5KRT9w1ZWzC1D/frWb0MphdY6x70VZM/ASRTPV5yIQT9xueUA\nLE+eINmGFxjN3T+XH7fO5trkmcyPkEIgXIdSipn9fiCg4TK6/t8Um+5pn7t5jp6ze5Nr2SQ+f8c2\nhcCapBg4kXbl2vF+x9fYGRLGwKG3SEuzfhs7z+3kmcgXSP5pPovDC1OmjPXbEMKkgNwBrB4RDe1e\np/VTGzh50vptJKYmEjarJ+m/juL5thaGDrV+G9YmxcDJvNT0H/Ru0og1gRb6Dkqw6iBcRy4fod2U\nrqRHTSBifB3q1bPeawvhSCoXrszcAdOJ79qLxmF7OHrUeq+dkpZC2E99OfprWQYE/R9vvWW917Yl\nKQZORinFD5YJtG9SjA1BPenYJYmLF3P+ukcvH6PJtx1IXv4Byz7v7VDd5IWwhY4VOjKp11ck9OpI\nk7B9rFmT89dMTkvGMnUI6zek8/dS0/jsU8cYkfRRZLsYKKX6KKX2K6XSlFJ1H7BcJ6XUIaXUUaXU\na9ltT/zJ08OTn3pN44lG/vzWvD21Gl9h9ersv17M4R3U+vIJfLb+HzsmjaBpU+tlFcKR9a3Wl2+7\nf0b64Hb0fnkd779Pts/H3Uy6RZ1PQlnz3wT+0yyc9971dppCADnbM9gL9AD+e78FlFKewHigE1AV\nGKCUqpKDNt1CTEzMQ5fx9vRmdu+f6d+sER7PNGbwK9sZOBB+++3R20lO1gz76kfaTulIy/jxHPl5\nJBUrZj+3LTzKunAXsi7+ZM11MbDGQGb3m45H/17M/X0CdetpFi/msfoi/Lx6H4FvNeTCkdJs/VcE\nTw91vpEcs10MtNaHtNZHHrJYQ+CY1vqE1joFmA3IFesP8agbuofy4JMOn/BJl/dI7d+Z2Epv07D5\nTXr1gnnz4EYW8+OkpcGePfC3d45T4NnuhMd+zk/t1rL08574+Vn3fViD/AH8k6yLP1l7XXQo34EN\nI9aRu+mPeA3txD/HHKJOHfjii4wvWFkVhvPn4duJCZR56gMGr2xN35KvcX7iD9Sq4WXVbPZi69Ql\ngdN3PY4FGtm4TbczoMYAWpRuwRur3+DwSxVJVCP5YsZgnnyqEsWLKYoVA19fuHglmd+SNuPZYBIp\nZRbzdMdX+LTXXHy8fEy/BSGMCykcwsbhG/li8xd8kt6CoDwdWLp3KJ+0bE1KkhdlykD+/HArXnP8\n6m/cKD0L1eB7atVqwsohv1KxSBnTbyFHHlgMlFIrgWJZ/OhNrXX0I7y+9CKzkyD/IGb0mMG+C/uY\nvGMycz3a4Nc8hQJ5KpGSnodLyVc5k3iY8gEVeLL2QIbX+ZKCuQuaji2EQ/H29ObVZq8yqt4opuya\nwsxcb3I9+ADl81fGk0LEJSdyPuk3PD1gaFULzzdYSs1ABxt+NJty3ANZKfUL8LLWekcWP2sMjNZa\nd8p8/AaQrrUem8WyUjiEECIbrNED2VqHie4XZBtQUSlVBjgL9AMGZLWgNd6MEEKI7MnJpaU9lFKn\ngcbAYqXU0sznSyilFgNorVOBF4HlwAFgjtb6YM5jCyGEsCaHGahOCCGEOcZ7ILtbpzSlVCml1C+Z\nHfb2KaX+nvl8QaXUSqXUEaXUCqVUgbt+543M9XNIKdXBXHrbUEp5KqV2KqWiMx+75bpQShVQSs1T\nSh1USh1QSjVy43Xxz8zPx16l1CyllI+7rAul1I9KqTil1N67nnvs966Uqpe5/o4qpf7z0Ia11sZu\ngCdwDCgDeAO7gComM9nhPRcDamfe9wMOA1WAccCrmc+/Bnyceb9q5nrxzlxPxwAP0+/DyuvkJWAm\nEJX52C3XBTANGJ553wvI747rgoxL0o8DPpmP5wBPucu6AFoAdYC9dz33OO/9jyM+W4GGmfeXAJ0e\n1K7pPQO365SmtT6vtd6Vef8WcJCMjd9Cxh8DMv/tnnk/DPhZa52itT5Bxn92Q7uGtiGlVBDQBZjE\nnxciuN26UErlB1porX+EjPNtWuvruOG6yOQF5FFKeQF5yLgAxS3WhdZ6HXD1nqcf5703UkoVB/Jp\nrbdmLjf9rt/JkulikFWntJKGsthd5lVWdYAtQKDWOi7zR3FAYOb9EmSslz+42jr6AngFSL/rOXdc\nF2WBi0qpKUqpHUqpiUqpvLjhutBanwE+A06RUQSuaa1X4obr4i6P+97vff4MD1knpouB2569Vkr5\nARHAP7TWN+/+mc7Yr3vQunGJ9aaU6gZc0Frv5D6XJ7vLuiDjm3Bd4FutdV0gHnj97gXcZV0opQLI\n+CZchow/an5KqcF3L+Mu6yIrj/Des8V0MTgDlLrrcSn+Ws1cklLKm4xCMENrvTDz6TilVLHMnxcH\nLmQ+f+86Csp8zhU0BSxKqd+Bn4E2SqkZuOe6iAVitda/Zj6eR0ZxOO+G66Id8LvW+rLOuDx9PtAE\n91wXf3icz0Rs5vNB9zz/wHViuhjc6ZSmlMpFRqe0KMOZbEoppYDJwAGt9Zd3/SiKjJNkZP678K7n\n+yulcimlygIVyTgx5PS01m9qrUtprcsC/YE1WushuOe6OA+cVkpVynyqHbAfiMbN1gVwEmislMqd\n+XlpR0Y/JXdcF394rM9E5vZ0I/OKNAUMuet3suYAZ847k3FFzTHgDdN57PB+m5NxfHwXsDPz1gko\nCKwCjgArgAJ3/c6bmevnENDR9Huw0XppyZ9XE7nlugBqAb8Cu8n4NpzfjdfFaDIurthLxglTb3dZ\nF2TsJZ8Fksk4pzosO+8dqJe5/o4BXz2sXel0JoQQwvhhIiGEEA5AioEQQggpBkIIIaQYCCGEQIqB\nEEIIpBgIIYRAioEQQgikGAghhAD+P4av4eGhyXqGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x103f1cac8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_predicted, = plt.plot(predicted, label='predicted')\n",
    "plot_test, = plt.plot(y['test'], label='test')\n",
    "plt.legend(handles=[plot_predicted, plot_test])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}