2-CNN_Model_Development&Hyperparameter Optimization.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A Transfer Learning and Optimized CNN Based Intrusion Detection System for Internet of Vehicles \n",
    "This is the code for the paper entitled \"**A Transfer Learning and Optimized CNN Based Intrusion Detection System for Internet of Vehicles**\" published in **IEEE International Conference on Communications (IEEE ICC)**, doi=[10.1109/ICC45855.2022.9838780](https://ieeexplore.ieee.org/document/9838780).    \n",
    "Authors: Li Yang (lyang339@uwo.ca) and Abdallah Shami (Abdallah.Shami@uwo.ca)  \n",
    "Organization: The Optimized Computing and Communications (OC2) Lab, ECE Department, Western University\n",
    "\n",
    "**Notebook 2: CNN Model Development**  \n",
    "Aims:  \n",
    "&nbsp; 1): Generate training and test images  \n",
    "&nbsp; 2): Construct CNN models (a CNN model by own, Xception, VGG16, VGG19, Resnet, Inception, InceptionResnet)  \n",
    "&nbsp; 3): Tune the hyperparameters of CNN models (hyperparameter optimization)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "from keras.preprocessing.image import  ImageDataGenerator\n",
    "from keras.layers import Dense,Flatten,GlobalAveragePooling2D,Input,Conv2D,MaxPooling2D,Dropout\n",
    "from keras.models import Model,load_model,Sequential\n",
    "from keras.applications.xception import  Xception\n",
    "from keras.applications.vgg16 import VGG16\n",
    "from keras.applications.vgg19 import VGG19\n",
    "from keras.applications.resnet50 import  ResNet50\n",
    "from keras.applications.inception_v3 import InceptionV3\n",
    "from keras.applications.inception_resnet_v2 import InceptionResNetV2\n",
    "from keras.applications.mobilenet import MobileNet\n",
    "import keras.callbacks as kcallbacks\n",
    "import keras\n",
    "from keras.preprocessing.image import load_img,img_to_array\n",
    "import math\n",
    "import random\n",
    "from keras.utils import plot_model\n",
    "import matplotlib.pyplot as plt\n",
    "import sklearn.metrics as metrics\n",
    "import numpy as np\n",
    "from PIL import Image\n",
    "import sklearn.metrics as metrics\n",
    "from sklearn.metrics import accuracy_score,precision_score,recall_score,f1_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate Training and Test Images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 23382 images belonging to 5 classes.\n",
      "Found 5845 images belonging to 5 classes.\n"
     ]
    }
   ],
   "source": [
    "#generate training and test images\n",
    "TARGET_SIZE=(224,224)\n",
    "INPUT_SIZE=(224,224,3)\n",
    "BATCHSIZE=128\t#could try 128 or 32\n",
    "\n",
    "#Normalization\n",
    "train_datagen = ImageDataGenerator(rescale=1./255)\n",
    "\n",
    "test_datagen = ImageDataGenerator(rescale=1./255)\n",
    "\n",
    "train_generator = train_datagen.flow_from_directory(\n",
    "        './train_224/',\n",
    "        target_size=TARGET_SIZE,\n",
    "        batch_size=BATCHSIZE,\n",
    "        class_mode='categorical')\n",
    "validation_generator = test_datagen.flow_from_directory(\n",
    "        './test_224/',\n",
    "        target_size=TARGET_SIZE,\n",
    "        batch_size=BATCHSIZE,\n",
    "        class_mode='categorical')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define the image plotting functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#plot the figures\n",
    "class LossHistory(keras.callbacks.Callback):\n",
    "    def on_train_begin(self, logs={}):\n",
    "        self.losses = {'batch':[], 'epoch':[]}\n",
    "        self.accuracy = {'batch':[], 'epoch':[]}\n",
    "        self.val_loss = {'batch':[], 'epoch':[]}\n",
    "        self.val_acc = {'batch':[], 'epoch':[]}\n",
    "    def on_batch_end(self, batch, logs={}):\n",
    "        self.losses['batch'].append(logs.get('loss'))\n",
    "        self.accuracy['batch'].append(logs.get('acc'))\n",
    "        self.val_loss['batch'].append(logs.get('val_loss'))\n",
    "        self.val_acc['batch'].append(logs.get('val_acc'))\n",
    "    def on_epoch_end(self, batch, logs={}):\n",
    "        self.losses['epoch'].append(logs.get('loss'))\n",
    "        self.accuracy['epoch'].append(logs.get('acc'))\n",
    "        self.val_loss['epoch'].append(logs.get('val_loss'))\n",
    "        self.val_acc['epoch'].append(logs.get('val_acc'))\n",
    "    def loss_plot(self, loss_type):\n",
    "        iters = range(len(self.losses[loss_type]))\n",
    "        plt.figure()\n",
    "        plt.plot(iters, self.losses[loss_type], 'g', label='train loss')\n",
    "        if loss_type == 'epoch':\n",
    "            # acc\n",
    "            plt.plot(iters, self.accuracy[loss_type], 'r', label='train acc')\n",
    "            # loss\n",
    "            plt.plot(iters, self.losses[loss_type], 'g', label='train loss')\n",
    "            # val_acc\n",
    "            plt.plot(iters, self.val_acc[loss_type], 'b', label='val acc')\n",
    "            # val_loss\n",
    "            plt.plot(iters, self.val_loss[loss_type], 'k', label='val loss')\n",
    "        plt.grid(True)\n",
    "        plt.xlabel(loss_type)\n",
    "        plt.ylabel('acc-loss')\n",
    "        plt.legend(loc=\"upper right\")\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "history_this= LossHistory()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Construct CNN models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Model 1: a CNN model by own (baseline)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def cnn_by_own(input_shape,num_class,epochs,savepath='./model_own.h5'):\n",
    "    model = Sequential()\n",
    "    model.add(Conv2D(64,(3,3),strides=(1,1),input_shape=input_shape,padding='same',activation='relu',kernel_initializer='glorot_uniform'))\n",
    "    model.add(Conv2D(64,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='glorot_uniform'))\n",
    "    model.add(MaxPooling2D(pool_size=(2,2)))\n",
    "    model.add(Conv2D(128,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='glorot_uniform'))\n",
    "    model.add(Conv2D(128,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='glorot_uniform'))\n",
    "    model.add(MaxPooling2D(pool_size=(2,2)))\n",
    "    model.add(Conv2D(256,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='glorot_uniform'))\n",
    "    model.add(Conv2D(256,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='glorot_uniform'))\n",
    "    model.add(Conv2D(256,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='glorot_uniform'))\n",
    "    model.add(GlobalAveragePooling2D())\n",
    "    model.add(Dense(256,activation='relu'))\n",
    "    model.add(Dropout(rate=0.5))\n",
    "    model.add(Dense(num_class,activation='softmax'))\n",
    "    model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])\n",
    "    #train model\n",
    "    earlyStopping=kcallbacks.EarlyStopping(monitor='val_acc', patience=2, verbose=1, mode='auto')\n",
    "    saveBestModel = kcallbacks.ModelCheckpoint(filepath=savepath, monitor='val_acc', verbose=1, save_best_only=True, mode='auto')\n",
    "    hist=model.fit_generator(\n",
    "        train_generator,\n",
    "        steps_per_epoch=len(train_generator),\n",
    "        epochs=epochs,\n",
    "        validation_data=validation_generator,\n",
    "        validation_steps=len(validation_generator),\n",
    "        callbacks=[earlyStopping,saveBestModel,history_this],\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "182/183 [============================>.] - ETA: 1s - loss: 0.0588 - acc: 0.9784Epoch 00001: val_acc improved from -inf to 0.99884, saving model to ./model_own.h5\n",
      "183/183 [==============================] - 303s 2s/step - loss: 0.0585 - acc: 0.9785 - val_loss: 0.0040 - val_acc: 0.9988\n",
      "Epoch 2/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0293 - acc: 0.9883Epoch 00002: val_acc did not improve\n",
      "183/183 [==============================] - 125s 685ms/step - loss: 0.0293 - acc: 0.9883 - val_loss: 0.0204 - val_acc: 0.9883\n",
      "Epoch 3/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0204 - acc: 0.9887Epoch 00003: val_acc did not improve\n",
      "183/183 [==============================] - 126s 686ms/step - loss: 0.0205 - acc: 0.9886 - val_loss: 0.0185 - val_acc: 0.9846\n",
      "Epoch 00003: early stopping\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8FPW9//HXZ3dzARJCAMULKqF6tFxCMCj0oFyO1YL8CngrtuL9cvydaqWeeorXWv2dU6u2tVp6rLV60FpRa7V4pNLSkxhtxYoWBRFbQD2EWuUWSAi57X5/f8xk2IRcdpOdJOj7yWMfM/Od78x8drLMZ78zs98x5xwiIiIAkd4OQERE+g4lBRERCSgpiIhIQElBREQCSgoiIhJQUhARkYCSgoiIBJQUREQkoKQgIiKBWG8HkK6hQ4e6ESNGdGnZPXv2MGDAgMwGlAGKKz2KK319NTbFlZ7uxPX6669vc84d1GlF59wB9SotLXVdVVZW1uVlw6S40qO40tdXY1Nc6elOXMAql8IxVqePREQkoKQgIiIBJQUREQkccBeaReSTq7GxkcrKSurq6no1joKCAt55551ejaEtqcSVm5vL8OHDycrK6tI2QksKZvYQ8H+Aj51zY9qYb8APgdOBWuAi59wbYcUjIn1fZWUl+fn5jBgxAu8Q0Tuqq6vJz8/vte23p7O4nHNs376dyspKioqKurSNME8f/Rcwo4P5M4Fj/NcVwH+GGIuIHADq6uoYMmRIryaEA5mZMWTIkG61tEJLCs65CmBHB1XmAI/4d0utBAaZ2aFhxSMiBwYlhO7p7v7rzWsKhwObk6Yr/bIPw9jYyy/Dww+PoKICYjGIRlMbplO3K+uI6FK/iPQhB8SFZjO7Au8UE8OGDaO8vDztdSxZcgSPPPKZDEeWGdHoFCKRBNGoC16RyP7jkQj7zets/v51abVM+/Pj8YN56qkNbdRtex3tzU+lTkfzW3/xqamp6dJnIGx9NS7ou7G1jqugoIDq6upei6eqqoqnnnqKSy65JO04zjrrLH72s58xaNCglOr/x3/8B3l5eXzta19LeRvxeDyluOrq6rr89+7NpLAFOCJperhfth/n3APAAwATJkxw06ZNS3tj06bBvHnlTJkyjXgcmprodJhKnUysY9OmzRx++FEZ3X59fdfWkUikvWtDF4m0bF0510hublYbrS9HLNo8dEQjEAvGHbFowhtGHNFIImmYIGrOHyb2DS3uT8e9cbzxKAli1uQP40SJE6OJbR/v5ZBh+UTMYeYwwAwM55d542aOSDC+b9iiTvIySeuKmAvm77dMW9vCS6y1mzcz9MgjWi3nMJfGeqzjeL1tdbyOFu/BYNOmjRxz9MhgeuOUSQzYU4vhgr9/T55M2v73v/PQAw/wL2edRW5ODrh9cTQ1NRGLtX/I/O3ixd5ITU1K28ppaCCnvp78FOsD7InFGJBC0snNzWX8+PEprzdZbyaFpcBVZrYEmAjscs6FcuqomRn+N1DIzg5zS+kpL3+PadOO2leQSOw7Sje/kqc7mtfZdCd1XWMT8cYETY2Od9dvYMQRI2hqdMSbHE0NCW/YxL7pON785mGToykO8UZHU9z2Tz5x2zedsH3TcfOmE0Y87g2b4hG/LGnYFKGhCVx9lCYX9cpdlLgzmogRJ9pimDze0Gpe58Oc/dYRPzAa1wes3/zmHWorh7UxJzlJuBZzOpq2/ZZtOd26/r/d+DU2vP8BJafOZtLEf2LKSTO5/6f/ztChh/DuX97i10++wTXf+BJ//6iS+vp65p/7L3zpzEsAx6mzR/HkIxXsrd3DP19zBseP+xx/futVhh18GIvufpzc3H4tYthWnU1tUzZ/2TKAd959k2/dcS1762o5cngR37n5RxQMHMQjS37C4796mFg0ytFFx/Lwj3/CqnUvcs0113jrMqOioiKjd0qFeUvq48A0YKiZVQLfArIAnHP3A8vwbkfdgHdL6sVhxQLAq69y5GOPeRcXMngQzcSyJzc2eomgubwXGd6HIgaMa69S8lf05ldH0y3Go5CbYt12piv//neGH3lkl5btbl0XiZKItJFALMYfX32dE06ciHPen9O5lq9EAu/7svOG7dZx++p5y4Bz1nGd/baTtAywZs3bjBo1Zv91pLSefetKuFbTLdbjlbe5nna2tWnTe4wYURRMDypwHHao11y95ZWvs277m60O4Slw7U+OGjyOW074QatP/L4GwY03fZdNl61j6XOvEYtlsXJlOWvXrWLZsjUcMdy7xfO7d/8XgwYNpm7vXs4860ROn3suhYVDsIgRy+tPJJLgg80b+f49S/j340q4ZsE8fvfKcmbPng8uKZ7sbMjOIT4gn3+77avceOO9nHDCVO679xZ++PAPuP76e/jJIz/kt8s3kZWdw+7dVTT0y+Huu+9m0aJFTJ48mZqaGnJzc9PdQx0KLSk4577cyXwHfDWs7e+nooKRDz64b7q52ZCpA0lOTpcPOn/72984oqgo/INbmsu+9MornDxtWst5feDK+IbycoZ34RRiJhgQ9V+tG5sDtw3g0LFDez6oFGQPqWfatL533315+UdMm/bZYPqdd4zDDvc+Y3l5RnbqZ1ZSMnCgMaKo/c+wRYzsbDj0sHry87P5383GxIknMm3ayKDOrbfexzPPPAPARx9tJpHYwDHHDCUWg88cbdTUGEVFRcyZ452+mTatlPr6Dzj22JYnwoYONfLyjMMO283evVVccME0AP71GxdxzjnnMGqUcfzxxfy/f5/P3LlzmTt3Ls7FmTx5Mtdeey3nnXceZ555JsOHD8/oPvr0tIUXLKBi3DimTJ/eZw5uzTaWl3NELx3kOhLPy4O8vN4OQz6l7plxT2+HANCiq+ry8nJWrFjBK6+8Qv/+/Zk2bVqbvwnIyckJxqPRKHv37u3Stp9//nkqKipYunQpt99+OytXrmThwoXMmjWLZcuWMWnSJFasWMFxxx3XpfW3pe8cGcOWlUUiOxuysvpUQhCRviM/P7/Du3t27dpFYWEh/fv3Z/369axcubLb2ywoKKCwsJCXXnoJgEcffZSpU6eSSCTYvHkz06dP584776Sqqoqamho2btzI2LFj+eY3v8mECRNYv359t2NI9ulpKYiIdGLIkCFMnjyZiRMnMmvWLGbNmtVi/owZM7j//vspLi7m2GOPZdKkSRnZ7uLFi7nyyiupra1l5MiRPPzww8TjcebPn8+uXbtwzvH1r3+dQYMGccMNN1BWVkYkEmH06NHMnDkzIzE0U1IQEUnyi1/8okUfQ8m3wOfk5PCb3/ymzeXef/99AIYOHcratWuD8m984xtt1r/11luD8ZKSkjZbHS+//HKL6erqau67775U3kaX6TyKiIgElBRERCSgpCAiIgElBRERCSgpiIhIQElBREQCSgoiIr6qqip+/OMfd2nZ008/naqqqgxH1POUFEREfB0lhaampg6XXbZsWcrPUujLlBRERHwLFy5k48aNTJ48meuuu47y8nKmT5/OV77yFYqLiwGYO3cupaWljB49mgceeCBYdsSIEWzbto3333+fz372s1x++eWMHj2a0047rc2+j5577jkmTpzI+PHj+fznP89HH30EeA8euvjiixk7dizFxcU8/fTTALzwwgucfPLJjBs3jlNOOSW0faBfNItI37RgAaxendl1lpTAPe13tHfHHXewdu1a/vCHP5Cfn095eTl/+tOfWLt2LUVFXtfZDz30EIMHD2bv3r2ccMIJnHXWWQwZMqTFev7617/y+OOP89Of/pQvfelLPP3008yfP79FnZNOOomVK1diZjz44IPceeedfO973+P222+noKCANWvWALBz5062bt3K5ZdfzrJlyxg7diw7duzI7H5JoqQgItKBE088MUgIAPfee2/QdfbmzZv561//ul9SKCoqoqSkBIDS0tKgC4xklZWVzJs3jw8//JCGhoZgGytWrGDJkiVBvcLCQp577jmmTJnCiBEjABg8eHAm32ILSgoi0jd18I2+J4XVdfbVV1/Ntddey+zZsykvL2/RF1Jv0jUFERFfT3advWvXLg4//HDA6yW12amnnsqiRYuC6Z07dzJp0iQqKiqCFkeYp4+UFEREfMldZ1933XX7zZ8xYwZNTU0UFxdz8803d6vr7FtvvZVzzjmHk08+maFD9z2x76abbmLnzp2MGTOGcePGUVZWxkEHHcQDDzzA/PnzGTduHPPmzevydjuj00ciIkl6quvsOXPmMGfOnP3K8/LyWrQcms2cOZOTTjopiCssaimIiEhASUFERAJKCiIiElBSEBGRgJKCiIgElBRERCSgpCAi4uvJrrNvvfVW7r777i5tK0xKCiIiPnWdraQgIhLoya6zk61evZpJkyZRXFzMGWecwc6dOwGv871Ro0ZRXFzMueeeC8CLL75ISUkJJSUljB8/vsNuObpCv2gWkT5pwQsLWP33zHadXXJICffM6BtdZye74IILuO+++5g6dSq33HIL3/72t7nnnnu44447eO+998jJyQlOTd19990sWrSIyZMnU1NTQ25ubgb2zD5qKYiIdKCtrrPHjRvHpEmTgq6zW0ul6+xmu3btoqqqiqlTpwJw4YUXUlFRAUBxcTHnnXceP//5z4nFvO/wkydP5tprr+Xee++lqqoqKM8UtRREpE/q6Bt9Twqr6+xUPP/881RUVLB06VJuv/12Vq5cycKFC5k1axbLli1j0qRJrFixguOOO65L629LqC0FM5thZu+a2QYzW9jG/CPNrMzM/mxmb5nZ6WHGIyLSkZ7sOrtZQUEBhYWFvPTSSwA8+uijTJ06lUQiwebNm5k+fTp33nknVVVV1NTUsHHjRsaOHcs3v/lNJkyYwPr167sdQ7LQWgpmFgUWAacClcBrZrbUObcuqdpNwJPOuf80s1HAMmBEWDGJiHQkuevsWbNmMWvWrBbzZ8yYwf33309xcTHHHntst7rOTrZ48WKuvPJKamtrGTlyJA8//DDxeJz58+eza9cunHN8/etfZ9CgQdxwww2UlZURiUQYPXo0M2fOzEgMzcI8fXQisME5twnAzJYAc4DkpOCAgf54AfC3EOMREelUT3WdnfyktZKSkjZbHS+//HKL6erqau67775U3kaXhZkUDgc2J01XAhNb1bkV+K2ZXQ0MAD4fYjwiItIJc86Fs2Kzs4EZzrnL/OnzgYnOuauS6lzrx/A9M/sc8DNgjHMu0WpdVwBXAAwbNqw0+aHW6aipqSEvL69Ly4ZJcaVHcaWvr8bWOq6CggKOPvroXozIE4/HiUajvR3GflKNa8OGDezatatF2fTp0193zk3odGHnXCgv4HPA8qTp64HrW9V5GzgiaXoTcHBH6y0tLXVdVVZW1uVlw6S40qO40tdXY2sd17p163onkFZ2797d2yG0KdW42tqPwCqXwrE7zLuPXgOOMbMiM8sGzgWWtqrzv8ApAGb2WSAX2BpiTCIi0oHQkoJzrgm4ClgOvIN3l9HbZnabmc32q/0rcLmZvQk8DlzkZzQREekFof54zTm3DO820+SyW5LG1wGTw4xBRERSp24uRES6oS9ewO8OJQUREQkoKYiI+BYuXMiiRYuC6eYH4dTU1HDKKadw/PHHM3bsWH796193uq72uth+4YUXOP744xk3bhynnHIK4N2ae/HFFzN27FiKi4t5+umnM//mUqQO8USkT1qwAFZntudsSkrgng762Zs3bx4LFizgggsuAODJJ59k+fLl5Obm8swzzzBw4EC2bdvGpEmTmD17NmbW7rra6mI7kUhw+eWXU1FRQVFRETt27ADg9ttvp6CggDVr1gAEz1PoDUoKIiK+8ePH8/HHH/Phhx+yadMmCgsLOeKII2hsbOSGG26goqKCSCTCli1b+OijjzjkkEPaXde9997LM888AxB0sb1161amTJkSdMU9ePBgAFasWEHyj3ILCwtDfJcdU1IQkT6po2/0YTrnnHN49tlnqaqqYt68eQA89thjbN26lddff52srCxGjBjRZpfZzVLtYrsv0jUFEZEk8+bN4+mnn+aXv/wl55xzDuB1mX3wwQeTlZVFWVkZH3zwQYfraK+L7UmTJlFRUcF7770HEJw+OvXUU1tcy+jN00dKCiIiSUaPHk1NTQ2HH344hx56KADnnXceq1atYsKECTz22GOdPtRmxowZNDU1UVxczM033xx0sX3QQQfxwAMPcOaZZzJu3LigJXLTTTexc+dOxowZw7hx4ygrKwv3TXZAp49ERFpZuXJl0HU2eN1hv/LKK23Wramp2a+soy62Z86cud8zEPLy8li8eHE3Is4ctRRERCSgpCAiIgElBRERCSgpiIhIQElBREQCSgoiIhJQUhAR6Yb2us4+ULvUVlIQEZGAkoKIiC+TXWc3c85x3XXXMWbMGMaOHcsTTzwBwIcffsiUKVMoKSlhzJgxvPTSS8TjcS666KKg7g9+8IOMv8fO6BfNItInLViwgNUZ7ju7pKSEezroaS+TXWc3+9WvfsXq1at588032bZtGyeccAJTpkzhF7/4BV/4whe48cYbicfj1NbWsnr1arZs2cLatWsBqKqqyswbT4OSgoiIL5NdZzd7+eWX+fKXv0w0GmXYsGFMnTqV1157jRNOOIFLLrmExsZG5s6dS0lJCSNHjmTTpk1cffXVzJo1i9NOO60H3nVLSgoi0id19I0+TJnoOjsVU6ZMoaKigueff57zzz+f6667jgsuuIA333yT5cuXs2jRIp588kkeeuihTLytlOmagohIkkx0nZ3s5JNP5oknniAej7N161YqKio48cQT+eCDDxg2bBiXX345l156KW+88Qbbtm0jkUhw1llncfvtt/PGG2+E9TbbpZaCiEiS9rrO/uIXv8iECRMoKSnptOvsZGeccQavvPIK48aNw8y48847OeSQQ1i8eDF33XUXWVlZ5OXl8cgjj7BlyxYuvvhiEokEAN/5zndCeY8dUVIQEWmlu11nJ5ebGXfddRd33XVXi/kXXnghF1544X7L9UbrIJlOH4mISEBJQUREAkoKItKnOOd6O4QDWnf3n5KCiPQZubm5bN++XYmhi5xzbN++ndzc3C6vQxeaRaTPGD58OJWVlWzdurVX46irq+vWgTUsqcSVm5vL8OHDu7wNJQUR6TOysrIoKirq7TAoLy9n/PjxvR3GfnoiLp0+EhGRQKhJwcxmmNm7ZrbBzBa2U+dLZrbOzN42s1+EGY+IiHQstNNHZhYFFgGnApXAa2a21Dm3LqnOMcD1wGTn3E4zOziseEREpHNhthROBDY45zY55xqAJcCcVnUuBxY553YCOOc+DjEeERHphIV165eZnQ3McM5d5k+fD0x0zl2VVOdZ4C/AZCAK3Oqce6GNdV0BXAEwbNiw0iVLlnQpppqamj75iDzFlR7Flb6+GpviSk934po+ffrrzrkJnVZ0zoXyAs4GHkyaPh/4Uas6/w08A2QBRcBmYFBH6y0tLXVdVVZW1uVlw6S40qO40tdXY1Nc6elOXMAql8KxO6XTR2Y22cwG+OPzzez7ZnZUJ4ttAY5Imh7ulyWrBJY65xqdc+/htRqOSSUmERHJvFSvKfwnUGtm44B/Az4AHulkmdeAY8ysyMyygXOBpa3qPAtMAzCzocA/AJtSjElERDIs1aTQ5Dc/5gA/dM79EMjvaAHnXBNwFbAceAd40jn3tpndZmaz/WrLge1mtg4oA65zzm3vyhsREZHuS/WW1Gozux6YD0wxswjedYAOOeeWActald2SNO6Aa/2XiIj0slRbCvOAeuBS59zf8a4P3NXxIiIicqBJuaWAd9oobmb/ABwHPB5eWCIi0htSbSlUADlmdjjwe+Bi4L/CCkpERHpHqknBnHO1wJnAfc65M4Ax4YUlIiK9IeWkYGafA84Dnk9zWREROUCkemBfgNdx3TP+baUj8W4hFRGRT5CULjQ7514EXjSzPDPLc85tAr4WbmgiItLTUu3mYqyZ/Rl4G1hnZq+b2ehwQxMRkZ6W6umjnwDXOueOcs4dCfwr8NPwwhIRkd6QalIY4JwLriE458qBAaFEJCIivSbVH69tMrObgUf96fmo4zoRkU+cVFsKlwAHAb/yXwf5ZSIi8gmS6t1HO9HdRiIin3gdJgUzew5o93mdzrnZ7c0TEZEDT2cthbt7JAoREekTOkwK/o/WWjCz451zb4QXkoiI9Jau9F/0YMajEBGRPqErScEyHoWIiPQJXUkK3854FCIi0iek2vfRGWZWAOCce9bMBpnZ3HBDExGRnpZqS+FbzrldzRPOuSrgW+GEJCIivSXVpNBWvVS7yBARkQNEqklhlZl938w+47++D7weZmAiItLzUk0KVwMNwBPAEqAO+GpYQYmISO9Ite+jPcDCkGMREZFelurdR78zs0FJ04Vmtjy8sEREpDekevpoqH/HERD0mnpwOCGJiEhvSTUpJMzsyOYJMxtBB72niojIgSnV20pvBF42sxfxurk4GbgitKhERKRXpHqh+QUzm4CXCP4MPAvsDTMwERHpeSklBTO7DLgGGA6sBiYBrwD/FF5oIiLS01K9pnANcALwgXNuOjAe2NrZQmY2w8zeNbMNZtbuLa1mdpaZOb81IiIivSTVpFDnnKsDMLMc59x64NiOFjCzKLAImAmMAr5sZqPaqJePl3ReTSdwERHJvFSTQqX/O4Vngd+Z2a+Bv3WyzInABufcJudcA94voee0Ue924Lt4v5IWEZFeZM6ld2epmU0FCoAX/IN9e/XOBmY45y7zp88HJjrnrkqqczxwo3PuLDMrB77hnFvVxrquwL/badiwYaVLlixJK+ZmNTU15OXldWnZMCmu9Ciu9PXV2BRXeroT1/Tp0193znV+it45F8oLOBt4MGn6fOBHSdMRoBwY4U+XAxM6W29paanrqrKysi4vGybFlR7Flb6+GpviSk934gJWuRSO3V158lqqtgBHJE0P98ua5QNjgHIzex/vjqalutgsItJ7wkwKrwHHmFmRmWUD5wJLm2c653Y554Y650Y450YAK4HZro3TRyIi0jNCSwrOuSbgKmA58A7wpHPubTO7zcxmh7VdERHpulCfnuacWwYsa1V2Szt1p4UZi4iIdC7M00ciInKAUVIQEZGAkoKIiASUFEREJKCkICIiASUFEREJKCmIiEhASUFERAJKCiIiElBSEBGRgJKCiIgElBRERCSgpCAiIgElBRERCSgpiIhIQElBREQCSgoiIhJQUhARkYCSgoiIBJQUREQkoKQgIiIBJQUREQkoKYiISEBJQUREAkoKIiISUFIQEZGAkoKIiASUFEREJKCkICIiASUFEREJhJoUzGyGmb1rZhvMbGEb8681s3Vm9paZ/d7MjgozHhER6VhoScHMosAiYCYwCviymY1qVe3PwATnXDHwS+DOsOIREZHOhdlSOBHY4Jzb5JxrAJYAc5IrOOfKnHO1/uRKYHiI8YiISCfCTAqHA5uTpiv9svZcCvwmxHhERKQT5pwLZ8VmZwMznHOX+dPnAxOdc1e1UXc+cBUw1TlX38b8K4ArAIYNG1a6ZMmSLsVUU1NDXl5el5YNk+JKj+JKX1+NTXGlpztxTZ8+/XXn3IROKzrnQnkBnwOWJ01fD1zfRr3PA+8AB6ey3tLSUtdVZWVlXV42TIorPYorfX01NsWVnu7EBaxyKRxjwzx99BpwjJkVmVk2cC6wNLmCmY0HfgLMds59HGIsIiKSgtCSgnOuCe+U0HK8lsCTzrm3zew2M5vtV7sLyAOeMrPVZra0ndWJiEgPiIW5cufcMmBZq7JbksY/H+b2RUQkPfpFs4iIBJQUREQkoKQgIiIBJQUREQkoKYiISEBJQUREAkoKIiISUFIQEZGAkoKIiASUFEREJKCkICIiASUFEREJKCmIiEhASUFERAJKCiIiElBSEBGRgJKCiIgElBRERCSgpCAiIgElBRERCSgpiIhIQElBREQCsd4OoKfcUnYLD7z6AIPWDiI3lku/WD/6ZfWjf1Z/8rLzyMvOY0DWAAbmDGRgzkAKcgsoyClgUO4gBvcbzKDcQQzpN4RBuYOIRJRLReST6VOTFN7f+T7bGraxdcdWEi7R7fUZRsQiRC1KNBIlFomRFc0iO5pNdjSbnGgOubFccmO59M/qT79YPwZkD/CST/YABmbvSz6VH1VS824Nhf0KGdxvsDfMHUx2LDsD71xEJHWfmqTwyJmPcMngS5g2bVpQVtdUx47aHezYu4Oq+ipvWFfFrrpd7K7fze763VQ3VFNTX8Oexj3sadxDbWMtexv3UtdUR128jvqmehriDcGrtrGWeCJOwiVIuAQOl1qA69suNgwza5F8YpEY2ZFssmNeAsqN5tIvq1/Q+hmQNYAB2QPIz84nLzuP/Jz8oNUzMGdgkHwG9xvM4NzB9M/u3/0dLCKfCJ+apNCW3Fguhw08jMMGHhbqdpoSTVTVVbG9dnuQeJpf1Q3VvPXuWww+dDDVDdXsadjDngYv+dQ21lIXr6OuyUs+9fF6GuONNCYaqW6oJl4fDxJQysmnAxGLBK2fWCQGCej3Wr+g5ZMTzSE3yzv11tz6aT71lp/jJaCCnAIKcr0EVJBTQGG/Qob0G0Jhv0IGZg/UqTeRPu5TnRR6SiwSY2j/oQztP7TN+eWN5S1aMF2VSCSoaahh+97tbN+7nV11u9hZtzNo/VTXV7O7YTfV9dXUNNRQ21gbtH7qmurY27S3RfKprfdaRdX11cTdvtZPdwWn3iJRohYlK5rltX6ak0/MS0DNyad/Vn/6Z3vXfvKz89n+4XZWxlYGrZ+C3AIKc73Wz5D+3nWfWEQfbZGu0P+cT5BIJMLA3IEMzB1IUWFRt9dXXt5+sqptqGVHnXfqbcfeHeyq29Wi9bO73ks+1Q3VXvLxWz97m/xTb0111MfraWhqoDHR2OLUW9zFcc513PrZ3Hn8zcmnOQHFIjGyIllkRbNaJJ/m6z7Nr7ysPPJyvATUfOqtufUzKGfQvlNv/QeTG8vt2s4V6aOUFKRL+md7396HDxwe6nYamhq8U297vVNvu+p38cc3/shhIw/zWj8N1UHyaW79BKfekpOi+HX7AAALo0lEQVSPf82nMd5IXVMdVYmqoPWTqVNvhhF5KYKZBdeCkocR8+YFdf2E1VwWTCfdxBCJRFqc1ktuYSXf6NDWMBaJeeORKNu3bufBHQ8G16RiFiMajRKzGLFoLBhmWRaRSISsSFZw80RzMm2ebk6sbQ2zo9nBdPNNF1mRLO/6V/N1sEi2TiP2YUoK0qdlx7I5OO9gDs47OCjrv6U/006YltHtJBIJdjfsDq77NCeg4NRbq+Szp2EPe+r3sKduD7X1tdTX11O1u4rs3GwSiX03GSQPEy6BM+e1hJLmBf+cw5nb10oy9hsHwFoFb52MNw+3ZnSXZc6L3sCSgjezoCxIrq0TbdJ4cjINkiwRIpGWCTiVZGsYe2r2ULixsNOEG4vE9pvu6JVKso1adF+ijbZMtFUNVaH/OZQUpFc552hsbKSxsZGGhoaUhqtWraK6ujrl+umsO51hItH96ys9rflg6k3sK0uen1zunGtRFpTjMPyhPz+5bjAOQVLbt4J9owmX2G/9rZOew0uWrcv2W8bRsp61qt963R0l147qpjIe0vLFpxUz97S5hCnUpGBmM4AfAlHgQefcHa3m5wCPAKXAdmCec+79MGP6JHLOEY/HM37gW79+PS+99FJoB9XGxkaamppC3TeRSITs7GyysrJSGg4YMKDTeu3N27RpE8cee6z37d7/Vt96vK2ydOZ3dV3vv/8+Rx11VEZjyUSsf/vb3zj00ENDjyXdWLdu3cqQIUMArxUZT8RJkCART5DAn/ZbhAmX8Mbxhs659stcIthe83jCJXCJpNZi8vzmMn/+qAGjCFtoScHMosAi4FSgEnjNzJY659YlVbsU2OmcO9rMzgW+C8wLK6bOxOPxUA58ndXZsmUL9913X7e2Ebbs7OyUD665ubnk5+enfDBNd7hmzRomTZrU6QE7KyuLaDQa+r5p1tGF+d7WV2NTXOkpLy8PfRththROBDY45zYBmNkSYA6QnBTmALf6478EfmRm5oKTp5nz0EMP8e1vf5tYLNbuATvs0wHtHRjj8TgDBw7cr3zAgAEZPZim++135cqVnHLKKUSj0RanGHpbIpGgtLS0t8MQ+UQKMykcTssbByuBie3Vcc41mdkuYAiwLdPBDB06lKKiIoYPHx76wbStYSwWa/fA2le/leTk5BCL6bKTyKeJhfCl3Fux2dnADOfcZf70+cBE59xVSXXW+nUq/emNfp1trdZ1BXAFwLBhw0qXLFnSpZhqamrIy8vr0rJhUlzpUVzp66uxKa70dCeu6dOnv+6cm9BpxeQLG5l8AZ8DlidNXw9c36rOcuBz/ngMr4VgHa23tLTUdVVZWVmXlw2T4kqP4kpfX41NcaWnO3EBq1wKx+4wf0HyGnCMmRWZWTZwLrC0VZ2lwIX++NnA//jBi4hILwjthLHzrhFchdcaiAIPOefeNrPb8DLWUuBnwKNmtgHYgZc4RESkl4R6FdE5twxY1qrslqTxOuCcMGMQEZHUqQMSEREJKCmIiEhASUFERAJKCiIiEgjtx2thMbOtwAddXHwoIfxaOgMUV3oUV/r6amyKKz3dieso59xBnVU64JJCd5jZKpfKL/p6mOJKj+JKX1+NTXGlpyfi0ukjEREJKCmIiEjg05YUHujtANqhuNKjuNLXV2NTXOkJPa5P1TUFERHp2KetpSAiIh34xCQFM5thZu+a2QYzW9jG/Bwze8Kf/6qZjUiad71f/q6ZfaGH47rWzNaZ2Vtm9nszOyppXtzMVvuv1j3Mhh3XRWa2NWn7lyXNu9DM/uq/Lmy9bMhx/SAppr+YWVXSvDD310Nm9rH/DJC25puZ3evH/ZaZHZ80L5T9lUJM5/mxrDGzP5rZuKR57/vlq81sVaZiSiO2aWa2K+nvdUvSvA4/AyHHdV1STGv9z9Rgf14o+8zMjjCzMv848LaZXdNGnZ77fKXSv3Zff+H1wroRGAlkA28Co1rV+Rfgfn/8XOAJf3yUXz8HKPLXE+3BuKYD/f3x/9sclz9d04v76yLgR20sOxjY5A8L/fHCnoqrVf2r8XrfDXV/+eueAhwPrG1n/unAbwADJgGv9sD+6iymf2zeFjCzOSZ/+n1gaC/ur2nAf3f3M5DpuFrV/SJed/6h7jPgUOB4fzwf+Esb/x977PP1SWkpBM+Dds41AM3Pg042B1jsj/8SOMXMzC9f4pyrd869B2zw19cjcTnnypxztf7kSmB4hrbdrbg68AXgd865Hc65ncDvgBm9FNeXgccztO0OOecq8Lp3b88c4BHnWQkMMrNDCXF/dRaTc+6P/jah5z5bzdvubH+1pzufzUzH1SOfL+fch865N/zxauAdvEcVJ+uxz9cnJSm09Tzo1ju1xfOggebnQaeybJhxJbsU79tAs1wzW2VmK81sboZiSieus/ym6i/N7Ig0lw0zLvzTbEXA/yQVh7W/UtFe7GHur3S0/mw54Ldm9rp5j7vtDZ8zszfN7DdmNtov6xP7y8z64x1cn04qDn2fmXdaezzwaqtZPfb50lPZ+wgzmw9MAKYmFR/lnNtiZiOB/zGzNc65jT0U0nPA4865ejP7Z7xW1j/10LZTcS7wS+dcPKmsN/dXn2Vm0/GSwklJxSf5++pg4Hdmtt7/Ft1T3sD7e9WY2enAs8AxPbj9znwR+INzLrlVEeo+M7M8vCS0wDm3O1PrTdcnpaWwBTgiaXq4X9ZmHTOLAQXA9hSXDTMuzOzzwI3AbOdcfXO5c26LP9wElON9g+iRuJxz25NieRAoTXXZMONKci6tmvYh7q9UtBd7mPurU2ZWjPf3m+Oc295cnrSvPgaeIXOnTFPinNvtnKvxx5cBWWY2lF7eX0k6+nxlfJ+ZWRZeQnjMOferNqr03Ocr0xdNeuOF1+LZhHc6ofni1OhWdb5KywvNT/rjo2l5oXkTmbvQnEpc4/EurB3TqrwQyPHHhwJ/JUMX3FKM69Ck8TOAlW7fha33/PgK/fHBPRWXX+84vIt+1hP7K2kbI2j/wuksWl4I/FPY+yuFmI7Eu0b2j63KBwD5SeN/BGZkcl+lENshzX8/vIPr//r7LqXPQFhx+fML8K47DOiJfea/70eAezqo02Ofr4x+CHrzhXd1/i94B9gb/bLb8L59A+QCT/n/Sf4EjExa9kZ/uXeBmT0c1wrgI2C1/1rql/8jsMb/T7EGuLSH4/oO8La//TLguKRlL/H34wbg4p6My5++Fbij1XJh76/HgQ+BRrzztpcCVwJX+vMNWOTHvQaYEPb+SiGmB4GdSZ+tVX75SH8/ven/jW/M5L5KMbarkj5fK0lKXG19BnoqLr/ORXg3nyQvF9o+wzut54C3kv5Wp/fW50u/aBYRkcAn5ZqCiIhkgJKCiIgElBRERCSgpCAiIgElBRERCSgpiPQgv3fQ/+7tOETao6QgIiIBJQWRNpjZfDP7k993/k/MLGpmNWb2PTN7w7xnXxzk1y3xO+F7y8yeMbNCv/xoM1vhd/r2hpl9xl99nt/J4Hoze8zvrVekT1BSEGnFzD4LzAMmO+dKgDhwHl73Bm84544HXgS+5S/yCPBN51wx3q9Nm8sfAxY558bh/eL6Q798PLAA71keI4HJob8pkRSpl1SR/Z2C1wHga/6X+H7Ax0ACeMKv83PgV2ZWAAxyzr3oly8GnjKzfOBw59wzAM65OgB/fX9yzlX606vx+uJ5Ofy3JdI5JQWR/Rmw2Dl3fYtCs5tb1etqHzH1SeNx9P9Q+hCdPhLZ3++Bs/1+8zGzwf5DfSLA2X6drwAvO+d2ATvN7GS//HzgRec9Qauy+WE/5j0jvH+PvguRLtA3FJFWnHPrzOwmvKdsRfB61PwqsAcYbWav4z25b56/yIXA/f5BfxNwsV9+PvATM7vNX8c5Pfg2RLpEvaSKpMjMapxzeb0dh0iYdPpIREQCaimIiEhALQUREQkoKYiISEBJQUREAkoKIiISUFIQEZGAkoKIiAT+PzkDmUuzwo52AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12bd292f940>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4VGX2wPHvSUISaqgC0lWkaUS6ogKKCuqPlQWxrKuuBXXXysqquyu23bVhWdAVwY6rKOoiAhYEIhaQIkU6oYOYACaBQHre3x8z9+ZOpiZkMsnM+TwPDzN37p28bzJzzz1vu2KMQSmllAKIi3QBlFJK1RwaFJRSStk0KCillLJpUFBKKWXToKCUUsqmQUEppZRNg4JSSimbBgWllFI2DQpKKaVsCZEuQEU1b97cdOzYsVLHHj16lPr161dtgWqQaK5fNNcNort+0Vw3qD31W7ly5UFjTItg+9W6oNCxY0dWrFhRqWPT0tIYPHhw1RaoBonm+kVz3SC66xfNdYPaUz8R2RXKftp8pJRSyqZBQSmllE2DglJKKVut61NQSkWvoqIi9u7dS35+fqSLErKUlBQ2btwY6WLYkpOTadu2LXXq1KnU8RoUlFI1xt69e2nYsCEdO3ZERCJdnJAcOXKEhg0bRroYABhjOHToEHv37qVTp06Veo+wNh+JyDAR2Swi6SLygI/XbxCRAyKy2v3v5nCWRylVs+Xn59OsWbNaExBqGhGhWbNmx5VphS1TEJF44CXgQmAvsFxEZhtjNpTb9X1jzB3hKodSqnbRgHB8jvf3F85MoR+QbozZbowpBGYAvwnjzwvo293fMm37NEpNaaSKoJRSNV44+xTaAHscz/cC/X3sN0pEzgO2APcaY/aU30FExgJjAVq2bElaWlqFC/PBng94d8+7XL3gahokNKjw8bVBbm5upX43tUE01w2iu34VqVtKSgpHjhwJb4ECyM7OZubMmdxyyy0hH1NSUsKRI0cYNWoUr732Go0bNw7puH/96180aNCAu+66q7LF9Ss/P7/Sn6dIdzR/CrxnjCkQkVuBt4Dzy+9kjJkKTAXo06ePqczswR2rdvDy9pc5rc9pdGzc8bgKXVPVlpmVlRHNdYPorl9F6rZx48aIdtoeOnSI119/nXHjxnm9VlxcTEKC9ynT6mj+8ssvK/SzkpKSSEpKCkt9k5OTOfPMMyt1bDibj/YB7RzP27q32Ywxh4wxBe6nrwK9w1WYxsmu6J2dnx2uH6GUquUeeOABtm3bRs+ePRk/fjxpaWkMGTKEa665htTUVAAuv/xyevfuTY8ePZg6dap9bMeOHTl48CA7d+6kW7du3HLLLfTo0YOLLrqIvLy8gD939erVDBgwgNTUVEaOHElWVhYAkyZNonv37qSmpnLVVVcB8PXXX9OzZ0969uzJmWeeWeWZVTgzheVAZxHphCsYXAVc49xBRFobY/a7n44AwjbYt0ndJgBk5WWF60coparQPZ/fw+pfVlfpe/Zs1ZMXhr3g9/Unn3ySdevWsXq16+empaWxbNky1q1bZw/xfP3112natCl5eXn07duXiy66yOtqf+vWrbz33ntMmzaNMWPG8NFHH3Httdf6/bnXXXcdkydPZtCgQUyYMIFHH32UF154gSeffJIdO3aQlJREdrbrgnbixIm89NJLDBw4kNzcXJKTk4/31+IhbJmCMaYYuAP4AtfJ/gNjzHoReUxERrh3u0tE1ovIGuAu4IZwlUczBaVUZfTr189jzP+kSZM444wzGDBgAHv27GHbtm1ex3Tq1ImePXsC0Lt3b3bu3On3/XNycsjOzmbQoEEAXH/99SxevBiA1NRUfve73/HOO+/YTVcDBw5k3LhxTJo0iezsbJ9NWscjrH0Kxph5wLxy2yY4Hj8IPBjOMliaJLszhXzNFJSqDQJd0Vcn57LYaWlpfPXVVyxZsoR69eoxePBgCgoKvI5JSkqyH8fHxwdtPvJn7ty5LF68mNmzZ/P444+zfv16HnjgAS699FLmzZvHgAED+Oqrr+jatWul3t+XmFn7SDMFpVQwDRs2DNhGn5OTQ5MmTahXrx6bNm1i6dKlx/0zU1JSaNKkCd988w0A06dPZ9CgQZSWlrJnzx6GDBnC008/TXZ2Nrm5uWzbto3TTz+d+++/nz59+rBp06bjLoNTpEcfVZuGSQ2JI077FJRSfjVr1oyBAwdy2mmnMXz4cC699FKP14cNG8aUKVNITU2lS5cuDBgwoEp+7ltvvcVtt93GsWPHOOmkk3jjjTcoKSnh2muvJScnB2MM9957L40bN+ahhx5i0aJFxMXF0aNHD4YPH14lZbDETFCIkzjqJ9TX5iOlVEDvvvuux3PncNqkpCQ+++wzj9etzMLqN2jevDnr1q2zX7/vvvt8/pxHHnnEftyzZ0+fWce3337rtW3y5MkBy3+8Yqb5CCAxLpHCksJIF0MppWqsmAoKglBSWhLpYiilVI0VU0EhXuIpRdc+UqomM8ZEugi12vH+/mIqKIiILoinVA2WnJzMoUOHNDBUknU/heOZ0BYzHc0AccRp85FSNVjbtm3Zu3cvBw4ciHRRQpafn1/ls4qPh3XntcqKraAgcZopKFWD1alTp9J3DIuUtLS0Si8+VxPFVvMR2nyklFKBxFRQiJM43l//Pvd96XvcsFJKxbqYCwoAzy55NsIlUUqpmimmgoKg935VSqlAYiooWJmCUkop32LqLBkXW9VVSqkKi6mzpGYKSikVWEydJbVPQSmlAoupoKCZglJKBRZTZ0ntU1BKqcBi6iypmYJSSgUWU2dJDQpKKRVYTJ0ltaNZKaUCi6mgoJmCUkoFFlNnSe1oVkqpwGLqLCmizUdKKRVITAWFeImPdBGUUqpGi6mgoB3NSikVWEwFBe1oVkqpwGLqLKkdzUopFVhMnSW1o1kppQILa1AQkWEisllE0kXkgQD7jRIRIyJ9wlkezRSUUiqwsJ0lRSQeeAkYDnQHrhaR7j72awjcDfwQrrJYDCbcP0IppWq1cF469wPSjTHbjTGFwAzgNz72exx4CsgPY1mUUkqFIJxBoQ2wx/F8r3ubTUR6Ae2MMXPDWA6llFIhSojUDxaROOA54IYQ9h0LjAVo2bIlaWlplfqZRcVF9uPKvkdNlpubG5X1guiuG0R3/aK5bhB99QtnUNgHtHM8b+veZmkInAakuUcFtQJmi8gIY8wK5xsZY6YCUwH69OljBg8eXKkCJWwoq25l36MmS0tLi8p6QXTXDaK7ftFcN4i++oWz+Wg50FlEOolIInAVMNt60RiTY4xpbozpaIzpCCwFvAKCUkqp6hO2oGCMKQbuAL4ANgIfGGPWi8hjIjIiXD83YJl09JFSSgUU1j4FY8w8YF65bRP87Ds4nGVRSikVnM7mUkopZdOgoJRSyhZbQUG7FJRSKqCYCgra0ayUUoHFVFBQSikVWMwGBWM0a1BKqfJiKig4m49KTWkES6KUUjVTTAUFJw0KSinlLWaDQokpiXQRlFKqxonZoKCZglJKedOgoJRSyhZTQcE54kiDglJKeYupoOBUUqp9CkopVV7MBgXNFJRSyltMBQWdp6CUUoHFVFBw0qCglFLeYjYo6DwFpZTyFlNBQZuPlFIqsJgKCk4aFGLTxO8nsiNrR6SLoVSNpUFBxYzMo5mMnz+eC6dfGOmiKFVjxWxQ0HkKsceavHik8EiES6JUzRWzQUEzhdgjIoDeS0OpQGIqKOgyF7FNcAUF/dsr5V9MBQXrShH0xBCL4sT1cdd7dSvlX0wFhXs630Pj5MaAzlOIZXpBoJR/MRUUWiS14LURrwGuE0NJaQmHCw5HuFSqulgZgvYpKOVfTAUFKGtCKDWljJ8/npQnUzhWdCzCpVLVwQoGmiko5V/MBoXP0z/nnbXvAHCkQIcoxgI7U9A+BaX8itmg8OCCB8uGKOpJIiZopqBUcDEXFOIl3n6ceTQT0JNErNA+BaWCi7mgYGUKTkUlRREoiapuVvDXzFAp/8IaFERkmIhsFpF0EXnAx+u3ichPIrJaRL4Vke7hLA/4DgqFJYXh/rGqBrAyBM0UlPIvbEFBROKBl4DhQHfgah8n/XeNMacbY3oCTwPPhas8lqJS76zA1zYVfbSjWangwpkp9APSjTHbjTGFwAzgN84djDHOSQL1Ifzf1qy8LK9t2nwUG7SjWangEsL43m2APY7ne4H+5XcSkT8B44BE4HxfbyQiY4GxAC1btiQtLa1SBcrNzeWHfT94bV+6fClZDb2DRW2Tm5tb6d9NTVcVdcvIzwCgtLS0xv2e9G9Xe0Vd/YwxYfkHjAZedTz/PfBigP2vAd4K9r69e/c2lbVo0SKz+eBmwyOYy2dcbngEwyOY73Z/V+n3rEkWLVoU6SKETVXUbWfWTsMjGHlEjr9AVUz/drVXbakfsMKEcO4OqflIRAaKSH3342tF5DkR6RDksH1AO8fztu5t/swALg+lPMfj1GanYh429D2xr71Nm49ig/YpKBVcqH0KLwPHROQM4C/ALuDtIMcsBzqLSCcRSQSuAmY7dxCRzo6nlwJbQyzPcUuIK2s5047m2GB01JFSQYXap1BsjDEi8hvg38aY10Tk+kAHGGOKReQO4AsgHnjdGLNeRB7DlcbMBu4QkaFAEZAFBHzPquQRFDRTiAmaISgVXKhB4YiIPAhcC5wnInFAnWAHGWPmAfPKbZvgeHx3BcpapTRTiD066kip4EJtProSKABuMsb8gqt/4Jmwlaoa1Ikri2k6eS02aPORUsGFnCngajYqEZFTga7Ae+ErVvhp81Hs0eYjpYILNVNYDCSJSBtgAfAH4M1wFao6aPNR7NFMQangQg0KYow5BvwWmGyMGQmcFr5ihZ9mCrFHMwWlggs5KIjIWcDvgLkVPLZGqhOvfQqxRjMFpYIL9cR+D/Ag8D/3sNKTgEXhK1b4afNR7NFMQangQupoNsZ8DXwtIg1EpIExZjtwV3iLFl7afBR7NFNQKrhQl7k4XURWAeuBDSKyUkR6hLdo4aWZQuzRTEGp4EJtPnoFGGeM6WCMaQ/8GZgWvmKFn2YKsUcnrykVXKhBob4xxu5DMMak4br/Qa2lk9dijzYfKRVcqJPXtovIQ8B09/Nrge3hKVL10Oaj2KPNR0oFF2qmcCPQAvjY/a+Fe1utpc1HsUczBaWCC3X0URa1fLRReZopxB7NFJQKLmBQEJFPCXDfZGPMiCovUTXRyWuxRzMFpYILlilMrJZSRIAzUygoKYhgSVR10UxBqeACBgX3pDUPItLLGPNj+IpUPZxBIb84P4IlUdVFMwWlgqvM+kWvVnkpIkCDQuzRTEGp4CoTFKTKSxEBznkKGhRig05eUyq4ygSFR6u8FBHgzBTyivIiWBJVXbT5SKngQl37aKSIpAAYY2aJSGMRuTy8RQsvbT6KPdp8pFRwoWYKDxtjcqwnxphs4OHwFKl6aFCIPZopKBVcqEHB136hLpFRIznnKazJWMMT3zwRwdKo6qCZglLBhRoUVojIcyJysvvfc8DKcBYs3JyZAsBfF/41QiVR1UUzBaWCCzUo3AkUAu8DM4B84E/hKlR1iJf4SBdBVTPNFJQKLtS1j44CD4S5LNXK2XykYoNmCkoFF+roo/ki0tjxvImIfBG+YoVfQlwCu+/ZzZ397ox0UVQ10UxBqeBCbT5q7h5xBNirpp4QniJVn3Yp7fTqMYbo5DWlggs1KJSKSHvriYh0JMDqqbWJLoYXO/QCQKngQh1W+jfgWxH5GtcyF+cCY8NWqmrknKNQUlpCfJx2QEcrbT5SKriQMgVjzOdAH2Az8B7wZyAq1oZwZgrHio5FsCQq3DRTUCq4UDuabwYW4AoG9+G6V/MjIRw3TEQ2i0i6iHiNXhKRcSKyQUTWisgCEelQseIfP+etOI8WHa3uH6+qkWYKSgUXap/C3UBfYJcxZghwJnAg0AEiEg+8BAwHugNXi0j3crutAvoYY1KBD4GnK1D2KvHCsBcQ98KvuYW51f3jVTXSTEGp4EINCvnGmHwAEUkyxmwCugQ5ph+QbozZbowpxDXp7TfOHYwxi4wxVpvNUqBt6EWvGu1T2vPRmI8AOFqomUI085cprM1YqyvlKuUWalDY656nMAuYLyKfAD8HOaYNsMf5Hu5t/twEfBZieapU/cT6gGYK0c5XppCTn8MZU87gulnXRaBEStU8oc5oHul++IiILAJSgM+rqhAici2ujuxBfl4fi3u0U8uWLUlLS6vUz8nNzfV57OaczQB8v+J7irYXeb1eW/irXzSoirqtObTGfmy918GCgwAsTF8Y0d+d/u1qr2irX4VXOvV132Y/9gHtHM/burd5EJGhuIa8DjLG+Jw0YIyZCkwF6NOnjxk8eHBFimxLS0vD17GN9jeC1dClRxcGd6nce9cE/uoXDaqibrlbcmGd6/GgQYMQEfYd3gdLISkxKaK/O/3b1V7RVr/K3HktVMuBziLSSUQSgauA2c4dRORM4BVghDEmM4xlCSg5IRmAJ799kkPHDkWqGCrMnM1HExZNYNrKafZzkai4y6xSxy1sQcEYUwzcAXwBbAQ+MMasF5HHRGSEe7dngAbATBFZLSKz/bxdWNVNqAvAkr1LGD1zdCSKoKqBs6P5H9/8g7FzxuowVaXKCeuNcowx84B55bZNcDweGs6fH6q6deraj9N2pkWuICqsfHU0W9usYclKxbpwNh/VGlamoKKbr6ygxJQA2nyklEWDAmV9Ciq6+coUikuLAc0UlLJoUAAS4xMjXQRVDXxmCqUlESiJUjWXBgW06SBW+MoUrOYjpZSLBgUVMwJlCnphoJSLBgUf9A5d0cnX39XuaNY+BaUADQo+aTtzdArY0ayZglKABgWfrBOFii6+mo8KSwojUBKlai4NCj5oUIhOvjIFa7l0bT5SykWDgg8aFKKTr0zhSOERQJuPlLJoUPBBg0J08pUpWPfQ0ExBKRcNCj5oUIhOvjIFvbGSUp40KLhl3pfJw4MeBjQoRKuAmYI2HykFaFCwtajfgk6NOwEaFKJVoExBm4+UctGg4JAQ51pJXINCdPI1eS2/OD8CJVGq5tKg4KBBIbr5aj6y5ik4m49KSksY8d4Ivtv9XbWVTamaQoOCgwaF6Bbq5LX9ufv5dMunjPlwTHUUS6kaRYOCgwaF6BYwU3D0KfjaT6lYoUHBQYNCdAuUKTibj6z94kS/Hir26KfeQYNCdPOVARSUFAC+MwUdkaRikQYFBw0K0c1XppC2M83vfjp3QcUiDQoOGhSim69MIfNoJuA9+gg0U1CxSYOCgxUUxn05jlmbZkW4NKqq+coULM4AoLfoVLFMg4KDFRRW/7KaTzZ/EuHSqKoW6h319MY7KpZpUHCwggKUrbOvokegoabOAGAHBW0+UjFIg4KDR1Ao0qAQbQI1HzlppqBimQYFB80UolvATAHvjmalYpEGBQfNFKJbwI5mbT5SCtCg4EEzhegW6vIV2nykYpkGBQdnUNA7ckWfUIekaqagYpkGBYdQmo9+zfu1uoqjqligTMEZMDRTULEsrEFBRIaJyGYRSReRB3y8fp6I/CgixSIyOpxlCUWw5qNFOxbR7OlmfLb1s+oslqoigTIF5xwGa/KaZgoqFoUtKIhIPPASMBzoDlwtIt3L7bYbuAF4N1zlqAhnUCgoKfAahbJ071LA93o5quYLNHnN+ZpmCiqWhTNT6AekG2O2G2MKgRnAb5w7GGN2GmPWAqFNNQ0zZ1AAOOu1s2jxTAv7eXxcPKBrI9VWgZqPnBcA2qegYllC8F0qrQ2wx/F8L9C/Mm8kImOBsQAtW7YkLS2tUgXKzc0NeGxeSZ7H8+U/Lwewj9m5ZycAu/bsqnQZwilY/Wqzqqhb+u50v68dPXbUfv81B9YAcOzYsWr7ferfrvaKtvqFMyhUGWPMVGAqQJ8+fczgwYMr9T5paWkEOja/OB++9d5uHbNm6RrYDq1PbM15g84jryiP+on1K1WWcAhWv9rseOqW/ms63V7qxsiuI/3uk5icaL9/xroM2AD169evtt+n/u1qr2irXzibj/YB7RzP27q31VhJ8Un0PbEvt/e53WO71bTgXFr7/vn30+CJBuQX55NbmEvPKT1Z8fOKai+zCm7p3qUUlxYzc8NMv/to85FSLuEMCsuBziLSSUQSgauA2WH8ecdNRFh2yzLGnz3eY3t2fjZQdnvGElPCKytfAVzzGVb8vII1GWv485d/rt4Cq5A0Tm4cdB/taFbKJWxBwRhTDNwBfAFsBD4wxqwXkcdEZASAiPQVkb3AFcArIrI+XOWpiLaN2nrcnzcrP4u8ojzeWvMW4DppWMMW84rySIxPBMru96tqllAGBjjvoWDtvy5zHRm5GWErl1I1UVjnKRhj5hljTjXGnGyM+ad72wRjzGz34+XGmLbGmPrGmGbGmB7hLE+o6sTXoW2jtvbzrLws7ph3Bz/s+wFwnUCsE8exomP2qJYNBzYw8fuJ9nELti/g8a8fr8aSh2bJniXIo8KyfcsiXZRqESwonNjwRJ/zFADOmHJG2MqlVE2kM5r9aNOwjf04Kz/LHokE8Paat+2s4GjRUfKKXaOWDhccZvz88WTlZQEwdPpQJqRNCHnNneoyb+s8AL5I/yLCJakewYLCOe3P8dmnAJBxVDMFFVs0KPjRpG4T+3FWXpZ9L9/yjhUdI6/IcyjrOW+c47F20uGCw2z7dRt7D++1tw3/73BeXPZiFZda+RIsKCTGJ3pkClYfklKxSIOCHw0TG9qP9x3Z5/eK8VjRMTtTsGw4sIFFOxbZzzOOZnDK5FPo8EIHwDWJasH2BfYMaRVeQYNCXKLdZLTp4Cb+tvBv1VEspWokDQp+JCUk2Y+3Htrqd79jRcdY+fNKr+3OE9HI913j462r0ZyCHIpKi0JaXO9Y0THeWPWGrtp6HCqSKfj6WyoVSzQo+JEUXxYUNh3a5He/ST9M4unvn/banl+cbz/ecGCD/dgYYzdFZeVnBS3HRxs+4sbZN/L7//0+pHIrb6EEBatPwfl3q81umHUDY2aOiXQxVC1UK2Y0R0JyQrL9+Jtd3/jdb9HORT637zvie55e5tFMOyhYmcJHGz7iuz3f8dzFz3ntfyjvEAA7s3eGVO5QhHqv4mgRUlAwJfya9ys3f3pzNZUqvKzh00pVlGYKfliZwoUnXegxRBGgfp3gS1t8vetrn9t3Zu/0CAoFxQWMnjma55c+7/PkpZ2exy9YUIiPi6fUlPLqj69WU4mUggvevoCL37k40sXwokHBD6tP4Zz253i9VlRaFPT4uVvmkhifyOyrPCdxZ+dnc+DoAcAVFN5Y/Yb92rJ9y7h9zu0s3rXY3paTnwPo7UGPR9CgIPGUlJbQqkGraiqRijV5RXleoxQX7ljIl9u+jFCJ/NOg4Eenxp0AOKH+CV6vFZUEDwoGQ48WPWjZoKXH9rziPDtTKDWlHmn+3xf+nSkrpzDozUGs+cW1Umd2gStTOFxwuHIV8SHW1vQJNVOoE1fH5+uB7sOgVCgaP9WY5s80j3QxQqJBwY8/nPkHZl4xk1t63eL1Wqht8q0btiYlKcVj24qfV/BL7i/287UZaznthNMAV/9E+5T2AMzdOpe317zNusx1QPCgEPdoHP/e+u+QyqV9Cp7iJA6D8dvJfKzoWDiKFTEvL3+Zn4/8HOli1Gj9pvXj7NfOrrL3KywprDWfIw0KfsRJHKO7jyY+Lp67+99dqfdoUa8FKcmeQeGf3/yT/6z4j/38WNExzm1/rv38qh5XcfoJpzNv6zyun3W9vfJqXnFewJObwTDr51mVKme0CxYUrGxwe9Z2n68fKThS5WWKlF3Zu/jjvD9yxcwrIl2UGm35z8tZsndJpIsRERoUQvDsRc9y6C+HKnzcCfVP8MoULPESbz8+tdmp9uPuLbrTrUU3Nh30Hgbr7+SkC/EFFiwodG/hukvsj7/86PP1qmy6izTravXQsYp/nlVs0KAQgvi4eJrWbcr/rvwf838/3+v1ZnWb0SS5idf2FvVaeAxtdTqpyUn24/Yp7e17NXRr0Y2UpBR7KKqTv5NTNJ20wiHUoLBq/yqfr0fTCDBrEmSdeN/9J0ppUKiAy7teztCThtrPW9Rz3b/5gys+YNkt3iuONkxqaK/J365RO4/XnJ2X7VPa23cF69KsC42SGnns27K+q7Pa38nfGqGkfAul+ahRUiP25+4HoG5CXY/XraDwzHfPII9Krep4Lr8YY06B67Pir1NdBb6XdyzQoHAcnMtr+2omsmbJbr1zK2tuW+PxWrN6zezH7VPa8+blb/LT7T+Rkpzi9V7tUlwB5Uih7+Yj64teUbHS4RzK/RSsTC8hLoEdd+/weM0KCg8seAAou9petX8VH2/8uCqLWiWcJ7Xyc2ysCwh/mcLkHybzyaZPwle4WqD8WmaxRoPCcXhk8CMAdGvezePqvm2jtjw2+DFuPPNGAE5peorHqqv3nXUf749+n7oJdUmKT6JFvRbUq1PPHoVUPlOwsoxgzUdxIfw5Dx47aI+XDuVkGQ1CqWfDJNcCiEnxSbRs0NJj6fSNBzeyJ2eP3Q+Uk59DXlEevab2YtQHo8JT6ACy8rLYlb3L7+vO+hYUF9iPDx07xNZfXet4+csU7vr8Li5//3Kfr3244UMmLJpQmSJXu+4vdefWT2+t1LFVnXn7yjxqcjaiy1xUQvuU9uzO2c2ILiMwD5f9cXfevZP84nxaNWjlNerI6Z4B99CmURua1G1CvTr1vG77WP5Ya5hqsOYj53pN/rR4poX9OFaCgvNeCf5YgdjqA3JeYT/69aM8+vWjJCckU1RaRHZ+tkefT35xvt++o3BInZLK3sN7PT57Ts6BBwUlBdTHNQPfOU7e6sOqCGvE0mNDHqvwsdVt48GNbDy4kVf+75UKH1vZzNsfX5Ndy2dwwRwpOMLI90dyz4B7uOzUy6qqaD5pplAJK25ZwepbV3tt79C4A12adwkYEADqJ7q/pPWa0yGlg9frFc0UrA9xXkmePVs6FLESFEKpp/U7t2ay++o3sDKF1CmpnPXaWfZ257yTyjpQcAB5VEjbmRZ0X+u+HE2faspjX3ufoAtKyrIDfyM8t64SAAAckklEQVTT4uPifW4PRbj6VIwxfLjhw4iPpiufKWw4sIGxn46t9PfFOf/FyhCcGZy17e8L/86MdTN8vseBYwdYsGNBtYwa06BQCS3qt+CMVpW/TaO1dtK/h/2bJy54wuv18n0KVqbgXEzPyRksTpjoPQPbn5gJCib0oGBnCj6yC+fJyvlFr4qJYGuyXX1OU1ZMCfmYrPwsHk572Gu784TjfOzka6Kes0nD33EQvtFuH2/8mCtmXsGz3z97XO9T2aaZN1a9Qb9p/TwyhZLSEm6YdQPTfpxmrzJQUc7lLazvnDNwW9v++c0/ufqjq32+x8FjBwHXhWS4aVCIAKuTb3DHwfRt09fr9fKZhtWh/beFf6PlxJZeX+hQT0rlg4Cv5Tq+3PYl/1z8z5DeLxK2Htpa4S99SJlCojtTiPefKfhb86qyQSHzaCbzt83nvDfOY3L6ZMA1afJ4hZIp+Jpd6/xc7Tuyj/vn38+WQ1u83sfXfUByC3NDaqYLxJqbc7zNN5XtKL5x9o0s/3m5x4VXQUkBdeu4RqOFcv8TX5y/V+tv4wy6hSWFTP5hcsD3sFoANCjEqPKZQtO6TT1WZi0/OiTUmZflF+TydbK8+J2L+fuiv4da1Gq15pc1nPriqTy7pGJXkhXpaC7fp3BP/3uCHrv/yP6QytH/1f68vPxl+/mF0y/koncu4pvd33C42D1YoBJBYdrKaR4n5PJ9Cr74CgrObd/u/panv3+ay2e4Op2dga98E0ZRSRENn2jIbXNuq3DZnQ4cc534WtRrwVfbv2LfYd/LzwMs3rXYbzNW+RtSfZ7+OX/45A8hl2Pbr9vsxwXFBTRObgz4Xw7fMn3NdNJ/Tffa7gxSVjBw/o3yivO46/O7Ar63ZgoxzhqCaklJTvE4sc3dOtd+XFxazA97fwjpfctnGOVPlr6uwC979zJSX0712v7Z1s94eJF300U4WctQfLv72wod5y8oPDb4McYNGAeUNR9ZWZx1kv3ruX/llz8H7jMIZXJbYUkhy/Yt44/z/mhvW5ux1ms/f0EhKy/L71X/2DljPVbbLH8VCt6Zj6+gcLSobCVeq2zWMGjnEiDlbyO7bJ9rjs6rq16luLSYktISbv30Vn7K+Mlnef2xrtDjJI4Lp1/IgNcGYIxh7pa5Hn/Dr7Z/xaA3B/H8kud9vk/5SYjD/zucN1e/GXJzaXpW2Yn9p8yfmLNlDgC7c3YDru/R+C/He/weSkpLuG7WdXSe3NkrmDkvxuxMwRGsQ8lANCjEuPIjWVKSUuwPUUpSCtPXTmfulrnkF+eT/ms6ecV51KtTL+j7egWFcm3tzhv5WF+guVvn8lOm95f7kncv4bHF1TsKxZpXUX60lv26MR7t3Z9u/pRDxw75PRk8NOghnr3YlXVYQcHa18oUEuMTadmgpdfkQ3CN4ElOSPbZxp6Rm8Ez3z2DMYZtv24jI7fsHt9jPx3LDbNu8Fmmndk7SduZRscXOpKdn01JaQklpSU0fbppwKvdlftdtxGd+P1Ej6XXrQBRvozll2IvKS2xT3rgWvvH2g54LLty1+d32cHm7TVvc84bZcvLbzq4ie1Z25n641R++8Fv/ZbXFytTsILD3sN7mb99Ppe9dxlPfFPW92Z9TtcdWOf1HsYYhv13mM/3D3VmuvNqf9Cbg+yAujtnN/O3zafz5M5MXDKRxxc/zoRFEyguLfa4i+LJk072eD+P5qNi7+ajUINCnbg6XoNQwkGDQjW6pPMlHstbhOLg+IN2mybA0xe6bv152XuXcfrLp9urqJ5+wun2Pv7S6vJtreX7FJxf/PInjXs/v9fnMhDV2VkdrC9h6sqppDyZwo6sHRwpOMKIGSNo81wbn1fk5VlNBNYX1PodWplDvzb9vI5JjE8kJSmFiUsmMm/rPIpLi8nIzeCnjJ+4fe7t/OWrv/DC0hc4ZfIp3D73dvu4aT9O83tntG92f8OQt4awK2cXTZ5qwkXvXGT/Xd796V2/5X9o0UOc8/o5jJ8/njs+u8PeXlhSyLSV02jylOcyLFZWYIxh/rb53P/V/Zz7RtnCjNZCjPtz9/PXBX9lfeZ6j+OT/5GMPCpcP+t6j+3rM9fbV7X+7it+yqRTuHm29x3urKDkbKaxAoTzwsT5mVu0YxE9p/S0P6/lf6azWS0rz/v2tzPXz2Tp3qUet8z1d0/2LYe2cNE7F9mjv+Ztncfjix/nm4PfeDSpFZQU8GjaoxSWFLI7Z7fH9y7zaCbrMtd5ZH3+RhTtyNrBqA9G0euVXmw+tJnm9Zr7vSCqShoUqtHca+ay7a5twXcErjvjOsBz5jPALb1u4ay2ruGQ6b+msy5zHXESxylNT7H38bdwXrDmo82HNtuPnU0JAC/88AKXvHuJ13tW5xIbVnn93Q/Calb7cf+PdmdlQUmB39VPnS7odAFQdhVqnUysSV6ju4/2OsYYQ4PEBgBc+u6l1Hm8Du2eb0fqlFQ7WM/cMNOjbBW1cMdCnvjWdZWcFJ8UMDB+t+c7r20FJQUe9xD/ferveWroUxSWFHLo2CHmbJnDRe9c5NFPk5KU4tG89MS3T/Deuvfoc2If/nX+v1x1Lzcbvm2jtsRJHOsPrOe/P/0X8N3JXVxazLasbby26jVeWfGK/VktKS2xf/fOoGC9PmvTLH738e8wxngMu7597u2syVjDmA/HUFRS5LVmmLMezqv5IwVHWLV/FWM+HMNZr51Fj//0sF/zte4YuAK2LwbjdcwjXz/CWa+dRYcXOng0dw777zBOf/l0j+9X+WOPFBzhlRWvcNKkk/h448es+mUVX23/qlqajkAnr9VYb/7mTd66vOxqcmTXkazNWIuI0LRuU3v7sn3L6Na8m8dkpJyCHDYf2ky35t3sDlQIHhSskSbg+05vvjKQ7Pxsr8AVLtYXyd/VkrUW1YFjByo8bLJD4w78+aw/07V5VwDeH/0+z3z/jP17HdNjDDuydrBy/0o+2vgR4DoZlP+dWCOUrJnDvgYBDD1pKAu2Lwh5mRHrJFtQUmD/7FCtz1zv0RwypscYu99iTcYan02ATw19itvmenYaZ+Vn0bV5V27vezt/XfhXr2PqxNWhe4vuPL74cXtbYUkhy/ct5+e8sk7q+dvKFpS8be5trPplFVMum8K+I/vsIOJsk7c6uItKi3j3p3fJysti9S+uOUJZeVn2fIt5W+cxdPpQNh8su7ABz4ub/q/2Z9gpw7ir3108v/R55m/3XtzS0rlpZ/tvWF7Tuk3p0qyL/bddnrWcnSt2eu33437XqrvOYcNWE5YzKy+fKTR60ruJ6EjhkWoLCpop1FDlT3wfX/kx6Xe5vtzOE9H3e75nYLuBHif4vYf30v/V/pz9+tmM+2Kc3X4ZKCjsO7zP7jAE15ep/AnP1yzYqlhB9Nnvn+XBrx4Mup8VqPxlCtZQ3t05uys1ln7iRRO5uZerWWNU91EsvXmp/XeIkzgePPdBPrjiA56/2NXBOajDoEpNtBrScQidmnTyXQfHyLN948pOjl2adQGo8H0Qxn05zuN5o6RGduD7+8K/281ETv3b9vf5Xt2ad6NxcmP7eKc7+t1hd9pbjhYepd+r/bhpxU2A6/NXPtuctcl1DxBnNmc1zwDMS5/nsf9n6Z/ZCxfuyN7hMSpq8a7FZBzN8Nj/zdVvejz/PP1zLnn3koABAbDvceLrFq277tnFH3qW9e98/svnTF873ef73NDzBgDO73S+x/Zb55QtweHsywtEg4LyyzlFPqcgh0EdB3mMoV/5s6vTcV3mOp5f+rz9xSs/JNV5TNvn27Jy/0p7hdCjhUe92med94CwVEVQuG/+fTz53ZNBZ8pa5SkfMCd+P5H6/6rP80tdJ+vtWdvDdmOcOInjtj63Me3/pvG/K/8XNCj4WlK9c9POnNzkZK/t4waMY/vdZSfHExueaD9+5bJXfN4aFqDviX2ZcukUlty0hKeHPs0jgx5h5hUz7SYxp+4tutMhpQMpSSks2buETo078dblb3Fnvzvt+TDOVWJTW5aNPLMC05rb1jC211h7+5U9ruTeAfdyfc/rubTzpfZ2KxPKL83n4ncuZsCrA7zKk3E0gy4vdrEzjAaJDTyu7q2rbYCB7QZ6HLs2Yy3Z+dnc1e8ue/nz8h5cEPxiwxfr/XwFwAaJDRjV3feaV385+y/MGDWD9DvTWXLTEl79v1fZfMdmHh7kf6TepGWTPJ4nxid6PLcyYA0Kyq/hpwy3H7eo14JR3UZ5XPWXH/P87e5vMcZw4+wbPbZvz9ru1YRk9U0cLTrKNR9d4/Gar0zB2U5bGc6OQF9jvJ2sk4WzzIUlhczZMsej7Xh/7v6AmcIferputVpZyQnJ3NzrZurWqeszKFid/s3qNmPXPWUL1037v2kAnN3ubG7tfSvndzqfYacMs08YY3qMoWndpgw9aShvX/62x3ue3e5s/vvb//osT6sGrbi1z60MaDuA8QPH8/DghxndfbQdFPqe2JdJwybxwegPaF6vOfFx8cz//XyuPu1qHh70MNedcR2Thk+ybz3bpG4Trk29ljYN27DmtjV884dv6N26NwPbu07KifGJvHjJi/Ycjgs6XYCIECdxzLlmDhv/tJGJF070KOOX275kTYZrRrB1J0NrGfoth7awcMdCoKzD3+nGnjcy5+o5zL1mLjNGzfD4XYJrEmj5gPH9jd/7/F05VzZ2LoMP8OEVH9qPrWzp8i6ueRojuoygW/Nu9oCDpnWb8uW1ZcOArQzvyaFPcuVpV3Jy05MZ0HYA8XHxnNrs1JAm9k04bwJN6zb1WuWgdcPWQFlwCDftU6iF7u5/N7/k/sJT3z1FjxN6kJSQ5HN2smV1xmp+PvKz18zbTQc3kfpyKivGljUhnNL0FH7K/InDBYe9OketoODs7Jy7dS4PfPUAK8euDLrmky+7cspOmj/u/9HjLnSWEe+N4FROpaSJ64t1rOgYGw5sYNuv2/jtB7+luLSYRkmN7ECQkZvht3lAEF7/zesVLqc/1lDh9DvTyS3MpecrPbnv7PvIPJrJeR3O8+jTubnXzdx45o3ESRyjuo/yuNo815xrn4icN3Kad8081maspU58HXq17uX188efPZ57B9zrs2zW7zKvOI87+9/p8VrfNn15d5TnaKaHznuIP/X9E83qNWP6yLLmkHPan+PxGQHXqKznhz3PHf3u8BpR17V5Vzo27sh98+/zKtNNZ97EQ+c9RElpCXf0u4OuL3leiTsnAn405iNGfTCKa1OvZUinIYArcJ7R6gy6Nu/KLZ+6gljHxh291hDrc2IfWjVo5bEu1b0D7uUf5/+Dy969jEZJjZg+cjpf7/qam2bfRLtG7bi8a9nqsOe0P4fVt64mtWUqV/S4gqZ1m1Inro5HlnrhyRfaj3fds4v0X9P99ned2+Fc+rfpzw/7fqBto7bc0usWzmx1JiNmjLD3GT9wPI8OeZQle8r6oW7oeQOZRzNZm7G22vruwhoURGQY8G8gHnjVGPNkudeTgLeB3sAh4EpjzM5wlikaiIidyltNCoGGhm7P2u5x8nXaeHCjxxhwqxN7+b7lXvsWlRZhjPFYtM1qs738/cvp1aqXPe4/VM4ON2e7clFJEQZDYUkhn275FICbz3S19x8tPOoxWgRc2dP7698HXKOonCOpnPy15VeWFYxbNmjJyYknk3lfptfQwdW3rmZblmvUmb/Jab6a5gCGdx7O8M6uzNA5wMA64T1xwRN+F7c7uamriap8s6E/IlLhE4/1M8pLTkhm3jXzyDyayQ2f3ADAMxc+w7izxhEncUy+xLWsw6dXf8qZrc6k7fOuK3irafSlS17it91+y8/jfravlK0ylm/S6di4oz0KLLVlKiO7jqROfB36nNiHOVvmcFKTk1h601L777Lw+oX2sZedepl9/4z4uHiu7HGlfcMra30zZzNeeel3prP0h6WkJKfQ+8TefvdLiEvgxUtepO+0vnRv0Z0JgyZ4zF2BsjXROjbuCLiaE5+68Cke+OoB5m2dV223UA1bUBCReOAl4EJgL7BcRGYbYzY4drsJyDLGnCIiVwFPAVeGq0zRxGq2OaGeKyiM7DqSz9I/49Rmp7Ll0Bbu6ncX9519H5OXTeaZ75/xGpVhaZTUiFdWvuLxHOCTza6lNG7pdQutGrTi8cWPszN7J5e+eymfpX/m9T5pO9NI25nG1adfzYMLHiQlKYX/XPofVxnLtYUXlhQy9O2h3D/wfjsoJMQlsCOr7OY2Q94awi+5v/DhmLKU3mo+cp7wx/YaS2rLVPq16WcHBad518zz6Nzs2aqnz99DZX19w9e8s/Yd+wvdor53in9GqzOOawFFp7+c/RdyCnJ4dPCjrPplVcDVTrs170bv1r15cuiTfvcJJyuYtf61NWf0O4OWDVp67WMtA/3kBU/SuVln+/4UY3u7+iycAaG8N37zBlNWTKFxcmO7KejpoU9z8SkXA67MZ86WOUwfOd3n38XinPg5Y7TvVUr9ObnpyeypuyekfXu17sUjgx7hD2e6OqlbNmiJedjw2o+v8c3ub+wLidYNW5NxXwbN6zUnTuK4u//dfLHtC3uYerhJuG72ICJnAY8YYy52P38QwBjzhGOfL9z7LBGRBOAXoIUJUKg+ffqYFSu8R0yEIi0tjcGDB1fq2Jpmy6EtdHmxC2tvW8vpLU/HGMPnCz9n+AXDMcbYH7Dpa6Zz3Sz/H6axvcYy9cepgGs0zTu/fYd2z7tm7/Zv05+lN7um8l8x8wo+3PChx7GPD3mchxY9FLCcjZMbM+XSKczaPIuRXUdSN6Euzy19jrSdaSTGJ3Jd6nV8svkTTmpyEskJyVx/xvV8uuVT/rfpfwBMGjbJ77ow57Y/l/m/n09SQhLZ+dleE7QAzMOGZfuW0TCxIbM3z+bWPrf6bLeOtGj6bJZXkbrdMvsWPtr4Eb/eX/HF55yfe0upKa2SRQYDqS1/OxFZaYzpE2y/cDYftQGcIXQvUH6sm72PMaZYRHKAZsDBMJYrKpza7FSPm6yICHXj69qPLf3a9CMxPtHvKJk7+9/J++vfp1OTTnxx7RckxCXQvUV3NhzY4HEzlb+d+zc+2fSJx4il8WePZ+X+lczaNIv6dep7jBqx2viz87O56qOrALzWii8sKeTVVa8ysN1A2qW0Y8a6GXy962uPfcoHhOSEZHto7YdjPrTvf+DrdqjO3wFAtxbd/O6jaoZpI6YxbcS04Dv64Ks9P9wBIRqFM1MYDQwzxtzsfv57oL8x5g7HPuvc++x1P9/m3udgufcaC4wFaNmyZe8ZMyqW4llyc3Np0KBBpY6tDfzVzxjDnrw9lJgSMvIzaJDQgC4Nu1BqSkmKTyK3OJfkuGS7IzmvJI/M/Ew61PfsvCssLeS7g9/ROrk1TRObckKyq1loyaEl1IuvR2pKKgsPLKRRQiNObXgq8RLPoYJDLD64mKzCLApLC2mV3IpTGpzCTzk/kVeSR7Ep5qxmZ9EooRFfZn5Jt4bdSJAETm5wMvvy9rEyayXnND+H3Tm7OaHBCTRJbML8jPm0SGrB1e2u9jgR/Jj1I00Tm7IwcyG9mvQiuyibwS0Gh+8XXoWi+bMZzXWD2lO/IUOGhJQpYIwJyz/gLOALx/MHgQfL7fMFcJb7cQKuDEECvW/v3r1NZS1atKjSx9YG0Vy/aK6bMdFdv2iumzG1p37AChPCuTucudVyoLOIdBKRROAqYHa5fWYD1opao4GF7sIrpZSKgLD1KRhXH8EduLKBeOB1Y8x6EXkMV8SaDbwGTBeRdOBXXIFDKaVUhIR1noIxZh4wr9y2CY7H+UDFFnNRSikVNto1r5RSyqZBQSmllE2DglJKKZsGBaWUUjYNCkoppWxhm9EcLiJyAPC95GdwzYnuJTSiuX7RXDeI7vpFc92g9tSvgzEm6E0Zal1QOB4issKEMs27lorm+kVz3SC66xfNdYPoq582HymllLJpUFBKKWWLtaAwNdIFCLNorl801w2iu37RXDeIsvrFVJ+CUkqpwGItU1BKKRVAzAQFERkmIptFJF1EHoh0eSpDRF4XkUz3zYmsbU1FZL6IbHX/38S9XURkkru+a0WkV+RKHpyItBORRSKyQUTWi8jd7u21vn4ikiwiy0Rkjbtuj7q3dxKRH9x1eN+9xDwikuR+nu5+vWMkyx8qEYkXkVUiMsf9PCrqJyI7ReQnEVktIivc22r959KfmAgKIhIPvAQMB7oDV4tI98iWqlLeBIaV2/YAsMAY0xlY4H4Orrp2dv8bC7xcTWWsrGLgz8aY7sAA4E/uv1E01K8AON8YcwbQExgmIgOAp4DnjTGnAFnATe79bwKy3Nufd+9XG9wNbHQ8j6b6DTHG9HQMPY2Gz6VvodyJp7b/I4S7wNWWf0BHYJ3j+Wagtftxa2Cz+/ErwNW+9qsN/4BPgAujrX5APeBHXPcrPwgkuLfbn1EqcUfCSP8D2uI6OZ4PzAEkWuoH7ASal9sWVZ9L57+YyBSANsAex/O97m3RoKUxZr/78S9AS/fjWltnd3PCmcAPREn93E0rq4FMYD6wDcg2xhS7d3GW366b+/UcoFn1lrjCXgD+ApS6nzcjeupngC9FZKX7fvEQJZ9LX8J6kx1VvYwxRkRq9XAyEWkAfATcY4w5LCL2a7W5fsaYEqCniDQG/gd0jXCRqoyIXAZkGmNWisjgSJcnDM4xxuwTkROA+SKyyflibf5c+hIrmcI+oJ3jeVv3tmiQISKtAdz/Z7q317o6i0gdXAHhv8aYj92bo6Z+AMaYbGARruaUxiJiXZg5y2/Xzf16CnComotaEQOBESKyE5iBqwnp30RJ/Ywx+9z/Z+IK6P2Iss+lU6wEheVAZ/doiERc94KeHeEyVZXZwPXux9fjaou3tl/nHg0xAMhxpLs1jrhSgteAjcaY5xwv1fr6iUgLd4aAiNTF1VeyEVdwGO3erXzdrDqPBhYadwN1TWSMedAY09YY0xHXd2uhMeZ3REH9RKS+iDS0HgMXAeuIgs+lX5Hu1Kiuf8AlwBZcbbl/i3R5KlmH94D9QBGutsqbcLXFLgC2Al8BTd37Cq4RV9uAn4A+kS5/kLqdg6vtdi2w2v3vkmioH5AKrHLXbR0wwb39JGAZkA7MBJLc25Pdz9Pdr58U6TpUoK6DgTnRUj93Hda4/623zh3R8Ln0909nNCullLLFSvORUkqpEGhQUEopZdOgoJRSyqZBQSmllE2DglJKKZsGBaV8EJGO4liNNoT9bxCRE0PY58XjL51S4aNBQamqcQMQMCgoVRtoUFDKvwQRecu9Lv6HIlJPRCaIyHIRWSciU90zV0cDfYD/utfcrysifUXke/c9FJZZs2KBE0Xkc/c6/E9HsG5K+aRBQSn/ugBTjTGpwGHgj8CLxpi+xpjTgLrAZcaYD4EVwO+MMT2BEuB94G7juofCUCDP/Z49gSuB04ErRaQdStUgGhSU8m+PMeY79+N3cC3FMcR9t7CfcC381sPHcV2A/caY5QDGmMOmbAnpBcaYHGNMPrAB6BDeKihVMbp0tlL+lV8DxgD/wbWezR4ReQTXOj4VUeB4XIJ+B1UNo5mCUv61F5Gz3I+vAb51Pz7ovu/DaMe+RwCr32Az0FpE+gKISEPHEtJK1Wj6QVXKv03A9SLyCq7VMF8GmuBa/XInriXZLW8CU0QkD9e9Eq4EJruXys7D1a+gVI2nq6QqpZSyafORUkopmwYFpZRSNg0KSimlbBoUlFJK2TQoKKWUsmlQUEopZdOgoJRSyqZBQSmllO3/AYdUgklE2Z5kAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12bdd5d5128>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cnn_by_own(input_shape=INPUT_SIZE,num_class=5,epochs=20)\n",
    "history_this.loss_plot('epoch')\n",
    "history_this.loss_plot('batch')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Validation accuracy of a CNN by own: 99.884%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model 2: Xception"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def xception( num_class, epochs,savepath='./xception.h5',history=history_this,input_shape=INPUT_SIZE):\n",
    "    model_fine_tune = Xception(include_top=False, weights='imagenet', input_shape=input_shape)\n",
    "    for layer in model_fine_tune.layers[:131]:\t\t#could be tuned to be 50, 100, or 131\n",
    "        layer.trainable = False\n",
    "    for layer in model_fine_tune.layers[131:]:\n",
    "        layer.trainable = True\n",
    "    model = GlobalAveragePooling2D()(model_fine_tune.output)\n",
    "    model=Dense(units=256,activation='relu')(model)\n",
    "    model=Dropout(0.5)(model)\n",
    "    model = Dense(num_class, activation='softmax')(model)\n",
    "    model = Model(model_fine_tune.input, model, name='xception')\n",
    "    opt = keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08)\n",
    "    model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])\n",
    "    #train model\n",
    "    earlyStopping = kcallbacks.EarlyStopping(\n",
    "        monitor='val_acc', patience=3, verbose=1, mode='auto')\t#patience could be tuned by 2 and 3\n",
    "    saveBestModel = kcallbacks.ModelCheckpoint(\n",
    "        filepath=savepath,\n",
    "        monitor='val_acc',\n",
    "        verbose=1,\n",
    "        save_best_only=True,\n",
    "        mode='auto')\n",
    "    hist = model.fit_generator(\n",
    "        train_generator,\n",
    "        steps_per_epoch=len(train_generator),\n",
    "        epochs=epochs,\n",
    "        validation_data=validation_generator,\n",
    "        validation_steps=len(validation_generator),\n",
    "        #use_multiprocessing=True, \n",
    "        callbacks=[earlyStopping, saveBestModel, history],\n",
    "    )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From D:\\py36cnn\\lib\\site-packages\\keras\\backend\\tensorflow_backend.py:2880: calling reduce_sum (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "keep_dims is deprecated, use keepdims instead\n",
      "Epoch 1/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0545 - acc: 0.9828Epoch 00001: val_acc improved from -inf to 0.96493, saving model to ./xception.h5\n",
      "183/183 [==============================] - 89s 486ms/step - loss: 0.0542 - acc: 0.9829 - val_loss: 0.1063 - val_acc: 0.9649\n",
      "Epoch 2/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0017 - acc: 0.9999Epoch 00002: val_acc improved from 0.96493 to 1.00000, saving model to ./xception.h5\n",
      "183/183 [==============================] - 78s 425ms/step - loss: 0.0017 - acc: 0.9999 - val_loss: 2.4231e-04 - val_acc: 1.0000\n",
      "Epoch 3/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 7.3764e-04 - acc: 1.0000Epoch 00003: val_acc did not improve\n",
      "183/183 [==============================] - 73s 398ms/step - loss: 7.3384e-04 - acc: 1.0000 - val_loss: 1.2173e-04 - val_acc: 1.0000\n",
      "Epoch 4/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 9.7458e-04 - acc: 0.9998Epoch 00004: val_acc did not improve\n",
      "183/183 [==============================] - 73s 399ms/step - loss: 9.7122e-04 - acc: 0.9998 - val_loss: 1.1143e-04 - val_acc: 1.0000\n",
      "Epoch 5/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 9.6899e-04 - acc: 0.9998Epoch 00005: val_acc did not improve\n",
      "183/183 [==============================] - 73s 399ms/step - loss: 9.6497e-04 - acc: 0.9998 - val_loss: 8.3518e-05 - val_acc: 1.0000\n",
      "Epoch 00005: early stopping\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8FfW9//HXJyEmQCAJARIKaKCilCUEApLKGpS9BapY7AWl3gqP/u7VilivuBaLj9a6VIXSWmq1aLXodaWFikUDcQEVLBYEvCBLCUIISyIBIoR8f3+cw3gIWU6WyQnyfj4e5+GZme/MvM/gmU9mOd8x5xwiIiIAUZEOICIijYeKgoiIeFQURETEo6IgIiIeFQUREfGoKIiIiEdFQUREPCoKIiLiUVEQERFPk0gHqKnWrVu7tLS0Ws175MgRmjdvXr+B6oFy1Yxy1VxjzaZcNVOXXGvXrt3vnGtTbUPn3Fn1yszMdLWVk5NT63n9pFw1o1w111izKVfN1CUXsMaFsY/V6SMREfGoKIiIiEdFQUREPGfdhWYR+fo6ceIEeXl5lJSURDRHQkICmzZtimiGioSTKy4ujg4dOhATE1OrdagoiEijkZeXR4sWLUhLS8PMIpbj8OHDtGjRImLrr0x1uZxzHDhwgLy8PDp16lSrdfh2+sjMnjSzfWa2oZLpZmZzzWyrmf3LzPr4lUVEzg4lJSUkJydHtCCczcyM5OTkOh1p+XlN4U/AqCqmjwa6BF/Tgd/5mEVEzhIqCHVT1+3n2+kj51yumaVV0WQ88HTw/tnVZpZoZu2cc3v8yiRAaSkcOXLayxUf4dDeL8nfU0b+Xkf+PmNvQTSf/fswbyW8Hd5y/XisayXLLCwq4q2ElfW+3LoqLCzkrcQVtZu5tl/kMOcrLPyCtxJzG2RdNZnvUGExbyW+6w2Pvrs5u6OKa7eeenTy5Em+iI58jvJi40vx+6xWJK8ptAd2hQznBcedURTMbDqBowlSUlJYsWJFrVZYXFxc63n9VD6XlZYSXVJC1LFjRJeUeK8zhktKiD52LPAKHVdSQtTRYxw+eh77j7bgwLGW7Psyif1fJrKvrDV7SSWfFPJJYS+p7ONbHCe2wmxGWQNtBRG49NbN7DkcuV8SHz5cyOuvP8dVV/1Xjee96aax3Hffs7RokRhW+wULZtO0aTzXXPPTsNfRzh3k8OHD1bYrKSmp9b7urLjQ7JxbACwA6Nu3rxs6dGitlrNixQpqO+8Zjh8/4y/uSl/FxVVOP1pQQDPnvhp3/HiFq3RAEQnkk8Ke0B17dFfym3yDfGvHXlLIL2tD/olWHHfnnbGM6KgyUuKPkJJQQkrScXokl5LS5nNSUiC1nZHSvgkpHc8jNS2Of/3f+2Rflh3+NqnJX5J1aFvpv2OETzvU6/9fp1R1VFODaV62elpefU3Lzc1l8ODB3qhN2+FbXf05kgvHjh2HWLLkd9x1x5QzLuiWlpbSpEnlu8x33/5b8F14+f/WDuLjoW+f8D/v4eKYsC6Ax8XF0bt377CXGyqSRWE30DFkuENwnD8KCoj/9FOIiqrVjvuM14kTNVt/XBw0b376Kz4el9qOz5u142TbdPJdW/LL2rD3RDL5xxPJP5bA3iMtyC9uRv4XTckvjOXL42deBooG2rYisGNPhe4pX71PKfe+VasooqJaANX/j2U7Y6CKL0HEmEW8ADSYqj5nTbZBdHTg1ciUxcZC06ZfjTALfEcjZNYdd/DZZ58xYNAgRo4cydixY7n33ntp164d69atY+PGjUyYMIFdu3ZRUlLCTTfdxPTp0wFIS0tjzZo1FBcXM3r0aAYOHMh7771H+/btee2112ga+jnhq/+Po6JYt24dP/7xjzl69Cjf/OY3efLJJ0lKSmLu3Lk8/vjjNGnShG7duvGHP/yBlStXctNNNwUXYeTm5tbrnVKR/MYvBm4ws0VAf6DI1+sJTz1F39tuq7pN06Zn7ribN4f27SseX9ErPh7XrDlfuBbkH4knv7g5ewtjyS+IJj8f9u6F/PzAa++2wH+//PLMKNHR0LZtcKf+TfhWJTv5lBRITo7o90jEFzNen8G6vevqdZkZqRk8OurRSqfff//9bNiwgXfffZcWLVqwYsUKPvjgAzZs2ODd4vnkk0/SqlUrjh07Rr9+/bjyyitJTk4+bTlbtmzhL3/5C3/4wx/4/ve/z0svvcSUKVMqXe+1117LvHnzGDJkCPfccw/33nsvjz76KPfffz/bt28nNjaWwsJCAB566CHmz5/PgAEDKC4uJi4urh62zFd8Kwpm9hdgKNDazPKAnwExAM65x4GlwBhgK3AUuM6vLABMmMD648fpmZXl7bxP25k3a1blntU5OHyYM3fsu0Leh4yv6I6wqKivdvSpqdC1a+B9cfFWBgy48LSdvXb0Io3DJZdccto9/3PnzuWVV14BYNeuXWzZsuWMotCpUycyMjIAyMzMZMeOHZUuv6ioiMLCQoYMGQLA1KlTueqqqwBIT09n8uTJTJgwgQkTJuCcY8CAAcycOZPJkydzxRVX0KFDh/r8uL7effSDaqY74L/9Wv8ZLrqIAwMHQsg5X+cCZ4327ql4x17+fWU7+jZtvtqZX3xx1X/RV3QEv2JFHkOHXujfZxc5C1X1F31DCu2qesWKFSxfvpxVq1bRrFkzhg4dWuFvAmJjv7pxIzo6mmPHjtVq3UuWLCE3N5fFixczZ84cVq9ezaxZsxg7dixLly4lKyuL5cuX07Vr11otvyKN8ISxP5Yvh8ceu4hHHz19h1/Rv9WpHf2pnflFF1W8k09NrXxHLyJnnxYtWlR5d09RURFJSUk0a9aMzZs3s3r16jqvMyEhgaSkJN5++20GDRrEM888w5AhQygrK2PXrl1kZ2czcOBAnnvuOYqLizl48CA9e/akZ8+erFq1is2bN6so1MamTfDuu63p0CGwM7/wwsovxrZurR29yLkoOTmZAQMG0L9/f8aOHcvYsWNPmz5q1Cgef/xx0tPTufjii8nKyqqX9S5cuNC70Ny5c2eeeuopTp48yZQpUygqKsI5x80330xiYiJ33HEHOTk5REVF0b17d0aPHl0vGU45Z4rCjTdCz57v1f8tgyLytfLcc8+d1sdQ6D4jNjaWv//97xXOd+q6QevWrdmw4avefX7604p/hzB79mzvfUZGRoVHHe+8885pw4cPH2bevHnhfIxa06VMERHxqCiIiIhHRUFERDwqCiIi4lFREBERj4qCiIh4VBRERIIKCwv57W9/W6t5x4wZ4/VPdDZTURARCaqqKJSWllY579KlS0lMDO9ZCo2ZioKISNCsWbMCXWcPGMCtt97KihUryM7O5j/+4z9IT08HYMKECWRmZtK9e3cWLFjgzZuWlsb+/fvZsWMH3/rWt5g2bRrdu3dnxIgRFfZ99Ne//pX+/fvTu3dvLr/8cvLz84HAQ7euu+46evbsSXp6Oi+99BIAr7/+OoMGDaJXr15cdtllvm2Dc+YXzSJylpkxA9bVb9fZZGTAo42j6+yBAweyevVqzIwnnniCBx54gIcffpg5c+aQkJDA+vXrATh06BAFBQVMmzaNpUuX0rNnTw4ePFi/2yWEioKISBX86jo7Ly+PSZMmsWfPHo4fP+6tY/ny5SxatMhrl5SUxF//+lcGDx5MWloaAK1atarPj3gaFQURaZyq+Iu+IfnVdfaNN97IzJkzGTduHCtWrDitL6RI0jUFEZGghuw6u6ioiPbt2wOBXlJPGT58OPPnz/eGDx06RFZWFrm5ud4Rh5+nj1QURESCQrvOvvXWW8+YPmrUKEpLS0lPT+fuu++uU9fZs2fP5qqrrmLQoEG0bt3aG3/XXXdx6NAhevToQa9evcjJyaFNmzYsWLCAKVOm0KtXLyZNmlTr9VZHp49EREI0VNfZ48ePZ/z48WeMj4+PP+3I4ZTRo0czcOBAL5dfdKQgIiIeFQUREfGoKIiIiEdFQUREPCoKIiLiUVEQERGPioKISFBDdp09e/ZsHnrooVqty08qCiIiQeo6W0VBRMTTkF1nh1q3bh1ZWVmkp6fzve99j0OHDgGBzve6detGeno6V199NQArV64kIyODjIwMevfuXWW3HLWhXzSLSKM04/UZrNtbv11nZ6Rm8OioxtF1dqhrr72WefPmMWTIEO655x7uvfdeHn30Ue6//362b99ObGysd2rqoYceYv78+QwYMIDi4mLi4uLqYct8RUcKIiJVqKjr7F69epGVleV1nV1eOF1nn1JUVERhYSFDhgwBYOrUqeTm5gKQnp7O5MmT+fOf/0yTJoG/4QcMGMDMmTOZO3cuhYWF3vj6oiMFEWmUqvqLviH51XV2OJYsWUJubi6LFy9mzpw5rF69mlmzZjF27FiWLl1KVlYWy5cvp2vXrrVafkV8PVIws1Fm9qmZbTWzWRVMP9/Mcszsn2b2LzMb42ceEZGqNGTX2ackJCSQlJTE22+/DcAzzzzDkCFDKCsrY9euXWRnZ/PAAw9QWFhIcXExn332GT179uS2226jb9++bN68uc4ZQvl2pGBm0cB8YDiQB3xoZoudcxtDmt0FvOCc+52ZdQOWAml+ZRIRqUpo19ljx45l7Nixp00fNWoUjz/+OOnp6Vx88cV16jo71MKFC/nxj3/M0aNH6dy5M0899RQnT55kypQpFBUV4Zzj5ptvJjExkTvuuIOcnByioqLo3r07o0ePrpcMp/h5+ugSYKtzbhuAmS0CxgOhRcEBLYPvE4DPfcwjIlKthuo6O/RJaxkZGRUedbzzzjunDR8+fJh58+aF8zFqzc+i0B7YFTKcB/Qv12Y28IaZ3Qg0By73MY+IiFTDnHP+LNhsIjDKOXd9cPgaoL9z7oaQNjODGR42s28DfwR6OOfKyi1rOjAdICUlJTP0odY1UVxcTHx8fK3m9ZNy1Yxy1VxjzVY+V0JCAhdeeGEEEwWcPHmS6OjoSMc4Q7i5tm7dSlFR0WnjsrOz1zrn+lY7s3POlxfwbWBZyPDtwO3l2nwCdAwZ3ga0rWq5mZmZrrZycnJqPa+flKtmlKvmGmu28rk2btwYmSDlfPHFF5GOUKFwc1W0HYE1Lox9t593H30IdDGzTmZ2HnA1sLhcm38DlwGY2beAOKDAx0wiIlIF34qCc64UuAFYBmwicJfRJ2b2czMbF2x2CzDNzD4G/gL8MFjRREQkAnz98ZpzbimB20xDx90T8n4jMMDPDCIiEj51cyEiUgeN8QJ+XagoiIiIR0VBRCRo1qxZzJ8/3xs+9SCc4uJiLrvsMvr06UPPnj157bXXql1WZV1sv/766/Tp04devXpx2WWXAYFbc6+77jp69uxJeno6L730Uv1/uDCpQzwRaZRmzIB19dtzNhkZ8GgV/exNmjSJGTNmcO211wLwwgsvsGzZMuLi4njllVdo2bIl+/fvJysri3HjxmFmlS6roi62y8rKmDZtGrm5uXTq1ImDBw8CMGfOHBISEli/fj2A9zyFSFBREBEJ6t27N/v27WPPnj1s27aNpKQkOnbsyIkTJ7jjjjvIzc0lKiqK3bt3k5+fT2pqaqXLmjt3Lq+88gqA18V2QUEBgwcP9rribtWqFQDLly8n9Ee5SUlJPn7KqqkoiEijVNVf9H666qqrePXVVyksLGTSpEkAPPvssxQUFLB27VpiYmJIS0ursMvsU8LtYrsx0jUFEZEQkyZN4qWXXuLFF1/kqquuAgJdZrdt25aYmBhycnLYuXNnlcuorIvtrKwscnNz2b59O4B3+mj48OGnXcuI5OkjFQURkRDdu3enuLiY9u3b065dOwAmT57MmjVr6Nu3L88++2y1D7UZNWoUpaWlpKenc/fdd3tdbLdp04YFCxZwxRVX0KtXL+9I5K677uLQoUP06NGDXr16kZOT4++HrIJOH4mIlLN69Wqv62wIdIe9atWqCtsWFxefMa6qLrZHjx59xjMQ4uPjWbhwYR0S1x8dKYiIiEdFQUREPCoKIiLiUVEQERGPioKIiHhUFERExKOiICJSB5V1nX22dqmtoiAiIh4VBRGRoPrsOvsU5xy33norPXr0oGfPnjz//PMA7Nmzh8GDB5ORkUGPHj14++23OXnyJD/84Q+9to888ki9f8bq6BfNItIozZgxg3X13Hd2RkYGj1bR0159dp19yssvv8y6dev4+OOP2b9/P/369WPw4ME899xzjBw5kjvvvJOTJ09y9OhR1q1bx+7du9mwYQMAhYWF9fPBa0BFQUQkqD67zj7lnXfe4Qc/+AHR0dGkpKQwZMgQPvzwQ/r168d//ud/cuLECSZMmEBGRgadO3dm27Zt3HjjjYwdO5YRI0Y0wKc+nYqCiDRKVf1F76f66Do7HIMHDyY3N5clS5ZwzTXXcOutt3Lttdfy8ccfs2zZMubPn88LL7zAk08+WR8fK2y6piAiEqI+us4ONWjQIJ5//nlOnjxJQUEBubm5XHLJJezcuZOUlBSmTZvGj370Iz766CP2799PWVkZV155JXPmzOGjjz7y62NWSkcKIiIhKus6+7vf/S59+/YlIyOj2q6zQ33ve99j1apV9OrVCzPjgQceIDU1lYULF/Lggw8SExNDfHw8Tz/9NLt37+a6666jrKwMgF/+8pe+fMaqqCiIiJRT166zQ8ebGQ8++CAPPvjgadOnTp3K1KlTz5gvEkcHoXT6SEREPCoKIiLiUVEQkUbFORfpCGe1um4/FQURaTTi4uI4cOCACkMtOec4cOAAcXFxtV6GLjSLSKPRoUMH8vLyKCgoiGiOkpKSOu1Y/RJOrri4ODp06FDrdagoiEijERMTQ6dOnSIdgxUrVtC7d+9IxzhDQ+TS6SMREfH4WhTMbJSZfWpmW81sViVtvm9mG83sEzN7zs88IiJSNd9OH5lZNDAfGA7kAR+a2WLn3MaQNl2A24EBzrlDZtbWrzwiIlI9P48ULgG2Oue2OeeOA4uA8eXaTAPmO+cOATjn9vmYR0REqmF+3fplZhOBUc6564PD1wD9nXM3hLR5Ffg/YAAQDcx2zr1ewbKmA9MBUlJSMhctWlSrTMXFxY3yEXnKVTPKVXONNZty1UxdcmVnZ691zvWttqFzzpcXMBF4ImT4GuA35dr8DXgFiAE6AbuAxKqWm5mZ6WorJyen1vP6SblqRrlqrrFmU66aqUsuYI0LY98d1ukjMxtgZs2D76eY2a/N7IJqZtsNdAwZ7hAcFyoPWOycO+Gc207gqKFLOJlERKT+hXtN4XfAUTPrBfwPsBN4upp5PgS6mFknMzsPuBpYXK7Nq8BQADNrDVwEbAszk4iI1LNwi0Jp8PBjPPCYc+4xoEVVMzjnSoEbgGXAJuAF59wnZvZzMxsXbLYMOGBmG4Ec4Fbn3IHafBAREam7cG9JPWxmtwNTgMFmFkXgOkCVnHNLgaXlxt0T8t4BM4MvERGJsHCPFCYBXwI/cs7tJXB94MGqZxERkbNN2EcKBE4bnTSzi4CuwF/8iyUiIpEQ7pFCLhBrZu2BN4HrgD/5FUpERCIj3KJgzrmjwBXAPOfc94Ae/sUSEZFICLsomNm3gcnAkhrOKyIiZ4lwd+wzCHRc90rwttLOBG4hFRGRr5GwLjQ751YCK80s3szinXPbgJ/4G01ERBpauN1c9DSzfwKfABvNbK2Zdfc3moiINLRwTx/9HpjpnLvAOXc+cAvwB/9iiYhIJIRbFJo757xrCM65FUBzXxKJiEjEhPvjtW1mdjfwTHB4Cuq4TkTkayfcI4X/BNoALwdfbYLjRETkayTcu48OobuNRES+9qosCmb2V6DS53U658ZVNk1ERM4+1R0pPNQgKUREpFGosigEf7R2GjPr45z7yL9IIiISKbXpv+iJek8hIiKNQm2KgtV7ChERaRRqUxTurfcUIiLSKITb99H3zCwBwDn3qpklmtkEf6OJiEhDC/dI4WfOuaJTA865QuBn/kQSEZFICbcoVNQu3C4yRETkLBFuUVhjZr82s28GX78G1voZTEREGl64ReFG4DjwPLAIKAH+269QIiISGeH2fXQEmOVzFhERibBw7z76h5klhgwnmdky/2KJiEgkhHv6qHXwjiPA6zW1rT+RREQkUsItCmVmdv6pATNLo4reU0VE5OwU7m2ldwLvmNlKAt1cDAKm+5ZKREQiItwLza+bWV8CheCfwKvAMT+DiYhIwwv3QvP1wJvALcBPCTyreXYY840ys0/NbKuZVXr3kpldaWYuWHhERCRCwr2mcBPQD9jpnMsGegMFVc1gZtHAfGA00A34gZl1q6Bdi+Dy369BbhER8UG4RaHEOVcCYGaxzrnNwMXVzHMJsNU5t805d5zAj97GV9BuDvArAj+IExGRCAq3KOQFf6fwKvAPM3sN+LyaedoDu0KXERznMbM+QEfn3JIwc4iIiI/MuZrdWWpmQ4AE4PXgEUBl7SYCo5xz1weHrwH6O+duCA5HAW8BP3TO7TCzFcBPnXNrKljWdIJ3O6WkpGQuWrSoRplPKS4uJj4+vlbz+km5aka5aq6xZlOumqlLruzs7LXOueqv2zrnfHkB3waWhQzfDtweMpwA7Ad2BF8lBI4++la13MzMTFdbOTk5tZ7XT8pVM8pVc401m3LVTF1yAWtcGPvu2jx5LVwfAl3MrJOZnQdcDSwOKUZFzrnWzrk051wasBoY5yo4UhARkYbhW1FwzpUCNwDLgE3AC865T8zs52Y2zq/1iohI7fn6oBzn3FJgablx91TSdqifWUREpHp+nj4SEZGzjIqCiIh4VBRERMSjoiAiIh4VBRER8agoiIiIR0VBREQ8KgoiIuJRURAREY+KgoiIeFQURETEo6IgIiIeFQUREfGoKIiIiEdFQUREPCoKIiLiUVEQERGPioKIiHhUFERExKOiICIiHhUFERHxqCiIiIhHRUFERDwqCiIi4lFREBERj4qCiIh4VBRERMSjoiAiIh4VBRER8agoiIiIR0VBREQ8vhYFMxtlZp+a2VYzm1XB9JlmttHM/mVmb5rZBX7mERGRqvlWFMwsGpgPjAa6AT8ws27lmv0T6OucSwdeBB7wK4+IiFTPzyOFS4CtzrltzrnjwCJgfGgD51yOc+5ocHA10MHHPCIiUg1zzvmzYLOJwCjn3PXB4WuA/s65Gypp/xtgr3PuvgqmTQemA6SkpGQuWrSoVpmKi4uJj4+v1bx+Uq6aUa6aa6zZlKtm6pIrOzt7rXOub7UNnXO+vICJwBMhw9cAv6mk7RQCRwqx1S03MzPT1VZOTk6t5/WTctWMctVcY82mXDVTl1zAGhfGvrtJrUpOeHYDHUOGOwTHncbMLgfuBIY45770MY+IiFTDz2sKHwJdzKyTmZ0HXA0sDm1gZr2B3wPjnHP7fMwiIiJh8K0oOOdKgRuAZcAm4AXn3Cdm9nMzGxds9iAQD/yvma0zs8WVLE5ERBqAn6ePcM4tBZaWG3dPyPvL/Vy/iIjUjH7RLCIiHhUFERHxqCiIiIhHRUFERDwqCiIi4lFREBERj4qCiIh4VBRERMSjoiAiIh4VBRER8agoiIiI55wpCs6nhwmJiHydnDNF4emnn+b666/ntttu48033+TLL/XoBhGR8s6ZopCUlESLFi145JFHuPzyy0lKSmLMmDE89thjbNq0SUcSIiL43HV2YzJu3DhatmxJ3759WbFiBW+88QbLli1jxowZAHTs2JERI0YwcuRIr2iIiJxrzpmicEp8fDzf+c53+M53vgPAjh07vALx4osv8sc//pGoqCj69evHyJEjGTFiBP3796dJk3NuU4nIOeicOX1UmbS0NKZPn85LL73E/v37effdd7n77rsxM+677z4GDhxIcnIyV1xxBb///e/ZsWNHpCOLiPhGf/6GaNKkCZdeeimXXnops2fP5uDBg7z11lssW7aMZcuW8corrwDQpUsX7ygiOzub+Pj4CCcXEakfKgpVaNWqFRMnTmTixIk45/j000+9U01PPvkkv/nNb4iJiWHAgAHe9YiMjAyios75AzAROUtp7xUmM6Nr16785Cc/YcmSJRw8eJA333yTm2++mcLCQu644w4yMzNJTU1l8uTJLFy4kD179kQ6tohIjehIoZZiY2MZNmwYw4YN41e/+hV79+5l+fLlLFu2jDfeeIPnnnsOgPT0dO8oYuDAgcTFxUU4uYhI5XSkUE9SU1OZMmUKzzzzDHv27OGf//wn999/P61bt2bu3LkMHz6cVq1aMXr0aB555BE2btyo30aISKNzzhwpLP50MX/+7M/sTt7NwI4DuSDxAt/WFRUVRUZGBhkZGdx2220cOXKElStXehesZ86cCUCHDh0YMWIEHTp0oGfPniQnJ/uWSUQkHOdMUZj3/jyW5y3nf/P+FwDDaBrTlOSmyXRo2YEurbqQnpJOVocs+n2jH+c1Oa/e1t28eXPGjBnDmDFjANi5cydvvPEGb7zxBi+//DKFhYXMmTOHfv36eaea+vfvT0xMTL1lEBEJxzlTFJ654hl+u/S3lCSVsLFgI9sPbSf/SD57i/ey64tdrMpbdVr7mKgYWsa2JDU+lbTENLq36U7mNzIZ2HEg32j5jTplueCCC5g2bRrTpk2jtLSUBQsWUFBQwLJly/jFL37BfffdR8uWLRk2bJh362vnzp3rtE4RkXCcM0UhNT6VYW2HMXTo0DOmlZSWsObzNbyf9z4f7/2YLQe3sPvwbg4cO8DGgo18UvAJS7Ys8dobRrOYZrRu1pqOLTvSJbkLvVJ6cWnHS+ndrjdNosLfrE2aNKFbt24MHTqUn/3sZxQWFvLmm296t76++uqrAFx44YXeUUR2djYtWrSo8zYRESnvnCkKVYlrEsfA8wcy8PyBFU7ffmg77/77XT7a+xGbCjaxo3AH+Ufy2X14NzuLdvLOrndOa39e1HkkxCWQGp9K56TOdG/bnb7t+jKg4wDaxretMktiYiJXXnklV155Jc45tmzZ4l2L+NOf/sRvf/tb70d2p44i+vTpo99GiEi9UFEIQ6ekTnRK6sSUXlPOmHb0+FFW717NB7s/YH3+erYc3MLnhz/n4LGDrN+3nvX71vPap6957aMsiuYxzWndrDXnJ5zPRckX0ezwAQzdAAAKtElEQVSLZiTuSSQ9Jf20nbuZcdFFF3HRRRdx44038uWXX/Lee+95RxF33nknd955J8nJyQwfPtwrEt/4Rt1Ob4nIuUtFoY6andeMYZ2GMazTsDOmlZWV8dmhz3hv13us3bOWzfs3s7NoJ/uO7OPfRf9me+F2Vu5cCcBjWx8DIDY6lsS4RNq1aEfnxM70bNuTvt/oy6UdL6VVs1ZkZ2eTnZ3NL3/5S/bt28c//vEP77cRixYtAqBHjx5egRg0aBBNmzZtuA0iImc1FQUfRUVF0SW5C12SuzA1Y+oZ078o+YJVeat44d0XONL8CFsPbuXzw59zqOQQ+UfyWbd3HS9vfvmr5VkU8efF06ZZG85POJ+urbuS0TWD/7n8f3jqqafYsGGDdxQxb948Hn74YeLi4hg8eDAjR45k5MiRdOvWDTNryM0gImcRX4uCmY0CHgOigSecc/eXmx4LPA1kAgeASc65HX5makxaxrVk5IUjic2LPeMCeFlZGZv2b2JV3io+2vMRn+7/lH9/8W8KjhSwvXA7nx36jJwdOafNExcdR2LTRNp/vz1jp46l+Z7mFG8sZtP7m7jlllu45ZZbaN++PSNGjGDEiBEMHz5cv40QkdP4VhTMLBqYDwwH8oAPzWyxc25jSLMfAYeccxea2dXAr4BJfmU6m0RFRdG9bXe6t+1e4fTCkkLe2/UeH+R9wIaCDWw7tC1wlHHsEHuL97KWtYGGnQKvqKIoYv8dS9FnRTzzwjM89dRTmBk9Mnrw3dHfZdSoUWRlZem3ESLnOD+PFC4BtjrntgGY2SJgPBBaFMYDs4PvXwR+Y2bm1P9DtRLjEhnTZQxjuow5Y1pZWRn/yv8Xq/JW8fHej9l8YDO7inZR0KaAI+lHKDtZBp+D2+pY/9l61v9yPb/4xS+wWCOhawIX9ruQIZcNIblJMic+OwEETl2dOu0UZVFEBXtIOXVh3DCiLOqr6cH3Vc0TRbllhsxf2Tgzo6CkgLwv8ogi6rRlnVp2Ze9PtatuHqm5srKyyqdR+bTSslJKy0r9iFQnjTVXVdu5vvhZFNoDu0KG84D+lbVxzpWaWRGQDOz3MdfXXlRUFBntMshol1Hh9H3F+1iVt4oPPv+Ajfs2suXzLeR9nMfhTYcp3FLImo/XsOaJNRAPNMSBQ0Nd4miI9ehyjfio+7jubBi2wdd1nBUXms1sOjAdICUlhRUrVtRqOcXFxbWe10+RyJVAAsOjhjM8dTikAn0C40+cPMGqrat487032fbpNq+9w1X4PmTkme0qOOA7Yzmu8ukVjndQ5soqvFhe2QFmbddTVfaKxrsyV6uL+FbTSlKbwuPw/QaDGn8OAv9mje7GB9dIcwEXJ13s/77COefLC/g2sCxk+Hbg9nJtlgHfDr5vQuAIwapabmZmpqutnJycWs/rJ+WqGeWqucaaTblqpi65gDUujH23nydQPwS6mFknMzsPuBpYXK7NYuDUvZoTgbeC4UVEJAJ8O33kAtcIbiBwNBANPOmc+8TMfk6gYi0G/gg8Y2ZbgYMECoeIiESIr9cUnHNLgaXlxt0T8r4EuMrPDCIiEj7dfyciIh4VBRER8agoiIiIR0VBREQ8KgoiIuKxs+1nAWZWAOys5eytaZxdaChXzShXzTXWbMpVM3XJdYFzrk11jc66olAXZrbGOdc30jnKU66aUa6aa6zZlKtmGiKXTh+JiIhHRUFERDznWlFYEOkAlVCumlGummus2ZSrZnzPdU5dUxARkaqda0cKIiJSha9lUTCzUWb2qZltNbNZFUyPNbPng9PfN7O0RpLrh2ZWYGbrgq/rGyjXk2a2z8wqfKSTBcwN5v6XmfVpJLmGmllRyPa6p6J29Zypo5nlmNlGM/vEzG6qoE2Db68wc0Vie8WZ2Qdm9nEw170VtGnw72OYuSLyfQyuO9rM/mlmf6tgmr/bK5yHLpxNLwLddH8GdAbOAz4GupVr81/A48H3VwPPN5JcPwR+E4FtNpjAs9c2VDJ9DPB3As/8ygLebyS5hgJ/a+Bt1Q7oE3zfAvi/Cv4dG3x7hZkrEtvLgPjg+xjgfSCrXJtIfB/DyRWR72Nw3TOB5yr69/J7e30djxQuAbY657Y5544Di4Dx5dqMBxYG378IXGb+P3svnFwR4ZzLJfA8i8qMB552AauBRDNr1whyNTjn3B7n3EfB94eBTQSeNR6qwbdXmLkaXHAbFAcHY4Kv8hcyG/z7GGauiDCzDsBY4IlKmvi6vb6ORaE9sCtkOI8zvxxeG+dcKVAEJDeCXABXBk85vGhmHX3OFK5ws0fCt4OnAP5uZt0bcsXBw/beBP7KDBXR7VVFLojA9gqeClkH7AP+4ZyrdHs14PcxnFwQme/jo8D/AGWVTPd1e30di8LZ7K9AmnMuHfgHX/01IBX7iMBP93sB84BXG2rFZhYPvATMcM590VDrrU41uSKyvZxzJ51zGUAH4BIz69EQ661OGLka/PtoZt8B9jnn1vq9rsp8HYvCbiC0oncIjquwjZk1ARKAA5HO5Zw74Jz7Mjj4BJDpc6ZwhbNNG5xz7otTpwBc4Cl/MWbW2u/1mlkMgR3vs865lytoEpHtVV2uSG2vkPUXAjnAqHKTIvF9rDZXhL6PA4BxZraDwCnmYWb253JtfN1eX8ei8CHQxcw6mdl5BC7ELC7XZjEwNfh+IvCWC161iWSucuedxxE4L9wYLAauDd5VkwUUOef2RDqUmaWeOpdqZpcQ+P/Z151JcH1/BDY5535dSbMG317h5IrQ9mpjZonB902B4cDmcs0a/PsYTq5IfB+dc7c75zo459II7CPecs5NKdfM1+3l6zOaI8E5V2pmNwDLCNzx86Rz7hMz+zmwxjm3mMCX5xkz20rgQubVjSTXT8xsHFAazPVDv3MBmNlfCNyZ0trM8oCfEbjwhnPucQLP2R4DbAWOAtc1klwTgf9nZqXAMeDqBijuA4BrgPXB89EAdwDnh+SKxPYKJ1cktlc7YKGZRRMoQi845/4W6e9jmLki8n2sSENuL/2iWUREPF/H00ciIlJLKgoiIuJRURAREY+KgoiIeFQURETEo6Ig0oAs0FPpGT1fijQWKgoiIuJRURCpgJlNCfa3v87Mfh/sPK3YzB42s4/M7E0zaxNsm2Fmq4Mdp71iZknB8Rea2fJgB3Qfmdk3g4uPD3awttnMnm2AHnpFwqaiIFKOmX0LmAQMCHaYdhKYDDQHPnLO9QFWEviFNcDTwG3BjtPWh4x/Fpgf7IDuUuBUVxe9gRlANwLP1xjg+4cSCdPXrpsLkXpwGYHOzz4M/hHflED3ymXA88E2fwZeNrMEINE5tzI4fiHwv2bWAmjvnHsFwDlXAhBc3gfOubzg8DogDXjH/48lUj0VBZEzGbDQOXf7aSPN7i7XrrZ9xHwZ8v4k+h5KI6LTRyJnehOYaGZtAcyslZldQOD7MjHY5j+Ad5xzRcAhMxsUHH8NsDL49LM8M5sQXEasmTVr0E8hUgv6C0WkHOfcRjO7C3jDzKKAE8B/A0eA7ma2lsDTriYFZ5kKPB7c6W/jq15RrwF+H+zh8gRwVQN+DJFaUS+pImEys2LnXHykc4j4SaePRETEoyMFERHx6EhBREQ8KgoiIuJRURAREY+KgoiIeFQURETEo6IgIiKe/w/d3EKkfrf5+wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12bdffca7f0>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8XHWd//HXJ5PJ5NI0LS2E0gstAoVCoUCAukUIK/ug0H2ArnhBWdTF7cPfT1m8reL64ybuYxFQEQShKsKCgigIXa2wggyVqy1QoFdaKNAUem/TTHOdmc/vj5lMpm2SSdMcJsl5Px+PPHou35nzmU9O55PzPed8j7k7IiIiACXFDkBERAYPFQUREclRURARkRwVBRERyVFREBGRHBUFERHJUVEQEZEcFQUREclRURARkZzSYgewr8aOHeuTJ0/u12t37dpFVVXVwAY0RCkXXZSLDOWhy3DMxYsvvrjF3Q8s1G7IFYXJkyezePHifr02Ho9TX18/sAENUcpFF+UiQ3noMhxzYWZv96Wduo9ERCRHRUFERHJUFEREJGfInVMQkeGro6ODhoYGWltbixpHTU0NK1asKGoM/VVeXs6ECROIRqP9er2KgogMGg0NDVRXVzN58mTMrGhxNDU1UV1dXbTt95e7s3XrVhoaGpgyZUq/3iOw7iMzu9PMNpnZ0l7a1JvZEjNbZmZPBRWLiAwNra2tjBkzpqgFYSgzM8aMGbNfR1pBnlO4C5jd00ozGwXcBpzn7scAHw8wFhEZIlQQ9s/+5i+wouDuC4FtvTT5NPCQu7+Tbb8pqFgAlm1axp1r72TTrkA3IyIypBXz6qMjgdFmFjezF83s4iA3tnzzcu555x4VBRHp0Y4dO7jtttv69dpzzz2XHTt29Ln91VdfzY033tivbQWpmCeaS4GTgA8DFcBzZva8u7++Z0MzmwvMBaitrSUej+/zxpZvXg7AokWL2FK1pf9RDxOJRKJfeRyOlIuMwZCHmpoampqairb9hoYGfvKTn/DpT396rziSySSlpT1/Zf7mN78B6HP8bW1tRKPRQD5va2trv3+XxSwKDcBWd98F7DKzhcDxwF5Fwd3nAfMA6urqvD+3n29ZvgWWQ11dHdNrp+9X4MPBcLyNv7+Ui4zBkIcVK1YU9aqf733ve6xdu5bTTz+ds88+mzlz5nDNNdcwbtw4lixZwvLly/nIRz7CunXraG1t5bLLLmPu3LlA1xA8iUSCc845h9NOO41nn32W8ePH88gjj1BRUbHbtmKxGLFYjOrqapYsWcIXv/hFmpub+cAHPsCdd97J6NGjufnmm7n99tspLS1l2rRp3H///Tz11FNcdtllQOb8wcKFC/fKWXl5OSeccEK/clDMovAI8BMzKwXKgFOBHwW1MSNz8sXxoDYhIgPoK49+hSUblgzoe844eAY3zb6px/XXXXcdS5cu5ZlnnqG6upp4PM7f/vY3li5dmrvE88477+SAAw6gpaWFk08+mY997GOMGTNmt/dZvXo19913Hz/72c/4xCc+wYMPPshFF13U43YvvvhibrnlFs444wyuvPJKrrnmGm666Sauu+461q5dSywWy3VN3Xjjjdx6663MmjWLRCJBeXn5AGSmS5CXpN4HPAdMNbMGM7vEzL5oZl8EcPcVwKPAq8DfgJ+7e4+Xrw5APGS3G9QmRGQYOuWUU3a75v/mm2/m+OOPZ+bMmaxbt47Vq1fv9ZopU6YwY8YMAE466STeeuutHt+/sbGRHTt2cMYZZwDw2c9+loULFwJw3HHH8ZnPfIZ7770313U1a9Ysvva1r3HzzTezY8eOXru0+iOwIwV3v7APbW4AbggqhnydRwoiMjT09hf9+yl/CO14PM7jjz/Oc889R2VlJfX19d3eExCLxXLTkUiElpaWfm37j3/8IwsXLmT+/Plce+21LFu2jMsvv5w5c+awYMECZs6cyeOPP85RRx3Vr/fvTujGPlL3kYj0pLq6utcTv42NjYwePZrKykpWrlzJ888/v9/brKmpYfTo0fz1r38F4J577uGMM84gnU6zbt06zjzzTK6//np27NhBIpHgjTfeYPr06XzrW9+irq6OlStX7ncM+UIzzIW6j0SkkDFjxjBr1ixOPfVU5syZw5w5c3ZbP3v2bG6//XaOO+44pk6dysyZMwdku3fffXfuRPNhhx3GL3/5S1KpFBdddBGNjY24O1/96lcZNWoUV1xxBU8++SQlJSUcc8wxnHPOOQMSQ6fwFAV1H4lIH/z617/ebeyj/CuyYrEYf/rTn7p9Xed5g7Fjx7J0adfp0W984xvdtr/66qtz0zNmzOj2qOPpp5/ea9ktt9xS6CPsF3UfiYhITmiKgrqPREQKC09R0H0KIkOC/nDbP/ubv/AUBY28KDLolZeXs3XrVhWGfup8nsL+3NAWmhPNnbSziQxeEyZMoKGhgc2bNxc1jtbW1gG/U/j90vnktf4KTVFQ95HI4BeNRvv9xLCBFI/H+z120FAXuu4jHSmIiPQsPEVB9ymIiBQUmqLQSd1HIiI9C01RUPeRiEhh4SkK6j4SESkoNEWhk7qPRER6FpqioO4jEZHCgnzy2p1mtsnMen2ampmdbGZJM7sgqFhA9ymIiPRFkEcKdwGze2tgZhHg+8D/BhhH57aC3oSIyJAXWFFw94XAtgLNLgUeBDYFFcee1H0kItKzog1zYWbjgY8CZwInF2g7F5gLUFtbSzwe3+ftvbL9FQBeevklUmtT+/z64SaRSPQrj8ORcpGhPHQJcy6KOfbRTcC33D1dqGvH3ecB8wDq6uo8/0lIfZVem4ZXM084OmPyGf0Id3iJx+P0J4/DkXKRoTx0CXMuilkU6oD7swVhLHCumSXd/eEgNqb7FERECitaUXD33FCIZnYX8IegCsJu29XVRyIiPQqsKJjZfUA9MNbMGoCrgCiAu98e1HZ7iYfstt/vTYuIDBmBFQV3v3Af2n4uqDg66T4FEZHCQndHs4iI9Cw0RaGTuo9ERHoWmqKg7iMRkcLCUxTUfSQiUlBoikIndR+JiPQsNEVB3UciIoWFpyjoPgURkYLCUxQ0zIWISEGhKQqd1H0kItKz0BQFdR+JiBQWnqKgE80iIgWFpyjoPgURkYJCUxQ6qftIRKRnoSkK6j4SESksPEVB3UciIgUFVhTM7E4z22RmS3tY/xkze9XMXjOzZ83s+KBiyafuIxGRngV5pHAXMLuX9WuBM9x9OnAtMC/AWNR9JCLSB0E+eW2hmU3uZf2zebPPAxOCigV0n4KISF8MlnMKlwB/CnIDGuZCRKSwwI4U+srMziRTFE7rpc1cYC5AbW0t8Xh8n7ezumk1AK8tfY2aDTX9CXVYSSQS/crjcKRcZCgPXcKci6IWBTM7Dvg5cI67b+2pnbvPI3vOoa6uzuvr6/d5W6M2jIKX4JhjjqH+6H1//XATj8fpTx6HI+UiQ3noEuZcFK37yMwmAQ8B/+zurwe+PZ1oFhEpKLAjBTO7D6gHxppZA3AVEAVw99uBK4ExwG3Zk8BJd68LMJ6g3lpEZNgI8uqjCwus/wLwhaC238t23+9NiogMGYPl6qPAqftIRKSw8BQFdR+JiBQUmqLQSd1HIiI9C01RUPeRiEhh4SkKGuZCRKSg8BQFDXMhIlJQaIpCJ3UfiYj0LDRFQd1HIiKFhaco6ESziEhB4SkKuk9BRKSg0BSFTuo+EhHpWWiKgrqPREQKC09R0IlmEZGCwlMUdJ+CiEhBoSkKndR9JCLSs9AUBXUfiYgUFlhRMLM7zWyTmS3tYb2Z2c1mtsbMXjWzE4OKBdR9JCLSF0EeKdwFzO5l/TnAEdmfucBPA4wlR91HIiI9C6wouPtCYFsvTc4H/tszngdGmdm4oOJR95GISGHFPKcwHliXN9+QXRYI3acgIlJYabED6Aszm0umi4na2lri8fg+v8fG1o0ArFy5knjjvr9+uEkkEv3K43CkXGQoD13CnItiFoX1wMS8+QnZZXtx93nAPIC6ujqvr6/f54290/gOvABTp06l/sR9f/1wE4/H6U8ehyPlIkN56BLmXBSz+2g+cHH2KqSZQKO7vxfUxtR9JCJSWGBHCmZ2H1APjDWzBuAqIArg7rcDC4BzgTVAM/D5oGLJxkN220FuRkRkSAusKLj7hQXWO/CloLa/J92nICJSWGjuaO6k7iMRkZ71qSiY2Swzq8pOX2RmPzSzQ4MNbWCp+0hEpLC+Hin8FGg2s+OBbwJvA/8dWFQBUPeRiEhhfS0Kyew5gPOBH7v7j4Hq4MIKjrqPRER61tcTzU1m9m3gIuB0MysheyXRUKHuIxGRwvp6pPBJoA24xN03kLnR7IbAogqA7lMQESmsz0cKZLqNUmZ2JHAUcF9wYQ28ziOFLy34EhsSG/jumd8tckQiIoNPX48UFgIxMxsPPEHmRrO7ggoqaNcuvLbYIYiIDEp9LQrm7s3APwG3uPtHgWODC2vg6eojEZHC+lwUzOyDwGeAP+7jaweFzu4jERHpWV+/2L8CfBv4vbsvM7PDgCeDC2vg6UhBRKSwPp1odvengKfMbISZjXD3N4F/CzY0ERF5v/V1mIvpZvYysAxYbmYvmtkxwYY2sNR9JCJSWF+7j+4Avubuh7r7JODrwM+CC2vgqftIRKSwvhaFKnfPnUNw9zhQFUhEAdGRgohIYX29ee1NM7sCuCc7fxHwZjAhiYhIsfT1SOFfgAOBh7I/B2aX9crMZpvZKjNbY2aXd7N+kpk9aWYvm9mrZnbuvgS/L9R9JCJSWF+vPtrOPl5tZGYR4FbgH4AGYJGZzXf35XnN/h/wgLv/1MymkXlE5+R92c4+xBPE24qIDCu9FgUz+x/oeQQ5dz+vl5efAqzJXr6Kmd1PZujt/KLgwMjsdA3wbh9iFhGRgBQ6UrhxP957PLAub74BOHWPNlcD/2tml5I5cX1Wd29kZnOBuQC1tbXE4/F9DqY52bzbfH/eYzhJJBKhz0En5SJDeegS5lz0WhSyN63txsxOdPeXBmj7FwJ3ufsPssNo3GNmx7p7eo845gHzAOrq6ry+vn6fN5RoT8AzXfP9eY/hJB6Phz4HnZSLDOWhS5hz0Z/xi37ex3brgYl58xOyy/JdAjwA4O7PAeXA2H7EVJBONIuIFNafotDXb9dFwBFmNsXMyoBPAfP3aPMO8GEAMzuaTFHY3I+YRERkAPSnKFzTl0bungS+DDwGrCBzldEyM/uumXWeoP468K9m9gqZh/Z8zgN6XqauPhIRKaxPl6Sa2UeBv7h7o7s/bGajgHp3f7i317n7AjKXmeYvuzJvejkwa9/D3nfqPhIRKayvRwpXuXtj54y77wCuCiakYOhIQUSksL4Whe7a9XWIDBERGSL6WhQWm9kPzewD2Z8fAi8GGdhAU/eRiEhhfS0KlwLtwG+A+4FW4EtBBRUEdR+JiBTW17GPdgF7DWgnIiLDS1+fvPbn7BVHnfOjzeyx4MIaeOo+EhEprK/dR2OzVxwBuVFTDwompGCo+0hEpLC+FoW0mU3qnDGzyfQyeupgpCMFEZHC+npZ6XeAp83sKTLDXHyI7KilIiIyfPT1RPOjZlZHphC8DDwMtAQZ2EBT95GISGF9HebiC8BlZEY6XQLMBJ4D/j640AaWuo9ERArr6zmFy4CTgbfd/UzgBIbYaKY6UhARKayvRaHV3VsBzCzm7iuBqcGFJSIixdDXE80N2fsUHgb+bGbb0fOURUSGnb6eaP5odvJqM3sSqAEeDSwqEREpin1+yI67P+Xu8929vVBbM5ttZqvMbI2ZdTtMhpl9wsyWm9kyM/v1vsYjIiIDJ7Dhr80sAtwK/APQACwys/nZB+t0tjkC+DYwy923m9mQuktaRGS46c/jOPvqFGCNu7+ZPaq4Hzh/jzb/CtyaHTYDd98UYDwiIlJAkEVhPLAub74huyzfkcCRZvaMmT1vZrMDjEdERAoo9tPTSoEjgHoyN8YtNLPp+YPvAZjZXLLDatTW1hKPx/d7wwPxHkNZIpEIfQ46KRcZykOXMOciyKKwHpiYNz8huyxfA/CCu3cAa83sdTJFYlF+I3efB8wDqKur8/r6+v5F9FTXZL/fY5iIx+Ohz0En5SJDeegS5lwE2X20CDjCzKaYWRnwKWD+Hm0eJnOUgJmNJdOd9GaAMYmISC8CKwrungS+DDwGrAAecPdlZvZdMzsv2+wxYKuZLQeeBP7d3bcGFZOIiPQu0HMK7r4AWLDHsivzph34WvZHRESKLMjuIxERGWJUFEREJEdFQUREckJbFLY0byl2CCIig05oi8JZ/31WsUMQERl0QlsUXtn4SrFDEBEZdEJbFEREZG8qCiIikqOiICIiOSoKIiKSo6IgIiI5KgoiIpKjoiAiIjkqCiIikqOiICIiOSoKIiKSE2hRMLPZZrbKzNaY2eW9tPuYmbmZ1QUZj4iI9C6womBmEeBW4BxgGnChmU3rpl01cBnwQlCx5LaFBb0JEZEhLcgjhVOANe7+pru3A/cD53fT7lrg+0BrgLEAKgoiIoUEWRTGA+vy5huyy3LM7ERgorv/McA4RESkj0qLtWEzKwF+CHyuD23nAnMBamtricfj/dpmiZWQ9nRuvr/vMxwkEolQf/58ykWG8tAlzLkIsiisBybmzU/ILutUDRwLxM0M4GBgvpmd5+6L89/I3ecB8wDq6uq8vr6+XwHZwt27j/r7PsNBPB4P9efPp1xkKA9dwpyLILuPFgFHmNkUMysDPgXM71zp7o3uPtbdJ7v7ZOB5YK+CICIi75/AioK7J4EvA48BK4AH3H2ZmX3XzM4Laru9iZZEi7FZEZEhI9BzCu6+AFiwx7Ire2hbH2QsALGSGM2p5tz8yi0rOWrsUUFvVkRkyAjVHc3lkfLd5v/uF39XpEhERAanUBeF7a3bixSJiMjgFKqiECuJFTsEEZFBTUVBRERyQlUU9uw+EhGR3YWqKJw0+qRihyAiMqiFqihcMP4CVl+6mqljphY7FBGRQSlURcHMOPyAwzlh3AnFDkVEZFAKVVHolEwnc9Puzjf//E0WrV9UxIhERAaHUBaFcw8/NzedTCe54dkb+OAvPljEiEREBodQFoXPzfgcHzv6YxxSfQgpTwGZriURkbALZVEwMw6pPoSWjpZcV5KeyiYiEtKiAFAZraS5o7mrKOhIQUQk3EWhLdVGW7INyDyVTUQk7EL7TVhRWgHAzradgIqCiAiEuChURiuBrqKgcwoiIgEXBTObbWarzGyNmV3ezfqvmdlyM3vVzJ4ws0ODjCdfZ1Foam/qjOX92rSIyKAVWFEwswhwK3AOMA240Mym7dHsZaDO3Y8DfgdcH1Q8e9KRgojI3oI8UjgFWOPub7p7O3A/cH5+A3d/0t07n4/5PDAhwHh2UxHNnFNobG0EdE5BRASCfUbzeGBd3nwDcGov7S8B/tTdCjObC8wFqK2tJR6P9yugRCKRe+3q7asBWLx0MQDpVLrf7zsU5eci7JSLDOWhS5hzEWRR6DMzuwioA87obr27zwPmAdTV1Xl9fX2/thOPx+l8bdm6MngVxh06DtZANBqlv+87FOXnIuyUiwzloUuYcxFkUVgPTMybn5BdthszOwv4DnCGu7cFGM9udE5BRGRvQXakLwKOMLMpZlYGfAqYn9/AzE4A7gDOc/dNAcayl877FHROQUSkS2DfhO6eBL4MPAasAB5w92Vm9l0zOy/b7AZgBPBbM1tiZvN7eLsB13mkcNvi2wAVBRERCPicgrsvABbssezKvOmzgtx+bzqLQifdpyAiojuac3ROQUQkxEWhvLR8t3kdKYiIhLgomBlPXPxEbl7nFEREQlwUAGpiNblpFQURkbAXhfKuorBnd5KISBiFuyjkHSkcPOLgIkYiIjI4hLso5B0pLHx7IVuatxQxGhGR4gt1USiLlPG9M7+Xm3/8zceLGI2ISPGFuigAfOf07+Smr4pfxcvvvVzEaEREiiv0RSHf61tf58R5JxY7DBGRolFREBGRHBWFbvx+xe/3WpZoT7C9ZXsRohGRILSn2nND50sXFYVu/NMD/8R/PPEfPLzyYW792624O0feciQHXH9AsUMTkQFy9r1nU3NdTeGGITMonrw2WDz9+ac57ZenAfBfT/9XbvnUsVN5L/FescISkQDE34oXO4RBSUcKeWZNmsX1Z12/1/J7X703N33RQxfR3NG8T++7YPUC3QMhMkil0qlul6/ZtoZJP5rEO43vvM8RFVegRcHMZpvZKjNbY2aXd7M+Zma/ya5/wcwmBxlPT9Z/bT2vf/l1AL7xd99gw9c3cOxBx+bW3/3K3bnpX732K6596lqS6SRAwS/77S3bmfPrOXz8tx8PIHKR/tmY2LjbfCqdoi31vj0Nd1Bpam/qdvkdi+9g3c513Pfafe9zRMUVWFEwswhwK3AOMA240Mym7dHsEmC7ux8O/Aj4flDx9OaQ6kM4YswRQGb01NoRtbz2f17j+Uue77b9dc9cx6k/P5Ur/nIFB95wIA8uf5BPP/hpVm1ZxTf//E22Nm9l2aZlACx+dzGQOVR9p/Ed3tz+Jqu2rOLtHW+zasuq3HtuTGwkmU6STCdxd15+72WeXfdsvz7PtpZtpD3dr9cOpCUblvDqxld3WxZ/K85f3/5rkSIKn2/87zf4xUu/2G3ZovWLOPgHB+/2ZfelBV9i9tOzcff3O8TAuDt/fuPPBT9TTyebB8P/oWII8pzCKcAad38TwMzuB84Hlue1OR+4Ojv9O+AnZmY+SPbMUyecSuPljXzolx/a68vtpfde4qX3XgLggt9eAMB9SzP/yW549gYAph80ndc2vZZ7zaE3HbrXNv7z7/+T2qpavvA/XwAyd1mPqRiTO4cxqWYS7za9y7x/nMfcP8zlitOv4OixR/OLl3/B9IOmM+fIORw2+jAAXtnwCks2LOHK+JV850Pf4YrTr+CZdc8wunw0I2MjqYhW0NzRzKV/upRjS47l5PaT2dm2k4//9uNMqpnE98/6Pjc+eyMA5009j7GVY5kwcgJVZVU0dzTTmmylpaOFjbs2MmHkBMpLyyktKaW0pBTDSHmKLc1bmDhyImWRMk644wQA/CpnY2IjW5q3cObdZwKw5tI1HDb6MN7Y/gZTRk0hUhLJ5aTz1+84LR0tVJVV7ZW39TvXc/CIg3OvS6aTrGtcR3WsmrGVY/dq35Zsoy3VxsjYyNyyHa07qInVkPIUm3Zt4qCqg3Z7jbvjOKl0imgkutd7Auxq35WLL+1p0p6mtKSU1mQrEYuws20nI8pG8MiqRzjn8HOojlXv9R4vNLzAQyse4rqzrqMj3UFZpKzbbXVKtCeIRWKYGb9f8XvmHDmHymglTW1NxEpjudenPc0PnvsBAJeceAmpdIodrTt4YNkDADz6xqOcPP5kDOOOF+8A4N2mdxk/cjzJdJLfr/g9Hz7sw5SXlvOXtX9h7fa1XHrqpb3G1hcbExtZvW01p006bb/fqzd3v3I3n3/k89z9kbu5+PiLd1uX/xXT1NZ1pJBMJ9nZtpNEMoHTtR92enD5g0yvnU5VtIrxI8f3uv3O/Wd/R2Bu7mje66FgQbKgvn/N7AJgtrt/ITv/z8Cp7v7lvDZLs20asvNvZNv02CdTV1fnixcv7ldM8Xic+vr6fr0WoCPVgePE34pz9r1nUxWtYkTZCDbuyhyKz5wwk+cbuj+6CJuK0gpaki1A5ql2+f+xOkVLonSkOwAYGRtJa7KVymglifYEkBmwcGvLVg6sPJBoJMq2lm2MjI0k0Z7IndcZXz2etKfZ1rIt1/0RsQjVsWrakm1Ux6pJppNsa9kGQG1VLZGSCIaxvmk91WXVNLU3YRgHVh3IiLIRub8Q3216l2Q6SSwSY1z1uNyXfiqdIu1pkukkm5s3UxWtoqa8hq3NW4FMYe+pS2LKqCmkPIW7k/Y0LcmWXGyV0UpaOlqYWDORiOUVyWxxrIxWsq1lG4n2BJXRStpSbbSn2nN52JDYQEW0gtqqWjYkNpBMJ3M5mTxqMttathW8BPOAigNyX4zdqYnVMKJsBJGSCC0dLThOWaSMtKeJRWI0tTdRWlJKiZWQTCepidXQkmyhLFJGxCK0p9pZt3MdABNHTsRxOlId7OrYRU2shqqyqr32l/zvqELL85e9l3iP1mQrVdEqRpWPYmfbTiqiFaTSKRzP5b3zD5y0p9mY2Miujl2UWilJT+ZyckDFAbR0tLC+af1uuYqWRHN/MERLornPuiGxgRIrobSklDEVY2hqb2J7y3Ym1Uzqtkg4jrtn4i2rorSklOaOZtyd9U3rmTxqMqUlpcw9cS7/Puvfe/0d9sTMXnT3ukLthsTVR2Y2F5gLUFtbSzwe79f7JBKJfr82XxllPHnGk92v/EDmn9caX2NK1RQqI5WUWAltqTZe2vESB8UOojxSTkNzA9GSKCOjI1m+cznHjDyGhpYGdnTsYPrI6bzd/Dajy0bzzJZnOLj8YMyMpo4mxleMpynZxK7kLra2b6Ul1cKBsQNpTjWT9jRjY2MZVz6Od1veJeWZE2jt6XZ2JncyqXIS29q3Ma58HCu2ryBWFgNgxqgZvJF4A8j8ddmebqfDOzCMqtIqUp4i5SlGlo7k1cZXaUu3MaZsDM2pZo6uPprN7ZtpTjZzSMUhuDulJaVsbd/K5tbN1JTV0JHuoLGjkbZUG2NiY1jZtJLTx55OmjQtqRYqIpkCUhGpoD3dnvlySTYRIULF6ApSnqLd26kcWUl7up1oSZR3dr1DTbSGWCRGCSUwAt5tfReAg8szI966O5GSCFGLsrlyM6PLRtOWbiNqUdKeZlzpOA6KHURjayMWyXzWpCdz/2nrqupo93ZwaE41Y2ZEiGBmlFgJJZSQqEoQsUjmCKGylKrSKlpSLbkjp8aORmpjtSzevpgZo2bQ4R2UUIKZYRhRi8LozO8o7WmqIlUkUond9ymHSCxCylNEy6O0pFqIWIQSK2Fp41KmjZxGtCTK5LLJlFrmyG3qmKmkSfNG4g1GR0dTU1ZDuixNIplgbNlYXt7xMkePPJpNrZsYVzGO57c+z+jS0Rw76lhiJTFiJZl9o8RKKLVSNrZtpD3VnvuiS3uassqy3D41lQ8FAAAG5klEQVRWQgkd3kFlVSUpT9GcaqY52Ux5pBwrNSIlEdwdKzMmRieypW0Lh5YfSqQkQqmVErEIzcnm3H7XKTed92DEbtfvMY3B4bHDc/lJe5p0LE11tJr2dDspT/FS8iVqy2sZXz4+U1QcxteMpzXVStSjVMeqea/1PcaWjc38zktLGFMyhrGxsby16y2Oqj4qV/wAkp6kNFZKylNMq5jGiMgI2tJttKfbaS9rJ1WVIhaJ9fi90lkMk+kkhhEpj5AixSgbxcFlB+PuNK1vGpDvsF65eyA/wAeBx/Lmvw18e482jwEfzE6XAlvIHr309HPSSSd5fz355JP9fu1wo1x0US4ylIcuwzEXwGLvw3d3kFcfLQKOMLMpZlYGfAqYv0eb+cBns9MXAH/JBi8iIkUQWPeRuyfN7MtkjgYiwJ3uvszMvkumYs0HfgHcY2ZrgG1kCoeIiBRJoOcU3H0BsGCPZVfmTbcCuoBfRGSQ0B3NIiKSo6IgIiI5KgoiIpKjoiAiIjkqCiIikhPYMBdBMbPNwNv9fPlYMjfIiXKRT7nIUB66DMdcHOruBxZqNOSKwv4ws8Xeh7E/wkC56KJcZCgPXcKcC3UfiYhIjoqCiIjkhK0ozCt2AIOIctFFuchQHrqENhehOqcgIiK9C9uRgoiI9CI0RcHMZpvZKjNbY2aXFzueIJnZRDN70syWm9kyM7ssu/wAM/uzma3O/js6u9zM7OZsbl41sxOL+wkGnplFzOxlM/tDdn6Kmb2Q/cy/yQ7vjpnFsvNrsusnFzPugWZmo8zsd2a20sxWmNkHw7hfmNlXs/83lprZfWZWHtZ9Yk+hKApmFgFuBc4BpgEXmtm04kYVqCTwdXefBswEvpT9vJcDT7j7EcAT2XnI5OWI7M9c4Kfvf8iBuwxYkTf/feBH7n44sB24JLv8EmB7dvmPsu2Gkx8Dj7r7UcDxZHISqv3CzMYD/wbUufuxZIb2/xTh3Sd215cn8Qz1H/rwFLjh/AM8AvwDsAoYl102DliVnb4DuDCvfa7dcPgBJpD5svt74A9kHu64BSjdc/+gH08DHCo/QA2wds/PE7b9AhgPrAMOyP6O/wCcHcZ9orufUBwp0LUTdGrILhv2soe6JwAvALXu/l521QagNjs93PNzE/BNIJ2dHwPscM8+mX33z5vLRXZ9Y7b9cDAF2Az8MtuV9nMzqyJk+4W7rwduBN4B3iPzO36RcO4TewlLUQglMxsBPAh8xd135q/zzJ89w/7SMzP7R2CTu79Y7FgGgVLgROCn7n4CsIuuriIgHPtF9pzJ+WSK5CFAFTC7qEENImEpCuuBiXnzE7LLhi0zi5IpCL9y94eyizea2bjs+nHApuzy4ZyfWcB5ZvYWcD+ZLqQfA6PMrPPJg/mfN5eL7PoaYOv7GXCAGoAGd38hO/87MkUibPvFWcBad9/s7h3AQ2T2kzDuE3sJS1FYBByRvbqgjMxJpflFjikwZmZknn+9wt1/mLdqPvDZ7PRnyZxr6Fx+cfZqk5lAY153wpDm7t929wnuPpnM7/0v7v4Z4EnggmyzPXPRmaMLsu2HxV/O7r4BWGdmU7OLPgwsJ3z7xTvATDOrzP5f6cxD6PaJbhX7pMb79QOcC7wOvAF8p9jxBPxZTyPTBfAqsCT7cy6ZftAngNXA48AB2fZG5uqsN4DXyFyVUfTPEUBe6oE/ZKcPA/4GrAF+C8Syy8uz82uy6w8rdtwDnIMZwOLsvvEwMDqM+wVwDbASWArcA8TCuk/s+aM7mkVEJCcs3UciItIHKgoiIpKjoiAiIjkqCiIikqOiICIiOSoKIt0ws8lmtnQf2n/OzA7pQ5uf7H90IsFRURAZGJ8jM2SCyJCmoiDSs1Izuzv7LIHfZe+AvdLMFmXH4Z+Xvdv3AqAO+JWZLTGzCjM72cyeNbNXzOxvZladfc9DzOzR7LMLri/iZxPploqCSM+mAvPc/ThgJ/B/gZ+4+8meGYe/AvhHd/8dmbuEP+PuM4AU8BvgMnc/nsxYOy3Z95wBfBKYDnzSzCYiMoioKIj0bJ27P5OdvpfM8CFnZp++9RqZwfWO6eZ1U4H33H0RgLvv9K4hmZ9w90Z3byUz3s6hwX4EkX1TWriJSGjtOQaMA7eRGQNonZldTWZcnH3RljedQv8HZZDRkYJIzyaZ2Qez058Gns5Ob8k+q+KCvLZNQOd5g1XAODM7GcDMqvOGZBYZ1LSjivRsJfBZM7uDzAiiPyUzquhrwFtkhmTvdBdwu5m1kHmU4yeBW8ysgsz5hLPev7BF+k+jpIqISI66j0REJEdFQUREclQUREQkR0VBRERyVBRERCRHRUFERHJUFEREJEdFQUREcv4/Rn4LZdA8gPAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12be19c16d8>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#default only 50, tf36cnn 99\n",
    "xception(num_class=5,epochs=20)\n",
    "history_this.loss_plot('epoch')\n",
    "history_this.loss_plot('batch')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Validation accuracy of Xception: 100.0%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model 3: VGG16"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def vgg16( num_class, epochs,savepath='./VGG16.h5',history=history_this,input_shape=INPUT_SIZE):\n",
    "    model_fine_tune = VGG16(include_top=False, weights='imagenet', input_shape=input_shape)\n",
    "    for layer in model_fine_tune.layers[:15]:\t#the number of frozen layers for transfer learning, have tuned from 5-18\n",
    "        layer.trainable = False\n",
    "    for layer in model_fine_tune.layers[15:]:\n",
    "        layer.trainable = True\n",
    "    model = GlobalAveragePooling2D()(model_fine_tune.output) #GlobalAveragePooling2D layer to convert the features to a single 1280-element vector per image\n",
    "    model=Dense(units=256,activation='relu')(model)\n",
    "    model=Dropout(0.5)(model)\n",
    "    model = Dense(num_class, activation='softmax')(model)\n",
    "    model = Model(model_fine_tune.input, model, name='vgg')\n",
    "    opt = keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08)\t#tuned learning rate to be 0.001\n",
    "    model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])\t#set the loss function to be binary crossentropy\n",
    "    #train model\n",
    "    earlyStopping = kcallbacks.EarlyStopping(\n",
    "        monitor='val_acc', patience=2, verbose=1, mode='auto')\t#set early stop patience to save training time\n",
    "    saveBestModel = kcallbacks.ModelCheckpoint(\n",
    "        filepath=savepath,\n",
    "        monitor='val_acc',\n",
    "        verbose=1,\n",
    "        save_best_only=True,\n",
    "        mode='auto')\n",
    "    hist = model.fit_generator(\n",
    "        train_generator,\n",
    "        steps_per_epoch=len(train_generator),\n",
    "        epochs=epochs,\n",
    "        validation_data=validation_generator,\n",
    "        validation_steps=len(validation_generator),\n",
    "        #use_multiprocessing=True, \n",
    "        #workers=2,\n",
    "        callbacks=[earlyStopping, saveBestModel, history],\n",
    "    )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0401 - acc: 0.9919Epoch 00001: val_acc improved from -inf to 1.00000, saving model to ./VGG16.h5\n",
      "183/183 [==============================] - 71s 389ms/step - loss: 0.0398 - acc: 0.9920 - val_loss: 1.0384e-07 - val_acc: 1.0000\n",
      "Epoch 2/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 1.6893e-05 - acc: 1.0000Epoch 00002: val_acc did not improve\n",
      "183/183 [==============================] - 65s 357ms/step - loss: 1.6802e-05 - acc: 1.0000 - val_loss: 1.0385e-07 - val_acc: 1.0000\n",
      "Epoch 3/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 2.4050e-07 - acc: 1.0000Epoch 00003: val_acc did not improve\n",
      "183/183 [==============================] - 67s 369ms/step - loss: 2.3990e-07 - acc: 1.0000 - val_loss: 1.0384e-07 - val_acc: 1.0000\n",
      "Epoch 00003: early stopping\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8VfWZ7/HPkwsJJCEJt8CALWAdVCAJgkIH5XKsFcqpaNViK4rWwvFMdaRMPcVrtczr1FE7dbBMrVo7aK1otVqsVKa0CdFWqkixImpF1CEUuScQQoAkz/yxV5abkMveSVYS9PvmtV+sy2+t9d2LzX72uuzfNndHREQEIKWrA4iISPehoiAiIiEVBRERCakoiIhISEVBRERCKgoiIhJSURARkZCKgoiIhFQUREQklNbVAZLVr18/Hzp0aJuWPXDgAFlZWR0bqAMoV3KUK3ndNZtyJac9uV599dVd7t6/1Ybuflw9xo4d621VUlLS5mWjpFzJUa7kdddsypWc9uQC1noC77E6fSQiIiEVBRERCakoiIhI6Li70CwiH19HjhyhvLycmpqaLs2Rm5vLm2++2aUZmpJIrszMTIYMGUJ6enqbtqGiICLdRnl5OTk5OQwdOhQz67Ic+/fvJycnp8u235zWcrk7u3fvpry8nGHDhrVpG5GdPjKzh8xsh5ltaGa+mdliM9tkZn8xs9OiyiIix4eamhr69u3bpQXheGZm9O3bt11HWlFeU/hPYFoL86cDJwWPecCPIswiIscJFYT2ae/+i+z0kbuXmdnQFprMBB4O7p9dY2Z5ZjbI3bdFkWf+fCgtLSYvL4q1t09FhXLFODjg3uLwvspT6Z1Tk3D7jhsOxpsZrqkpIDPzvU7ZU8nqrtka5/rOA7WkHDnUhYli6j2NFOv6HI2lZ6SSMyLabXTlNYXBwJa48fJg2jFFwczmETuaoKCggNLS0qQ3Vl7+GerqelJRUdFCq4/+o5sfPR7l9NT6Og5UVHT6dlubnuX11GLJrQfi3jAJ31CPnt7EepLQO6nWnSezqwO0oLtmOyZXXS4cPtwVUQDYt7+CXz//C7568dykl5133YXc/S8/oXdOYp+k7r3//9OrZzZXXfZPiW8k1dm/f3+rzWpqatr0PgnHyYVmd78fuB9g3LhxPmXKlKTXMWXDDzm86mZ6pKRAXR3U18f+jh/25N6cuqWUFEhN/ejvhkf8eILD+6qr6Z2X16Zl2zycQLu3N21ixCmndF6mBLOWrl5NW16bnaG0tLRbZmuc680332TEKV13gff993fz1HMP8a2b5x5zQbe2tpa0tObfMlf/4b+S2la/X2eQnZ3BiHGJP99EL4BnZmYyZsyYpPI06MqisBU4IW58SDAtGiNGsHPKFAafcEKnvXklOrx23TrGjR/f/nWaxR4dZF03fSPZVlrKiG6YS45/Cxcu5N1332XixImce+65zJgxg9tvv51Bgwaxfv16Nm7cyPnnn8+WLVuoqanhuuuuY968eQAMHTqUtWvXUlVVxfTp0znzzDP54x//yODBg/nVr35Fz549m93u+vXrufrqq6murubEE0/koYceIj8/n8WLF3PfffeRlpbGqaeeygMPPMDq1au57rrrgNj1g7Kysg69U6ori8Jy4BozWwaMByqjup4AwDnn8E56OoO74ZtJ1b59MHp0V8cQ6VbmPz+f9R+u79B1Fg8s5p5p9zQ7/4477mDDhg384Q9/ICcnh9LSUl5++WU2bNgQ3uL50EMP0adPHw4ePMjpp5/OhRdeSN++fY9azzvvvMNjjz3GAw88wJe//GWeeuopZs+e3ex2L7/8cu69914mT57Mrbfeyu23384999zDHXfcwXvvvUdGRkZ46vvuu+9myZIlTJw4kaqqKjIzO/bkYJS3pD4GvASMMLNyM7vKzK42s6uDJiuAzcAm4AHgH6PKIiLSVmecccZR9/wvXryYoqIiJkyYwJYtW3jnnXeOWWbYsGEUFxcDMHbsWN5///1m119ZWUlFRQWTJ08GYM6cOZSVlQFQWFjIpZdeys9+9rPw1NXEiRNZsGABixcvpqKiosVTWm0R5d1HX2llvgPfiGr7InJ8a+kTfWeK76q6tLSUVatW8dJLL9GrVy+mTJnS5HcCMjIywuHU1FQOHjzYpm0/99xzlJWVsXz5chYtWsSaNWtYuHAhM2bMYMWKFUyYMIFVq1Zx8sknt2n9TVHfRyIigZycnBbv7qmsrCQ/P59evXrx1ltvsWbNmnZvMzc3l/z8fF544QUAHnnkESZPnkx9fT1btmxh6tSp3HnnnVRUVFBVVcW7777L6NGj+fa3v824ceN466232p0h3nFx95GISGfo27cvEydOZPz48cyYMYMZM2YcNX/atGncd999FBYWMmLECCZMmNAh2126dGl4oXn48OH89Kc/pa6ujtmzZ1NZWYm7881vfpO8vDxuvPFGSkpKSElJYeTIkUyfPr1DMjRQURARifPzn//8qFs/4+/Ay8jI4De/+U2TyzVcN+jXrx8bNnzUu8+3vvWtJtvfdttt4XBxcXGTRx0vvvjiUeP79+/n3nvvTeRptJlOH4mISEhFQUREQioKIiISUlEQEZGQioKIiIRUFEREJKSiICISqKio4D/+4z/atOwXvvCFVrrmPz6oKIiIBFoqCrW1tS0uu2LFCvK6469lJUlFQUQkEN919vXXX09paSlTp07lq1/9KoWFhQCcf/75jB07lpEjR3L//feHyw4dOpRdu3bx/vvvc8oppzB37lxGjhzJ5z//+Sb7Pnr22WcZP348Y8aM4XOf+xzbt28HoKqqiiuvvJLRo0dTWFjIU089BcDzzz/PWWedRVFREWeffXZk+0DfaBaR7mn+fFjfsV1nU1wM93SPrrPPPPNM1qxZg5nx4IMPcuedd/L973+fRYsWkZuby+uvvw7A3r172blzJ3PnzmXFihWMHj2aPXv2dOx+iaOiICLSgqa6zn766acBwq6zGxeFRLrOLi8vZ9asWWzbto3Dhw+H21i1ahXLli0L2+Xn5/Pss88yadIkhg4dCkCfPn068ikeRUVBRLqnFj7Rd6aous6+9tprWbBgAeeddx6lpaVH9YXUlXRNQUQk0JldZ1dWVjJ48GAg1ktqg3POOYclS5aE43v37mXChAmUlZWFRxxRnj5SURARCcR3nX399dcfM3/atGnU1tZSWFjILbfc0q6us2+77TYuvvhizjrrLPr16xdOv/nmm9m7dy+jRo2iqKiIkpIS+vfvz/3338/s2bMpKipi1qxZbd5ua3T6SEQkTmd1nT1z5kxmzpx5zPTs7OyjjhwaTJ8+nTPPPDPMFRUdKYiISEhFQUREQioKIiISUlEQEZGQioKIiIRUFEREJKSiICIS6Myus2+77TbuvvvuNm0rSioKIiIBdZ2toiAiEurMrrPjrV+/ngkTJlBYWMgFF1zA3r17gVjne6eeeiqFhYVccsklAKxevZri4mKKi4sZM2ZMi91ytIW+0Swi3dL85+ez/sOO7Tq7eGAx90zrHl1nx7v88su59957mTx5Mrfeeiu3334799xzD3fccQfvvfceGRkZ4ampu+++myVLljBx4kSqqqrIzMzsgD3zER0piIi0oKmus4uKipgwYULYdXZjiXSd3aCyspKKigomT54MwJw5cygrKwOgsLCQSy+9lJ/97GekpcU+w0+cOJEFCxawePFiKioqwukdRUcKItIttfSJvjNF1XV2Ip577jnKyspYvnw5ixYtYs2aNSxcuJAZM2awYsUKJkyYwKpVqzj55JPbtP6mRHqkYGbTzOxtM9tkZgubmP8pMysxsz+b2V/M7AtR5hERaUlndp3dIDc3l/z8fF544QUAHnnkESZPnkx9fT1btmxh6tSp3HnnnVRUVFBVVcW7777L6NGj+fa3v824ceN466232p0hXmRHCmaWCiwBzgHKgVfMbLm7b4xrdjPwhLv/yMxOBVYAQ6PKJCLSkvius2fMmMGMGTOOmj9t2jTuu+8+CgsLGTFiRLu6zo63dOlSrr76aqqrqxk+fDg//elPqaurY/bs2VRWVuLufPOb3yQvL48bb7yRkpISUlJSGDlyJNOnT++QDA2iPH10BrDJ3TcDmNkyYCYQXxQc6B0M5wJ/izCPiEirOqvr7PhfWisuLm7yqOPFF188anz//v3ce++9iTyNNouyKAwGtsSNlwPjG7W5DfgvM7sWyAI+F2EeERFphbl7NCs2uwiY5u5fD8YvA8a7+zVxbRYEGb5vZp8FfgKMcvf6RuuaB8wDKCgoGBv/o9bJqKqqIjs7u03LRkm5kqNcyeuu2Rrnys3N5TOf+UwXJoqpq6sjNTW1q2McI9FcmzZtorKy8qhpU6dOfdXdx7W6sLtH8gA+C6yMG78BuKFRmzeAE+LGNwMDWlrv2LFjva1KSkravGyUlCs5ypW87pqtca6NGzd2TZBG9u3b19URmpRorqb2I7DWE3jvjvLuo1eAk8xsmJn1AC4Bljdq89/A2QBmdgqQCeyMMJOIiLQgsqLg7rXANcBK4E1idxm9YWbfNbPzgmb/DMw1s9eAx4ArgoomIiJdINIvr7n7CmK3mcZPuzVueCMwMcoMIiKSOHVzISLSDt3xAn57qCiIiEhIRUFEJLBw4UKWLFkSjjf8EE5VVRVnn302p512GqNHj+ZXv/pVq+tqrovt559/ntNOO42ioiLOPvtsIHZr7pVXXsno0aMpLCzkqaee6vgnlyB1iCci3dL8+bC+Y3vOprgY7mmhn71Zs2Yxf/58Lr/8cgCeeOIJVq5cSWZmJk8//TS9e/dm165dTJgwgfPOOw8za3ZdTXWxXV9fz9y5cykrK2PYsGHs2bMHgEWLFpGbm8vrr78OEP6eQldQURARCYwZM4YdO3awbds2Nm/eTH5+PieccAJHjhzhxhtvpKysjJSUFLZu3cr27dsZOHBgs+tavHgxTz/9NEDYxfbOnTuZNGlS2BV3nz59AFi1ahXxX8rNz8+P8Fm2TEVBRLqllj7RR+niiy/mmWeeoaKiglmzZgHw6KOPsnPnTl599VXS09MZOnRok11mN0i0i+3uSNcURETizJo1i6eeeoonn3ySiy++GIh1mT1gwADS09MpKSnhgw8+aHEdzXWxPWHCBMrKynjvvfcAwtNH55xzzlHXMrry9JGKgohInJEjR1JVVcXgwYMZNGgQAJdeeilr165l3LhxPProo63+qM20adOora2lsLCQW265Jexiu3///tx///186UtfoqioKDwSufnmm9m7dy+jRo2iqKiIkpKSaJ9kC3T6SESkkTVr1oRdZ0OsO+yXXnqpybZVVVXHTGupi+3p06cf8xsI2dnZLF26tB2JO46OFEREJKSiICIiIRUFEREJqSiIiEhIRUFEREIqCiIiElJREBFph+a6zj5eu9RWURARkZCKgohIoCO7zm7g7lx//fWMGjWK0aNH8/jjjwOwbds2Jk2aRHFxMaNGjeKFF16grq6OK664Imz7gx/8oMOfY2v0jWYR6Zbmz5/P+g7uO7u4uJh7WuhpryO7zm7wy1/+kvXr1/Paa6+xa9cuTj/9dCZNmsTPf/5zzj33XG666Sbq6uqorq5m/fr1bN26lQ0bNgBQUVHRMU88CSoKIiKBjuw6u8GLL77IV77yFVJTUykoKGDy5Mm88sornH766Xzta1/jyJEjnH/++RQXFzN8+HA2b97Mtddey4wZM/j85z/fCc/6aCoKItIttfSJPkod0XV2IiZNmkRZWRnPPfccl112Gddffz2XX345r732GitXrmTJkiU88cQTPPTQQx3xtBKmawoiInE6ouvseGeddRaPP/44dXV17Ny5k7KyMs444ww++OADCgoKmDt3LldddRXr1q1j165d1NfXc+GFF7Jo0SLWrVsX1dNslo4URETiNNd19he/+EXGjRtHcXFxq11nx7vgggt46aWXKCoqwsy48847GThwIEuXLuWuu+4iPT2d7OxsHn74YbZu3cqVV15JfX09AN/73vcieY4tUVEQEWmkvV1nx083M+666y7uuuuuo+bPmTOHOXPmHLNcVxwdxNPpIxERCakoiIhISEVBRLoVd+/qCMe19u4/FQUR6TYyMzPZvXu3CkMbuTu7d+8mMzOzzevQhWYR6TaGDBlCeXk5O3fu7NIcNTU17XpjjUoiuTIzMxkyZEibt6GiICLdRnp6OsOGDevqGJSWljJmzJiujnGMzsil00ciIhKKtCiY2TQze9vMNpnZwmbafNnMNprZG2b28yjziIhIyyI7fWRmqcAS4BygHHjFzJa7+8a4NicBNwAT3X2vmQ2IKo+IiLQuyiOFM4BN7r7Z3Q8Dy4CZjdrMBZa4+14Ad98RYR4REWmFRXXrl5ldBExz968H45cB4939mrg2zwB/BSYCqcBt7v58E+uaB8wDKCgoGLts2bI2ZaqqquqWP5GnXMlRruR112zKlZz25Jo6deqr7j6u1YbuHskDuAh4MG78MuCHjdr8GngaSAeGAVuAvJbWO3bsWG+rkpKSNi8bJeVKjnIlr7tmU67ktCcXsNYTeO9O6PSRmU00s6xgeLaZ/ZuZfbqVxbYCJ8SNDwmmxSsHlrv7EXd/j9hRw0mJZBIRkY6X6DWFHwHVZlYE/D/gA+DhVpZ5BTjJzIaZWQ/gEmB5ozbPAFMAzKwf8PfA5gQziYhIB0u0KNQGhx8zgX93938HclpawN1rgWuAlcCbwBPu/oaZfdfMzguarQR2m9lGoAS43t13t+WJiIhI+yV6S+p+M7sBmA1MMrMUYtcBWuTuK4AVjabdGjfswILgISIiXSzRI4VZwCHgKnf/kNj1gbtaXkRERI43CR8pEDttVGdmfw+cDDwWXSwREekKiR4plAEZZjYY+B1wJfCfUYUSEZGukWhRMHevBr4E3OvuFwCjooslIiJdIeGiYGafBS4FnktyWREROU4k+sY+n1jHdU8Ht5UOJ3YLqYiIfIwkdKHZ3VcDq80s28yy3X0z8E/RRhMRkc6WaDcXo83sz8AbwEYze9XMRkYbTUREOluip49+DCxw90+7+6eAfwYeiC6WiIh0hUSLQpa7h9cQ3L0UyIokkYiIdJlEv7y22cxuAR4JxmejjutERD52Ej1S+BrQH/hl8OgfTBMRkY+RRO8+2ovuNhIR+dhrsSiY2bNAs7/X6e7nNTdPRESOP60dKdzdKSlERKRbaLEoBF9aO4qZnebu66KLJCIiXaUt/Rc92OEpRESkW2hLUbAOTyEiIt1CW4rC7R2eQkREuoVE+z66wMxyAdz9GTPLM7Pzo40mIiKdLdEjhe+4e2XDiLtXAN+JJpKIiHSVRItCU+0S7SJDRESOE4kWhbVm9m9mdmLw+Dfg1SiDiYhI50u0KFwLHAYeB5YBNcA3ogolIiJdI9G+jw4ACyPOIiIiXSzRu49+a2Z5ceP5ZrYyulgiItIVEj191C+44wgIe00dEE0kERHpKokWhXoz+1TDiJkNpYXeU0VE5PiU6G2lNwEvmtlqYt1cnAXMiyyViIh0iUQvND9vZuOIFYI/A88AB6MMJiIinS/RC81fB34H/DPwLWK/1XxbAstNM7O3zWyTmTV795KZXWhmHhQeERHpIoleU7gOOB34wN2nAmOAnS0tYGapwBJgOnAq8BUzO7WJdjnB+v+URG4REYlAokWhxt1rAMwsw93fAka0sswZwCZ33+zuh4l96W1mE+0WAf9K7AtxIiLShRItCuXB9xSeAX5rZr8C/tbKMoOBLfHrCKaFzOw04AR3fy7BHCIiEiFzT+7OUjObDOQCzwdHAM21uwiY5u5fD8YvA8a7+zXBeArwe+AKd3/fzEqBb7n72ibWNY/gbqeCgoKxy5YtSypzg6qqKrKzs9u0bJSUKznKlbzumk25ktOeXFOnTn3V3Vu/buvukTyAzwIr48ZvAG6IG88FdgHvB48aYkcf41pa79ixY72tSkpK2rxslJQrOcqVvO6aTbmS055cwFpP4L27Lb+8lqhXgJPMbJiZ9QAuAZbHFaNKd+/n7kPdfSiwBjjPmzhSEBGRzhFZUXD3WuAaYCXwJvCEu79hZt81s/Oi2q6IiLRdpD+U4+4rgBWNpt3aTNspUWYREZHWRXn6SEREjjMqCiIiElJREBGRkIqCiIiEVBRERCSkoiAiIiEVBRERCakoiIhISEVBRERCKgoiIhJSURARkZCKgoiIhFQUREQkpKIgIiIhFQUREQmpKIiISEhFQUREQioKIiISUlEQEZGQioKIiIRUFEREJKSiICIiIRUFEREJqSiIiEhIRUFEREIqCiIiElJREBGRkIqCiIiEVBRERCSkoiAiIiEVBRERCUVaFMxsmpm9bWabzGxhE/MXmNlGM/uLmf3OzD4dZR4REWlZZEXBzFKBJcB04FTgK2Z2aqNmfwbGuXsh8CRwZ1R5RESkdVEeKZwBbHL3ze5+GFgGzIxv4O4l7l4djK4BhkSYR0REWmHuHs2KzS4Cprn714Pxy4Dx7n5NM+1/CHzo7v/SxLx5wDyAgoKCscuWLWtTpqqqKrKzs9u0bJSUKznKlbzumk25ktOeXFOnTn3V3ce12tDdI3kAFwEPxo1fBvywmbaziR0pZLS23rFjx3pblZSUtHnZKClXcpQred01m3Ilpz25gLWewHt3WptKTmK2AifEjQ8Jph3FzD4H3ARMdvdDEeYREZFWRHlN4RXgJDMbZmY9gEuA5fENzGwM8GPgPHffEWEWERFJQGRFwd1rgWuAlcCbwBPu/oaZfdfMzgua3QVkA78ws/VmtryZ1YmISCeI8vQR7r4CWNFo2q1xw5+LcvsiIpIcfaNZRERCKgoiIhJSURARkZCKgoiIhFQUREQkpKIgIiIhFQUREQmpKIiISEhFQUREQioKIiISUlEQEZGQioKIiIRUFEREJKSiICIiIRUFEREJqSiIiEhIRUFEREIqCiIiElJREBGRkIqCiIiEVBRERCSkoiAiIiEVBRERCakoiIhISEVBRERCaV0doLMsWLmAn6z9Cb3/3JueaT3J6pFFdno2vTN6k5uZS15mHvk98+nfqz/9evajf1Z/BuUMYmDWQPr16kdKiuqniHz8fWKKwsEjBzlcf5gdB3ZQW19LvdcnvY5USyU1JZX0lHR6pPYgMy2Tnuk9yUrPIrtHNjk9cuid2Zu8jDz69OxD31596duzb6zAZA+iILuAgVkD6ZHWI4JnKCLSfp+YovCj//0jZmXPYsqUKeG02vpadlXv4m/7/8b2qu3srN7JzgM72X1wN3sP7qWipoKKmgr2H95P1eEqqo9UU32kmpraGg7VHaL6YDW1B2IFxvGk8qRYCqmWSlpKGqmeStbaLDLTMumV3uujAhMcxfTp2SdWZHr2ZUDWAAqyCyjIKmBQziCye2R38J4SkU+yT0xRaEpaShoDswcyMHtgu9dVX1/PvsP72LpvK9sPbGfngZ3sOLCD3dW72VOzh4qDFeyt2RsWmAOHD3DgyAFqamuoPlTNvkP72H1wN3X1dUkXGMNITYkVmB6pPchIzQiPYrLTs2NFJiMndoosM5/8nvlhgRmQNYCCrAIG5gykT2YfnSYT+YT7RBeFjpSSkkJeZh55mXmMZGRSy5aWlh51BANQfbiabVXb+LDqQ3Yc2MGOAzvYVb2LPTV7wqOYfYf2sf/Qfg4c+ajA1NTWcODIASpqKqjzuqRPkxlGiqWQlpJGiqfQ8+We4RFMr/ReZKd/VGDyMmLXYfr07MOArAH0z+rPgKwB/F323zEgewBpKXp5iRxvIv1fa2bTgH8HUoEH3f2ORvMzgIeBscBuYJa7vx9lpuNFrx69OLHPiZzY58R2r6u2vpYPqz4MC8z2qu2xAnMwVmD21OwJC0zVkdhRzMHag1QdrOJQ7SEOHD7A9vrtbT5NlmZppKWmfXQdJq0nvdJ7kZWeRU5GDrkZueRm5pLfM59+PfvRt1df+vcKCkzO3zEoZxCZaZnt3g8i0rrIioKZpQJLgHOAcuAVM1vu7hvjml0F7HX3z5jZJcC/ArOiyvRJlZaSxpDeQxjSe0hSyzV1BFNfX8+emj18uP9Dth/YzocHPmTXgV3hdZg9B/dQeaiS/Yf2s/9w7Cjm4JGD4VFMRU0Fu+p3tes0WQopZK7JJDM1k8z0THql9SKrR9ZH12EycsnrmUffnn3DAtNwsX9QziB69+it02QizYjySOEMYJO7bwYws2XATCC+KMwEbguGnwR+aGbm7sm9W0inSUlJoV+vfvTr1Y9RjGr3+qoOV7Ft/7ZYgan6MLzQv7t6N3tr9lJZU8m+Q/vYd2gfB44coPpINfuq91FPPfsP72dvzd52nyZLT00Pr8M0nCbL6ZETHsXk9/zoOky/Xv0oyCoIr8MM6DVABUY+VqIsCoOBLXHj5cD45tq4e62ZVQJ9gV0dHWb+/PmUlpaSl5fX0atut4qKCuVqRVbwB5rP5e4crjvMobpDHKk7wuH6w9TW1XKk/ghH6o9QW19LXX1deEtynddRX19PjddQTTXuHjt6actHEmv4y9rxLEVaNvzE4WyasinSbRwXVwLNbB4wD6CgoIDS0tKk11FeXk5dXR0VFRUdnK79lCs5ieRKJZWe9IyNGLGrWqnJbafWYwWl1mupra+N/e211Hld+AiLC/XUe323LQqOd8tsypWcdNLb9P6XFHeP5AF8FlgZN34DcEOjNiuBzwbDacSOEKyl9Y4dO9bbqqSkpM3LRkm5kqNcyeuu2ZQrOe3JBaz1BN67ozwZ+gpwkpkNM7MewCXA8kZtlgNzguGLgN8H4UVEpAtEdvrIY9cIriF2NJAKPOTub5jZd4lVrOXAT4BHzGwTsIdY4RARkS4S6TUFd18BrGg07da44Rrg4igziIhI4nQvnYiIhFQUREQkpKIgIiIhFQUREQmpKIiISMiOt68FmNlO4IM2Lt6PCLrQ6ADKlRzlSl53zaZcyWlPrk+7e//WGh13RaE9zGytu4/r6hyNKVdylCt53TWbciWnM3Lp9JGIiIRUFEREJPRJKwr3d3WAZihXcpQred01m3IlJ/Jcn6hrCiIi0rJP2pGCiIi04GNTFMxsmpm9bWbdWR6UAAAGtUlEQVSbzGxhE/MzzOzxYP6fzGxo3Lwbgulvm9m5nZxrgZltNLO/mNnvzOzTcfPqzGx98Gjc7XjUua4ws51x2/963Lw5ZvZO8JjTeNmIc/0gLtNfzawibl6U++shM9thZhuamW9mtjjI/RczOy1uXiT7K4FMlwZZXjezP5pZUdy894Pp681sbUdlSiLbFDOrjPv3ujVuXouvgYhzXR+XaUPwmuoTzItkn5nZCWZWErwPvGFm1zXRpvNeX4n86EJ3fxDrmvtdYDjQA3gNOLVRm38E7guGLwEeD4ZPDdpnAMOC9aR2Yq6pQK9g+P825ArGq7pwf10B/LCJZfsAm4O/84Ph/M7K1aj9tcS6ZI90fwXrngScBmxoZv4XgN8Q+523CcCfOmF/tZbpHxq2BUxvyBSMvw/068L9NQX4dXtfAx2dq1HbLxL7jZdI9xkwCDgtGM4B/trE/8dOe319XI4UzgA2uftmdz8MLANmNmozE1gaDD8JnG1mFkxf5u6H3P09YFOwvk7J5e4l7l4djK4BhnTQttuVqwXnAr919z3uvhf4LTCti3J9BXisg7bdIncvI/abH82ZCTzsMWuAPDMbRIT7q7VM7v7HYJvQea+thm23tr+a057XZkfn6pTXl7tvc/d1wfB+4E1iv18fr9NeXx+XojAY2BI3Xs6xOzVs4+61QCXQN8Flo8wV7ypinwYaZJrZWjNbY2bnd1CmZHJdGByqPmlmJyS5bJS5CE6zDQN+Hzc5qv2ViOayR7m/ktH4teXAf5nZqxb7DfSu8Fkze83MfmNmI4Np3WJ/mVkvYm+uT8VNjnyfWey09hjgT41mddrrK9If2ZHEmdlsYBwwOW7yp919q5kNB35vZq+7+7udFOlZ4DF3P2Rm/4fYUdb/6qRtJ+IS4El3r4ub1pX7q9sys6nEisKZcZPPDPbVAOC3ZvZW8Cm6s6wj9u9VZWZfAJ4BTurE7bfmi8Af3D3+qCLSfWZm2cSK0Hx339dR603Wx+VIYStwQtz4kGBak23MLA3IBXYnuGyUuTCzzwE3Aee5+6GG6e6+Nfh7M1BK7BNEp+Ry991xWR4Exia6bJS54lxCo0P7CPdXIprLHuX+apWZFRL795vp7rsbpsftqx3A03TcKdOEuPs+d68KhlcA6WbWjy7eX3Faen11+D4zs3RiBeFRd/9lE0067/XV0RdNuuJB7IhnM7HTCQ0Xp0Y2avMNjr7Q/EQwPJKjLzRvpuMuNCeSawyxC2snNZqeD2QEw/2Ad+igC24J5hoUN3wBsMY/urD1XpAvPxju01m5gnYnE7voZ52xv+K2MZTmL5zO4OgLgS9Hvb8SyPQpYtfI/qHR9CwgJ274j8C0jtxXCWQb2PDvR+zN9b+DfZfQayCqXMH8XGLXHbI6Y58Fz/th4J4W2nTa66tDXwRd+SB2df6vxN5gbwqmfZfYp2+ATOAXwX+Sl4HhccveFCz3NjC9k3OtArYD64PH8mD6PwCvB/8pXgeu6uRc3wPeCLZfApwct+zXgv24CbiyM3MF47cBdzRaLur99RiwDThC7LztVcDVwNXBfAOWBLlfB8ZFvb8SyPQgsDfutbU2mD482E+vBf/GN3Xkvkow2zVxr681xBWupl4DnZUraHMFsZtP4peLbJ8RO63nwF/i/q2+0FWvL32jWUREQh+XawoiItIBVBRERCSkoiAiIiEVBRERCakoiIhISEVBpBMFvYP+uqtziDRHRUFEREIqCiJNMLPZZvZy0Hf+j80s1cyqzOz7ZrbOYr990T9oWxx0wvcXM3vazPKD6Z8xs1VBp2/rzOzEYPXZQSeDb5nZo0FvvSLdgoqCSCNmdgowC5jo7sVAHXApse4N1rn7acBq4DvBIg8D33b3QmLfNm2Y/iiwxN2LiH3jelswfQwwn9hveQwHJkb+pEQSpF5SRY51NrEOAF8JPsT3BHYA9cDjQZufAb80s1wgz91XB9OXAr8wsxxgsLs/DeDuNQDB+l529/JgfD2xvnhejP5pibRORUHkWAYsdfcbjppodkujdm3tI+ZQ3HAd+n8o3YhOH4kc63fARUG/+ZhZn+BHfVKAi4I2XwVedPdKYK+ZnRVMvwxY7bFf0Cpv+LEfi/1GeK9OfRYibaBPKCKNuPtGM7uZ2K9spRDrUfMbwAFgpJm9SuyX+2YFi8wB7gve9DcDVwbTLwN+bGbfDdZxcSc+DZE2US+pIgkysyp3z+7qHCJR0ukjEREJ6UhBRERCOlIQEZGQioKIiIRUFEREJKSiICIiIRUFEREJqSiIiEjofwA2hbPacOf3XAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12be1eae668>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYFPWd7/H3dy4w3OQ2OmscFVgRg4BDGBVDThgS1wO6j5cTT9TIUXPc8LiJezQmbvDJCasx+6yJxiCGGElMTLzmYtZwlOgGpUHjDYlE7gqIMiTKdQYGZmAuv/NH1xQ9Q89M00x1V1d9Xs8zD9XV1V3f3zjOZ76/qq4y5xwiIiIARfkuQEREwkOhICIiPoWCiIj4FAoiIuJTKIiIiE+hICIiPoWCiIj4FAoiIuJTKIiIiK8k3wUcrfLycjdixIisXrt//34GDBjQuwWFSJTHF+WxQbTHF+WxQeGMb8WKFTudc8f3tF3BhcKIESN48803s3ptIpGgpqamdwsKkSiPL8pjg2iPL8pjg8IZn5m9n8l2mj4SERGfQkFERHwKBRER8RXcMQURia7m5mZqa2tpamrKdykZGzx4MOvWrct3Gb6ysjIqKyspLS3N6vUKBREJjdraWgYNGsSIESMws3yXk5F9+/YxaNCgfJcBgHOOXbt2UVtby8iRI7N6D00fiUhoNDU1MXz48IIJhLAxM4YPH35MnZZCQURCRYFwbI71+xerUGhqbeKRvzyCbkEqIpJerEJh/qb5XPP0NSx9f2m+SxGREKqrq+NHP/pRVq+98MILqaury3j722+/nXvuuSerfQUpVqGw6+AuAPYd3JfnSkQkjLoLhZaWlm5fu2jRIoYMGRJEWTkVq1AQEenO7Nmz2bRpE1VVVdx6660kEgmmTZvGF77wBSZMmADApZdeyqRJkzjzzDNZsGCB/9oRI0awc+dOtmzZwsc//nG+9KUvceaZZ3LBBRfQ2NjY7X5XrlzJ5MmTmTBhApdddhl79uwBYN68eYwdO5YJEyZw5ZVXArB06VKqqqqoqqpi4sSJ7NvXu3/k6pRUEQmlm5+7mZUfruzV96z6uyrmTp/b5fN33XUXq1evZuXK5H4TiQRvvPEGq1ev9k/x/NnPfsawYcNobGzk7LPP5oILLjjilNR3332XJ554gp/85Cd8/vOf56mnnmLmzJld7veaa67h/vvvZ+rUqcyZM4c77riDuXPnctddd/Hee+/Rt29ff2rqnnvuYf78+UyZMoWGhgbKysqO9dvSgToFEZFunHPOOR3O+Z83bx5nnXUWkydPZuvWrWzatOmI14wcOZKqqioAJk2axJYtW7p8//r6eurq6pg6dSoA1157LcuWLQNgwoQJXH311Tz66KOUlCT/hp8yZQq33HIL8+bNo66uzl/fW9QpiEgodfcXfS6lXhY7kUiwePFiXn31Vfr3709NTQ0HDx484jV9+/b1l4uLi3ucPurKs88+y7Jly1i4cCF33nkna9asYfbs2Vx00UUsWrSIyZMns3jxYs4444ys3j8ddQoiIp5BgwZ1O0dfX1/P0KFD6d+/P+vXr+e111475n0OHjyYoUOH8tJLLwHwyCOPMHXqVNra2ti6dSvTpk3je9/7HnV1dTQ0NLBp0ybGjx/PN77xDaqrq1m/fv0x15BKnYKIiGf48OFMmTKFcePGMWPGDC666KIOz0+fPp0f//jHTJgwgTFjxjB58uRe2e8vfvELbrjhBg4cOMCoUaP4+c9/TmtrKzNnzqS+vh7nHF/96lcZMmQI3/rWt1iyZAlFRUWceeaZzJgxo1dqaKdQEBFJ8fjjj3d4nHoDnb59+/KHP/yhw/PtnUX7cYPy8nJWr17tP//1r3897X5uv/12f7mqqipt1/Hyyy8fse7+++/vtv5jpekjERHxKRRERMQXaCiY2XQz22BmG81sdprnTzGzJWb2lpm9bWYXBlmPiISfrk12bI71+xdYKJhZMTAfmAGMBa4ys7GdNvu/wK+dcxOBK4HsLjqSIYd+2ETCrKysjF27dikYstR+P4Vj+UBbkAeazwE2Ouc2A5jZk8AlwNqUbRxwnLc8GPhrgPX4dGlekXCqrKyktraWHTt25LuUjDU1NfX6p4qPRfud17IVZCicBGxNeVwLnNtpm9uB/zKzfwEGAOcHWI+IhFxpaWnWdwzLl0QiwcSJE/NdRq/J9ympVwEPO+e+b2bnAY+Y2TjnXFvqRmY2C5gFUFFRQSKRyGpnLa3JqxyuWrWKgX8deCx1h1JDQ0PW35uwi/LYINrji/LYIHrjCzIUtgEnpzyu9Naluh6YDuCce9XMyoByYHvqRs65BcACgOrqapd63vDRKFmVHO748eOpOT279wizRCJBtt+bsIvy2CDa44vy2CB64wvy7KPlwGgzG2lmfUgeSF7YaZsPgM8CmNnHgTKgcCYTRUQiJrBQcM61ADcCzwPrSJ5ltMbMvm1mF3ubfQ34kpn9BXgCuM7ptAMRkbwJ9JiCc24RsKjTujkpy2uBKUHWICIimYvlJ5rVjIiIpBerUDD0+QQRke7EKhRERKR7CgUREfEpFERExKdQEBERn0JBRER8CgUREfEpFERExBerUNBNdkREuherUGinm+yIiKQXy1AQEZH0FAoiIuJTKIiIiE+hICIiPoWCiIj4FAoiIuJTKIiIiE+hICIiPoWCiIj4YhkKukeziEh68QwFXQNJRCSteIaCOgURkbTiGQrqFERE0opnKKhTEBFJK1ahoA5BRKR7sQqFdgoHEZH04hkKmj4SEUkrVqFgJO+4pk5BRCS9WIVCO3UKIiLpxTMU1CmIiKQVz1BQpyAiklY8Q0GdgohIWvEMBXUKIiJpxTMU1CmIiKQVz1BQpyAiklY8Q0GdgohIWoGGgplNN7MNZrbRzGZ3sc3nzWytma0xs8eDrKedOgURkfRKgnpjMysG5gP/ANQCy81soXNubco2o4HbgCnOuT1mdkJQ9aRSpyAikl6QncI5wEbn3Gbn3CHgSeCSTtt8CZjvnNsD4JzbHmA9PnUKIiLpBRkKJwFbUx7XeutSnQ6cbmZ/MrPXzGx6gPX41CmIiKQX2PTRUex/NFADVALLzGy8c64udSMzmwXMAqioqCCRSGS1s5aWFgDWr19Poi679wizhoaGrL83YRflsUG0xxflsUH0xhdkKGwDTk55XOmtS1ULvO6cawbeM7N3SIbE8tSNnHMLgAUA1dXVrqamJquCilcVAzBmzBhqJmb3HmGWSCTI9nsTdlEeG0R7fFEeG0RvfEFOHy0HRpvZSDPrA1wJLOy0zdMkuwTMrJzkdNLmAGsCNH0kItKVwELBOdcC3Ag8D6wDfu2cW2Nm3zazi73Nngd2mdlaYAlwq3NuV1A1pdQW9C5ERApSoMcUnHOLgEWd1s1JWXbALd5XzqhTEBFJL56faFanICKSVjxDQZ2CiEha8QwFdQoiImnFMxTUKYiIpBXPUFCnICKSVjxDQZ2CiEha8QwFdQoiImnFMxTUKYiIpBXPUFCnICKSVqxCwTBAnYKISFdiFQrt1CmIiKQXz1BQpyAiklasQkFhICLSvViFQjtNH4mIpBfPUFDHICKSVkahYGZTzGyAtzzTzO41s1ODLS046hRERNLLtFN4ADhgZmcB/wq8D/wysKoCFtZO4bXa19i2t/NtrEVEcifTUGjx7pJ2CXCfc+4+YFBwZQUrrJ3CeQ+dx2n3n5bvMkQkxjK9Hec+M7sNmAl82syKgNLgygpWWDsFgKaWpnyXICIxlmmncAVwELjeOfchUAncHVhVAQtrpyAikm8Zdwokp41azex04AzgieDKClaYOwURkXzKtFNYBvQ1s5OAF4AvAg8HVVTQ1CmIiKSXaSiYc+4A8D+A+51zlwHjgisrWOoURETSyzgUzOw84Grg2aN8beioUxARSS/TX+w3A7cB/+mcW2Nmo4AlwZUVLHUKIiLpZXSg2Tm3FFhqZgPNbKBzbjPwf4ItLTjqFERE0sv0MhfjzewtYA2w1sxWmNmZwZbWux5Y/gCv734dUKcgItKVTKePHgRucc6d6pw7Bfga8JPgyup9+5v3+8vqFERE0ss0FAY45/xjCM65BDAgkIoCUmThPi6uoBKRMMj0w2ubzexbwCPe45nA5mBKCkaxFfvLYZw+anNt+S5BRCTjTuF/A8cDv/O+jvfWFYziopRQCOFf5QoFEQmDTM8+2kMBn20EHaeP1CmIiKTXbSiY2f+Drn+DOucu7vWKAtJh+kidgohIWj11CvfkpIocUKcgItKzbkPB+9BaB2b2Cefcn4MrKRiFdEyhzbWF/mwpEYmmbH7z/LTXq8iBQuoUmlub81iJiMRZNqFgvV5FDhTSMYVDrYfyWImIxFk2oXBHphua2XQz22BmG81sdjfbfc7MnJlVZ1FPRjpMH4WwU2h1rf7ywdaDeaxEROIs02sfXWZmgwGcc0+b2RAzu7SH1xQD84EZwFjgKjMbm2a7QcBNwOtHW/zR6DB9pE5BRCStTDuFf3PO1bc/cM7VAf/Ww2vOATY65zY75w4BTwKXpNnuTuC7QKB3rC+kTzQrFEQkXzK9zEW68OjptScBW1Me1wLnpm5gZp8ATnbOPWtmt3b1RmY2C5gFUFFRQSKRyKTmDtbuWOsvf/DBB1m9R5B2HtzpL7/0ykts6b/lqN+joaEhdOPqLVEeG0R7fFEeG0RvfJmGwptmdi/J6SCArwArjmXHZlYE3Atc19O2zrkFwAKA6upqV1NTc9T7q1tfB14uVJ5cSTbvEaTavbXwWnJ50tmTGHv8ETNtPUokEqEbV2+J8tgg2uOL8tggeuPLdProX4BDwK9ITgM1kQyG7mwDTk55XOmtazeI5H2eE2a2BZgMLAzqYHMhnX2kU1JFJF8yvfbRfqDLs4e6sBwYbWYjSYbBlcAXUt6zHihvf2xmCeDrzrk3j3I/GSmkzym0tLXksRIRibNMzz76o5kNSXk81Mye7+41zrkW4EbgeWAd8Gvv/s7fNrOcXzOpkD7RrFAQkXzJ9JhCuXfGEZC8aqqZndDTi5xzi4BFndbN6WLbmgxryUrYLxuhUBCRMMj0N2WbmZ3S/sDMRtDN1VPDqJBOSW1u0zEFEcmPTDuFbwIvm9lSkpe5+G94p4gWCk0fiYj0LNMDzc95ZwXNAt4CngYagyyst+lAs4hIzzIKBTP7J5KXoqgEVpI8ffRV4DPBlda7dEqqiEjPMj2mcBNwNvC+c24aMBHYEVhVAQh7p9DadviCeOoURCRfMg2FJudcE4CZ9XXOrQfGBFdW79MxBRGRnmV6oLnW+5zC08AfzWwP8Nfgyup9OvtIRKRnmR5ovsxbvN3MlgCDgecCqyoAhXTpbHUKIpIvmXYKvnT3bS4EYb/JjkJBRMIg3B/z7UWF1Cno7CMRyZfYhEIhHVNQpyAi+RKbUCikTkGhICL5EptQSD2m8NO3fsrOAzu72Tp3Zv5uJp986JM6+0hEQiE+oZAyfQTw/Ve+n6dKOnps1WO8WvuqOgURCYXYhELnS2ebWZ4qSU+hICJhEJtQSJ0+CiOdfSQiYRCbUDiiUyBcnUKr07WPRCT/YhMKnY8phI2mj0QkDOITCp2mj8J8TEFnH4lIvsQmFMI+faROQUTCIDah0Hn6KMydgkJBRPIlNqHQuVMIG519JCJhEO7flL2okE5JbXHJTuG7L38Xu8MUEiKSM/EJhc7TRyE+pvDwyoepa6rjOy99B4ADzQfyVZaIxExsQqGQpo8Anlj1hH/hvrAd/xCR6Ar3b8peVEinpAKcdNxJ/iW+W9ta071ERKTXxSYUOk8Xhe3y2Z1DAQ7XqLORRCRX4hMKnTqDsH1ArD0UJlRMAJJnILWvS70EhohIkGITCp2F7a/v9gD4j8/+B5AMLU0fiUiuxTYUwnaaZ/sv/rKSMiBZn6aPRCTXSvJdQL6E7Rdte6fQt7gvAF/8/Rf9aSNNH4lIrsS3UwjpMYW+JclQSA0CTR+JSK7ENhTC3imkClutIhJdsQ2FsHcKqTR9JCK5Et9QCNmBZnUKIhIGsQ2FsP2i7bZT0DEFEcmRWIXC3LPm+suhnT5K0ylo+khEciXQUDCz6Wa2wcw2mtnsNM/fYmZrzextM3vBzE4Nsp6zhpzlL4etU2j/xV9SVHLExfvCVquIRFdgoWBmxcB8YAYwFrjKzMZ22uwtoNo5NwH4LfC9oOrpLGzHFLbWbwWSV3MtLSrt8Jymj0QkV4LsFM4BNjrnNjvnDgFPApekbuCcW+Kca79ZwGtAZYD1dBC2v77nvTEP8EKhuFMoaPpIRHIkyE80nwRsTXlcC5zbzfbXA39I94SZzQJmAVRUVJBIJLIqqKGhwV/esXtH1u8TlFEDRvHay69hbR0v3rfirRWwpefXNzQ0hG5MvSXKY4Nojy/KY4PojS8Ul7kws5lANTA13fPOuQXAAoDq6mpXU1OT1X4SiQQ7bt3BjMdm0Ke4D9m+T69amvxnUJ9BbPr6JgCKXi+ClEZm3Phx1JxW0+NbJRKJcIwpAFEeG0R7fFEeG0RvfEFOH20DTk55XOmt68DMzge+CVzsnDsYYD0AlPcvp7x/Oa9sfYXG5sagd5ex1IPLh1oPdXhO00cikitBhsJyYLSZjTSzPsCVwMLUDcxsIvAgyUDYHmAtHTy38TkAHnrroVztskep93vofLps2I5/iEh0BRYKzrkW4EbgeWAd8Gvn3Boz+7aZXextdjcwEPiNma00s4VdvF2vuuncmwDYe3BvLnaXkdQ7w3UOAZ19JCK5EugxBefcImBRp3VzUpbPD3L/XZk7fS4PrniQPY178rH7tLq7Z7Smj0QkV2L1ieZU5f3L2dW4K99l+DrfQzqVpo9EJFdiHQo7D+zMdxm+bjsFTR+JSI7ENhSG9xseqlDofGmLVOoURCRXYhsKhTR9pGMKIpIrsQ2FsHUKmj4SkTCIbSgMKRvC3oN7cc7luxRAB5pFJBxiGwrH9T2OlrYWGlvC8anm1E7h74f+fYfnNH0kIrkS61CA/H6ALbVLST3QvPYra9k7+3Bdmj4SkVxRKOQzFDgcCqnTR32K+zCo7yDqZ9cDmj4SkdyJbSgMLhsM5DcU2m/BCekPNBdbMaDpIxHJndiGQnunUN9Un7caOoRCmgPNJUXJq5Bo+khEciX2oRCWTiHdh9eKi5KdgqaPRCRXFAohCYXupo+27dvG1Ien8mHDhzmrTUTiKbahMLhvuI4ppOsUzIwiK+LBFQ+y7P1lLFixIJfliUgMxTYU2juF3Y2781ZDT8cU4HC3AFBaVBp4TSISb7ENhdLiUiqPq2Rz3ea81dDT9BEcPtgMyZpFRIIU21AAOH346byz65287T+TTiH1swypASEiEoRYh8LoYaPDEwpddApNLU2Ht+nm+kgiIr0h1qEwaugodjfuZt/BfXnZfyadQqqwXKdJRKIr1qEwrN8wAOqa6vKy/57OPgL45+p/9pcPNB8IvCYRibdYh8KQsiFAOEKhq+mje//7vf6yQkFEgqZQICSh0MX0UVlJmb/c2KzpIxEJVqxDof0DbLsbd+fl+kKZdAqpDrSoUxCRYMU6FNo7hUt/dSkTfjwh5/vP9EDzjlt3UDGgQtNHIhI4hYJn7Y61Od9/pp1Cef9yPjboY+xu3M2uA7tyUZqIxFSsQ6H9ngr5ksnZR+36l/Zn8ebFlN9dHnRZIhJjsQ6FPsV96FfSL2/7P5rPKfQv7R90OSIi8Q4FOHxhvHw4mgPNqaGgm+6ISFBiHwr9Sg93Cs2tzezYvyNn+z6aTiE1vPY37w+sJhGJt9iHQurnAPp8pw+nzj01Z/u+77X7/OWejimkhsLW+q2B1SQi8Rb7UOh8TKGxpTFn91hY8OfDN805mlAY98A4nZ4qIoGIfSikdgrtNu3elPM6ejqm0PnYx3t73guyHBGJqdiHQuoxhVMHJ6eONu3JQygcxTEFgI27NwZZjojEVOxD4cLTLvSXq/6uCoANOzdk9NqmlqaszwTq/Lqj7RQUCiIShNiHwi3n3cKtn7wVgIoBFYwZPobF7y3m7j/dzaHWQ92+tt+/9+Nzv/5cVvttONTQ4XFPncLAPgM7PH7m3WdwznWxtYhIdmJ/f0czY8zwMQCMGDKC/c37eWzVY7z8wcu0ulZmf2p22tc1tzYD8PsNv89qv/UH64+oozudb8WZ2JJg9fbVjK8Yn9X+RUTSCbRTMLPpZrbBzDaa2RG/Xc2sr5n9ynv+dTMbEWQ9Xbm26lrmXzifr33ya0w6cZK//pl3nunyNR82fJj1/nbs38Gjbz/aYV1PZx+lez4f12sSkWgLLBTMrBiYD8wAxgJXmdnYTptdD+xxzp0G/AD4blD1dKekqIQvn/1l+hT3ofpj1f76VdtX4ZzjxfdeZMTcEZz30HnsadwDwLZ92454nxc2v8Bnf/lZHl/1uD+1k26K5/LfXM43X/xmh3U9TR+NGjrKX76u6joArnzqSrbtPbIOEZFsBTl9dA6w0Tm3GcDMngQuAVL/vL0EuN1b/i3wQzMzl8fJ8oknTvSX9x7cS9WDVbz90dsAvF//Pk+vf5rjBxxP7d5af7uWtha21m/lkicvYX/zfl5870WG9RvG8m3L+eXbv+TpK57mjPIzKC4qBuCVra8csd+epo/OKD+D929+nwGlAxhcNpiHVz4MwA3P3sAjlz1Cm2vjo6aP2L5/O0PKhtCnuI//2ubWZkqLS7P+ngRt3Y51NLY08okTP5HvUkRiL8hQOAlI/ehtLXBuV9s451rMrB4YDuwMsK5uDewzkJvPvZny/uXcuexO1u1Y1+H5Wc/MoqWtpcO6UfeNYuvejp8yvuqpq/w7uo17YBzFVsxpw07DzI54PcAJA07osbZTBp/iLy+9bik3PHMDz7zzDMffffzh93w92flUHldJkRXR2tbKB/UfcNqw00IZDM451u1Mfo/PKD+jy2m0/fv3M2DtgFyWllNRHl+Uxwa5Hd+cT8/hinFXBLoPC+qPcjO7HJjunPsn7/H/As51zt2Yss1qb5ta7/Emb5udnd5rFjALoKKiYtKTTz6ZVU0NDQ0MHDiw5w09ba4Nw/hN7W/46OBHNLY20tjayJDS5H0Y9rcmr0HU2tZKK62UWilDSofQp6gPuw/tptW18unjP82yHcswMw62HQRgYPFA+hb3pV9xP0YPHE1dcx3nn3A+A0qO7ger1bWyYd8GXt75MiVWwpZ9WxhaNpT+Jf3ZdTB53wWHo7SoNNR3bWujjVaX/P51paWlhZKS6J4XEeXxRXlskNvxXXTiRZw97OysXjtt2rQVzrnqHjd0zgXyBZwHPJ/y+Dbgtk7bPA+c5y2XkOwQrLv3nTRpksvWkiVLsn5tIYjy+KI8NueiPb4oj825whkf8KbL4Hd3kGcfLQdGm9lIM+sDXAks7LTNQuBab/ly4EWveBERyYPAeh6XPEZwI8luoBj4mXNujZl9m2RiLQQeAh4xs43AbpLBISIieRLoRJhzbhGwqNO6OSnLTcD/DLIGERHJXOwvcyEiIocpFERExKdQEBERn0JBRER8CgUREfEF9onmoJjZDuD9LF9eTh4voZEDUR5flMcG0R5flMcGhTO+U51zx/e0UcGFwrEwszddJh/zLlBRHl+UxwbRHl+UxwbRG5+mj0RExKdQEBERX9xCYUG+CwhYlMcX5bFBtMcX5bFBxMYXq2MKIiLSvbh1CiIi0o3YhIKZTTezDWa20cxm57uebJjZz8xsu3dzovZ1w8zsj2b2rvfvUG+9mdk8b7xvm1mo73VpZieb2RIzW2tma8zsJm99wY/PzMrM7A0z+4s3tju89SPN7HVvDL/yLjGPmfX1Hm/0nh+Rz/ozZWbFZvaWmT3jPY7E+Mxsi5mtMrOVZvamt67gfy67EotQMLNiYD4wAxgLXGVmY/NbVVYeBqZ3WjcbeME5Nxp4wXsMybGO9r5mAQ/kqMZstQBfc86NBSYDX/H+G0VhfAeBzzjnzgKqgOlmNhn4LvAD59xpwB7gem/764E93vofeNsVgpuA1PvXRml805xzVSmnnkbh5zK9TO7EU+hfZHAXuEL5AkYAq1MebwBO9JZPBDZ4yw8CV6XbrhC+gN8D/xC18QH9gT+TvF/5TqDEW+//jJLFHQnz/QVUkvzl+BngGcCiMj5gC1DeaV2kfi5Tv2LRKQAnAVtTHtd666Kgwjn3N2/5Q6DCWy7YMXvTCROB14nI+LyplZXAduCPwCagzjnX4m2SWr8/Nu/5emB4bis+anOBfwXavMfDic74HPBfZrbCu188ROTnMp3o3k07hpxzzswK+nQyMxsIPAXc7Jzba2b+c4U8PudcK1BlZkOA/wTOyHNJvcbM/hHY7pxbYWY1+a4nAJ9yzm0zsxOAP5rZ+tQnC/nnMp24dArbgJNTHld666LgIzM7EcD7d7u3vuDGbGalJAPhMefc77zVkRkfgHOuDlhCcjpliJm1/2GWWr8/Nu/5wcCuHJd6NKYAF5vZFuBJklNI9xGR8Tnntnn/bicZ6OcQsZ/LVHEJheXAaO9siD4k7wW9MM819ZaFwLXe8rUk5+Lb11/jnQ0xGahPaXdDx5ItwUPAOufcvSlPFfz4zOx4r0PAzPqRPFayjmQ4XO5t1nls7WO+HHjReRPUYeScu805V+mcG0Hy/60XnXNXE4HxmdkAMxvUvgxcAKwmAj+XXcr3QY1cfQEXAu+QnMv9Zr7ryXIMTwB/A5pJzlVeT3Iu9gXgXWAxMMzb1kiecbUJWAVU57v+Hsb2KZJzt28DK72vC6MwPmAC8JY3ttXAHG/9KOANYCPwG6Cvt77Me7zRe35UvsdwFGOtAZ6Jyvi8MfzF+1rT/rsjCj+XXX3pE80iIuKLy/SRiIhkQKEgIiI+hYKIiPgUCiIi4lMoiIiIT6EgkoaZjbCUq9FmsP11ZvaxDLb54bFXJxIchYJI77gO6DYURAqBQkGkayVm9gvvuvi/NbP+ZjbHzJab2WozW+B9cvVyoBp4zLvmfj8zO9vMXvHuofBG+6digY+Z2XPedfi/l8exiaSlUBDp2hhggXNuArAX+DLwQ+fc2c65cUA/4B+dc78F3gSuds5VAa3Ar4CbXPIeCud+dfJzAAAA/0lEQVQDjd57VgFXAOOBK8zsZERCRKEg0rWtzrk/ecuPkrwUxzTvbmGrSF747cw0rxsD/M05txzAObfXHb6E9AvOuXrnXBOwFjg12CGIHB1dOluka52vAeOAH5G8ns1WM7ud5HV8jsbBlOVW9P+ghIw6BZGunWJm53nLXwBe9pZ3evd9uDxl231A+3GDDcCJZnY2gJkNSrmEtEio6QdVpGvrgWvN7EGSV8N8ABhK8uqXW0hekr3dw8CPzayR5L0SrgDu9y6V3UjyuIJI6OkqqSIi4tP0kYiI+BQKIiLiUyiIiIhPoSAiIj6FgoiI+BQKIiLiUyiIiIhPoSAiIr7/D1vWP2m22OWiAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12bf9a331d0>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "vgg16(num_class=5,epochs=20)\t#tf36cnn\n",
    "history_this.loss_plot('epoch')\n",
    "history_this.loss_plot('batch')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Validation accuracy of VGG16: 100.0%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model 4: VGG19"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def vgg19( num_class, epochs,savepath='./VGG19.h5',history=history_this,input_shape=INPUT_SIZE):\n",
    "    model_fine_tune = VGG19(include_top=False, weights='imagenet', input_shape=input_shape)\n",
    "    for layer in model_fine_tune.layers[:19]:\t#the number of frozen layers for transfer learning, have tuned from 5-18\n",
    "        layer.trainable = False\n",
    "    for layer in model_fine_tune.layers[19:]:\n",
    "        layer.trainable = True\n",
    "    model = GlobalAveragePooling2D()(model_fine_tune.output)\n",
    "    model=Dense(units=256,activation='relu')(model)\n",
    "    model=Dropout(0.5)(model)\n",
    "    model = Dense(num_class, activation='softmax')(model)\n",
    "    model = Model(model_fine_tune.input, model, name='vgg')\n",
    "    opt = keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08)\t#tuned learning rate to be 0.001\n",
    "    model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])\t#set the loss function to be binary crossentropy\n",
    "    #train model\n",
    "    earlyStopping = kcallbacks.EarlyStopping(\n",
    "        monitor='val_acc', patience=2, verbose=1, mode='auto')\t#set early stop patience to save training time\n",
    "    saveBestModel = kcallbacks.ModelCheckpoint(\n",
    "        filepath=savepath,\n",
    "        monitor='val_acc',\n",
    "        verbose=1,\n",
    "        save_best_only=True,\n",
    "        mode='auto')\n",
    "    hist = model.fit_generator(\n",
    "        train_generator,\n",
    "        steps_per_epoch=len(train_generator),\n",
    "        epochs=epochs,\n",
    "        validation_data=validation_generator,\n",
    "        validation_steps=len(validation_generator),\n",
    "        #use_multiprocessing=True, \n",
    "        #workers=2,\n",
    "        callbacks=[earlyStopping, saveBestModel, history],\n",
    "    )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0090 - acc: 0.9968Epoch 00001: val_acc improved from -inf to 1.00000, saving model to ./VGG19.h5\n",
      "183/183 [==============================] - 78s 429ms/step - loss: 0.0089 - acc: 0.9968 - val_loss: 1.0405e-07 - val_acc: 1.0000\n",
      "Epoch 2/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 4.5221e-06 - acc: 1.0000Epoch 00002: val_acc did not improve\n",
      "183/183 [==============================] - 75s 411ms/step - loss: 4.4981e-06 - acc: 1.0000 - val_loss: 1.0396e-07 - val_acc: 1.0000\n",
      "Epoch 3/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0051 - acc: 0.9992Epoch 00003: val_acc did not improve\n",
      "183/183 [==============================] - 75s 409ms/step - loss: 0.0051 - acc: 0.9992 - val_loss: 1.0384e-07 - val_acc: 1.0000\n",
      "Epoch 00003: early stopping\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X98FOW59/HPlRCICAYEBV9gBaxHK5AEgxKL8uNRK8iroFWLVhSthcfnVI+UUx7xF1U556lH7akHyqlVaw/+RCtFsVI5pU2ItFIBi4KIFRFLOKiAJCUiapLr+WMn42ZJNrvJTjbY75vXvHbmnvueuXaY7LXzY+8xd0dERAQgJ9sBiIhIx6GkICIiISUFEREJKSmIiEhISUFEREJKCiIiElJSEBGRkJKCiIiElBRERCTUKdsBpKt3794+YMCAVrX96KOPOPzwwzMbUAYorvQorvR11NgUV3raEte6det2u/tRLVZ090NqKCkp8dYqKytrddsoKa70KK70ddTYFFd62hIXsNZT+IzV6SMREQkpKYiISEhJQUREQofchWYR+eL67LPPqKys5MCBA1mNo6CggDfeeCOrMTQllbjy8/Pp378/eXl5rVqHkoKIdBiVlZV0796dAQMGYGZZi2Pfvn107949a+tvTktxuTt79uyhsrKSgQMHtmodkZ0+MrOHzOwDM9vYzHwzs3lmtsXMXjOzU6KKRUQODQcOHKBXr15ZTQiHMjOjV69ebTrSivKawn8B45LMHw+cEAzTgZ9GGIuIHCKUENqmrdsvstNH7l5hZgOSVJkEPBzcP7vazHqY2THuvjOKeGbMgPLyYnr0iGLpLfFGL4nT1VWFFBTUpVS3yel06qbRdt++f6B7t33tvt6Wpvd/dBxdu77fuvWGL5lve+BAH/Lz36F5Sf5Yk/4dt7bd5xUOHDiG/Py/tsM602v38cf9OOywynDmD35aS07tp8lWkJ5Wfj7W1+eRk9NcHNlLWp06dyLqs1rZvKbQD9geN10ZlB2UFMxsOrGjCfr06UN5eXnaK9tbkcMRm3Kpbyho9Ghqj/03J34ANFEvrbYpKki7RfvoeGdUY7pmO4Bm5Gc7gCQ6amyHJRbUFsAnn2QjFAD+tq+KX7/wS7518bS0206//kLu+Zefc0T31L55zr///9H1sG5cffk/pbwOM2ffvtoW6x04cKBVn5NwiFxodvf7gfsBhg8f7mPGjEl7GWP+ZRnvzZtH32OOAbODh5ycpstTHdrQ/u1t2zj++OPbfb0ttd2wcSNDCwvbfb0tDX9as4YRpaUda3sB5eXlpLxvujc9nql5CdMVK1cyatSorK2/uXmrVq3ijJEjw+I33nuPE09sZQpLXEcrbNv2MYt//XNm3XAl3RK6k6itraVTp+Y/Mlf+/rm04ul9dC7dDs/hxCFJ7hJKWMa+Tz6hewpJJz8/n2HDhrVYrynZTAo7gGPjpvsHZdE47zw2d+1K31YklKhtLy/n+A4Y154ePaADxvXxe+/BCSdkO4y2CRLJQeMRqe/SBQ476Ht51tUWFEDv3p8X7NoFrbyVMhNmz5nD21u38tUxYzj33HOZMGECt99+O8cccwzr169n06ZNnH/++Wzfvp0DBw5w/fXXM336dAAGDBjA2rVrqampYfz48Zxxxhn88Y9/pF+/fjz77LMclrj9O3WKvdf8fNavX88111zD/v37Of7443nooYfo2bMn8+bN47777qNTp06cfPLJPPDAA6xcuZLrr78eADOjoqIio3dKZTMpLAWuNbNFwAigOqrrCSJy6JnxwgzWv7c+o8ss7lvMvePubXb+nXfeycaNG/nDH/5A9+7dKS8v5+WXX2bjxo3hLZ4PPfQQRx55JB9//DGnnnoqF154Ib169Wq0nLfeeosnnniCBx54gG9+85ssXryYKVOmNLveK664gvnz5zN69GjmzJnD7bffzr333sudd97JO++8Q5cuXaiqqgLgnnvuYcGCBYwcOZKamhry8zN7cjDKW1KfAF4CTjSzSjO72syuMbNrgirLgK3AFuAB4B+jikVEpLVOO+20Rvf8z5s3j6KiIkpLS9m+fTtvvfXWQW0GDhxIcXExACUlJWzbtq3Z5VdXV1NVVcXo0aMBmDp1KhUVFQAUFhZy2WWX8eijj4anrkaOHMnMmTOZN28eVVVVSU9ptUaUdx9d2sJ8B74b1fpF5NCW7Bt9e4rvqrq8vJwVK1bw0ksv0bVrV8aMGdPkbwK6dOkSjufm5vLxxx+3at3PP/88FRUVLF26lLlz57J69Wpmz57NhAkTWLZsGaWlpaxYsYKTTjqpVctvivo+EhEJdO/enX379jU7v7q6mp49e9K1a1c2b97M6tWr27zOgoICevbsyYsvvgjAI488wujRo6mvr2f79u2MHTuWu+66i6qqKmpqanj77bcZOnQoN9xwA8OHD2fz5s1tjiHeIXH3kYhIe+jVqxcjR45kxIgRTJgwgQkTJjSaP27cOO677z4KCws58cQTKS0tzch6Fy5cGF5oHjRoEL/4xS+oq6tjypQpVFdX4+5873vfo0ePHtx0002UlZWRk5PD4MGDGT9+fEZiaKCkICIS5/HHH2/Ux1D8bcZdunThN7/5TZPtGq4b9O7dm40bP+/d5/vf/36T9W+77bZwvLi4uMmjjlWrVjWa3rdvH/Pnz0/lbbSaTh+JiEhISUFEREJKCiIiElJSEBGRkJKCiIiElBRERCSkpCAiEqiqquI///M/W9X2vPPOC/snOpQpKYiIBJIlhdra5M8xWLZsGT2y8xSvjFJSEBEJzJ49m7fffpuRI0cya9YsysvLGTt2LN/61rcoLCwE4Pzzz6ekpITBgwdz//33h20HDBjA7t272bZtG1/5yleYNm0agwcP5mtf+1qTfR8999xzjBgxgmHDhnH22Wfz/vvvA1BTU8NVV13F0KFDKSwsZPHixQC88MILnHnmmRQVFXHWWWdFtg30i2YR6ZhmzID1me06m+JiuLdjdJ19xhlnsHr1asyMBx98kLvuuosf/ehHzJ07l4KCAjZs2ADA3r172bVrF9OmTWPZsmUMHTqUDz/8MLPbJY6SgohIEk11nb1kyRKAsOvsxKSQStfZlZWVTJ48mZ07d/Lpp5+G61ixYgWLFi0K6/Xs2ZPnnnuOUaNGMWDAAACOPPLITL7FRpQURKRjSvKNvj1F1XX2ddddx8yZM5k4cSLl5eWN+kLKJl1TEBEJtGfX2dXV1fTr1w+I9ZLa4JxzzmHBggXh9N69eyktLaWioiI84ojy9JGSgohIIL7r7FmzZh00f9y4cdTW1lJYWMitt97apq6zb7vtNi6++GLOPPNMesc9p/qWW25h7969DBkyhKKiIsrKyjjqqKO4//77mTJlCkVFRUyePLnV622JTh+JiMRpr66zJ02axKRJkw4q79atW6Mjhwbjx4/njDPOCOOKio4UREQkpKQgIiIhJQUREQkpKYiISEhJQUREQkoKIiISUlIQEQm0Z9fZt912G/fcc0+r1hUlJQURkYC6zlZSEBEJtWfX2fHWr19PaWkphYWFXHDBBezduxeIdb538sknU1hYyCWXXALAypUrKS4upri4mGHDhiXtlqM19ItmEemQZrwwg/XvZbbr7OK+xdw7rmN0nR3viiuuYP78+YwePZo5c+Zw++23c++993LnnXfyzjvv0KVLl/DU1D333MOCBQsYOXIkNTU15OfnZ2DLfE5HCiIiSTTVdXZRURGlpaVh19mJUuk6u0F1dTVVVVWMHj0agKlTp1JRUQFAYWEhl112GY8++iidOsW+w48cOZKZM2cyb948qqqqwvJM0ZGCiHRIyb7Rt6eous5OxfPPP09FRQVLly5l7ty5rF69mtmzZzNhwgSWLVtGaWkpK1as4KSTTmrV8psS6ZGCmY0zszfNbIuZzW5i/pfMrMzM/mxmr5nZeVHGIyKSTHt2nd2goKCAnj178uKLLwLwyCOPMHr0aOrr69m+fTtjx47lrrvuoqqqipqaGt5++22GDh3KDTfcwPDhw9m8eXObY4gX2ZGCmeUCC4BzgEpgjZktdfdNcdVuAZ5y95+a2cnAMmBAVDGJiCQT33X2hAkTmDBhQqP548aN47777qOwsJATTzyxTV1nx1u4cCHXXHMN+/fvZ9CgQfziF7+grq6OKVOmUF1djbvzve99jx49enDTTTdRVlZGTk4OgwcPZvz48RmJoUGUp49OA7a4+1YAM1sETALik4IDRwTjBcD/RBiPiEiL2qvr7PgnrRUXFzd51LFq1apG0/v27WP+/PmpvI1WizIp9AO2x01XAiMS6twG/LeZXQccDpwdYTwiItICc/doFmx2ETDO3b8TTF8OjHD3a+PqzAxi+JGZnQ78HBji7vUJy5oOTAfo06dPSfxDrdNRU1NDt27dWtU2SoorPYorfR01tsS4CgoK+PKXv5zFiGLq6urIzc3NdhgHSTWuLVu2UF1d3ahs7Nix69x9eIuN3T2SATgdWB43fSNwY0Kd14Fj46a3AkcnW25JSYm3VllZWavbRklxpUdxpa+jxpYY16ZNm7ITSIK//e1v2Q6hSanG1dR2BNZ6Cp/dUd59tAY4wcwGmlln4BJgaUKdvwJnAZjZV4B8YFeEMYmISBKRJQV3rwWuBZYDbxC7y+h1M7vDzCYG1f4ZmGZmrwJPAFcGGU1ERLIg0h+vufsyYreZxpfNiRvfBIyMMgYREUmdurkQEWmDjngBvy2UFEREJKSkICISmD17NgsWLAinGx6EU1NTw1lnncUpp5zC0KFDefbZZ1tcVnNdbL/wwguccsopFBUVcdZZZwGxW3Ovuuoqhg4dSmFhIYsXL878m0uROsQTkQ5pxgxYn9mesykuhnuT9LM3efJkZsyYwRVXXAHAU089xfLly8nPz2fJkiUcccQR7N69m9LSUiZOnIiZNbusprrYrq+vZ9q0aVRUVDBw4EA+/PBDAObOnUtBQQEbNmwACJ+nkA1KCiIigWHDhvHBBx+wc+dOtm7dSs+ePTn22GP57LPPuOmmm6ioqCAnJ4cdO3bw/vvv07dv32aXNW/ePJYsWQIQdrG9a9cuRo0aFXbFfeSRRwKwYsUK4n+U27NnzwjfZXJKCiLSISX7Rh+liy++mGeeeYaqqiomT54MwGOPPcauXbtYt24deXl5DBgwoMkusxuk2sV2R6RrCiIicSZPnszixYt5+umnufjii4FYl9lHH300eXl5lJWV8e677yZdRnNdbJeWllJRUcE777wDEJ4+Oueccxpdy8jm6SMlBRGROIMHD6ampoZ+/fpxzDHHAHDZZZexdu1ahg8fzmOPPdbiQ23GjRtHbW0thYWF3HrrrWEX20cddRT3338/3/jGNygqKgqPRG655Rb27t3LkCFDKCoqoqysLNo3mYROH4mIJFi9enXYdTbEusN+6aWXmqxbU1NzUFmyLrbHjx9/0DMQunXrxsKFC9sQceboSEFEREJKCiIiElJSEBGRkJKCiIiElBRERCSkpCAiIiElBRGRNmiu6+xDtUttJQUREQkpKYiIBDLZdXYDd2fWrFkMGTKEoUOH8uSTTwKwc+dORo0aRXFxMUOGDOHFF1+krq6OK6+8Mqz74x//OOPvsSX6RbOIdEgzZsxgfYb7zi4uLubeJD3tZbLr7Aa/+tWvWL9+Pa+++iq7d+/m1FNPZdSoUTz++OOce+653HzzzdTV1bF//37Wr1/Pjh072LhxIwBVVVWZeeNpUFIQEQlksuvsBqtWreLSSy8lNzeXPn36MHr0aNasWcOpp57Kt7/9bT777DPOP/98iouLGTRoEFu3buW6665jwoQJfO1rX2uHd92YkoKIdEjJvtFHKRNdZ6di1KhRVFRU8Pzzz3P55Zcza9YsrrjiCl599VWWL1/OggULeOqpp3jooYcy8bZSpmsKIiJxMtF1drwzzzyTJ598krq6Onbt2kVFRQWnnXYa7777Ln369GHatGlcffXVvPLKK+zevZv6+nouvPBC5s6dyyuvvBLV22yWjhREROI013X217/+dYYPH05xcXGLXWfHu+CCC3jppZcoKirCzLjrrrvo27cvCxcu5O677yYvL49u3brx8MMPs2PHDq666irq6+sB+OEPfxjJe0xGSUFEJEFbu86OLzcz7r77bu6+++5G86dOncrUqVMPapeNo4N4On0kIiIhJQUREQkpKYhIh+Lu2Q7hkNbW7aekICIdRn5+Pnv27FFiaCV3Z8+ePeTn57d6GbrQLCIdRv/+/amsrGTXrl1ZjePAgQNt+mCNSipx5efn079//1avQ0lBRDqMvLw8Bg4cmO0wKC8vZ9iwYdkO4yDtEZdOH4mISCjSpGBm48zsTTPbYmazm6nzTTPbZGavm9njUcYjIiLJRXb6yMxygQXAOUAlsMbMlrr7prg6JwA3AiPdfa+ZHR1VPCIi0rIojxROA7a4+1Z3/xRYBExKqDMNWODuewHc/YMI4xERkRZYVLd+mdlFwDh3/04wfTkwwt2vjavzDPAXYCSQC9zm7i80sazpwHSAPn36lCxatKhVMdXU1HTIR+QprvQorvR11NgUV3raEtfYsWPXufvwFiu6eyQDcBHwYNz05cBPEur8GlgC5AEDge1Aj2TLLSkp8dYqKytrddsoKa70KK70ddTYFFd62hIXsNZT+OxO6fSRmY00s8OD8Slm9u9mdlwLzXYAx8ZN9w/K4lUCS939M3d/h9hRwwmpxCQiIpmX6jWFnwL7zawI+L/Au8DDLbRZA5xgZgPNrDNwCbA0oc4zwBgAM+sN/AOwNcWYREQkw1JNCrXB4cck4D/c/T+A7skauHstcC2wHHgDeMrdXzezO8xsYlBtObDHzDYBZcAsd9/TmjciIiJtl+otqfvM7EZgCjDKzHKIXQdIyt2XAcsSyubEjTswMxhERCTLUj1SmAx8Alzt7u8Ruz5wd/ImIiJyqEn5SIHYaaM6M/sH4CTgiejCEhGRbEj1SKEC6GJm/YDfAVcB/xVVUCIikh2pJgVz9/3AN4D57n4BMCS6sEREJBtSTgpmdjpwGfB8mm1FROQQkeoH+wxiHdctCW4rHUTsFlIREfkCSelCs7uvBFaaWTcz6+buW4F/ijY0ERFpb6l2czHUzP4MvA5sMrN1ZjY42tBERKS9pXr66GfATHc/zt2/BPwz8EB0YYmISDakmhQOd/fwGoK7lwOHRxKRiIhkTao/XttqZrcCjwTTU1DHdSIiXzipHil8GzgK+FUwHBWUiYjIF0iqdx/tRXcbiYh84SVNCmb2HNDs8zrdfWJz80RE5NDT0pHCPe0ShYiIdAhJk0Lwo7VGzOwUd38lupBERCRbWtN/0YMZj0JERDqE1iQFy3gUIiLSIbQmKdye8ShERKRDSLXvowvMrADA3Z8xsx5mdn60oYmISHtL9UjhB+5e3TDh7lXAD6IJSUREsiXVpNBUvVS7yBARkUNEqklhrZn9u5kdHwz/DqyLMjAREWl/qSaF64BPgSeBRcAB4LtRBSUiItmRat9HHwGzI45FRESyLNW7j35rZj3ipnua2fLowhIRkWxI9fRR7+COIyDsNfXoaEISEZFsSTUp1JvZlxomzGwASXpPFRGRQ1Oqt5XeDKwys5XEurk4E5geWVQiIpIVqV5ofsHMhhNLBH8GngE+jjIwERFpf6leaP4O8Dvgn4HvE3tW820ptBtnZm+a2RYza/buJTO70Mw8SDwiIpIlqV5TuB44FXjX3ccCw4BdyRqYWS6wABgPnAxcamYnN1Gve7D8P6URt4iIRCDVpHDA3Q8AmFkXd98MnNhCm9OALe6+1d0/Jfajt0lN1JsL/BuxH8SJiEgWpZoUKoPfKTwD/NbMngX+p4U2/YDt8csIykJmdgpwrLs/n2IcIiISIXNP785SMxsNFAAvBEcAzdW7CBjn7t8Jpi8HRrj7tcF0DvB74Ep332Zm5cD33X1tE8uaTnC3U58+fUoWLVqUVswNampq6NatW6vaRklxpUdxpa+jxqa40tOWuMaOHbvO3Vu+buvukQzA6cDyuOkbgRvjpguA3cC2YDhA7OhjeLLllpSUeGuVlZW1um2UFFd6FFf6Ompsiis9bYkLWOspfHa35slrqVoDnGBmA82sM3AJsDQuGVW7e293H+DuA4DVwERv4khBRETaR2RJwd1rgWuB5cAbwFPu/rqZ3WFmE6Nar4iItF6kD8px92XAsoSyOc3UHRNlLCIi0rIoTx+JiMghRklBRERCSgoiIhJSUhARkZCSgoiIhJQUREQkpKQgIiIhJQUREQkpKYiISEhJQUREQkoKIiISUlIQEZGQkoKIiISUFEREJKSkICIiISUFEREJKSmIiEhISUFEREJKCiIiElJSEBGRkJKCiIiElBRERCSkpCAiIiElBRERCSkpiIhISElBRERCSgoiIhJSUhARkZCSgoiIhJQUREQkpKQgIiKhSJOCmY0zszfNbIuZzW5i/kwz22Rmr5nZ78zsuCjjERGR5CJLCmaWCywAxgMnA5ea2ckJ1f4MDHf3QuBp4K6o4hERkZZFeaRwGrDF3be6+6fAImBSfAV3L3P3/cHkaqB/hPGIiEgLzN2jWbDZRcA4d/9OMH05MMLdr22m/k+A99z9X5qYNx2YDtCnT5+SRYsWtSqmmpoaunXr1qq2UVJc6VFc6euosSmu9LQlrrFjx65z9+EtVnT3SAbgIuDBuOnLgZ80U3cKsSOFLi0tt6SkxFurrKys1W2jpLjSo7jS11FjU1zpaUtcwFpP4bO7U6tSTmp2AMfGTfcPyhoxs7OBm4HR7v5JhPGIiEgLorymsAY4wcwGmlln4BJgaXwFMxsG/AyY6O4fRBiLiIikILKk4O61wLXAcuAN4Cl3f93M7jCziUG1u4FuwC/NbL2ZLW1mcSIi0g6iPH2Euy8DliWUzYkbPzvK9YuISHr0i2YREQkpKYiISEhJQUREQkoKIiISUlIQEZGQkoKIiISUFEREJKSkICIiISUFEREJKSmIiEhISUFEREJKCiIiElJSEBGRkJKCiIiElBRERCSkpCAiIiElBRERCSkpiIhISElBRERCSgoiIhJSUhARkZCSgoiIhJQUREQkpKQgIiIhJQUREQkpKYiISEhJQUREQkoKIiISUlIQEZGQkoKIiIQiTQpmNs7M3jSzLWY2u4n5XczsyWD+n8xsQJTxiIhIcp2iWrCZ5QILgHOASmCNmS11901x1a4G9rr7l83sEuDfgMlRxPOvFf/Kz17+Gd1f705uTi45lkOuBa85ueRabqPXhvm5Obl0yunUaLyTdSI3N5dO1ik2nWTIy82LvebkhdN5OXl0yo2V5eXksXn3Zj7Z8gl5uXl0zu1MXk4euTm54Xjn3M7hvM45nencKfbaKacTOTk62BORzIksKQCnAVvcfSuAmS0CJgHxSWEScFsw/jTwEzMzd/dMB7Phgw1UflwJH8emnYyvom1ez9yiDIu9moXTiePxrzmW0+R4XW0dXdZ0wSxW3tTQkFjD8Zy4ZJuQaBNfkyXbXMv9PPHmdgpf8yyP7X/dzupVqxsl2obEm5eT9/l0kIAbkmtebl6jxHsoJtv6+npq62upJ/baMDSU13ot7v55uddTV1/Hln1bOGLnEdTV11FbX0ud18XaeC119XWxeh6b19C+zuti40Gd+Hp19XU4Hiv3unA9DW0a6iROOx6uu97r+WvlX3nqo6fCOg3Ld3fq64NxvHG514fra5hX7/WN5jVXHj/P3RvVA8Lxj/Z/xGEbDwvru8eGej5vF//a0D5+2Q3La/isaapdQ5uGsvh6DeMNdXCY0HcCY8aMiXQfizIp9AO2x01XAiOaq+PutWZWDfQCdmc6mL6r+lJYXkiPHj2S1ov/T240Hr9DJIyHO0KynaeJHaLh9cAnB+jcuXPT84OdodmyJnaupsYb6jd6j3y+IwcjjZJlvddTYzUHlTdq83nB3xdLnDQcDxNyi186/t6216HImituZkb8vCb2jybrWvN1mpr/1qC3YFrSqNssyqSQMWY2HZgO0KdPH8rLy9NeRmVlJXV1dVRVVWU2tuBfTrqXZ+zz17rOdeTm5mY0rkyoq8tsXAclz7iklmy8oW3DdF19HWbWdIJspix+Gjho2Yll4TqbSKjx8xPnxf9hW+O/6IPLkk0bzS+Lg48GE+s0GjfD3cMjwObW01T7+CPMppbb1Lx0yuvr68N9rOFvKazTRGyJ76GpZSaWtUam9/1MOe6w41r1+ZeW8FtshgfgdGB53PSNwI0JdZYDpwfjnYgdIViy5ZaUlHhrlZWVtbptlBRXehRX+jpqbIorPW2JC1jrKXx2R3nidA1wgpkNNLPOwCXA0oQ6S4GpwfhFwO+D4EVEJAsiO33ksWsE1xI7GsgFHnL3183sDmIZaynwc+ARM9sCfEgscYiISJZEek3B3ZcByxLK5sSNHwAujjIGERFJXce8705ERLJCSUFEREJKCiIiElJSEBGRkJKCiIiE7FD7WYCZ7QLebWXz3kTQhUYGKK70KK70ddTYFFd62hLXce5+VEuVDrmk0BZmttbdh2c7jkSKKz2KK30dNTbFlZ72iEunj0REJKSkICIiob+3pHB/tgNohuJKj+JKX0eNTXGlJ/K4/q6uKYiISHJ/b0cKIiKSxBcmKZjZODN708y2mNnsJuZ3MbMng/l/MrMBcfNuDMrfNLNz2zmumWa2ycxeM7PfmdlxcfPqzGx9MCR2Ox51XFea2a649X8nbt5UM3srGKYmto04rh/HxfQXM6uKmxfl9nrIzD4ws43NzDczmxfE/ZqZnRI3L5LtlUJMlwWxbDCzP5pZUdy8bUH5ejNbm6mY0ohtjJlVx/1/zYmbl3QfiDiuWXExbQz2qSODeZFsMzM71szKgs+B183s+ibqtN/+lcpDFzr6QKxr7reBQUBn4FXg5IQ6/wjcF4xfAjwZjJ8c1O8CDAyWk9uOcY0Fugbj/6chrmC6Jovb60rgJ020PRLYGrz2DMZ7tldcCfWvI9Yle6TbK1j2KOAUYGMz888DfkPsGWulwJ/aYXu1FNNXG9YFjG+IKZjeBvTO4vYaA/y6rftApuNKqPt1Ys94iXR/PerJAAAFZ0lEQVSbAccApwTj3YG/NPH32G771xflSOE0YIu7b3X3T4FFwKSEOpOAhcH408BZZmZB+SJ3/8Td3wG2BMtrl7jcvczd9weTq4H+GVp3m+JK4lzgt+7+obvvBX4LjMtSXJcCT2Ro3Um5ewWxZ340ZxLwsMesBnqY2TFEuL1aisnd/xisE9pv32pYd0vbqzlt2TczHVe77F/uvtPdXwnG9wFvEHt+fbx227++KEmhH7A9brqSgzdqWMfda4FqoFeKbaOMK97VxL4NNMg3s7VmttrMzs9QTOnEdWFwqPq0mR2bZtso4yI4zTYQ+H1ccVTbKxXNxR7l9kpH4r7lwH+b2TqLPQM9G043s1fN7DdmNjgo6xDby8y6EvtwXRxXHPk2s9hp7WHAnxJmtdv+FelDdiR1ZjYFGA6Mjis+zt13mNkg4PdmtsHd326nkJ4DnnD3T8zsfxM7yvpf7bTuVFwCPO3udXFl2dxeHZaZjSWWFM6IKz4j2FZHA781s83Bt+j28gqx/68aMzsPeAY4oR3X35KvA39w9/ijiki3mZl1I5aEZrj73zK13HR9UY4UdgDHxk33D8qarGNmnYACYE+KbaOMCzM7G7gZmOjunzSUu/uO4HUrUE7sG0S7xOXue+JieRAoSbVtlHHFuYSEQ/sIt1cqmos9yu3VIjMrJPb/N8nd9zSUx22rD4AlZO6UaUrc/W/uXhOMLwPyzKw3Wd5ecZLtXxnfZmaWRywhPObuv2qiSvvtX5m+aJKNgdgRz1ZipxMaLk4NTqjzXRpfaH4qGB9M4wvNW8ncheZU4hpG7MLaCQnlPYEuwXhv4C0ydMEtxbiOiRu/AFjtn1/YeieIr2cwfmR7xRXUO4nYRT9rj+0Vt44BNH/hdAKNLwS+HPX2SiGmLxG7RvbVhPLDge5x438ExmVyW6UQW9+G/z9iH65/DbZdSvtAVHEF8wuIXXc4vD22WfC+HwbuTVKn3favjO4E2RyIXZ3/C7EP2JuDsjuIffsGyAd+GfyRvAwMimt7c9DuTWB8O8e1AngfWB8MS4PyrwIbgj+KDcDV7RzXD4HXg/WXASfFtf12sB23AFe1Z1zB9G3AnQntot5eTwA7gc+Inbe9GrgGuCaYb8CCIO4NwPCot1cKMT0I7I3bt9YG5YOC7fRq8H98cya3VYqxXRu3f60mLnE1tQ+0V1xBnSuJ3XwS3y6ybUbstJ4Dr8X9X52Xrf1Lv2gWEZHQF+WagoiIZICSgoiIhJQUREQkpKQgIiIhJQUREQkpKYi0o6B30F9nOw6R5igpiIhISElBpAlmNsXMXg76zv+ZmeWaWY2Z/cjMXrHYsy+OCuoWB53wvWZmS8ysZ1D+ZTNbEXT69oqZHR8svlvQyeBmM3ss6K1XpENQUhBJYGZfASYDI929GKgDLiPWvcEr7n4KsBL4QdDkYeAGdy8k9mvThvLHgAXuXkTsF9c7g/JhwAxiz/IYBIyM/E2JpEi9pIoc7CxiHQCuCb7EHwZ8ANQDTwZ1HgV+ZWYFQA93XxmULwR+aWbdgX7uvgTA3Q8ABMt72d0rg+n1xPriWRX92xJpmZKCyMEMWOjuNzYqNLs1oV5r+4j5JG68Dv0dSgei00ciB/sdcFHQbz5mdmTwUJ8c4KKgzreAVe5eDew1szOD8suBlR57glZlw8N+LPaM8K7t+i5EWkHfUEQSuPsmM7uF2FO2coj1qPld4CNgsJmtI/bkvslBk6nAfcGH/lbgqqD8cuBnZnZHsIyL2/FtiLSKekkVSZGZ1bh7t2zHIRIlnT4SEZGQjhRERCSkIwUREQkpKYiISEhJQUREQkoKIiISUlIQEZGQkoKIiIT+P6hB4VvispccAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12be6ce7dd8>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuUVOWZ7/Hv01egJXK1RRoFIxrRYBtaxRC1zSQu0CyNiROvEz3HCclET0wcMwtXEgZ1kjETY4yGGDGjuXrJJBMPyZCgx1AqJioXieEqiArtmEHkIg009OU5f9SusrrounTRm6ra9fusVYvau96qet6iqIf3efd+t7k7IiIiAFXFDkBEREqHkoKIiCQpKYiISJKSgoiIJCkpiIhIkpKCiIgkKSmIiEiSkoKIiCQpKYiISFJNsQPor1GjRvn48eMLeu7u3btpaGgY2IBKSJT7F+W+QbT7F+W+Qfn0b9myZVvdfXSudmWXFMaPH8/SpUsLem4sFqO1tXVgAyohUe5flPsG0e5flPsG5dM/M3s9n3YqH4mISJKSgoiIJCkpiIhIUtnNKYhIdHV2dtLW1kZHR0exQ8nb4Ycfzpo1a4odRtKgQYNoamqitra2oOcrKYhIyWhra2Po0KGMHz8eMyt2OHnZtWsXQ4cOLXYYALg7b7/9Nm1tbUyYMKGg11D5SERKRkdHByNHjiybhFBqzIyRI0ce1EhLSUFESooSwsE52M+vYpLC4k2LeeDVB9jfvb/YoYiIlKyKSQp/3PxHfrrpp3R2dxY7FBEpUTt27OD73/9+Qc89//zz2bFjR97t58yZwx133FHQe4Up1KRgZtPNbJ2ZbTCzWRnafMrMVpvZKjN7KKxYqizeVcfDegsRKXPZkkJXV1fW5y5YsIBhw4aFEdYhFVpSMLNqYC4wA5gEXG5mk9LaTARuBqa5+0nAF0OLh3idrcd7wnoLESlzs2bN4pVXXqG5uZkvf/nLxGIxzj33XK644gomT54MwMc//nGmTJnCSSedxLx585LPHT9+PFu3buW1117jxBNP5DOf+QwnnXQS5513Hnv37s36vitWrGDq1KlMnjyZiy++mO3btwNw9913M2nSJCZPnsxll10GwFNPPUVzczPNzc2ceuqp7Nq1a0A/gzAPST0d2ODuGwHM7BHgImB1SpvPAHPdfTuAu28JK5jE5Iu7Rgoi5eCLv/8iK/66YkBfs/nIZu6aflfGx2+//XZWrlzJihXx943FYrzwwgusXLkyeYjnAw88wIgRI9i7dy+nnXYa55133gGHpK5fv56HH36Y+++/n0996lP86le/4qqrrsr4vp/+9Ke55557OOecc5g9eza33HILd911F7fffjuvvvoq9fX1ydLUHXfcwdy5c5k2bRrt7e0MGjToYD+WXsIsH40FNqdstwX7Uh0PHG9mz5rZc2Y2PaxgVD4SkUKcfvrpvY75v/vuuznllFOYOnUqmzdv5pVXXjngORMmTKC5uRmAKVOm8Nprr2V8/Z07d7Jjxw7OOeccAK6++mqefvppACZPnsyVV17Jz372M2pq4v+HnzZtGjfeeCN33303O3bsSO4fKMU+ea0GmAi0Ak3A02b2fnfvNVtjZjOBmQCNjY3EYrF+v9GGtg0APP3M07yn9j0HFXSpam9vL+izKQdR7htEu3/96dvhhx+eLIfcNu22UOLJVm5pb2+np6cn2WbPnj3U19cnt5955hkWLlzI448/zpAhQzj//PPZu3cvu3btwt1pb2+nvb2d2tra5HO6urrYvXv3Ae+7b9++ZDt3Tz6eGsMjjzzCs88+y4IFC7jlllt44YUXuO6662htbeXxxx/njDPOYP78+Rx//PG9Xrujo6Pg71OYSeENYFzKdlOwL1Ub8Ly7dwKvmtnLxJPEktRG7j4PmAfQ0tLihSxT+9LzL8Er8Sw7csjIfj+/HJTLEr6FiHLfINr960/f1qxZU9Szg8eMGcPu3buTMQwZMoSamprkdmdnJ6NGjaKxsZG1a9eyZMkSqqqqGDp0KGbGYYcdBpDcB1BfX09nZ+cB/aqvr6e+vp6mpiZGjBjBihUrOOuss/j1r3/NueeeS0NDA5s2beKCCy7gvPPOo6mpCTNjy5YtTJ06lalTp7J8+XI2b97MlClTer32oEGDOPXUUwv6DMJMCkuAiWY2gXgyuAy4Iq3NY8DlwINmNop4OWljGMGofCQiuYwcOZJp06Zx8sknM2PGDC644IJej0+fPp0f/OAHTJ48mRNOOIGpU6cOyPv++Mc/5nOf+xx79uzh2GOP5cEHH6S7u5urrrqKnTt34u586UtfYtiwYXzta19j0aJFVFVVcdJJJzFjxowBiSEhtKTg7l1mdj2wEKgGHnD3VWZ2K7DU3ecHj51nZquBbuDL7v52GPHo6CMRycdDD/U+Mj51lFNfX8/vfve7Xo8nyj6JeYNRo0axcuXK5OM33XRTn+8zZ86c5P3m5maee+65A9osXrz4gH333HNP1vgPVqhzCu6+AFiQtm92yn0HbgxuodLRRyIiuVXMGc0qH4mI5FYxSUHlI5HyoNH8wTnYz69ykoLKRyIlb9CgQbz99tv6d1qgxPUUDuaEtmKfp3DIqHwkUvqamppoa2vjrbfeKnYoeevo6Bjws4oPRuLKa4WqmKSg8pFI6autrS34imHFEovFCj4noBSpfCQiIkkVkxRUPhIRya1ikoLKRyIiuVVOUlD5SEQkp4pJCiofiYjkVjFJQeUjEZHcKicpqHwkIpJTxSQFlY9ERHKrmKSg8pGISG6VkxRUPhIRyalikoLKRyIiuVVMUlD5SEQkt8pJCiofiYjkVDlJIRgpqHwkIpJZxSSFxJyCykciIplVTFJQ+UhEJLfKSQoqH4mI5FQxSSF5SKpGCiIiGVVMUkiUjzSnICKSWahJwcymm9k6M9tgZrP6ePwaM3vLzFYEt78PLRaVj0REcqoJ64XNrBqYC3wUaAOWmNl8d1+d1vRRd78+rDgSVD4SEcktzJHC6cAGd9/o7vuBR4CLQny/rFQ+EhHJLbSRAjAW2Jyy3Qac0Ue7T5rZ2cDLwJfcfXN6AzObCcwEaGxsJBaL9TuYl7a9BMDyF5fTubGz388vB+3t7QV9NuUgyn2DaPcvyn2D6PUvzKSQj98AD7v7PjP7LPBj4MPpjdx9HjAPoKWlxVtbW/v9Rt0bu+Ev0NzczFnHnHVwUZeoWCxGIZ9NOYhy3yDa/Yty3yB6/QuzfPQGMC5luynYl+Tub7v7vmDzh8CUsIJR+UhEJLcwk8ISYKKZTTCzOuAyYH5qAzMbk7J5IbAmrGB09JGISG6hlY/cvcvMrgcWAtXAA+6+ysxuBZa6+3zgC2Z2IdAFbAOuCSseHX0kIpJbqHMK7r4AWJC2b3bK/ZuBm8OMIUHlIxGR3CrnjGaVj0REcqqYpKDykYhIbhWTFFQ+EhHJrXKSgspHIiI5VUxSUPlIRCS3ikkKKh+JiORWOUlB5SMRkZwqJimofCQiklvFJAWVj0REcqucpKDykYhIThWTFFQ+EhHJrWKSgspHIiK5VU5SUPlIRCSnykkKwUhB5SMRkcwqJikk5hRUPhIRyaxikoLKRyIiuVVOUlD5SEQkp4pJCslDUjVSEBHJqGKSQqJ8pDkFEZHMKicpqHwkIpJTxSQFlY9ERHKrmKSg8pGISG6VkxRUPhIRySnUpGBm081snZltMLNZWdp90szczFrCikXlIxGR3EJLCmZWDcwFZgCTgMvNbFIf7YYCNwDPhxULqHwkIpKPMEcKpwMb3H2ju+8HHgEu6qPdbcA3gY4QY1H5SEQkDzUhvvZYYHPKdhtwRmoDM/sAMM7d/8vMvpzphcxsJjAToLGxkVgs1u9gtu7bCsDadWuJ7er/88tBe3t7QZ9NOYhy3yDa/Yty3yB6/QszKWRlZlXAncA1udq6+zxgHkBLS4u3trb2+/3e3PUmPAcTj59Ia0v/n18OYrEYhXw25SDKfYNo9y/KfYPo9S/M8tEbwLiU7aZgX8JQ4GQgZmavAVOB+WFNNqt8JCKSW5hJYQkw0cwmmFkdcBkwP/Ggu+9091HuPt7dxwPPARe6+9IwgtHRRyIiuYWWFNy9C7geWAisAX7h7qvM7FYzuzCs981ERx+JiOQW6pyCuy8AFqTtm52hbWuYsah8JCKSW8Wc0azykYhIbhWTFFQ+EhHJrXKSgspHIiI5VUxSUPlIRCS3ikkKKh+JiORWOUlB5SMRkZzySgpmNs3MGoL7V5nZnWZ2TLihDSyVj0REcst3pHAvsMfMTgH+CXgd+EloUYVA5SMRkdzyTQpdHq+7XAR8192/S3ztorKh8pGISG75ntG8y8xuBq4Czg5WOK0NL6yBlxgpqHwkIpJZviOFS4F9wLXu/lfiK55+K7SoQpCYU1D5SEQks7xHCsTLRt1mdjzwPuDh8MIaeCofiYjklu9I4Wmg3szGAk8C/wv4UVhBhUHlIxGR3PJNCubue4BPAPe4+8XEL5BTNpKHpGqkICKSUd5JwczOBK4E/qufzy0JifKR5hRERDLL94f9i8DNwK+DC+UcCywKL6zwqHwkIpJZXhPN7v4U8JSZHWZmh7n7RuAL4YY28KqoUvlIRCSLfJe5eL+ZvQisAlab2TIzOync0MKh8pGISGb5lo/uA25092Pc/WjgH4H7wwsrHGam8pGISBb5JoUGd0/OIbh7DGgIJaIQqXwkIpJdvievbTSzrwE/DbavAjaGE1K4VD4SEcks35HC/wZGA/8Z3EYH+8qKykciItnle/TRdsrwaKN0Kh+JiGSXNSmY2W8g83+t3f3CHM+fDnwXqAZ+6O63pz3+OeA6oBtoB2a6++r8Qi+MykciIpnlGincUegLm1k1MBf4KNAGLDGz+Wk/+g+5+w+C9hcCdwLTC33PPGJS+UhEJIusSSE4aa0XM/uAuy/P47VPBzYEJ7phZo8Qv0hPMim4+zsp7RvIMioZCCofiYhkl+/RR6l+CHwgj3Zjgc0p223AGemNzOw64EagDvhwAfH0i8pHIiKZFZIUbCADcPe5wFwzuwL4KnD1AW9oNhOYCdDY2EgsFiv4/Ta/sfmgnl/K2tvb1bcyFeX+RblvEL3+FZIUbsmz3RvAuJTtpmBfJo8A9/b1gLvPA+YBtLS0eGtra54h9Fb9bDVjjxpLoc8vdbFYTH0rU1HuX5T7BtHrX75rH11sZocDuPtjZjbMzD6e42lLgIlmNsHM6oDLgPlprzsxZfMCYH3+oRdG5SMRkczyPXntn919Z2LD3XcA/5ztCe7eBVwPLATWAL8Ilt2+NTjSCOB6M1tlZiuIzyscUDoaSDr6SEQku3zLR30lj5zPdfcFwIK0fbNT7t+Q5/sPCB19JCKSXb4jhaVmdqeZvTe43QksCzOwsKh8JCKSWb5J4f8A+4FHiU8IdxA/E7msVFs13d5d7DBEREpWvmsf7QZmhRxL6KqsSklBRCSLfI8+esLMhqVsDzezheGFFY5qq6arp6vYYYiIlKx8y0ejgiOOgOSqqUeEE1J4qqyK7h6NFEREMsk3KfSY2dGJDTMbT8jrFIVBcwoiItnle0jqV4DFZvYU8WUuziJYdqKcVFGl8pGISBb5TjT/3sxaiCeCF4HHgL1hBhaGaqtW+UhEJIu8koKZ/T1wA/H1i1YAU4E/cQhWNR1IVaaRgohINvnOKdwAnAa87u7nAqcCb4UWVUg0pyAikl2+SaHD3TsAzKze3dcCJ4QXVjh0SKqISHb5TjS3BecpPAY8YWbbgf8OL6xwaE5BRCS7fCeaLw7uzjGzRcDhwO9DiyokOvpIRCS7fl9kp6/rNpcLzSmIiGSX75xCJOjoIxGR7CoqKWhOQUQku4pKChopiIhkV1FJQXMKIiLZVVRS0EhBRCS7ikoK1WhOQUQkm8pKCjqjWUQkq4pKCrocp4hIdhWVFHRIqohIdqEmBTObbmbrzGyDmc3q4/EbzWy1mb1kZk+a2TFhxqOJZhGR7EJLCmZWDcwFZgCTgMvNbFJasxeBFnefDPwS+Lew4gEdkioikkuYI4XTgQ3uvtHd9wOPABelNnD3Re6+J9h8jvhFfEKjkYKISHZhJoWxwOaU7bZgXybXAr8LMR4dkioikkO/V0kNg5ldBbQA52R4fCbx60PT2NhILBYr6H26u7rZ37W/4OeXuvb2dvWtTEW5f1HuG0Svf2EmhTeAcSnbTcG+XszsI8BXgHPcfV9fL+Tu84B5AC0tLd7a2lpQQPdvvB83p9Dnl7pYLKa+lako9y/KfYPo9S/M8tESYKKZTTCzOuAyYH5qAzM7FbgPuNDdt4QYC6A5BRGRXEJLCu7eBVwPLATWAL9w91VmdquZXRg0+xZwGPAfZrbCzOZneLkBUW3V9HgP7h7m24iIlK1Q5xTcfQGwIG3f7JT7Hwnz/dNVWTwHdns3NVYS0ykiIiWl4s5oBnQEkohIBhWVFKqC7mpeQURKxed++znsFit2GEkVlRSSIwWd1SwiJeK+ZfcVO4ReKiopJOYUNFIQEelbRSUFzSmIiGRXkUlBIwURKTWlcqh8RSWF1ENSRURKSY/3FDsEoMKSQmKksHjT4iJHIiLSm5JCEXT2dAJw+a8uL3IkIiK9lUoFo6KSglE6xwKLiKTSSKEIph85nZqqGlrHtxY7FBGRXkrlqMiKSgq1VbWcfczZdHZ3FjsUEZFeNFIoktqqWh2SKiIlR3MKRVJTVaOkICIlRyOFIlFSEJFSpDmFIqmpqkkemioicqj9Zt1vOOJbR7C3c2+v/RopFEltteYURKR4bnz8Rt7a8xab39nca7/mFIpE5SMRKabEiCCx7E76/mJTUhAROYQyJQXNKRRJjdXoPAURKZrEj79GCiVCcwoiUkyJH//0kYHmFIpE5SMRKaZEUkj/HdJIoUiUFEQkDN9c/E2Wv7k8Z7vkSCFtZFARcwpmNt3M1pnZBjOb1cfjZ5vZcjPrMrNLwowlQecpiEgYZj05iynzpuRsl0gG6Ukg8iMFM6sG5gIzgEnA5WY2Ka3ZJuAa4KGw4kintY9EZKD151KaGUcKJTKnUBPia58ObHD3jQBm9ghwEbA60cDdXwseO2QpMlE+cnfMdH0FETl46T/oid+Y2uraA9pW8pzCWCD1lL22YF9R1VTF82Cp/AWISPlL/4E/68GzqPuXuj7bZjz6qETmFMIcKQwYM5sJzARobGwkFosV9Drt7e1s2rYJgCdjT1JX1fdfWrlqb28v+LMpdVHuG0S7f1HuG8T7t+ipRcntWCzGc23PJe+n6+yKz2kuWbaEvRveXf9oybIl7F6/O9xg8xBmUngDGJey3RTs6zd3nwfMA2hpafHW1taCAorFYpww4gR4FT74oQ9yWN1hBb1OqYrFYhT62ZS6KPcNot2/KPcN4v1rntoMi+Pbp33wNHgqfr+vftsfDbphcvNkzj7m7GTb5lOb+eC4Dx6aoLMIs3y0BJhoZhPMrA64DJgf4vvlJVE+0mSziAyU1N+T8x86P3m/rwnoRJmo4o4+cvcu4HpgIbAG+IW7rzKzW83sQgAzO83M2oC/Be4zs1VhxZOgpCAiAy319+T1Ha8n73d0dRzQttTPUwh1TsHdFwAL0vbNTrm/hHhZ6ZBJJIX0tcxFRAqVmhRGN4zm9Z3xxNC+v53BtYN7ta3ko49KUm1V/BCxo+86usiRiEhUpP7Apy64ubvzwIljrX1UYhIjBRGRgdIrKaSsmNC+v/2Atk58niE9CWikUCSpSWHXvl1FjEREoiJ9pDBs0DCg76SQUKrnKVR0UnjP7e8pYiQiEhXpI4Xhg4YD2ZOC5hRKRPpp5/1Zs0REpC+ZRgq792c+Ga1U1z6quKRQbdW9trNlchGRfKQmha6eroLKRxopFMm+7n29tnfu21mkSEQkKg4oHw1W+ahspP8l7ejYUaRIRCQq0stH+cwplOrJaxWXFNJrfDs7NFIQkYOTPlJQ+aiMjBk6pte2RgoicrDSRwr11fUMrhnc58lrfT0HNNFcNJ888ZPcOPXGd7d/8Ul+8uefFDEiESl3qT/w3d5NbXUth9UddsBIIXV0oJPXSoSZccHxFyS393Xv4+rHri5iRCJS7tJLQTVVNX0mhdQDXXTyWgmpq47WxXVEpLjSS0G1VX2PFFJXTdVIoYSUyocvItFwQFKorqWhruGAOYXU1Zk1p1BCtCieiAykgkYKOvqodJzZdCb/+jf/WuwwRCQi+hop9Ld8pDmFIjIzZn1oVvLaCiIiByPfkcLerszlI40USkB1VXXuRiIiOaT/wNdU1dBQ25CzfJS6IKfmFEpAlb3bfa2WKiKFKrR8lDo60EihBKSumJrtzEMRkWz6Kh8NGzSMPZ17el2eM/3oo9REoDmFEpBaPnpn3ztFjEREyllfI4XEonjbO7Yn96eXj1JLRt/+07exW4yzHjwr5Gizq+ikcOKoE5P3tQaSiBSqr5HCiMEjANi2d1tyf7by0ZvtbwKweNPiMEPNqaKTwvzL53PTmTcB8Or2V9nfvb/IEYlIOepzpBBcU2H73ndHCqlHH3X3dJdMyShVqEnBzKab2Toz22Bms/p4vN7MHg0ef97MxocZT7pRQ0bxhTO+AMDHHv4Y9f9Szy2xWw5lCCISAX2NFBLlo8RIwd35xjPfAOLzmelzCqmKOekcWlIws2pgLjADmARcbmaT0ppdC2x39+OA7wDfDCueTNKX0p7z1Bza97fz1T98lW//8duHOhwRKUN9HZKaKB8l5hT+/D9/5vWdrwPQUNdAt3fzxMYn+ny9ObE5fOOZb7Dsv5fxnT99h3uevyfE6HsLc72H04EN7r4RwMweAS4CVqe0uQiYE9z/JfA9MzM/hMeH9rXkxXNtz/H1Z74OwD+c9g/M/M1M/m7y33Hi6BM5+vCjeXX7q6x6axUXTLwAM8v42i+//TJrt67lwhMuBOL/U1i8aTEtR7UwuHbwAe3vXXIvzUc2c+a4Mw+6X6/veJ0xQ8do8T85pF544wW6e7qZMHwCIwaPKPvv397OvQyqGZTx3/nCDQt5adtLdA3LXD5a9OoiHlv7GMvfXJ58vKG2gfuX38/9y+/v83Vve/o2AL7+zNfZ07kHgOtOv67XYfRhCTMpjAU2p2y3AWdkauPuXWa2ExgJbA0xrgNcd9p1vLPvHU5pPIWbnriJTzz6ieRjo781mj2de/j5X34OwNGHH82mnZsAOGroUTTUNlBlVcm/LOfdfLZ261oA3jv8vezv3s+u/bvY0bGD4YOGM7phNMa7X7Qe72H9tvVA7wnw/tizZw9DVg2hq6eL9dvWM3rIaEY3jC7otUrN7t27aVjdUOwwQnOo+7evax+b39nMscOPTX4PU7+70Pe5O7naJL7DACMGj+CIhiOS38vU93lr91sc0XBE1v9UFVt3Tzfrt63nqKFHMbRuaJ9t1mxdA8DIwSN77U8ckgrwwIoHDnheYlI5l0RCAHjf997HbefexqUnX5rXcwtVFivDmdlMYCZAY2MjsVisoNdpb2/v87mXDLkEhgD74bPHfpZNezZRV1VHY30jW/ZtYXdX/ByGHnowjBOOOIERdSPYtn8b3cSPIHB3Er/xiS//UUccxf7u/dRV11FXV0dNQw07O3dSW1Xb5z+4ptFNVFs1nd55wGP56KrvosZqoBpGjxwdX8YjIufkDa8bTo2Xxde1IIe8fzVw3Kjj6OjpOOAhe/eL3Hs7/fF3dyQ1jW6i27sZXD2YKqtib/deuuq7ep0ThMMx7zmGju6O0v5+GowdNZYeeg5IhglHjj6Szq5OqqurGTY8ngR6vIe3177N4pcX8/n3fp7Onk6OHHQkK3eupNu7Oe6w4xhcPZil25dSV1VHp3cyYcgE2rva6ejpYH/Pftq72g94z27vZtPLm4htjYXbb3cP5QacCSxM2b4ZuDmtzULgzOB+DfERgmV73SlTpnihFi1aVPBzy0GU+xflvrlHu39R7pt7+fQPWOp5/HaHWaBaAkw0swlmVgdcBsxPazMfSFz27BLgD0HwIiJSBKGNVz0+R3A98dFANfCAu68ys1uJZ6z5wL8DPzWzDcA24olDRESKJNQiprsvABak7Zudcr8D+NswYxARkfxV9BnNIiLSm5KCiIgkKSmIiEiSkoKIiCQpKYiISJKV22kBZvYW8HqBTx/FIV5C4xCLcv+i3DeIdv+i3Dcon/4d4+45170pu6RwMMxsqbu3FDuOsES5f1HuG0S7f1HuG0SvfyofiYhIkpKCiIgkVVpSmFfsAEIW5f5FuW8Q7f5FuW8Qsf5V1JyCiIhkV2kjBRERyaJikoKZTTezdWa2wcxmFTueQpjZA2a2xcxWpuwbYWZPmNn64M/hwX4zs7uD/r5kZh8oXuS5mdk4M1tkZqvNbJWZ3RDsL/v+mdkgM3vBzP4c9O2WYP8EM3s+6MOjwRLzmFl9sL0heHx8MePPl5lVm9mLZvbbYDsS/TOz18zsL2a2wsyWBvvK/nuZSUUkBTOrBuYCM4BJwOVmNqm4URXkR8D0tH2zgCfdfSLwZLAN8b5ODG4zgXsPUYyF6gL+0d0nAVOB64K/oyj0bx/wYXc/BWgGppvZVOCbwHfc/ThgO3Bt0P5aYHuw/ztBu3JwA7AmZTtK/TvX3ZtTDj2Nwveyb/lciafcb+RxFbhyuQHjgZUp2+uAMcH9McC64P59wOV9tSuHG/B/gY9GrX/EL/y6nPj1yrcCNcH+5HeUAq5IWOwb0ET8x/HDwG+JX6QzEv0DXgNGpe2L1Pcy9VYRIwVgLLA5Zbst2BcFje6euAr4X4HG4H7Z9jkoJ5wKPE9E+heUVlYAW4AngFeAHe7eFTRJjT/Zt+DxnUDvK8OXnruAfwJ6gu2RRKd/DjxuZsuC68VDRL6XfYnuldArkLu7mZX14WRmdhjwK+CL7v6O2btXhS/n/rl7N9BsZsOAXwPvK3JIA8bMPgZscfdlZtZa7HhC8CF3f8PMjgCeMLO1qQ+W8/eyL5UyUngDGJey3RTsi4L/MbMxAMGfW4L9ZddnM6slnhB+7u7/GeyOTP8A3H0HsIh4OWWYmSX+Y5Yaf7JvweOHA28f4lD7YxpwoZm9BjxCvIT0XSLSP3d/I/hzC/GEfjrDmgiKAAACr0lEQVQR+16mqpSksASYGBwNUUf8WtDzixzTQJkPXB3cv5p4LT6x/9PB0RBTgZ0pw92SY/Ehwb8Da9z9zpSHyr5/ZjY6GCFgZoOJz5WsIZ4cLgmapfct0edLgD94UKAuRe5+s7s3uft44v+2/uDuVxKB/plZg5kNTdwHzgNWEoHvZUbFntQ4VDfgfOBl4rXcrxQ7ngL78DDwJtBJvFZ5LfFa7JPAeuD/ASOCtkb8iKtXgL8ALcWOP0ffPkS8dvsSsCK4nR+F/gGTgReDvq0EZgf7jwVeADYA/wHUB/sHBdsbgsePLXYf+tHXVuC3Uelf0Ic/B7dVid+OKHwvM910RrOIiCRVSvlIRETyoKQgIiJJSgoiIpKkpCAiIklKCiIikqSkINIHMxtvKavR5tH+GjM7Ko823zv46ETCo6QgMjCuAbImBZFyoKQgklmNmf04WBf/l2Y2xMxmm9kSM1tpZvOCM1cvAVqAnwdr7g82s9PM7I/BNRReSJwVCxxlZr8P1uH/tyL2TaRPSgoimZ0AzHP3ycA7wOeB77n7ae5+MjAY+Ji7/xJYClzp7s1AN/AocIPHr6HwEWBv8JrNwKXA+4FLzWwcIiVESUEks83u/mxw/2fEl+I4N7ha2F+IL/x2Uh/POwF4092XALj7O/7uEtJPuvtOd+8AVgPHhNsFkf7R0tkimaWvAePA94mvZ7PZzOYQX8enP/al3O9G/walxGikIJLZ0WZ2ZnD/CmBxcH9rcN2HS1La7gIS8wbrgDFmdhqAmQ1NWUJapKTpiyqS2VrgajO7j/hqmPcCw4mvfvka8SXZE34E/MDM9hK/VsKlwD3BUtl7ic8riJQ8rZIqIiJJKh+JiEiSkoKIiCQpKYiISJKSgoiIJCkpiIhIkpKCiIgkKSmIiEiSkoKIiCT9f4TdBHRdoKROAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12be6ce7a90>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "vgg19(num_class=5,epochs=20)\t#binary classificaiton\n",
    "history_this.loss_plot('epoch')\n",
    "history_this.loss_plot('batch')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Validation accuracy of VGG19: 100.0%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model 5: ResNet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def resnet( num_class, epochs,savepath='./resnet.h5',history=history_this,input_shape=INPUT_SIZE):\n",
    "    model_fine_tune = ResNet50(include_top=False, weights='imagenet', input_shape=input_shape)\n",
    "    for layer in model_fine_tune.layers[:120]:\t#the number of frozen layers for transfer learning, have tuned from 50-150\n",
    "        layer.trainable = False\n",
    "    for layer in model_fine_tune.layers[120:]:\t#the number of trainable layers for transfer learning\n",
    "        layer.trainable = True\n",
    "    model = GlobalAveragePooling2D()(model_fine_tune.output)\n",
    "    model=Dense(units=256,activation='relu')(model)\n",
    "    model=Dropout(0.5)(model)\n",
    "    model = Dense(num_class, activation='softmax')(model)\n",
    "    model = Model(model_fine_tune.input, model, name='resnet')\n",
    "    opt = keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08)\t#tuned learning rate to be 0.001\n",
    "    model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy']) #set the loss function to be binary crossentropy\n",
    "    #train model\n",
    "    earlyStopping = kcallbacks.EarlyStopping(\n",
    "        monitor='val_acc', patience=2, verbose=1, mode='auto')\t#set early stop patience to save training time\n",
    "    saveBestModel = kcallbacks.ModelCheckpoint(\n",
    "        filepath=savepath,\n",
    "        monitor='val_acc',\n",
    "        verbose=1,\n",
    "        save_best_only=True,\n",
    "        mode='auto')\n",
    "    hist = model.fit_generator(\n",
    "        train_generator,\n",
    "        steps_per_epoch=len(train_generator),\n",
    "        epochs=epochs,\n",
    "        validation_data=validation_generator,\n",
    "        validation_steps=len(validation_generator),\n",
    "        #use_multiprocessing=True, \n",
    "        callbacks=[earlyStopping, saveBestModel, history],\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0422 - acc: 0.9957Epoch 00001: val_acc improved from -inf to 0.94573, saving model to ./resnet.h5\n",
      "183/183 [==============================] - 109s 594ms/step - loss: 0.0431 - acc: 0.9956 - val_loss: 0.8699 - val_acc: 0.9457\n",
      "Epoch 2/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.2350 - acc: 0.9852Epoch 00002: val_acc did not improve\n",
      "183/183 [==============================] - 86s 471ms/step - loss: 0.2351 - acc: 0.9852 - val_loss: 0.8699 - val_acc: 0.9457\n",
      "Epoch 3/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.2356 - acc: 0.9853Epoch 00003: val_acc improved from 0.94573 to 0.94573, saving model to ./resnet.h5\n",
      "183/183 [==============================] - 86s 472ms/step - loss: 0.2357 - acc: 0.9853 - val_loss: 0.8699 - val_acc: 0.9457\n",
      "Epoch 4/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.2584 - acc: 0.9836Epoch 00004: val_acc did not improve\n",
      "183/183 [==============================] - 86s 470ms/step - loss: 0.2603 - acc: 0.9835 - val_loss: 0.9563 - val_acc: 0.9241\n",
      "Epoch 5/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.4339 - acc: 0.9728Epoch 00005: val_acc improved from 0.94573 to 0.97293, saving model to ./resnet.h5\n",
      "183/183 [==============================] - 88s 481ms/step - loss: 0.4327 - acc: 0.9729 - val_loss: 0.3270 - val_acc: 0.9729\n",
      "Epoch 6/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.2345 - acc: 0.9854Epoch 00006: val_acc improved from 0.97293 to 0.98652, saving model to ./resnet.h5\n",
      "183/183 [==============================] - 86s 472ms/step - loss: 0.2346 - acc: 0.9854 - val_loss: 0.2187 - val_acc: 0.9865\n",
      "Epoch 7/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.2351 - acc: 0.9853Epoch 00007: val_acc did not improve\n",
      "183/183 [==============================] - 87s 477ms/step - loss: 0.2346 - acc: 0.9854 - val_loss: 0.2161 - val_acc: 0.9865\n",
      "Epoch 8/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.2357 - acc: 0.9853Epoch 00008: val_acc did not improve\n",
      "183/183 [==============================] - 86s 471ms/step - loss: 0.2347 - acc: 0.9854 - val_loss: 0.2161 - val_acc: 0.9865\n",
      "Epoch 00008: early stopping\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVPX+x/HXFwRRURYXTE3FfWNxQe1qLqmldq+5BFqa5r22WFm23Wv9yiy7LbZ5NTPLFjPNcOtqWpYmkTctN8y1VNxwCxVQRFTg8/tjBkRlGWCGmYHP8/GYh3NmzpzznonmM+d7zvkcIyIopZRSAB7ODqCUUsp1aFFQSimVQ4uCUkqpHFoUlFJK5dCioJRSKocWBaWUUjm0KCillMqhRUEppVQOLQpKKaVyVHB2gKKqUaOGNGzYsFivPX/+PFWqVLFvIAdyp7zulBXcK687ZQX3yutOWaFkeTdv3nxKRGoWOqOIuNWtffv2Ulxr164t9mudwZ3yulNWEffK605ZRdwrrztlFSlZXmCT2PAdq8NHSimlcmhRUEoplUOLglJKqRxut6NZKVV2Xb58mYSEBNLT00tlfX5+fuzevbtU1mUPtuT18fGhXr16eHl5FWsdDisKxpiPgb8Cf4pImzyeN8B/gP5AGnCviGxxVB6llOtLSEigatWqNGzYEMtXhGOdO3eOqlWrOnw99lJYXhHh9OnTJCQkEBwcXKx1OHL46FOgbwHP9wOaWm/3AzMdmEUp5QbS09OpXr16qRSEssgYQ/Xq1Uu0peWwoiAiscCZAma5A/jMerTUBsDfGHODo/IopdyDFoSSKennZ8SBl+M0xjQEvs5n+Ohr4DURWWedXgP8S0Q25THv/Vi2JggKCmq/YMGCImeptn07vuvX4+HvT6aPD5mVK1+5VapERqVKV6Z9fMDD+fvgU1NT8fX1dXYMm7hTVnCvvO6UFUqW18/PjyZNmtg5Uf4yMzPx9PQstfWVlK159+3bR0pKylWP9ezZc7OIdCjstW6xo1lEPgA+AOjQoYP06NGj6AvZuBG++ML2+atUgapVLTdf3yv385oubB5f32IVmZiYGIr1Xp3AnbKCe+V1p6xQsry7d+8u1TH+a8fok5OTmT9/Pg899FCRl9W/f3/mz5+Pv7+/TfNPmjQJX19fnnrqqWLnzY+Pjw9t27a1ebm5ObMoHAVuzDVdz/qYYzz9NDHt2tEjIgLOnbtyS021ffr4cfjjjyvT58/bvv7KlW0vItbpWn/8ASdOXL+s/LbuHP14Ac8F7d4NCQmW4ufhAZ6eV+4X9TFHz6vDE8WSng7JyZCUBMmnMkg6mkbS8XSST14k6c/LJJ/JJDkJzpzxZY7f3mKt4++TL3PAlM6RRwCZmR6c8ryyvoSjJ/nPOzPod8vfr5s3IyODChXy/8qc8fYSkk5A0gnb8iedyuBS2mUO7LH9/VbxzcLRNdOZRWEZ8IgxZgHQCUgRkeMOXaOnJ1SrZrnZQ1aWpTAUp8CcOwcnT8K+fVc/n0sr+6QsFS2dHaCIbq5YEfz8rt/is/VWrdqV+5Uru0Whycqy/Jkln8kiKeE8yccsX+pJJy6SfOoySaeySE4WkpI9SDrnSfJ5b5IuVCT5YiWSLvmSLhVzLa0CUM16s6jMefxJpgLexc5498V0zqWW5md59Vfgv19/nkNH4uk3oBOdO/Xi5q59ef/Df1OzRm1+/+M3lkbHMf6pSE6cTODixXSGD3uYOwePAaDfgGbM/+xn0tJSefixO2gb9hfifttArVp1+M+bi/DxqXTVui5dggqehnOphj2/b+Pl18aRnp5GvXqNeOn5WVSrFsC8BTNYtORDPD0r0Ci4JZ++O5Mff/yRxx57DLDsP4iNjbXr1pUjD0n9AugB1DDGJAAvAF4AIvI+sBLL4aj7sBySOtpRWRzGw+PKF4M9ZGVBWlpOkfh1wwY6dux4/XwFfQHl95y9Hs/nuQ3r19O5UyfLe8jMtPx77c3Rj9v6msxMju7ZQ/2AgCsF+ezZQot0vjw88h5SLG6h8c7/S/XyJSHpaBrJCalX/1JPzLB82ScJSSkeJJ/zJOm8N8kXKpJ0sTLJl6uQnFmVLDyxHF9S1XrL9Z+VLPxJJoAk/E0KARUSqVPxPP5VLxJQ5SL+vpkEVMvEP8AQEGgIqOGJf1BFAmpXxL9OZbxr+kG1aqzfuJGbbrrJts/uGrvPQstmlvvjVz9F3MltxVpOfsKDwpja+82c6dTz5/HN1WDu/emT+evgnezY8isAMT/+yK7dm9ixeXPOIZ6LvphFYGAgFy5cIKJLFx57ZBDVq1fHywtat7T82Rw+so8lX35GeNhMooYP54+9Sxlx991XZQmqZfmzCQ2BEaP/wfS336Z7t25MfPFFFi39N1PffJO+A97kwJ49VKxYkeTkZDx9KvDmk28yY8YMunTpQmpqKj4+Pnb9jBxWFETkrkKeF+BhR63fLWV/ufj6wg03kHbsGLRo4exUNkk/fBhKcQdhScXHxFC/sHHva7cE87qdPZv/c6dOXT198aJN2TK9fPitcmf+59mNdZmd2X6hESmZx0nKrEYaVSDndj0fLli+2D3O4u+VSlDFJFpUS8e/yiUCfDPw9xMCAoSAQA/8a1QgIMgb/9o+BNStTNU6VfEI8AO/uuDTuEifZ24XjxyBBg2K9+Ldu6GidYskewjQnjw9rywfkEuXrpqmYkXLj57sx7y96dixI8G5/j+cNmsWS5cuBeBIQgJ7Dx+mep06V153+TLBwcGEW3/QtY+I4ODRo1evB6BCBahQgZT0dJJTUujepw8Ao/7xDyIjI6FiRUJDQxn+978zcOBABg4ciIjQpUsXnnjiCYYPH87gwYOpV6+eXT8it9jRrJRT2HtL8PLlPAvK+cQ0fvmtEut2+LPuj1psOFKHcymWX383VkqkdZXd1Ak4jn/VDAKsX+r+gZ4E1KyAfy1vAm7wwb9uFQLqVcEnyA+q1IYycHT31L5TnR0B4KpW1TExMaxevZr169dTuXJlevTokec5ARVzFQBPT08uXLhQrHWvWLGC2NhYli1bxuTJk9mwYQMTJkzg9ttvZ+XKlXTu3JnVq1fTwo4/HrUoKFVavLwgMJATlwL53yZYt85y27rVMrplDISGwj1joGtX6NIF6tevSUzMTrc6+sidVa1alXPnzuX7fEpKCgEBAVSuXJk9e/awYcOGEq/Tz8+PgIAAfvrpJ26++Wbmzp1L9+7dycrK4siRI/Ts2ZOuXbsyf/58UlNTOXPmDCEhIYSEhLB+/Xr27NmjRUGp4srIyODnn38u9i+3ohKBPXssX/7/+5/l3/37Lc9VqgSdOsEzz1gKwE03WfZ9K+epXr06Xbp0oU2bNvTr14/bb7/9quf79u3L+++/T2hoKM2bN6dz5852We+cOXN48MEHSUtLo1GjRnzyySdkZmYyYsQIUlJSEBEef/xx/P39efbZZ1m7di0eHh60bt2afv362SVDDlsuuuBKN73Ijmty9azHjx+XyZMnS7169QSQqKgoh6wnPV3kf/8Tef11kb/9TSQwUMRSGkRq1hQZNEjkrbdEfvlF5OJF25bp6p/ttUqSd9euXfYLYoOzZ8+W6vpKyta8eX2O2HiRnXKzpXD6NBw+XJk9e5ydxHbulPfYMR8uXLD8+nUVIsKPP/7IzJkzWbJkCRkZGfTp04egoCBiYmLIysrCo4Q7Ms+cgZ9/vrIlsHHjlf3JzZvDwIGWoaCuXS374d3gyFVVzpWbovDxx/DPf+ZxeKdLc6e8ls3ogACoUwfq1rX8m/t+9r9BQZYDLxwlJSWFzz77jJkzZ7J7924CAgJ49NFHefDBB2natClz585l5MiR/Prrr0Xa/BeBAweuDAOtWwe7dlme8/KC9u1h3DjLUFCXLlCz8KvhKuVyyk1R+Otf4dy5XbRq5T6nhO3a5T554+L2UK1aC44dg6NH4dgxyxfm8eOWnai5eXhYCkNeBSN3MQkMLNov661btzJz5kzmzZtHWloaHTt25NNPPyUqKopKuTZhBgwYgJeXF9HR0QUWhYwM2Lbt6v0Bx62nV/r5wV/+AsOHW7YCIiJcaytJqeIqN0WhZUu45ZY/6dHDPb5kAWJi3Cdv7don6NHj+iMgMjMhMfFKocj+N/v+wYOWL9zTp69fZsWK+ReM7PvVq6ezYsVC3nvvPTZs2EClSpW4++67GTt2LO3bt88zq5+fHxERESxatIg333wzZwjp3Dn45ZcrWwEbNlzpZNKgAdxyi2ULoGtXaN3aJXomKmV35aYoKOfw9ITatS23fL6jAUtfnePHry4Yuf/duhW+/tpywrfFfmAW8DFwGm/v5jRrNpW2bUdSq1YA69fD4cNXCkhQkGWIJ1v37t159dVXeeWVX0lM7My6dRAXZzlfzcPDcmjo6NFXDg218/lBSrksLQrKJfj4QHCw5ZafjIxMFi1awYwZ77Fu3So8PDxp1WogTZo8hDE9OX7c8PPPsHixZegnN2OgVi1LgahVC7ZufRR4i+efX0jlyp3p3Bn+7/8sRaBzZ/u1x1LK3WhRUC7vxIkTfPTRR8yaNYsjR45Qp04dJk2axJgxY6hbt+5182dlWYas8tvqOHECmjf3pEaN2zh1aiGHDr1BxYo6FqRKt3W2q9KioFySiBAbG8vMmTNZvHgxGRkZ9O7dm6lTp/K3v/2twIuSZ+/IDgqC/FrKx8Ts5MiRSEaOXM7WrUU7CkmVXcnJybz33nt5FoXCWmevXLnSkdFKjf48Ui4lJSWFd999lzZt2tCjRw9WrVrFuHHj+P333/n+++8ZPHhwgQWhKAYMGIC3tzcLFy60y/KU+5swYQL79+8nPDycp59+mpiYGHr27Mndd99NaGgoAAMHDqR9+/a0bt2aDz74IOe1DRs25NSpUxw8eJCWLVty33330bp1a2699dY8z6Bfvnw5nTp1om3btvTu3ZuTJ08ClivXjR49mpCQEEJDQ1m8eDEA3377LTfffDNhYWH06tXLYZ+BbikolxAXF5dzOOn58+fp0KEDH3/8MUOHDqVy5coOWaefnx+33nprzlFIem1gFzN+vGXvvz2Fh8PU/Bvtvfbaa+zYsYM463pjYmL49ddf2bFjR07r7I8//vhK6+yICIYMGUL16tWvWs7evXv54osv+PDDD4mKimLx4sWMGDHiqnm6du3Khg0bMMYwe/ZspkyZwltvvcXkyZPx8/Nj+/btACQlJZGYmMh9993HypUrCQkJ4cyZM/b8VK6iRUE5TXp6OosWLeK9995j/fr1+Pj4cNdddzF27FgiIiJKJUNkZCRff/01v/76K506dSqVdSr30rFjx5yCADBt2rQrrbOPHGHv3r3XFYXg4GDCw8MBaN++PQcPHrxuuQkJCQwdOpTjx49z6dKlnHWsXr2a3NehDwgIYPny5XTr1o2GDRsCEBgYaM+3eBUtCqrUxcfHM2vWLD766CNOnz5N06ZNefvttxk1apRD/9jzkj2EFB0drUXB1RTwi740Oap19rhx43jiiScYMGAAMTExTJo0ySH5i0r3KahSkZmZyfLly+nXrx9NmjThrbfeonv37nz//ffs2bOHxx9/vNQLAoC/v3/OEJIUdG1qVS6UZuvslJSUnKPn5syZk/N4nz59mDFjRs50UlISnTt3JjY2NmeLw5HDR1oUlEOdPHmSV155hUaNGjFgwAC2bdvGxIkTOXToEIsXL6Z3794lbkpXUpGRkRw+fJhff/3VqTmU8+Vunf30009f93zfvn3JyMggNDSU559/vkRHrU2aNInIyEhuvvlmatSokfP4c889R1JSEm3atCEsLIy1a9dSs2ZNPvjgA0aMGEFYWBhDhw4t9noLZUsrVVe6aets15Q7a1ZWlsTGxsqwYcPEy8tLALnllltk0aJFcunSJeeFzCV33qSkJPHy8pInn3zSeYEK4E5/ByLaOtuRSqN1tm4pKLs5e/YsM2bMICQkhG7duvHNN9/w0EMPsXv3btasWcOQIUPsdjipPfn7+3PbbbexcOFCHUJS5V652dG8ZcsWlixZknOYlzvYu3ev2+T9/vvv+eGHHzh//jzt2rVj9uzZDBs27KqddK5Mj0JSyqLcFIU1a9Ywffp0Z8cos7y9vXO6k0ZERLjdMf/Z7bQXLlyoRUGVa+WmKDzyyCM0a9aMrl27OjuKzdatW+c2eTdt2sRtt93m7BjFln0U0sKFC3njjTfcrqgpZS/lpihUqlQJPz+/604ycWXulDf3cdnuKioqihUrVugQkirXdEezUla5h5CUKq+0KChlpSeyqewuqcXRv39/kpOTbZ5/0qRJvPnmm8ValyNpUVAql8jISA4dOsTGjRudHUU5QUFFIePaKzddY+XKlW5/LQXQoqDUVe644w68vLyIjo52dhTlBKXZOju3uLg4OnfuTGhoKIMGDSIpKQmwNN9r1aoVoaGhDBs2DIAff/yR8PBwwsPDadu2bYFtOYqj3OxoVsoWuYeQ9Cgk5xr/7XjiTti3dXZ47XCm9nWN1tm5jRw5kunTp9O9e3cmTpzIiy++yNSpU3nttdc4cOAAFStWzBmaevPNN5kxYwZdunQhNTUVHx+fkn4sV9EtBaWuoUNIKre8WmeHhYXRuXPnnNbZ17KldXa2lJQUkpOT6d69OwCjRo0iNjYWgNDQUIYPH87nn3+ec9W3Ll268MQTTzBt2jSSk5MLvBpcceiWglLXyB5CWrhwIR07dnR2nHKroF/0pclRrbNtsWLFCmJjY1m2bBmTJ09mw4YNTJgwgdtvv52VK1fSuXNnVq9eTYsWLYq1/LzoloJS1/D396dPnz7aC6kcKs3W2dn8/PwICAjgp59+AmDu3Ll0796drKwsjhw5Qs+ePZkyZQrJycmkpqayf/9+QkJC+Ne//kWHDh3Ys2dPiTPk5tCiYIzpa4z53RizzxgzIY/n6xtj1hpjthpjfjPG9HdkHqVsFRUVpUNI5VBpts7Obc6cOTz99NOEhoYSFxfHxIkTyczMZMSIEYSEhNC2bVsef/xx/P39mTp1Km3atCE0NJRKlSrRr18/u2TIYUsr1eLcAE9gP9AI8Aa2Aa2umecDYKz1fivgYGHL1dbZrsmdsooUnje7nfZTTz1VOoEKUNY+24Jo6+yCuXvr7I7APhGJF5FLwALgjmtrElDNet8POObAPErZTIeQVHnlyKJQFziSazrB+lhuk4ARxpgEYCUwzoF5lCqS7KOQNm3a5OwoSpUaZx99dBfwqYi8ZYy5CZhrjGkjIlm5ZzLG3A/cDxAUFERMTEyxVpaamlrs1zqDO+V1p6xgW97q1atToUIF3nrrLR588MHSCZaHsvjZ5sfPz8/uJ2MVJDMzs1TXV1K25k1PTy/+34wtY0zFuQE3AatyTT8DPHPNPDuBG3NNxwO1Clqu7lNwTe6UVcT2vP3795cGDRpIVlaWYwMVoKx+tnnRfQoFc/d9ChuBpsaYYGOMNzAMWHbNPIeBXgDGmJaAD5DowExKFYkOIanyxmFFQUQygEeAVcBuIFpEdhpjXjLGDLDO9iRwnzFmG/AFcK+1oinlEnKfyKZUeeDQ8xREZKWINBORxiLyb+tjE0VkmfX+LhHpIiJhIhIuIt85Mo9SRRUQEECfPn2Ijo7Wo5BUnnx9fZ0dwa70jGalCqFDSKo80aKgVCF0CKn8mDBhAjNmzMiZzr4QTmpqKr169aJdu3aEhITw3//+t9Bl5ddi+9tvv6Vdu3aEhYXRq1cvwHLE1ujRowkJCSE0NJTFixfb/83ZyNmHpCrl8gICAujduzcLFy7k9ddf13bapWT8eIizb+dswsNhagF99oYOHcr48eN5+OGHAYiOjmbVqlX4+PiwdOlSqlWrxqlTp+jcuTMDBgwo8G8hrxbbWVlZ3HfffcTGxhIcHMyZM2cAmDx5Mn5+fmzfvh0g53oKzqBFQSkbREZG8ve//51NmzYRERHh7DjKQdq2bcuff/7JsWPHSExMJCAggBtvvJHLly/z7LPPEhsbi4eHB0ePHuXkyZPUrl0732VNmzaNpUuXAuS02E5MTKRbt245rbgDAwMBWL16NQsWLMh5bUBAgAPfZcG0KChlg4EDB/LAAw+wcOFCLQqlpKBf9I4UGRnJokWLOHHiBEOHDgVg3rx5JCYmsnnzZry8vGjYsGGeLbOz2dpi2xXpPgWlbJB7CEmPQirbhg4dyoIFC1i0aBGRkZGApWV2rVq18PLyYu3atRw6dKjAZeTXYrtz587ExsZy4MABgJzhoz59+ly1L8OZw0daFJSyUWRkJAcPHmTz5s3OjqIcqHXr1pw7d466detyww03ADB8+HA2bdpEhw4dmDdvXqEXtcmvxXbNmjX54IMPGDx4MGFhYTlbIs899xxJSUm0adOGsLAw1q5d69g3WQAdPlLKRnfccQcVKlQgOjqaDh06ODuOcqDsHb7ZatSowfr16/OcNzU19brHKlasyDfffJPn/P369bvuGgi+vr7MmTOnmGntS7cUlLJRYGCgttNWZZ4WBaWKQIeQVFmnRUGpIsgeQtIT2VRZpUVBqSIIDAykd+/e2gtJlVlaFJQqoqioKB1CUmWWFgWlikiHkFRZpkVBqSLKHkLSo5AU5N86211bamtRUKoYIiMjOXDgAFu2bHF2FKXsSouCUsUwcODAnBPZVNlhz9bZ2USEp59+mjZt2hASEsKXX34JwPHjx+nWrRvh4eG0adOGn376iczMTO69996ced955x27v8fC6BnNShVD7iGk1157TdtpO8D48eOJs3Pv7PDwcKYW0GnPnq2zsy1ZsoS4uDi2bdvGqVOniIiIoFu3bsyfP5/bbruN//u//yMzM5O0tDTi4uI4evQoO3bsACA5Odk+b7wIdEtBqWLSIaSyJ3fr7G3btuW0zhYRnn32WUJDQ+ndu3dO62xbrFu3jrvuugtPT0+CgoLo3r07GzduJCIigk8++YRJkyaxfft2qlatSqNGjYiPj2fcuHF8++23VKtWzcHv+Hq6paBUMeVup92+fXtnxylzCvpF70j2aJ1ti27duhEbG8uKFSu45557ePrppxk5ciTbtm1j1apVzJgxg+joaD7++GN7vC2b6ZaCUsUUGBhIr1699ES2MsYerbNzu/nmm/nyyy/JzMwkMTGR2NhYOnbsyKFDhwgKCuK+++7jH//4B1u2bOHUqVNkZWUxZMgQJk+e7JStUN1SUKoEoqKicv6H1q2FsiG/1tl/+9vf6NChA+Hh4YW2zs5t0KBBrF+/nrCwMIwxTJkyhdq1azNnzhzeeOMNvLy88PX15bPPPuPo0aOMHj2arKwsAF599VWHvMeCaFFQqgR0CKlsKmnr7NyPG2N44403eOONN656ftSoUYwaNeq61zl7H5UOHylVAtlDSHoimyortCgoVUKRkZHEx8c7/ReeUvagRUGpEso+kU17IdmHbnGVTEk/Py0KSpVQ9erVdQjJTnx8fDh9+rR+jsUkIpw+fRofH59iL0N3NCtlB5GRkYwZM4atW7fSrl07Z8dxW/Xq1SMhIYHExMRSWV96enqJvkBLmy15fXx8qFevXrHXoUVBKTvIPgopOjpai0IJeHl5ERwcXGrri4mJoW3btqW2vpIqjbw6fKSUHegQkiortCgoZSdRUVHEx8ezdetWZ0dRqti0KChlJwMHDsTT01OPQlJuTYuCUnaSPYSkvZCUO3NoUTDG9DXG/G6M2WeMmZDPPFHGmF3GmJ3GmPmOzKOUo2WfyKZDSMpdOawoGGM8gRlAP6AVcJcxptU18zQFngG6iEhrYLyj8ihVGgYNGqRDSMqt2VQUjDFdjDFVrPdHGGPeNsY0KORlHYF9IhIvIpeABcAd18xzHzBDRJIAROTPosVXyrXoUUjK3Rlb/nCNMb8BYUAoMBf4CBgsIt0LeM2dQF8RGWOdvgfoJCKP5JrnK+APoAvgCUwSkW/zWNb9wP0AQUFB7RcsWGDzG8wtNTUVX1/fYr3WGdwprztlBcfmXbFiBW+++SazZs2iWbNmJV6efraO405ZoWR5e/bsuVlEOhQ6o4gUegO2WP+dCPwj92MFvOZOYHau6XuAd6+Z52tgKeAFBANHAP+Cltu+fXsprrVr1xb7tc7gTnndKauIY/MmJiaKp6enTJgwwS7L08/Wcdwpq0jJ8gKbxIbve1v3KZwzxjwDjABWGGM8rF/kBTkK3Jhrup71sdwSgGUicllEDmDZamhqYyalXFKNGjV0CEm5LVuLwlDgIpathBNYvuDfKPglbASaGmOCjTHewDBg2TXzfAX0ADDG1ACaAfE2ZlLKZUVGRrJ//37i4uKcHUWpIrF5SwH4j4j8ZIxpBoQDXxT0AhHJAB4BVgG7gWgR2WmMeckYM8A62yrgtDFmF7AWeFpEThfnjSjlSvRENuWubC0KsUBFY0xdYA0wGvi0sBeJyEoRaSYijUXk39bHJorIMut9EZEnRKSViISISPH2ICvlYmrUqMEtt9yiJ7Ipt2NrUTAikgYMBqaLyCCgjeNiKeX+oqKidAhJuR2bi4Ix5iZgOLCiiK9VqlzSISTljmz9Yh+P5czjpdb9Ao2w7ANQSuUjewhJj0JS7sSmoiAiP4rIAGCGMcZXLGcpP+rgbEq5vcjISPbt26dDSMpt2NrmIsQYsxXYCewyxmw2xrR2bDSl3J/2QlLuxtbho1nAEyLSQETqA08CHzoullJlgw4hKXdja1GoIiI5+xBEJAao4pBESpUx2UNI27Ztc3YUpQpla1GIN8Y8b4xpaL09h555rJRNsoeQoqOjnR1FqULZWhT+DtQEllhvNa2PKaUKUaNGDXr27KlDSMot2Hr0UZKIPCoi7ay3x8R6DQSlVOGioqJ0CEm5hQoFPWmMWQ7k+9PGepiqUqoQgwYNYuzYsSxcuJDw8HBnx1EqXwUWBeDNUkmhVBmXPYQUHR3Nyy+/jDHG2ZGUylOBw0fWk9auugHnct1XStlIj0JS7qA4/Ytm2z2FUuWAnsim3EFxioJu9ypVDDVr1tSjkJTLK05ReNHuKZQqJyIjI9m7d68OISmXZWvvo0HGGD8AEfnKGONvjBno2GhKlT06hKRcna1bCi+ISEr2hIgkAy84JpJSZVcuX7ypAAAbn0lEQVTNmjXp0aOHDiEpl2VrUchrvsIOZ1VK5SEqKoq9e/fy22+/OTuKUtextShsMsa8bYxpbL29DWx2ZDClyiodQlKuzNaiMA64BHwJLADSgYcdFUqpsix7CCk6OlqHkJTLsbX30XkRmSAiHUQkQkSeFZHzjg6nVFmVfRSSDiEpV2Pr0UffG2P8c00HGGNWOS6WUmXb4MGD8fDw0CEk5XJsHT6qYT3iCLB0TQVqOSaSUmWfnsimXJWtRSHLGFM/e8IY05ACuqcqpQoXGRnJH3/8oUNIyqXYWhT+D1hnjJlrjPkc+BF4xnGxlCr7Bg0apENIyuXYuqP5W6AD8DvwBfAkcMGBuZQq82rVqqVDSMrl2LqjeQywBksxeAqYC0xyXCylyofsIaTt27c7O4pSgO3DR48BEcAhEekJtAUSHZZKqXIiewgpOjo6z+cfXP4gozaO4s/UP0s5mSqvbC0K6SKSDmCMqSgie4DmjoulVPlQq1atPHshpWek025WO2ZtmcXhtMM0md6EQ8mHnJhUlRe2FoUE63kKXwHfG2P+CxxzXCylyo+oqKirhpAOJR+i3tv12HpiK2FBYQyuO5hzl87RckZLdifudnJaVdbZuqN5kIgki8gk4HngI0BbZytlB7mPQlq1bxXNpjfj9IXT3BNyD3EPxjGuyThe7PEiFzIuED4rnE3HNjk7sirDinyRHev1mZeJyKXC5jXG9DXG/G6M2WeMmVDAfEOMMWKM6VDUPEq5u+whpPfnvE/fz/tyOesy0/pO47PBn+XMM7H7RKbeNpVLmZe46aObiDkY47zAqkwrzpXXbGKM8QRmAP2AVsBdxphWecxXFcuO7F8clUUpV5aVlcWZ4DOcOnIKr1NexIyKYVyncdfN91jnx/j0jk/JzMqk12e9WPb7MiekVWWdw4oC0BHYJyLx1q2KBcAdecw3GXgdS+dVpcqV1EuptH6vNXEBcWDgocoP0a1ht3znHxU+ioWRlpPdBi4YyLzt80orqionHFkU6gJHck0nWB/LYYxpB9woIiscmEMpl7Q7cTf13q7HntN7+EuLv9CzZ0++XfZtoSeyDWk1hG+Hf4uH8WDEkhHM3DizlBKr8sA46kxKY8ydQF8RGWOdvgfoJCKPWKc9gB+Ae0XkoDEmBnhKRK7bi2aMuR+4HyAoKKj9ggULipUpNTUVX1/fYr3WGdwprztlBefnjU2M5aXdL5Epmdxxwx2MbzaeZcuW8c477zB79mwaN25caNZdKbt4bNtjZEgG9zW8j7sb3F2abyFfzv5si8KdskLJ8vbs2XOziBS+31ZEHHIDbgJW5Zp+Bngm17QfcAo4aL2lYznMtUNBy23fvr0U19q1a4v9Wmdwp7zulFXEuXmfWf2MMAnxeNFDPt36ac7jJ0+eFA8PD3nuueeumr+grNtPbBefl32ESciE7yc4KnKRuNPfgjtlFSlZXmCT2PDd7cjho41AU2NMsDHGGxgG5OwZE5EUEakhIg1FpCGwARggeWwpKFUWZGVlcdvc23h13atU8arCxvs2Mip8VM7ztWrVonv37kXqhdQmqA07xu6gilcVXvvfazy04iFHxVflhMOKgohkAI8Aq4DdQLSI7DTGvGSMGeCo9Srlis6knaHRtEZ8F/8dDfwacHj8Ydrd0O66+aKiovj999/ZsWOHzctuHNiYPx75A38ff2ZumsnwxcPtGV2VM47cUkBEVopIMxFpLCL/tj42UUSuO5ZORHroVoIqi7Yc30L9qfU5lHKIWxvdSvyj8QRWDsxz3uJeka1OtTrsH7efWpVrMX/HfG6fd7s9oqtyyKFFQanybk7cHCI+jOD85fM80/UZVt2zCg+P/P+3yx5Cio6OLnI77cDKgex/bD83VruRlftW0u2TbmRlZZX0LahyRouCUg4ybuU47v3vvXgYDxZFLuKVXq/Y9LrIyMgiDyFl8/X2Zd+4fTQNbMpPh3+i/YfttTCoItGioJSdZWRl0OWjLry78V2qVazGbw/+xpBWQ2x+fXGHkLJ5V/Bm18O7CK8dTtyJOFq+15JLGYV2pVEK0KKglF0dO3uMG9+5kZ8TfqZF9RYcfeIoLWu2LNIygoKCinwU0rUqeFRg832bubn+zfxx+g+aTG9C6qXUYi1LlS9aFJSyk9iDsTSe1pgTqScY0nIIOx/aia938U40ioyMZM+ePcUaQsrm4eFB7OhY+jfpz5GzR2j8n8acSTtT7OWp8kGLglJ2MP2X6fSY04OLmRd5tderLIpaVOAO5cKUdAgptxXDV3B3m7v5M+1PGk9vzLGzeikUlT8tCkqV0MglI3n020fx8vDim+HfMKFrvl3ibRYUFES3bt1KNISU27wh8xjbYSzJ6ck0e7cZ+8/sL/EyVdmkRUGpYkrPSCf8/XDmbp9L9UrV+WPcH9zW5Da7LT8qKoo9e/Zw8OBBuyzvvdvfY0KXCZy/fJ42M9uw42Txh6ZU2aVFQaliOJB0gLpv12XbyW20q92OhCcSaODfwK7ryB5CWrt2rd2W+WrvV3m116uWa0B/0I71R9bbbdmqbNCioFQRfbP3G5q/25wzF85wb/i9bH5gMz4VfOy+nuwhpO+++464uDi7LXdC1wm81/89Lmdd5uZPbmZN/Bq7LVu5Py0KShXBv2P/Tf/5/cnIymBG/xl8cscnDl3fs88+y/nz52nbti1RUVHs3r3bLssdGzGWzwd/TpZkcevnt7J091K7LFe5Py0KStkgKyuLQQsG8dza5/Cp4MNPo3/ioQjHdyTt06cPX3zxBc899xzffPMNbdq0YdSoUcTHx5d42cNDhvPVsK8AGBI9hDlxc0q8TOX+tCgoVYiz6WdpMaMFX/3+FTf43sCBxw7QpX6XUlu/r68vkydPJj4+nieeeILo6GiaN2/Ogw8+SEJCQomWPaD5ANaMXIOnhyf3/vdepv8y3U6plbvSoqBUAXb+uZMbp97I3jN7ubn+zRx+/DC1fWs7JUvNmjV544032L9/Pw888AAff/wxTZo0Yfz48Zw8ebLYy+3RsAfr/7Eeb09vHv32UV768SU7plbuRouCUvmI3hlN2PthnL14lvGdxhM7OpYKHhWcHYs6derw7rvvsnfvXkaMGMG7775Lo0aNeOaZZzhzpnhnLHeo04G4B+KoVKESL8S8wJPfPWnn1MpdaFFQKg//+v5fDF00FEGYO3Au7/R9x9mRrtOgQQNmz57Nrl27GDhwIK+//jrBwcG89NJLnD17tsjLa1mzJbsf3k1V76q8vf5txiwb44DUytVpUVAql6ysLHp/1pspP0+hilcVNt+3mRFhI5wdq0DNmjVj3rx5bNu2jV69evHCCy8QHBzMlClTSEtLK9KyGvg3YN+4fQRWCuSjrR8RuTDSQamVq9KioJTVqbRTBE8LZs2BNTTyb0TCEwmE3xDu7Fg2CwkJYcmSJWzcuJFOnTrxr3/9i0aNGjF9+nQuXrxo83Jq+dZi/6P7qe1bm0W7FnHbXPudpa1cnxYFpYBNxzbRYGoDDqccpl+Tfuwdtxd/H39nxyqWDh06sHLlSn766SdatGjBo48+StOmTfnwww+5fPmyTcvw9/Fn/7j9NPRvyHfx33HT7Jv0Yj3lhBYFVe59tOUjOs3uRNrlNJ7v9jwrh68sUYdTV9G1a1fWrl3L6tWrqVOnDvfffz8tW7bk888/JzMzs9DXV/auzN5xe2lZoyUbjm4g7P0wMrIySiG5cib3/8tXqgQeWvEQY5aPwcN48NXQr3ipZ9k6HNMYQ69evVi/fj3Lly/H19eXe+65h9DQUBYvXlzor/8KHhXYMXYHHW7owI7EHTR/tznpGemllF45gxYFVS5dyrpE59mdmblpJv4+/uwYu4M7Wtzh7FgOY4zhr3/9K1u2bCE6OpqsrCzuvPPOnKGmgtpze3h48MuYX+gV3Iv4pHgaT2vM2fSiH92k3IMWBVXuJJxNYNiGYfxy9Bda1WjF0ceP0rxGc2fHKhUeHh5ERkayY8cO5syZQ3JyMrfffjtdunThhx9+KPB1q0euZmDzgRw7d4xG0xpxKu1UKSZXpUWLgiqTzqafJeZgDFM3TOXB5Q/S57M+tJzRklpv1KLB1AYkXU5iaOuh7Hx4J5W9Kzs7bqnz9PRk5MiR/P7778yaNYsjR47Qq1evnKGm/CwdtpRRYaM4feE0jac15nDK4VJMrUqD80/PVKqIMrIy2JW4i83HNrMrcRd7z+zlSMoRTp4/SXJ6MhcyLpAl14+VGww+FXwIqhLE4KDBvHvnu05I71q8vLy4//77GTlyJLNmzeKVV17hL3/5C/379+fll1+mbdu2173m04Gf4lfRj2m/TqPljJbEPRBH0+pNnZBeOYIWBeVyEs4msPnYZrad3MYfp//gYPJBjp87zpn0M6ReSs33CBhvT298vX2pW60udavWpaF/Q1rWaEloUCgRdSIIrByYM29MTEwpvRv34OPjw2OPPcaYMWOYPn06U6ZMoV27dgwZMoQXX3yR1q1bXzX/f/r9hwCfAF6MfZGQmSFs+McGtzqnQ+VPi4IqVamXUtlybAvbTm5jV+Iu9ift5+jZoySmJXLu4jnSM/M+ssXTeFLZqzJ1q9YlqEoQDfwb0LR6U0JrhdKhTgeC/YPLxGGkzlalShUmTJjA2LFjeeedd3j77bdZsmQJd999N5MmTaJJkyY5807qOYlqPtV48rsniZgdwdpRa+lav6sT0yt70KKg7CYrK4tdibvYcmILO/7cwb4z+ziccpiT50+SdCGp0GEd/0r+1Kxck3rV6tEksAmta7YmvHY4YbXDHHJlM5U/Pz8/Jk2axLhx45gyZQrTp09nwYIFjB49mueff5769esD8MRNT1CtYjXuX34/PT7twfK7llOJSk5Or0pCi4ILOpV2ij5z+3Do9CG8NnohXDlcUJCrDh/Mfi6vx/J73JZ5bX7O+s/lrMtk/pj3CVFeHl5UrViVOlXrUKdqHYIDgmlevTlhQWFE1I2gRuUaeb5OOV/16tV5/fXXefzxx3nllVeYNWsWn332GQ888ADPPvsstWvXZky7Mfj7+BO1MIrb599O/cr1qbzTPXbep6WluU1WgN5+velBD4euQ4uCizmQdICw98M4d+kc3h7eeIonxpic5w3mqvmzn8v9eF7zG2Ouul/YvFdNZy/bWO7n9ZrMy5k0qN6A+n71aVa9Ga1rtabDDR1oGthUh3XKgNq1azNt2jSeeuopXn75Zd577z1mz57NuHHj+Oc//8mdre7km+HfMOjLQRxOO4y5YApfqAsQEbfJChBcIdjxKxERt7q1b99eimvt2rXFfm1p2Hxss/hM9hEmIeO/Ge/yeXNzp6wi7pXXFbPu3btXRowYIcYYqVq1qkycOFGSk5NFxDXz5sedsoqULC+wSWz4jtUtBRfx/f7vcy4I/3rv1/lnl3/qETLKZTVp0oS5c+cyYcIEJk2axEsvvcT06dN5+umn8fb2tny5VKiAp6fnVbeSPJZ7C1U5jhYFF/DF9i8YvmQ4AJ/e8Smjwkc5OZFStmndujULFy5ky5YtTJw4kWeffdZh6/Lw8LBLgck9nZKSQmBgYOErdxE9e/akR48eDl2HQ4uCMaYv8B/AE5gtIq9d8/wTwBggA0gE/i4ihxyZydVM3TCVx1c9jqfxZNldy+jftL+zIylVZO3atePrr79mz549fPfdd4SEhJCZmXnVLSMjw6Ueu3jxIunp6aSmpjr747NZRobju9Q6rCgYYzyBGUAfIAHYaIxZJiK7cs22FeggImnGmLHAFGCoozK5mmdWP8Nr/3sNb09v1o1eR0TdCGdHUqpEWrRowYkTJxz+a9ZeYmJi3CYrlM5Jl47cUugI7BOReABjzALgDiCnKIjI2lzzbwBc+7qHdnTvV/cyZ9scqnhVYesDW7VNgFLKJRgpoGVuiRZszJ1AXxEZY52+B+gkIo/kM/+7wAkReTmP5+4H7gcICgpqv2DBgmJlSk1NxdfXt1ivtZesrCwm7JjAxqSNVKtQjU8jPiXAOyDPeV0hr63cKSu4V153ygruldedskLJ8vbs2XOziHQodEZbDlEqzg24E8t+hOzpe4B385l3BJYthYqFLdedD0m9nHlZ2s9qL0xCGrzTQM5dPFfg/M7OWxTulFXEvfK6U1YR98rrTllF3P+Q1KPAjbmm61kfu4oxpjfwf0B3EbH96uJuJu1SGiHvhxCfFE9IrRC2PLCFCh568JdSyrU48lTTjUBTY0ywMcYbGAYsyz2DMaYtMAsYICJ/OjCLU51KO0XwtGDik+K5peEtxD0QpwVBKeWSHFYURCQDeARYBewGokVkpzHmJWPMAOtsbwC+wEJjTJwxZlk+i3NbB5IO0HhaY/48/yfDWg9jzag12vZBKeWyHPpzVURWAiuveWxirvu9Hbl+Z9tyfAtdPupCemY64zuN552+7zg7klJKFUh/sjrImvg1dJrdifTMdF7v/boWBKWUW9CBbQfQthVKKXelRcHO/rPhP4xfNV7bViil3JIWBTvSthVKKXenRcFOtG2FUqos0KJQQllZWdw+/3a+3f8t1StVZ8dDO6jtW9vZsZRSqli0KJRARlYGnWd3ZvPxzTTwa8COh3bg6+0+fVSUUupaWhSKKe1SGqHvh7I/ab+2rVBKlRl6nkIxZLet2J+0n54Ne2rbCqVUmaFFoYgOJR+6qm3FD6N+0LYVSqkyQ7/NiiDueBwt3m3B2YtnGd9pPF/c+YWzIymllF1pUbDRmvg1RMyO0LYVSqkyTQfCbfDF9i8YsXQEIqJtK5RSZZoWhULkblux/O7l2rZCKVWmaVEogLatUEqVN1oU8jH6v6P5NO5TbVuhlCpXtCjkof+8/nyz7xttW6GUKne0KOSSkZXBTbNvYtPxTdT3q8/Oh3Zq2wqlVLmiRcFK21YopZSepwDAmbQz2rZCKaXQosCh5EMETwvWthVKKUU5Lwpxx+NoMUPbViilVLZyWxR+OPCDpW1FhratUEqpbOVy4PzLHV9y95K7tW2FUkpdo9wVBW1boZRS+StXReHD+A+Zf2Q+3p7exN4bS6d6nZwdSSmlXEq5KQoPLn+Q+Ufma9sKpZQqQLkpCmG1wwj0CmTnozu1bYVSSuWj3BSFsRFjaXm+pRYEpZQqQLk9JFUppdT1tCgopZTKoUVBKaVUDocWBWNMX2PM78aYfcaYCXk8X9EY86X1+V+MMQ0dmUcppVTBHFYUjDGewAygH9AKuMsY0+qa2f4BJIlIE+Ad4HVH5VFKKVU4R24pdAT2iUi8iFwCFgB3XDPPHcAc6/1FQC9jjHFgJqWUUgVwZFGoCxzJNZ1gfSzPeUQkA0gBqjswk1JKqQIYEXHMgo25E+grImOs0/cAnUTkkVzz7LDOk2Cd3m+d59Q1y7ofuB8gKCio/YIFC4qVKTU1FV9f97m8pjvldaes4F553SkruFded8oKJcvbs2fPzSLSobD5HHny2lHgxlzT9ayP5TVPgjGmAuAHnL52QSLyAfABgDEmsWfPnoeKmakGcKrQuVyHO+V1p6zgXnndKSu4V153ygoly9vAlpkcWRQ2Ak2NMcFYvvyHAXdfM88yYBSwHrgT+EEK2XQRkZrFDWSM2WRLpXQV7pTXnbKCe+V1p6zgXnndKSuUTl6HFQURyTDGPAKsAjyBj0VkpzHmJWCTiCwDPgLmGmP2AWewFA6llFJO4tDeRyKyElh5zWMTc91PByIdmUEppZTtytsZzR84O0ARuVNed8oK7pXXnbKCe+V1p6xQCnkddvSRUkop91PethSUUkoVoNwUhcL6MLkSY8zHxpg/redxuDRjzI3GmLXGmF3GmJ3GmMecnSk/xhgfY8yvxpht1qwvOjuTLYwxnsaYrcaYr52dpSDGmIPGmO3GmDhjzCZn5ymMMcbfGLPIGLPHGLPbGHOTszPlxRjT3PqZZt/OGmPGO2x95WH4yNqH6Q+gD5YzqzcCd4nILqcGy4cxphuQCnwmIm2cnacgxpgbgBtEZIsxpiqwGRjoip+ttYVKFRFJNcZ4AeuAx0Rkg5OjFcgY8wTQAagmIn91dp78GGMOAh2uPfnUVRlj5gA/ichsY4w3UFlEkp2dqyDW77KjWE7yLe75WgUqL1sKtvRhchkiEovlEF2XJyLHRWSL9f45YDfXtzNxCWKRap30st5c+leRMaYecDsw29lZyhJjjB/QDcth8YjIJVcvCFa9gP2OKghQfoqCLX2YVAlZW5+3BX5xbpL8WYdi4oA/ge9FxGWzWk0F/glkOTuIDQT4zhiz2dqaxpUFA4nAJ9ahudnGmCrODmWDYcAXjlxBeSkKysGMMb7AYmC8iJx1dp78iEimiIRjabvS0RjjssNzxpi/An+KyGZnZ7FRVxFph6Vd/sPWYVBXVQFoB8wUkbbAecDV9zV6AwOAhY5cT3kpCrb0YVLFZB2fXwzME5Elzs5jC+tQwVqgr7OzFKALMMA6Vr8AuMUY87lzI+VPRI5a//0TWIpl2NZVJQAJubYUF2EpEq6sH7BFRE46ciXlpSjk9GGyVtthWPouqRKy7rz9CNgtIm87O09BjDE1jTH+1vuVsBx4sMe5qfInIs+ISD0RaYjlb/YHERnh5Fh5MsZUsR5ogHUY5lbAZY+eE5ETwBFjTHPrQ70Alzs44hp34eChI3BwmwtXkV8fJifHypcx5gugB1DDGJMAvCAiHzk3Vb66APcA261j9QDPWlucuJobgDnWIzg8gGgRcenDPN1IELDUeo2sCsB8EfnWuZEKNQ6YZ/2hGA+MdnKefFkLbR/gAYevqzwckqqUUso25WX4SCmllA20KCillMqhRUEppVQOLQpKKaVyaFFQSimVQ4uCUqXIGNPD1budqvJNi4JSSqkcWhSUyoMxZoT12gtxxphZ1kZ6qcaYt4wxW4wxa4wxNa3zhhtjNhhjfjPGLDXGBFgfb2KMWW29fsMWY0xj6+J9c/Xxn2c9K1wpl6BFQalrGGNaAkOBLtbmeZnAcKAKlt4z7YAfgResL/kM+JeIhALbcz0+D5ghImHAX4Dj1sfbAuOBVkAjLGeFK+USykWbC6WKqBfQHtho/RFfCUur7SzgS+s8nwNLrH35/UXkR+vjc4CF1j5AdUVkKYCIpANYl/eriCRYp+OAhlgu+KOU02lRUOp6BpgjIs9c9aAxz18zX3F7xFzMdT8T/f9QuRAdPlLqemuAO40xtQCMMYHGmAZY/n+50zrP3cA6EUkBkowxN1sfvwf40XoVugRjzEDrMioaYyqX6rtQqhj0F4pS1xCRXcaY57BcRcwDuAw8jOVCLK2NMZuBFCz7HQBGAe9bv/Rzd9u8B5hljHnJuozIUnwbShWLdklVykbGmFQR8XV2DqUcSYePlFJK5dAtBaWUUjl0S0EppVQOLQpKKaVyaFFQSimVQ4uCUkqpHFoUlFJK5dCioJRSKsf/A+cnp7NIMke3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12be6f83c88>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJztnXecFEX2wL+PJYcDhBOVIKio5KiAqCxGMKDoqaiYUFF/plMxHyp654kgBhQBAyqeKKKnKAgKsiAnKCCLAgosSRZEcmYJu/X7Y6abmdkJPTPdM7Ps+/LZDx2qq95UV9erV6+CGGNQFEVRFIAy6RZAURRFyRxUKSiKoig2qhQURVEUG1UKiqIoio0qBUVRFMVGlYKiKIpio0pBURRFsVGloCiKotioUlAURVFsyqZbgHipXbu2adiwYULP7t69mypVqrgrkAeonO5REmSEkiFnSZARVM5IzJs3b5Mx5q8xAxpjStRfu3btTKJMmzYt4WdTicrpHiVBRmNKhpwlQUZjVM5IAHONgzpWu48URVEUG1UKiqIoio0qBUVRFMWmxDmaFUU5fDlw4AD5+fkUFBQkHEf16tX59ddfXZTKG7ySs2LFitSrV49y5col9LwqBUVRMob8/HyqVatGw4YNEZGE4ti5cyfVqlVzWTL38UJOYwybN28mPz+fRo0aJRSHdh8pipIxFBQUUKtWrYQVQmlHRKhVq1ZSlpYqBUVRMgpVCMmRbP6pUlCUEkju+lxm589OtxjKYYgqBUUpgbQZ0YZOb3VKtxiHHdu2bWPYsGEJPXvBBRewbds2x+GfffZZBg8enFBaXqJKQVEUxU80pXDw4MGoz06cOJEaNWp4IVZKUaWgKIri55FHHmH58uW0bt2aBx98kJycHLp27co111xDy5YtAbj00ktp164dzZo1Y+TIkfazDRs2ZNOmTaxatYomTZpw66230qxZM8477zz27t0bNd3c3Fw6duxIy5Yt6dmzJ1u3bgXglVdeoWnTprRs2ZJevXoBMH36dFq3bk3r1q1p06YNO3fudDUPdEiqoigZyd8n/Z3c9blxP1dYWEhWVlbYe62Pas1L3V6K+Oxzzz3HwoULyc31pZuTk8OPP/7IwoUL7SGeb7/9NkcccQR79+7llFNO4fLLL6dWrVpB8SxbtowxY8bwxhtvcOWVV/LJJ5/Qu3fviOlef/31DB06lC5duvDEE08wYMAAXnrpJZ577jlWrlxJhQoV7K6pwYMH89prr9G5c2d27dpFxYoV48qfWKiloCiKEoVTTz01aMz/K6+8QqtWrejYsSNr1qxh2bJlxZ5p1KgRrVu3BqBdu3asWrUqYvzbt29n27ZtdOnSBYAbbriBGTNmANCyZUuuvfZa3n//fcqW9bXhO3fuzP33388rr7zCtm3b7OtuoZaCoigZSbQWfTTcnhQWuLx1Tk4OU6ZMYdasWVSuXJns7OywcwIqVKhgH2dlZcXsPorEhAkTmDFjBuPHj+eZZ55h0aJFPPLII1x44YVMnDiRjh07MmXKFE4++eSE4g+HWgqKoih+qlWrFrWPfvv27dSsWZPKlSvz22+/MXt28sOCq1evTs2aNfnuu+8AGD16NF26dKGoqIg1a9bQtWtXnn/+ebZt28auXbtYvnw5LVq04OGHH6Z9+/b89ttvScsQiFoKiqIofmrVqkXnzp1p3rw53bt358ILLwy6361bN4YPH07Lli056aST6Nixoyvpvvvuu9x+++3s2bOH4447jlGjRlFYWEjv3r3Zvn07xhjuu+8+atSoQf/+/Zk2bRplypShWbNmdO/e3RUZLFQpKIqiBPDBBx8EnWdnZ9vHFSpU4Kuvvgr7nOU3qF27NgsXLrSv9+vXL2z4xx57zO7mat26dVirY+bMmcWuDR06NKr8yaLdR4qiKIqNKgVFURTFRpWCoigZhW87YSVRks0/VQqKomQMFStWZPPmzaoYEsTaTyGZCW3qaFYUJWOoV68e+fn5bNy4MeE4CgoKXJ/l6wVeyWntvJYoqhQURckYypUrl/COYRY5OTm0adPGJYm8I1Pl9Kz7SETeFpENIrIwwn0RkVdEJE9EfhaRtl7JoiiKojjDS5/CO0C3KPe7A439f32B1z2URVEURXGAZ0rBGDMD2BIlyCXAe8bHbKCGiBztlTyKoihKbNI5+qgusCbgPN9/TVEURUkTJcLRLCJ98XUxUadOHXJychKKZ9euXQk/m0pUTvcoCTJC4nKm8rcd7nmZajJWTmOMZ39AQ2BhhHsjgKsDzpcAR8eKs127diZRpk2blvCzqUTldI+SIKMx8cvJUxiewhthInC45mW6SLWcwFzjoN5OZ/fReOB6/yikjsB2Y8wfaZRHURSl1ONZ95GIjAGygdoikg88CZQDMMYMByYCFwB5wB7gJq9kURRFUZzhmVIwxlwd474B7vQqfUVRFCV+dO0jRVEUxUaVgqIoimKjSkFRPOaH/B+YsXpGusVQFEeUiHkKilKS6fiWbx9f86QuB61kPmopKIqiKDaqFBRFURQbVQqKoiiKjSoFRVEUxUaVgqIoimKjSkFRFEWxUaWgKIqi2KhSUBRFUWxUKSiKoig2qhQURVEUG1UKiqIoio0qBUVRFMVGlYKiKIpio0pBURRFsVGloCiKotioUlAURVFsVCkoigsUmSKGzBrCrv27ko7rQOEBBv1vEPsL97sgmaLEhyoFRXGBL5Z8wQNfP0C/r/slHdewOcN4aMpDvDjrRRckU5T4UKWgKC5QcLAAgK0FW5OOa/u+7QCuWB2KEi+qFBTFBcqI71MqMkVJx2WMCYpTUVKJljpFcQERAdxRClYcqhSUdKClTlFcwKrArVZ+MqhSUNKJljpFcQE3u49UKSjpxNNSJyLdRGSJiOSJyCNh7jcQkWkiMl9EfhaRC7yUR1G8wgulYHVJKUoq8UwpiEgW8BrQHWgKXC0iTUOC/QMYa4xpA/QChnklj6J4ieCeT8GgjmYlfXhZ6k4F8owxK4wx+4EPgUtCwhjgL/7j6sA6D+VRFM+wfQp451MYMXcEv2//Pen4FSUaZT2Muy6wJuA8H+gQEuYp4GsRuRuoApwTLiIR6Qv0BahTpw45OTkJCbRr166En00lKqd7pErGhZsXArBp06aI6UWTI1DO1b+vBmDlipXkHPBd23FgB7d/fzv1K9XnvVPfcxSn25SE9w0qZ7J4qRSccDXwjjHmBRHpBIwWkebGBNvgxpiRwEiA9u3bm+zs7IQSy8nJIdFnU4nK6R6pkrEgrwAWQo0jahRPb7rvv2hyBMo5ft94yIfGJzQmu5Pv2sbdG+F72Ct7feEcxOk2JeF9g8qZLF52H60F6gec1/NfC+RmYCyAMWYWUBGo7aFMiuIJqRqSGi7+52Y+x8h5I5NOV1HAW6UwB2gsIo1EpDw+R/L4kDC/A2cDiEgTfEpho4cyKYoneDL6CGejjx6d+ii3fXlb0ukqCnioFIwxB4G7gMnAr/hGGS0SkadFpIc/2APArSKyABgD3GjcaGopSopxdfSRLnOhpBFPfQrGmInAxJBrTwQcLwY6eymDoqQCr0cf6ZwFJVVoU0RRXERnNCslHS11iuICloWgSkEp6WipUxQXsCryQJfYqPmjuHjMxfb50B+Gkr8j33Fc2mWkpANVCoriApYyCLQU+ozvw5dLv7TP75l0Dxd+cGFy6bjgs1CUaKhSUBQXcNp9tK1gm+M4nQ5JVRQ3UaWgKC5gKQM3fAqKkk5UKSiKC1jdR15176jVoKQKVQqK4gJujj5SlHSiSkFRXMC2FGJMyI9nwr46lZV0oEpBUVzAHpLqQkUebiiqKgglVahSUBQXcNp9lOzcA10aTPEaVQqKkiQfL/qYGatnAPH5FPK25HHmqDOZtWYWAJPyJnHGqDPYsHtDsbCBymDRhkVJSqwokUn3JjuKUuK5ctyV9nE8Lfku73Rh3c51nPb2aUzrMo3u/+keMWxg91Hz15snJqiiOEAtBUVxkXgsha17t0a9r11FSjpQpaAoLhJLKSRa0auCUFKFKgVFcZF4Rgkl4nTWUUiK16hSUBQXiaf7KNYs5UAFoMpASRWqFBTFRdwckhrYZWQd63IXiteoUlAUF4mn7z+WgghnHajFoHiNKgVFSYDtBdt59rtni1kGrnYfGe0+UlJPqZmn8Pb8t3l6ztMs7ryYyuUqp1scpYRz3+T7GJU7ijXb1wRdd3NBPF1cT0kHjiwFEeksIlX8x71FZIiIHOutaO6yec9mVu9ZrUP7FFfYuX8nAMPnDQ+6Hs+Q1MDuo437NhYPS3GfgqJ4jdPuo9eBPSLSCngIWA2855lUHqJmuJIpBHYfhVMm2n2kpAOnSuGg8ZXQS4CXjTEvA9W8E8t9dBN0JRUkWnk7dSqrxaB4jVOfwk4ReRToDZwpImWAct6J5R36USlukOjQ0MDGSczRR2GGpCqK1zi1FK4C9gE3G2PWA/WAQZ5J5QE6vltJBXENSQ0ok+Ge0y4jJR04thTwdRsVisiJwMnAGO/E8g790BQvSXSZi3DPBfoZtNwqqcKppTADqCAidYGpwE3AO7EeEpFuIrJERPJE5JEIYa4UkcUiskhEPnAqeLyoT0Fxk0TLU1zbcWqXkZIGnFoKYozZIyI3A0ONMc+LyIKoD4hkAa8B5wL5wBwRGW+MWRwQpjHwKNDZGLNVRI5M7Gc4Rz80xUsS7j6K4WjWcqukCqeWgohIJ+BaYILDZ08F8owxK4wx+4EP8Y1eCuRW4DVjzFYAY0zxLadcQn0KJZf3f36fTxZ/km4xHPHn7j95efbLUcP89MdPDJ87nM17N0cNZ4xh4MyBjF00NmL30f7C/QnLqijhcGop/B1fi/6/xphFInIcMC3GM3WBwOme+UCHkDAnAojI/4As4CljzKTQiESkL9AXoE6dOuTk5DgU+xB5+XkAfDfzO6qWrRr386lk165dCf3GVJMqOa+bfh0A07rEKnLF8UrGDRsit1/+PvnvtCpoFfbevn37aDeyXbHru3fvLnZt+YrlvLHyDQDGdPC58A4ePBgU5tGPHrWPvX4XWi7dJVPldKQUjDHTgekiUlVEqhpjVgD3uJR+YyAb34imGSLSwhizLST9kcBIgPbt25vs7Oy4E8qdnQvL4fTTT6dGxRrJyu0pOTk5JPIbU03K5Jzu+y+RtLyScfim4VB8ErJNdna2LXcgFStW9I3jC6FS5UrFrjVs1BBW+o47dOgAP0BW2SwoPBTm2OOPhWUBaXqIlkt3yVQ5nS5z0UJE5gOLgMUiMk9EmsV4bC1QP+C8nv9aIPnAeGPMAWPMSmApPiXhGdo3q6QCN8qZkzh0fSTFbZz6FEYA9xtjjjXGNAAeAN6I8cwcoLGINBKR8kAvYHxImM/wWQmISG183UkrHMoUF+pTUFJJvJV1TEdzBJ9CYVFh2OuKkihOlUIVY4zdoWuMyQGqRHvAGHMQuAuYDPwKjPX7I54WkR7+YJOBzSKyGJ+P4kFjTHTvW5LoeG8lFRSa8JV1pNZ/2MlrDmY0R0pHURLFqaN5hYj0B0b7z3vjoEVvjJkITAy59kTAsQHu9/95is5TUNwkVnmKtwVfRJgF8Rw0YLT7SHEbp5ZCH+CvwKf+v7/6r5U41KegpIJ4W/DhyqWTGc3afaS4jdPRR1txZ7RR2lCfgpJK3LAUwsURqjy0+0hxm6iWgoh8ISLjI/2lSkg3UZ9CZHJW5fBu7rvpFiMtzPx9JiPmjnAtvm9Xfhv2+tqdoQPwfIxePbrYtTd+OjSWw1IG1uY+Fsu2LAsb3+/bf6ff1/087V7ad3AfD379IDv37YwdOA18vfxrPlz4YbrFKHHEshQGp0SKFKA+hdh0fbcrADe0viHNkqSeM0adAcBt7W9zFD6W5XnZ2MvCXo9USc/YNKPYtT93/2kfR2rMvP/z+2Gv9/60N9/9/h2XN7mcTvU7RZU1Ud6a/xaDZw3GYBh8XuZVFee/fz4AvZr3SrMkJYuoSsE/aS0IEWlrjPnJO5EURUmWA0UHvE+j8EDQ/8rhgVNHcyBvui5FClFHs1ISSbTcanepEi+JKIUS2Q+jjmbFTTK9OzKV5V0Vz+FFIkphgOtSpBAtwEpJJBPLbaYrRiUxnK591FNEqgMYYz4TkRoicqm3ormLFmClNOJld6l2xR6eOLUUnjTGbLdO/KuYPumNSN6iBVkpiWRyudWu2cMLp0ohXDinS2RkBFpwlZJMot1HqbCQM7FrS0kcp0phrogMEZHj/X9DgHleCuYVqS7A2wu2c9+k+9h3MMwi+gnw2W+fMW7xuGLXh80ZxvdrvncljUCmrpjKqPmjANiydwsPTH4g5hDEIbOG8NMfzkct79y3k/sm3cfeA3tjhjXGcMv4W7ji4yvS0nresncLp711Gqu2rUppuks3L40ZJjA/ZuXPKnYtHO8teI+vln2VkEyJKJzc9bkM/t43p2H0gtGc8sYpfLzo44TSzzSe/e5ZFm9cHDtgBD5c+CFfLv3SRYkSw2lr/26gP/ARYIBvgDu9EsoL0uVTeCrnKV764SVOrHUid5xyR9Lx9fyoJwDmyeCP/c6Jd4a9niznjD4HgJva3MQDXz/AO7nv0P6Y9lzd4uqIzzzw9QNxyfLvmf/mpR9eon71+tzfKfraiFv2buGt+W8BsGjjIpof2dxRGm7x4cIP7Qo3lVjvPRoGE7dFfMNnvomKbpebSLQZ0QaAfqf14/rPrgfgynFXYpqVbGtj9/7dPP7t47ww6wU2P5TYQs9Xf+L7plL1LiLhdO2j3cAjHsuSElLdurQmER0Oa9RYFsLBooMxQsaHFZ+TeIMWiUuDpZDJfftFpogyksiAQiVZrB4It3oE0onT0UffiEiNgPOaIjLZO7HcJ90+hUyuTJxiVTjp7ENOd/91utOPRtg9GVIg7+FQtpVDOG1W1A7cN9m/auqR3ojkLan+qNOtjNzE6oJL5xr+gWnrMONgdG+F9GEpxsOhTDpVCkUi0sA6EZGGkMFNpjCk+2VlcgvTKbalkMaWYborvkxuFYcrY6lolKT728oErLw/HBqBTh3NjwMzRWQ6vmUuzgD6eiaVh6T6oz6cPhirwHtVMTt5N+mulDNZuYd7L9p9pMSLI0vBGDMJaA8sAcYADwCxxw9mEOnW4IfDh2PlYan2KWTwe9y1f1e6RSi1ZHK5iBenjuZbgKn4lEE/fHs1P+WdWN6hPoXE8ar7KJ48Ckw7HXmbbqUUjYenPJzS9A6nsp0sdvfRYdAz4NSncC9wCrDaGNMVaANs9EwqD0j3y3JSmYxfMp435r0RMxz4JjP1/7Z/1Ap6696t3D3xbgoOFjiWE2DC0gnFrj38zaEKJ7SbYvyS8RE3e3nrp7eYnDfZlueer+5xNGxvx74d3D3x7mIT2sKlfcaoMyLudBaLL5d+6Wi3uYKDBZzwyglMXp65g+427yk+Pj5a+fh6+ddR4yssKqTf1/3I35EfM+1I73bC0glh89frlnXg5MnHpz7Ov7/7N/P/mB/1mRVbV/DIlEf4ce2PDPrfIPv6zN9nUv/F+sxdN9e+NnrB6KCJZrajOYqiHPrDUK7773WcMeoMe5LbKz+8wszfZ0aV66OFH4WdsOoVTn0KBcaYAhFBRCoYY34TkZM8lcwjMtmncMmHlwAwrcu0mGG7vd+NldtWclv726j3l3phwzwx7QlenfMqzY9s7nhHMYCLxlxUbALN898/z6l1TwWKKzhL7t4texeL65YvbvE986ThsamPMXzecFrVacXNbW+OKsPT05/m1Tmv0rhW46DrgfM9RMRO++z3zk5o0s/FYy52FG7l1pUs37qc5VuXx51Gqoh3joK1M1kkvl/zPS/MeoH56+cz9fqpUcNGercXjbkIKL6bn9cWV7uR7ezjZ2c+65Px28eilpHLx15O7vpcBv5vIAAPdn4QOLQr3ylvnGI/b028s86d/J57Jh3a5j77nWzGnjKWeyfdGxRPOHp90itmGDdxWory/fMUPgO+EZHPgXXeieU+6TZ13VZG+wv3xwxjTQZzyzFsxRMtvmi/04nMFpFkT/foI6+5oukVCX/84RogyVjIVl47mVQYb1nLxD74eMpnKNbvdprfqdgZL1Gczmi25tg/JSLTgOrAJM+k8pDDzafgaMSOS7/ZSitamtHSKsL34URq0TqRs7Co5M8M94pwZS2ZytfLbyWTfTOJkIlKLlHiXuk03L7NJYGS4FOIBye/x6vfHO23RPs4rNZUqFIIJ2ekeNJhKRQcLODs985OSVrJvDOv3ndcAwEOs8reKYfTPAVdKMVjvFZGTj5Ct1oxVlpRu4+iWQoRlEI0Qj+yIJ9Cij7AOWvn8MeuP1KSVjK4nR9Oy43BxF3OM7FlnUz+ZeLvSZRSpxTS9fLSOYzTbaJ2HzmwFJxUIJGUSzq6j1LZ+k3mvYa1uDyUPZkGz+FmUcTrU8hkPFUKItJNRJaISJ6IRFxlVUQuFxEjIu09kyVNlaiVbr9v+rkbr7/wpVLJWWklailYz8dlKYR8ZOnoPioprcBUr5CalL8iA/M0GUUV77OZ+PstPCtFIpIFvAZ0B5oCV4tI0zDhquGbB/GDV7IEUhJaKE4KjJPZxV4pQrd9CvHEE9h9VBLeZbwk5VPw6H3HI1M8XU6HE07mKUR6JtPwsmlxKpBnjFlhjNkPfAhcEibcM8BAIL4ZVnGSiWbduMXjeHr608WuW6N05q2bxz9n/BNjDA998xC563PtMNbwuVd+eCXo2Td/erPY5LNIH+D+wv3cOeFONu52Ng/RimfaqmlMXTGVT9d+GjEMUGxCWaBSeCf3HR6Z8ggDZw60x4U/OvVRthdsD3rm0amPho0D4JPFn0SVd1LeJPrO68vnv30e9n6kCUG563N5bOpjGfvRRkJEMMYETTQc9P0gHv7mYd766S2mrZzGy7NfjhrHsDnD+Gb5N0Dwu3xkyiPcOOfGiHkSWBmOmj+qWJ7LAOGmz28KChPI5LzJyAAhb0seA3IGRJ1olrs+l2s+uYZtBdt49rtnmbN2TtTfFIj1LfUa14tFGxYxcOZAZufPLvYbAGasnsFjUx9zFK9VLg8UHeCGz25g/JLx/N+E/2PPgT0Rn5m6Ifzcj8l5k+16IZlhsoni5T7LdYE1Aef5QIfAACLSFqhvjJkgIg9GikhE+uJfgK9OnTrk5OTELcxv638DYNasWayqtCru5xMlP//QbNBQua+YfgUAZ5ozg65v27mNnJwcuk7vCkDb/W0Z9P0gXv3hVTuM5fh8cfaL9KjQw75+6xe3Ar4JcOvW+aaSLFu2jJw9wWkDfPPnNwz7bRh5a/J49OTgyjdcHu/cuRPwzQK2ZnNelnNZ0DP7Cg/NaA0csZOTk8OfG/4E4NfFv/LU4qeKxQ9wxwd30Pe4vqxduxYg6KPKyclh0fZF9vkTOU9Elbn79O4AXPrRpWEnBFr5HxrH+d+dz/6i/WRLNuXLlCd3W26xcF6xYcOGhMo3wM5NO/nk6094fvbz9rXAd2XRqqBVsWetNO+c7tvBb1qXaeRu9f3u7du2M3CVT3GP/mo0DSr7FkzOW5sHwNq1azlofPMUlixdwovLXrTjCOSd3Hfs4/+b+H9B97r9pxsA2W9ms3bvWv454598c+Y3YX9n/4X9mbl5Jm2kDY//8jiPf/t4UFq7dkVeA+qDSR8w6EffbOVJSyex/cB2W9bdu3cHhe3yTpdiz4e+G+v8zwJf2d6xbwfvLXiP9xa8B4BsFa6oV7ycHTx4kH/99q+w8Vp5caY5k283fBs2jJd4qRSiIiJlgCHAjbHCGmNGAiMB2rdvb7Kzs+NOb3XualgCHTt2pFHNRnE/nyhf7v/Spw6BYnJPD7nuP69cpbLvmv/89DNOh/9BVlYWhPGzBoYNvDZuzzhYByeccALZHbKLPbf257XwG9Q+snaxOEJlAqhatSoEfzdBz2VnZ/sq8TCz9rOzs6n1Zy3YBC2at4AIW9k2aNCA7OxsPtnzSbHpkdnZ2ZT7vRxEqKMj5W/YeyH3A8OZ73yt4S5ndqFC2QqwClgQPk2nPNP1GfpP6x8zXJ0j6xR7F62Pah1kJUaiW5tudG7SGWZHDxfu3YZey87OpnBFIfwMNWrUAP9uKqd1PI0TjjgBgIU/LoQ8qFu3rm9Xvj/gpBNPgmUBccY5gN2U9VsiEuGdAZXWVILN0KpVK/glRH6iV54dOnSAH/1JZAkcOPR8lcVVIHLDPjidkG939bbVYTvAjzvuOLJPC34GoGzZskHfcri86tKlC+sXrYdfQ9L2GC+7j9YC9QPO6/mvWVQDmgM5IrIK6AiM99LZDCVj8lro1p2J9Fc6werbdzwLNcnhr/YIjSi/I9bubqnc1tTNsuK0KypcN2eVclUcPZuK7TizJCvq/WTzzFqnq1yZcknFEwmv/C5eDIBIl9/FyxI0B2gsIo1EpDzQCxhv3TTGbDfG1DbGNDTGNMTXvulhjJkbPrrkyESfQiRCKz6vhrvFqxScENUJTezRR7FWYk3l6CMrLTd8C04/8HCVVqVylRw9mxKlUKa4UjAm/nkKkbCVQlZspZDIewmUMzSvUznE1kn4IlOUllGTnpUgY8xB4C5gMj4DaKwxZpGIPC0iPaI/7R2ZvCCeRahSsM7jLSCxwserFJysgZPs6KNYlZpb8xSi/ZbQSXpuKKJkyl35rPKOwhWZIk8aP4Fxev39xGMpJGI1RvsmMm2IbboGOnjqUzDGTAQmhlx7IkLYbC9lKUnTz4spBX9FmOgHH6lVEu+ey06WvE52RnMspeCWpTBlxZSYYay0zhl9TtLpObYUwrzjsmWcfaYHCg8wdUX01UzjIZzM4SpiEcGtng4rz51YCk4aKaF41WPghQXrlZKPRdoczemiJPoUEu1Hj1WgAi2FH9f+GDM+J8Pj3LIUvPYpOPmIXe1WS6LV57R/ffCswWzasynhdCIRWIZj5YlbrVsnijARpeBV95oX9crh6FPIKEqyT8Eq/PEqmFirmgb233d4s0PYMIEEDjeNmKYDSyHau0iVT8FJ5ZAqX0sg4d6xU0vBbYVgvYPA8hh2H2hjXLfEnSjChCyFADlDy2FSPgUPunoOO59CppJOn8I7ue/4hu6FMHYI6Q8mAAAgAElEQVTR2KDzQlNoj3MGaDy0cbG4YnGg8ACvzvHNa3h4ysMs2rCIQf8bRPXnqjPof4P4Zvk39hDJz5cUn9z10cKP6PlRz6BrG3ZvKBZu3rp5QeeR8veKj6+IudMXQP9p/SNWxv/7/X9RfQq3f3l7xHvWZKnd+3dz6/hbi02SC8TJch7xUrZMWUeKaNW2VWGfdZPqz1XngckPBF17ZvozYXfcA8hZlWMfj100lnnr5tH/2/4xd6G77KPLot6PxsptK1m8cTGnvnEqF4+5mC17txQLE7j96BUfX8HFYy7m142/RlUWd0y4wz4OjPMf3/7DUd3w5LQnGTZnmH0+dtFY3s19l6E/Do35bCA79u0IOpcB4VcKDsz7cHWHF5Sa7qNM8Cnc9PlNrN2xlsfPfDzo+lXjruLKZlfa54WmkD6f9bHPrW6bSL8hXGH+ffvv9vG+wn2c/d7Z/LnbN8HmoSkPxZTV2u0pHHWq1LHjav9G8AjiSC3iwNnDsT6+UEVjcfqo0/nvVf+N+NyIeSMYftHwsPf6jO/DTW1uYuS8kbw5/01+XBe7uyxepXDJSZewr3Afu/bvKrbF4rUtruVf3/2rWBfcl1d/ae9OBvDd798B8MJ5LzD4+8GMu3Icw+eG/00A/Tr1o0O9DlzxcfEJUpHYsW8HQ2YPCboWOhEQwjdC+k/r72i+BcB/f4v8rpzQbFgz+3h2/mwuaHxB0P1ABWqVr5///Jnb60duHExYFl7x/eu7f3FirRNjyvT0jOAVCK4ad1XMZxKlyBQxbO4hBTRh2QQuPflSz9KzKH2WQpp9Cpv3Ft9HN5RI/eaRLIVwvym08nFzurw1eSmsLC5s+iMiEcMk23qP5/kiUxSXZVn/L/X56tqveL9n8f2qjz/ieLssvNrdZ8HVrVaXC0+8MGxc93e6n3UPrOO0+qdFtRAHdB1Az5N7RryfDIlY1ele0+hg0cGEewMS6Y7yktC8TNWCh6VGKWSKT8FJV0BEpRCHpRCqBNyc9OVkLkLU52N8tNGsOreGpDr1KSRSyUUqa1aa1lh/N8qkIClfHTWsHB5/X04tfWOMvXZYvKRjnaFohDZgVCl4RLrnKSSjFCLhxFJwcx+CRPdTsMM4sBQi4VY/v5NKpsgUuZpv1u+yyoDTii5aOBFJyw57mYrx/0uETFMKod+SKgWXyQSfAiQ3KSeejz8dloIxzj5IJ5aCk6WzkyHaBxY4eS0RJRSprNmWgn+pCDc+8kwp117jtOwbYxJuOGSaUlBLIUWk26fgZFKOJ91HKbAUnPbBx/povfQpBKYRiyJT5Koytd5fvN1H0cJlQrdo4LtK93LjyVgKqRrd4xT1KXhMJnw8kKRPIQlHcyoWknPaBx+z+ygFPgUnCjZhSyGWT8FvKbjRys8US8FrORxv3mNMwopJLQV/OilJJYNId0vGUgrhNraxXroXlkIqJmIVmkJnPoVY3Ucp8Ck4dTR76lNwainE8Cl4Rbq/lUDiWck3YUuhyDtLIaGRXOpT8JZMaVGVLVOWP3f9yZGDjwy6LgPELvj9F4UfB25trBNK6CQzgLu/ujtJSSMTaUmMPp/3oc/4PmHvBXLZ2OiTmr5f8z1v/PRG2Ht5W/Jixl9kiqg3pF7Ye9Y49R/Wht/9tczTZez3MHHZRGoMrBEzvVBi+hTKuGcpeMX1/73e3uwlFmMWjrHH0z/4TcS9spLi7q/uRgYIJ716UtSZ9Zv2bOKZX5/xRIZEePCbB8ldn5uQojrqhaOCzs9+72z2HtjrlmgRKTVKIV3Uqlwr6Lx25dr2xK9I7C+Kz4z9Ku+rYtdWb18dVxxuMGbhGCblTUo6nsBZp6E8O/PZmM//8ucvrN25Nuy9aauK78AWiXsn3RszTKd6nezjAV0HAHBMtWO4t8O9vH7h63So24EPLvsACPAphDiav7r2K46rclzENKJaCh4pltE/j3YcdlvBNvt470FvKq2V21YCsHTzUmasnuFJGl5gMJz+9umuWV3v/1x8DozblJoZzRbpdjQr3pNK/9FbPd6i6bCmnFz7ZI6odISd/kvdXgLg9vaHZteGWgrWoINuJ3RjUMtBXD7r8rBpZLqjWYlOor6pcKSiC6nUWArp+nhClVC6FrlSvMVJS9Aqg5al4MaaRlqWMp9EJ0GGQ5WCB6TaeRaaXip3DlO8J9ZS3+HCWgMJAueslInyKabL0ay4Q7zLpURDlYKLpKtFFdZS0A/5sMF6l44sBX8ZtMbDB1oK8ZaJnif3pODxgrieUdKDWgoZTqp9CmoppJ5UNgCstOKxFKzhwkFKIU5roIyUoULZCnHJqqQHNy2FVDQoS41SUJ+C4gXxbGlqhbWUQuDsdi0Thy/JzJ0IRS0FD0i3TyGTJgQdjjhdf8ktbEvBwXu1PmhrkpTT7iNVGCUf9SlkIOpTKB3c+PmNtBreKuy9fl/3cz0926fgQBFZo46sdfsDHc3qTD68qfrvqq7Eo0rBAzLBp6DWgncEbmMayguzXnA9vXgshQnXTOC+jvdxc5ubub3d7bxx8aFZ204mqPU4qUfEMKc3ON2pyEoJRpWCi2SST8HLNVaU+Dir0VlA4h9bPENSm/y1CUPOH0K5rHK8ftHr1P1LXfuek/J5/vHn8/EVH4e9d3mT8BPfYtHsr81iB1IyhlT0eJQapWCRbp9CkSmizYg2KZVBiYz1kSWqFOIZkupEjmhpREsnUfmdLOWuZA5qKbhIJvkUlMzBqnCTLR/Jdks6+dgDFUKoZZGo/E42fVIyB1UKHpBun0JJ3OLwcMaqTBPtXrS7jzy0QJ1U+GoplA5KvFIQkW4iskRE8kTkkTD37xeRxSLys4hMFZFjPZTFq6ijopZCySDRlnY8k9eiEW2ZC4toaSRavtVSKFmUaKUgIlnAa0B3oClwtYg0DQk2H2hvjGkJjAOe90oei0zwKSiZg919lGCl6ppPIcmVUK1hrvGilkLJokQrBeBUIM8Ys8IYsx/4ELgkMIAxZpoxZo//dDYQfmcUF1CfghKNZC2FZN+rk/SjKZ7d+3cnlK4bK7UqqaOkL3NRF1gTcJ7vvxaJm4Hiu8W4TLp9CqoUMotkRx9ZzyVbuUZTCvaeziIRHeO79u9KKF0rbkWxyIhmgoj0BtoDXSLc7wv0BahTpw45OTlxp7Fw00IA5s6dy45qOxIVNW7yVgZvH7l8xfKUpR0vx1U5jhW7V6Ql7Vrla7F5/+aoYY6peAzrCta5mu6WLVsAKCyMfy/mLrW7sHjOYno36M25dc5NqFxa7N59qKUfGs85Zc9hxdEraLyzMWV2laHHMT3oVb1XULjlqxIrV1s2b0noOSU6jas2ZtmuZa7Hu2DBAirmV3Q93kC8VAprgfoB5/X814IQkXOAx4Euxpiwm68aY0YCIwHat29vsrOz4xZmx5IdsAjatWtHu2Paxf18onxe8HmQvdSwYUNYlbLk4+L1nq9z/vvnJ/Rs26Pb8tMfPyWc9uDug7np85si3jdPGnbv3+3acgEWtWrVgq1gJH4L8p6z7qFrk650pWvScgRW8OHK98VcbB+fe9a5xe5/VvCZzxaPk9q1a0MUvXDucefyzYpv4o/YAdkNs8lZleNJ3Olm6QNLkQGJd/WceeyZYbcdbdGyBdmNs5OQLDZedh/NARqLSCMRKQ/0AsYHBhCRNsAIoIcxZoOHsqTNp7DnwJ6g80zuPkqmKyFRR6eFk+4bL5yi9h4HCcwyT4XTzymJ5n+sPmovf6N2XUUmUndkKgbKePbGjTEHgbuAycCvwFhjzCIReVpErEVcBgFVgY9FJFdExkeIzj25UuxTCN3IPKOVQpn0KQUnStuL4ZPxLH0dSiYpBWvjnniJle9eOjaTKW+HOxGVQgrqL099CsaYicDEkGtPBByf42X6gaRrnkJJshSSqeSSVgoO3k+mrRaaSUrBK0uhsCh+X4tT1FKITCSlkIr6I3NK9WFKwcHgLRMzWSmks/uoJJJRSsF4k/9evtdMyr9M47DsPspUUj15zdqk3SKjlUIau4/SRTLlIZMqtYQthRjdR6oU0kOkBloquo9KzVtJl6M5VAlk8tpHpdFSCFXa8ZBJlVrCPoVY3UdJ5E8sMin/Mg21FFJIqivlUKWQakuhQpbzzd2TsRSS/cAzRWnHQyZtk1mnSp2Enstdnxv1vpfKPtN8RJlEOh3NpUYppKsAuqkUqpSrEvczOTfm0LFeR/u8y7FdGHnRSD7v9XlQuLMbnc3JtU/msiaXxRV/3Wp1+c9l/+Hr3l/HLVvtyrUj3gscabTwjoX28XUtr+Phzg/HnVYk4nWkBu6W5nZL9/lznmfWzbMSenbguQMTem7F1hXc3/H+iPfbHtXWPr6uwXWc0eCMhNKx6Nu2L2ceeyaPnv5owqPJ7jrlroj3mh/ZPOz1kReNTCitQJ47+zmGdh9arNyeVOukpOMOpXxW+bDX1VLwgJK8IN4vd/wS9zNN/9qUWTfP4riaxwEwIHsAt7a7lR4n9cA8aey/KddPoXK5ynRtGN9ErPz787mmxTU0+WsTjq1efJHbaF1S/ToV3zO5jJTBPGlYctcSABrWaEizIw/tDvZez/d47pznHMl2cu2T7ePb2t3Gaxe8VixMvO+jU71OtD6qtS2rmzzY+cEgBR4PlctVto9rVKwRdK9VnfB7VlucVv80AJrUbhJ0/ckuT/KXCn+xz/s06sOMm2bwQKcH7Gvmyfi+pxEXj2D6jdN59uxnI84NaVijYdQ4Bp03KOK9X+74hWldphW7fmu7W+OSM5QeJ/Xg4dMf5q5T72LjgxuD7o24aIT9fblFJAtfRx+5SKZ0TyTVXZGAtWPvF+BwjZ9k8imcfNHWBArXGnJrKWoIVshFpihsV0i8febls8onvV6S14QqYqcT80Lz3BgT9j24VTFF6pqKVQbTke/R0vSiF6JC2fBKQbuPPKC0+RSsAmsV6lgfVDIfXLhnoymFQB9GpCWs3VLmkZRCvO+jQtkKxfI00wiVK5YTOlKlZjBh88dzpRCjkk1HAy/au/aiHGj3UQrIFJ9CMi81kY8hdGexmJZCEvkUTr5oSiFcXrj5wQc2AIwx4S2FOH0KJcJSKJOYpRCa98aYsO/IrYopUUshHd9yVEsBcb2yjqgU1FJwn5T7FNK8n0ImWwrhCnjoB+/WR1BEUVgFEO/7CPxYM1YphHYfJThc1WtLIZJcaimopZASSrtPwalSSKVPIRCrsLtqKXjgU6iQlfndR4laCqEYk57uIy+t2USJVi69kCeSUlBHswek06cgSNosBaddHqm0FAKx3kum+xQCu48ylXh9ChbFHM1eWwoRlFXM7qMMtBTcVgzafZQCMsGnkFUmK6kPKpEKO15LIZnlqSuWLb75h9MJcVa+hIsjUapXrH5IDslypQyUyyp3aF/mDJudbo37b1mnZdD1SuUqRX0uUoPBGBP2/bm1ummk4cqRRt5YlAafQqWy4d9ZKrqPMmLntVSSjnkKJxxxAte2uJY3f3qzmFK4uc3NHCw6SI+TenB2o7NZu3MtzYb5xuXX/0t9erfsTbcTujHz95nUrVaXl7u9zC9//sIJR5xA7cq1ueWLWwDfmP/BswYDcF/H+3hx9otA8VE9sT6oXs178db8t8Ju8BHKM82eCTr/9MpP6fpuV65vdT3rd63njZ/eoFyZcjzT9RmOqXYM2wu206B6A/728d/svLGoVLYST2c/7Xjy3MybZnL6qNPt8yHnDeGzJZ/R5dguPDPDJ9d/r/ovz3z2DJWPrMw/zvwH5bPKs2XvFi4+8WLOe/88wLeJTO76XO7tcC8Nqjdg2eZlDJ83HIDHTn+MFnVaULlcZc5ocIbdsrWHzaa4LMXip9t+YnLeZG5rfxvD5gzj4Sm+SX7TbphG9jvZ/LHrj2LPPHjag/Zx4yMa0/Pknvzru38BPqU3IHsAL//wctAz/zzrn3y78lt7vsKr3V/lrq98E8q6ndCNJrWbULlcZZr9tRnXfHoNAG/1eKvY4pBjLh9Dw5cbFpPpnUve4ZUfX2HTnk18ufRLjqxyJN1P6M67C94tFnbUJaOoVr4alcpVYuPujcXuX9PiGs5qeBbgKyMDpg9g+77t4TPQT4WsCuwrDN7vK1QpfHfTd5wx6oxi9zrX72zny8iLRnJU1aMYu3gs7//8PuD7NpduXkqrOq14duazgG+ew7HVj+WJnCeoVr4aTf/alNva38bv239nYt5ECg4WkLfFt4NjiV86O5NIp0+hSe0mPJX9FG/Pf7vYS32zx5tB59UrVuf9U9+n94+92V+4n2fP9hWcM489E4B7Otxjh91zYI+tFAadN8hWCkPOH3JIKYS0AmNVZOWzyvPS+S/RdmTbqOEGnjOQUw+cGnStUc1GrPr7KgCWbl7KGz+9gYjwjzP/YYeJ1pXRv0t/+ziWnJ0bdA4679OmD/d1ug/AVgr1/lKPqxtcHbST2SvdXwl6zsqXOlXqcH+n+/nlz18YPm84x9c8nn+d/a+waWeqpdD8yOb2jN6HOj9kK4UTa51Iv9P68cDXhyad9W7Zm/d/fp8WR7awr4kI/zzrn4eUgjFB1pZFjYo1WHznYvv8zlPvtJXChGsmBFWSllLo06ZPsXiOrVF8siNA3b/UZdQlo8jbkseXS7+kavmqvHPpO2GVQq/mvcJalze0uoF3F7zLOY3O4aY2vh397ut0H1NWTmHisol8cfUXXDzm4mLPAXxy5SdcNOaioGuhSuH0BqdTtXxVdu3fFdTQeq/ne/ZENmvC3MUnXWwrhSHnD7HDWkqhb7u+AJx/QvCuhwPPHcjAcwcydcVUzhnt22VAHc0ekA6fQqBj0kn3Ufkyvv7E/YX7o4Zz0p0U6lNwkr6jvQ1iKNlIjuNAH0PQkNEI78VpV0HSay/FYVFlqqUQD1bXTTRHe7qUnpWvTgYfxFrEMfQ9Wl1s0Ron4d592GsBDS4v88rJd+ImpUYppNOnENifH49SCDVhQ3Fi/RSzFBwUKieLoMXKz4iO40gTpSJUsE4r3kSVQmj8ThzymWopRCP0d9pKIWCYbmiYRJSeF/NMopW1eP0bls8s3hFZZcJUlVYZSWUvhI4+8oB0zFNIWCkcjK4U4rIU4th20pFSSNBScEq8StytVVpDLatoYUuypWDlV6EpjDqjOV7cbHw5KUPxli9HlkKYOMOVr8AegFQpBu0+cpF0+hTiVQrlxFdwY42hj6ebx5LBNaWQoKUQK7x9HmfhT9hSSKDbqkRaCiGyWi3saDO60630nJSheBWaE0shXJzhypdtKYikrCxo95EHpMWnENAKDayUx1w+Juwzlml/VNWjosYdj6Xg1NEMqbMUAmWJJFeqfQrxtE7TXWkmQ0b7FHDuU4hF6LNOLIVwhLUUwnQ1et341CGpLpIpPoXAD61X815hnxERPr3yU9oeHX0EUDwFMB5Hc6LLIgSSrKUQL275FCw5DnufwmFiKcSLrRSiWQoOu4/S4VPQIakekI55CqHdRw2qN4i5b0HPJj1jxm19LIFr3kfCWm/frQ8s1iQ36zdHmoQTC+t5pzvHJfq7rN9hVRaW0owWn7VvRNXyVRNKMx2Evi+rPJSRMvZvD500Fk8FJLjfhWJVtomUIcsvF+qITvSdhRv2elTVo9i4ZyMiYsvoReMzUCGlwtFcapSCNXHm3QXv0qVhl5SlG86nEKgokmX4hcPp2ii6ggHf+OmhPwx1tInLWY3O4qHTHmJrwVaMMRxb41j6Tzs0h+D+jvdzS9tbmD1zdsQ4mtRuQv8z+3Nzm5uL3Rt1ySjq/6U+P//5s30tVFk3rNGQJ7s8yfWtro+Yxovnv8gRlY7gj51/BOXnF1d/we79u6P+xnFXjKNcVjnOanQWBQcLuOOUO4LkiNb6e/3C17mg8QW0P6Z91DQyidvb3859k++zzx8/43EOFB7g1na3Uq5MOR47/TF7nscpx5zCnHVz7Lx48+I3aVyrMUUrI1dIubfnMjlvcrHrwy8cHnE3NIDZN8/mx7U/sm7nOrqd0I3pq6fbW4ueXPtk+p/Z357jMPfWucz8faaj3/v8uc9TvWJ1rmp2VdD1J7o8QZEpok+bPrQ7uh1z182lUc1G7Ni3g2OqHcPSzUuLVextj27LU9lPFf9tFw3nvQXvcXzN4/ni6i8Y/fPosBtNgW8iZahVNqLtCHbXjl5OwbdbYt+2ffllwy80qtEoZviksZbHLSl/7dq1M4nw8uyXDU9heIqEnk+URi81Mr0/7W2MMabl6y3NpR9eauq+UNf0+axPxGemTZuWUFqBv8/t32rFFxhnonJaDP1hqB3n6AWjk5QwPPHKOP+P+YanMC1fb+mJPJFINi/DEfq+7pl4j6Ny8dKslwxPYe6eeLfnMiZKtN+RrJyT8yaHLe9uk+r8BOYaB3VsqXE0X9fyOgBub3d7StMNaylgdNNygvdhNhnitLXkyPRF7xLBafdOqNO9tHE4vvt4KDVKoWalmlQoU4FqFaqxaMMithVsS0m6gfMUrFVSjTGlvuCB8xVUU4kTR/PhjptbopZESnuDrVSV/HJlylFwsIDmrzfnrHfPSkmaRabIng2ZVSaLg0UHgxRFaSbSkhfpxImjuaTitOVf2i2F0o6nNZOIdBORJSKSJyKPhLlfQUQ+8t//QUQaeilP+TLl2bV/FwDz18/3MimbwLWP1u1cx6S8Sew5sOewrHTiJXBETKZUQNp9pJbC4fju48EzpSAiWcBrQHegKXC1iDQNCXYzsNUYcwLwIjDQK3nAN1N4897N9vmKrSu8TA4I9ilYrdAd+3aU+oIHIT6FDKmAvBgbnymopeCMw/Hdx4OXlsKpQJ4xZoUxZj/wIXBJSJhLAGs93HHA2eLhG6mQVYFFGxbZ582HHRoqt71guycfgQkYfjruinH29Z37d7qeVkkjmQ19vMIqA6W5e6+0WwqlHS89fXWBNQHn+UCHSGGMMQdFZDtQC9jkhUCn1DyFT9Z+Yp/vPbiXukPqcqDwABv3bKR8Vnka1WjEgaIDHCg8wIGiA2RJVtg15Z2ycc9G+yM7te6h/QcmLJuQ+A85TAiclJQpTmdrslP1Com/80wl1g5sFtYktkhbQh7uZEpZTBfilYkoIn8DuhljbvGfXwd0MMbcFRBmoT9Mvv98uT/MppC4+gJ9AerUqdPuww8/TEim1VtW89XWr/go/yMAWlZvSb1K9SgjZZi8fjKNqzbmyIpHUlbKUlbK2o7hgsKCGDFHQeDyupfTorpvM5Mt+7fwzOJnuOuEuzi+6vFhH9m1axdVq8Y/83Ly+skcU+kYWlRvQd6uPH7Z/gs968aeGe2EJTuXMHH9RBpUasDl9S5PSk6LQlPIq3mvsq9oH/c3vt+TjzFeGY0xfLjmQ7od1Y2a5Wu6Lk8kks3LcPy09SeyJItWNVoB8MfeP7hz/p30Pa4v3Y7qFvG5A0UHGLVqFNc2uJYqZat4KmOiLNi2gD8K/gj7O5KVs8gUMWrVKI6t7JuIdk6dcxKOKxqpzs+uXbvOM8bEnnHpZDJDIn9AJ2BywPmjwKMhYSYDnfzHZfFZCBIt3kQnrxmTWZNvoqFyukdJkNGYkiFnSZDRGJUzEmTA5LU5QGMRaSQi5YFewPiQMOOBG/zHfwO+9QuvKIqipAHPOs+Mz0dwFz5rIAt42xizSESexqexxgNvAaNFJA/Ygk9xKIqiKGnCU4+KMWYiMDHk2hMBxwXAFV7KoCiKojin9I67UxRFUYqhSkFRFEWxUaWgKIqi2KhSUBRFUWxUKSiKoig2ns1o9goR2QisTvDx2ni0hIbLqJzuURJkhJIhZ0mQEVTOSBxrjPlrrEAlTikkg4jMNU6meacZldM9SoKMUDLkLAkygsqZLNp9pCiKotioUlAURVFsSptSGJluARyicrpHSZARSoacJUFGUDmTolT5FBRFUZTolDZLQVEURYlCqVEKItJNRJaISJ6IPJJGOeqLyDQRWSwii0TkXv/1I0TkGxFZ5v+/pv+6iMgrfrl/FpG2KZY3S0Tmi8iX/vNGIvKDX56P/MuiIyIV/Od5/vsNUyRfDREZJyK/icivItIpE/NSRO7zv++FIjJGRCpmQl6KyNsissG/4ZV1Le78E5Eb/OGXicgN4dJyWcZB/nf+s4j8V0RqBNx71C/jEhE5P+C6p3VAODkD7j0gIkZEavvP05KXjnCy6UJJ/8O3dPdy4DigPLAAaJomWY4G2vqPqwFLgabA88Aj/uuPAAP9xxcAXwECdAR+SLG89wMfAF/6z8cCvfzHw4E7/Mf/Bwz3H/cCPkqRfO8Ct/iPywM1Mi0v8W07uxKoFJCHN2ZCXgJnAm2BhQHX4so/4Ahghf//mv7jmh7LeB5Q1n88MEDGpv7vuwLQyP/dZ6WiDggnp/96fXxbCKwGaqczLx39jlQmlq4/HOwCl0bZPgfOBZYAR/uvHQ0s8R+PAK4OCG+HS4Fs9YCpwFnAl/4CvCngY7TzlQR20XNBvur+ylZCrmdUXnJoL/Ij/HnzJXB+puQl0DCkwo0r/4CrgREB14PCeSFjyL2ewH/8x0HftpWXqaoDwskJjANaAas4pBTSlpex/kpL95H1UVrk+6+lFX+3QBvgB6COMeYP/631QB3/cTplfwl4CCjyn9cCthljDoaRxZbTf3+7P7yXNAI2AqP8XVxvikgVMiwvjTFrgcHA78Af+PJmHpmVl4HEm3/p/r764Gt1E0WWtMgoIpcAa40xC0JuZZScgZQWpZBxiEhV4BPg78aYHYH3jK+JkNZhYSJyEbDBGDMvnXLEoCw+c/11Y0wbYDe+7g6bDMnLmsAl+JTYMUAVoPiO8xlIJuRfNETkceAg8J90yxKKiFQGHgOeiBU2kygtSmEtvn49i3r+a2lBRMrhUwj/McZ86r/8p4gc7b9/NLDBfz1dsncGeojIKuBDfF1ILwM1RMTasS9QFltO//3qwGaPZcwH8o0xP84vgLUAAANpSURBVPjPx+FTEpmWl+cAK40xG40xB4BP8eVvJuVlIPHmX1ryVURuBC4CrvUrr0yT8Xh8DYEF/u+oHvCTiByVYXIGUVqUwhygsX+0R3l8zrvx6RBERATf3tS/GmOGBNwaD1gjDW7A52uwrl/vH63QEdgeYNp7hjHmUWNMPWNMQ3z59a0x5lpgGvC3CHJa8v/NH97TFqYxZj2wRkRO8l86G1hMhuUlvm6jjiJS2f/+LTkzJi9DiDf/JgPniUhNv1V0nv+aZ4hIN3xdmz2MMXtCZO/lH8HVCGgM/Ega6gBjzC/GmCONMQ3931E+vkEm68mgvAwneKn4w+ftX4pvBMLjaZTjdHzm+M9Arv/vAnx9xlOBZcAU4Ah/eAFe88v9C9A+DTJnc2j00XH4PrI84GOggv96Rf95nv/+cSmSrTUw15+fn+EbsZFxeQkMAH4DFgKj8Y2OSXteAmPw+TkO4Ku0bk4k//D16+f5/25KgYx5+PrerW9oeED4x/0yLgG6B1z3tA4IJ2fI/VUccjSnJS+d/OmMZkVRFMWmtHQfKYqiKA5QpaAoiqLYqFJQFEVRbFQpKIqiKDaqFBRFURQbVQqKEgYRaRhutcso4W8UkWMchHk1eekUxTtUKSiKO9yIbwkLRSnRqFJQlMiUFZF3/evdj/PPSH5CROaIb1+Ekf4ZqX8D2gP/EZFcEakkIqeIyPciskBEfhSRav44jxGRSf618p9P429TlLCoUlCUyJwEjDTGtAR24Nvn4FVjzCnGmOZAJeAiY8w4fLOqrzXGtAYKgY+Ae40xrfCtfbTXH2dr4CqgBXCViNRHUTIIVQqKEpk1xpj/+Y/fx7dESVfx7Yb2C75FApuFee4k4A9jzBwAY8wOc2iJ7KnGmO3GmAJ86x8d6+1PUJT4KBs7iKKUWkLXgDHAMHzr1KwRkafwrVMUD/sCjgvRb1DJMNRSUJTINBCRTv7ja4CZ/uNN/v0w/hYQdie+7VXBv4uWiJwCICLVApbIVpSMRguqokTmN+AGERmBb8XQ1/GtwvoLvhUv5wSEfQcYLiJ78W39eBUwVEQq4fMnnJM6sRUlcXSVVEVRFMVGu48URVEUG1UKiqIoio0qBUVRFMVGlYKiKIpio0pBURRFsVGloCiKotioUlAURVFsVCkoiqIoNv8PFeMMDl3wxLwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12c05dc5b70>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "resnet(num_class=5,epochs=20)\t#binary classificaiton\n",
    "history_this.loss_plot('epoch')\n",
    "history_this.loss_plot('batch')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Validation accuracy of Resnet: 98.652%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model 6: Inception"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def inception( num_class, epochs,savepath='./inception.h5',history=history_this,input_shape=INPUT_SIZE):\n",
    "    model_fine_tune = InceptionV3(include_top=False, weights='imagenet', input_shape=input_shape)\n",
    "    for layer in model_fine_tune.layers[:35]:\t#the number of frozen layers for transfer learning, have tuned from 50-150\n",
    "        layer.trainable = False\n",
    "    for layer in model_fine_tune.layers[35:]:\t#the number of trainable layers for transfer learning\n",
    "        layer.trainable = True\n",
    "    model = GlobalAveragePooling2D()(model_fine_tune.output)\n",
    "    model=Dense(units=256,activation='relu')(model)\n",
    "    model=Dropout(0.5)(model)\n",
    "    model = Dense(num_class, activation='softmax')(model)\n",
    "    model = Model(model_fine_tune.input, model, name='resnet')\n",
    "    opt = keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08)\t#tuned learning rate to be 0.001\n",
    "    model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy']) #set the loss function to be binary crossentropy\n",
    "    #train model\n",
    "    earlyStopping = kcallbacks.EarlyStopping(\n",
    "        monitor='val_acc', patience=2, verbose=1, mode='auto')\t#set early stop patience to save training time\n",
    "    saveBestModel = kcallbacks.ModelCheckpoint(\n",
    "        filepath=savepath,\n",
    "        monitor='val_acc',\n",
    "        verbose=1,\n",
    "        save_best_only=True,\n",
    "        mode='auto')\n",
    "    hist = model.fit_generator(\n",
    "        train_generator,\n",
    "        steps_per_epoch=len(train_generator),\n",
    "        epochs=epochs,\n",
    "        validation_data=validation_generator,\n",
    "        validation_steps=len(validation_generator),\n",
    "        #use_multiprocessing=True, \n",
    "        callbacks=[earlyStopping, saveBestModel, history],\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0314 - acc: 0.9910Epoch 00001: val_acc improved from -inf to 0.61608, saving model to ./inception.h5\n",
      "183/183 [==============================] - 163s 892ms/step - loss: 0.0315 - acc: 0.9909 - val_loss: 6.0914 - val_acc: 0.6161\n",
      "Epoch 2/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0145 - acc: 0.9932Epoch 00002: val_acc improved from 0.61608 to 0.99973, saving model to ./inception.h5\n",
      "183/183 [==============================] - 92s 500ms/step - loss: 0.0144 - acc: 0.9932 - val_loss: 0.0021 - val_acc: 0.9997\n",
      "Epoch 3/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0019 - acc: 0.9995Epoch 00003: val_acc improved from 0.99973 to 1.00000, saving model to ./inception.h5\n",
      "183/183 [==============================] - 93s 509ms/step - loss: 0.0019 - acc: 0.9995 - val_loss: 5.0573e-05 - val_acc: 1.0000\n",
      "Epoch 4/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0039 - acc: 0.9994Epoch 00004: val_acc did not improve\n",
      "183/183 [==============================] - 93s 506ms/step - loss: 0.0039 - acc: 0.9994 - val_loss: 0.2731 - val_acc: 0.9811\n",
      "Epoch 5/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 3.7142e-04 - acc: 0.9999Epoch 00005: val_acc did not improve\n",
      "183/183 [==============================] - 93s 507ms/step - loss: 3.6954e-04 - acc: 0.9999 - val_loss: 9.7136e-05 - val_acc: 0.9999\n",
      "Epoch 00005: early stopping\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VPXZ///XNUlIWEIIW0AIAqUuLFkAIcoWRBTQcrOIgLLOVOu3alVuabG39ra37V212Fr90e9dbgkCImBBLQKFr3gTQG9jBQFBwIqALCKLkJAYtmQ+vz9mMmRnMuTMmeV69jHNzJlzzuc9B+eak3Mm1xFjDEoppSKfw+4ASimlgkMLvlJKRQkt+EopFSW04CulVJTQgq+UUlFCC75SSkUJLfhKKRUltOArpVSU0IKvlFJRItbuAOW1bNnSdOzYMaBlv//+exo3bly/geqB5qobzVU3mqtuIjHX1q1bTxljWvk1szEmZG69evUygdqwYUPAy1pJc9WN5qobzVU3kZgL2GL8rLF6SEcppaKEFnyllIoSWvCVUipKhNRJW6VU5Lp06RJHjhzh/PnztmVISkpiz549to1fE39yJSQk0L59e+Li4gIeRwu+Uioojhw5QmJiIh07dkREbMlQWFhIYmKiLWPX5kq5jDF89913HDlyhE6dOgU8jh7SUUoFxfnz52nRooVtxT6ciQgtWrS46t+OtOArpYJGi33g6mPbhX3BP3fuHLNnz2bbtm12R1FKqZBmacEXkWYislxE9orIHhG5ub7HiIuL4w9/+APLly+v71UrpSJEfn4+f/7znwNadsSIEeTn5/s9/zPPPMPs2bMDGstqVu/h/wlYa4y5AUgH6v30eGxsLFOnTiUvL49jx47V9+qVUhGgtoJfUlJS67Jr1qyhWbNmVsQKOssKvogkAQOBeQDGmIvGGP8/Jutg+vTpuN1uFi5caMXqlVJhbtasWXz11Vf069ePmTNnkpuby+DBg7n33ntJS0sDYNSoUfTq1Ytu3boxd+5c37IdO3bk1KlTHDx4kBtvvJH777+fbt26cfvtt3Pu3Llax92+fTtZWVmkpaUxevRozpw5A8DLL79M165dSUtLY8KECQBs3LiRjIwMMjIyyMzMpLCwsN63g3haMdQ/EckA5gK78ezdbwUeNcZ8X2m+B4AHAFJSUnotXbo0oPEefvhhzp49y4IFC0LqxFBRURFNmjSxO0YVmqtuNFfdVJcrKSmJLl26APCLDb9g58md9Tpmj1Y9eH7w89U+9/XXX3PPPffwv//7v8TExLB582bGjRtHXl4eZQ0bT58+TfPmzTl37hzZ2dmsWbOGFi1a0L17dzZu3EhRUREZGRls3LiRtLQ0pk6dyvDhw30Fu8x//ud/0qRJE372s59x88038/vf/57+/fvzm9/8hsLCQp5//nmuu+46du7cSXx8PPn5+SQmJjJx4kRmzJhBVlYWRUVFJCQkEBtb8Zvz+/bto6CgoMK0wYMHbzXG9PZnG1n5PfxYoCfwiDHmYxH5EzALeLr8TMaYuXg+GOjdu7fJzs4OaLC77rqL559/nri4OPr3739VwetTbm4ugb4mK2muutFcdVNdrj179vi+a96gQQNiYmLqdcwGDRrU+F32Jk2a4HA4iImJITExkUaNGtGnTx969Ojhm+fFF1/k7bffBuDo0aN8++23vr8ZKPvw6tSpE/369QOgb9++HD9+vMqY8fHxxMfH43a7OXv2LMOHDwfggQceYNy4cSQmJpKens6DDz7IqFGjGDVqFMYYBg0axFNPPcV9993HmDFjSE5OrvI6EhISyMzMDHgbWVnwjwBHjDEfex8vx1PwLTFo0CDmzJnDvHnzQqrgK6WqemnYS3ZHqNCOODc3l/Xr1/PRRx/RqFEjsrOzq/3Oe3x8vO9+TEzMFQ/p1GT16tVs2rSJlStX8uyzz5KXl8esWbO48847WbNmDVlZWaxfv54bbrghoPXXxLJj+MaYb4HDInK9d9IQPId3LNGwYUMmTJjAm2++acmxL6VU+EpMTKy1LhQUFJCcnEyjRo3Yu3cveXl5Vz1mUlISycnJbN68GYBFixYxaNAg3G43hw8fZvDgwbzwwgvk5+dTVFTEV199RY8ePfjFL35B79692bt371VnqMzqb+k8AiwWkc+ADOA/rRzM5XJRXFzMsmXLrBxGKRVmWrRoQb9+/ejbty8zZ86s8vywYcMoKSkhLS2Np59+mqysrHoZd8GCBcycOZO0tDS2b9/Or371K0pLS5k0aRI9evQgMzOTxx9/nGbNmvHSSy/RvXt30tLSaNiwoe9QUL3yt3F+MG5XewEUt9ttunbtam6++eaA11PfIvGCC1bSXHUTTrl2794d/CCVnD171u4I1fI3V3XbkGi9AIqI4HQ6+eijj0KyI55SStkpogo+wOTJk4mNjSUnJ8fuKEopFVIiruC3bt2aH/3oRyxcuJBLly7ZHUcppUJGxBV88Jy8PXHiBKtWrbI7ilJKhYyILPh33HEHbdu21cM6SilVTkQW/NjYWKZNm8aaNWv45ptv7I6jlFIhISILPoDT6dSGakopILjtkUNZxBb8Ll26MHDgQHJycjAWNYhTSoUHbY/sEbEFHzwnb7/88kvfnzYrpaJTMNsjv/vuu/Tt25fMzExuu+02jh8/Dng6iE6fPp0ePXqQlpbGihUrAFi7di0DBgwgPT2dIUOGWLodrGyeZruxY8fy8MMPk5OTw8CBA+2Oo5Qq89hjsH17/a4zIwNeqr4p23PPPceuXbv48MMPSUxMJDc3l3/84x/s2rWLTp06AZCTk+Nrj3zTTTcxduxYWrRoUWE9X375JUuWLOG///u/ueeee1ixYgWTJk2qME///v3Jy8tDRHj11Vd54YUXePHFF3n22WdJSkpi505PW+gzZ85w8uRJ7r//ftasWUOPHj04ffp0/W6TSiK64Ddu3JiJEyfy+uuv8/LLL9O0aVO7IymlQkSfPn18xR48FyUpa498+PBhvvzyyyoFv1OnTmRkZADQq1cvDh48WGW9R44cYfz48Rw7doyLFy/6xli/fj3lr/eRnJzMu+++y8CBA309+Zs3b16fL7GKiC744Dl5O3fuXJYtW8b9999vdxylFNS4Jx5MVrVHfuSRR5gxYwYjR44kNzeXZ555xpL8gYjoY/jg+RTv1q0b8+bNszuKUsomwWyPXFBQQLt27QBPt8wyQ4cOZc6cOb7HZ86cISsri02bNvl+U7D6kE7EF3wRweVy8fHHH/P555/bHUcpZYNgtkd+5plnGDduHAMGDKBly5a+6U899RRnzpyhe/fupKens2HDBlq1asXcuXOZNGkS6enpjB8/PuBx/eJvW81g3K62PXJNTpw4YeLi4syMGTMCXn+gwql9bSjQXHUTTrm0PXLNtD1yPWrVqhUjR45k4cKFXLx40e44Silli6go+OA5eXvq1CltqKaUilpRU/DvuOMO2rVrpydvlVJRK2oKfkxMDNOmTWPt2rUcPXrU7jhKKRV0UVPwAaZPn47b7a7wVSmllIoWUVXwf/CDH5Cdna0N1ZRSUSmqCj54Tt5+9dVXbNq0ye4oSqkgCWZ75GeeeYbZs2cHNJbVoq7gjx07lqZNm+rJW6WiiLZH9rC04IvIQRHZKSLbRWSLlWP5q1GjRkycOJHly5dTUFBgdxylVBAEsz1yedu3bycrK4u0tDRGjx7NmTNnAE+jtq5du5KWlsaECRMA2LhxIxkZGWRkZJCZmVlrK4hABaN52mBjzKkgjOM3l8vFX/7yF5YuXcpPfvITu+MoFXUeW/sY27+t3/bIGW0yeGmY/e2Ry5syZQqvvPIKgwYN4le/+hW//vWveemll3juuec4cOAA8fHxvsNFs2fPZs6cOfTr14+ioiISEhLqactcFnWHdAB69+5Njx499CLnSkWx6tojp6enk5WV5WuPXJk/7ZHLFBQUkJ+fz6BBgwCYOnWq79xhWloa9913H6+//jqxsZ797n79+jFjxgxefvll8vPzfdPrk9V7+Ab4fyJigL8YY+ZeaYFgEBGcTiePP/44u3btonv37nZHUiqq1LQnHkxWtUf2x+rVq9m0aRMrV67k2WefJS8vj1mzZnHnnXeyZs0asrKyWL9+PTfccENA66+J1QW/vzHmqIi0Bt4Tkb3GmApfjxGRB4AHAFJSUsjNzQ1ooKKiojot26lTJ2JjY/n1r3/NQw89FNCYVuQKFs1VN5qrbqrLlZSUZMlxaX+dPXuW0tJSCgsLKS4upqSkxJfn22+/JTExkdLSUrZu3UpeXh7FxcUUFhZijKGoqIiioiLcbrdvmQsXLnDhwoUqr+nChQvExcXhcDhISkpi3bp13HLLLbz66qvcfPPNFBQUcPjwYXr37k16ejqLFy/m7NmzHDp0iM6dO/PTn/6UzZs3s23bNl+b5TLnz5+/qn9vSwu+Meao9+cJEXkb6ANsqjTPXGAuQO/evU12dnZAY+Xm5lLXZUeNGkVubi6LFy+mQYMGAY1rRa5g0Fx1o7nqprpce/bsITEx0ZY8iYmJ9O/fn1tuuYU777yTO++8k9jYWF+e0aNHs2DBAvr168f1119PVlYWjRo1IjExERGhSZMmADgcDt8y8fHxXLp0qcprio+PJz4+nsTERBYtWsSDDz5IcXExnTt3Zv78+TRq1IgHH3yQgoICjDHMmDGD5s2bM3v2bDZs2IDD4aBbt26MGTOmwm8UAAkJCWRmZga8HSwr+CLSGHAYYwq9928H/sOq8QLhcrlYvnw5K1eu5O6777Y7jlLKQm+88QaFhYW+Al3+Ayk+Pp6///3v1S5Xdpy+ZcuW7Nq1yzf9iSeeqHb+8le4ysjIqPZiKh988EGFx4WFhbzyyiv+vIyrYuVJ2xTgAxHZAfwDWG2MWWvheHU2dOhQ2rdvrydvlVJRwbKCb4zZb4xJ9966GWN+a9VYgSprqLZu3TqOHDlidxyllLJUVH4tszxtqKaUihZRX/A7d+7M4MGDycnJwe122x1HKaUsE/UFHzwnb/fv38/GjRvtjqKUUpbRgg+MGTOGpKQkPXmrlIpoWvCBhg0bcu+997J8+fI6tUFVSkWusu/eRxIt+F5Op5Pz58+zdOlSu6MopZQltOB79erVi7S0NO2Tr1QEmjVrFnPmzPE9LrtISVFREUOGDKFnz5706NGDv/3tb1dcV01tlNeuXUvPnj1JT09nyJAhgKfFxPTp0+nRowdpaWmsWLGi/l9cHQSjPXJYEBFcLhePPvoon332ma9HtlKq/j32GGyv3+7IZGTASzX0ZBs/fjyPPfYYU6ZMAeDNN99k3bp1JCQk8Pbbb9O0aVNOnTpFVlYWI0eORERqHKe6Nsput5v777+fTZs20alTJ06fPg3As88+S1JSEjt37gTw9cO3i+7hl3PffffRoEEDPXmrVITJzMzkxIkTHDt2jB07dpCcnExqairGGH75y1+SlpbGbbfdxtGjRzl+/Hit66qujXJeXh4DBw70tVtu3rw5AOvXr6/QnDE5Odm6F+kH3cMvp0WLFowaNYpFixbx/PPPV2lcpJSqHzXtiVtp3LhxvPPOO+Tn5zN+/HgAFi9ezMmTJ9m6dStxcXF07Nix2rbIZfxtoxyqdA+/EqfTyenTp1m5cqXdUZRS9Wj8+PGsWLGC5cuXM27cOMBzkZLWrVsTFxfHhg0b+Prrr2tdR0FBAcnJyTRq1Ii9e/f6GqNlZWWxadMmDhw4AOA7pDN06NAK5w70kE6Iue2220hNTdWTt0pFmG7dulFUVES7du1o27Yt4DmMu2XLFnr37s3ixYuveMGRYcOGUVJSQlpaGk8//TRZWVkAtGrVirlz5zJmzBjS09N9v0E89dRTnDlzhu7du5Oens6GDRusfZFXoId0KilrqPab3/yGw4cPk5qaanckpVQ9ycvLq9C/vmXLlnz00UfVzltUVFRlWm1tlIcPH87w4cMrTGvSpElI9enSPfxqTJ8+HWMMr732mt1RlFKq3mjBr0anTp0YMmQI8+fP14ZqSqmIoQW/Bk6nkwMHDoTk9UKVUioQWvBrMHr0aJo1a6Ynb5VSEUMLfg3KGqqtWLHC9q9SKaVUfdCCXwuXy8WFCxdYsmSJ3VGUUuqqacGvRc+ePcnIyNBWC0pFoZraI4dz22Qt+FfgdDrZunUrO3bssDuKUkpdFS34V6AN1ZQKf/XZHrmMMYaZM2fSvXt3evTowbJlywA4duwYAwcOJCMjg+7du7N582ZKS0uZNm2ab94//vGP9f4a/aF/aXsFzZs3Z/To0bz++uu88MIL2lBNqXrw2GOPsb2e+yNnZGTwUg1d2eqzPXKZt956i+3bt7Njxw5OnTrFTTfdxMCBA3njjTe44447+Ld/+zdKS0spLi5m+/btHD16lF27dgHYdmU9y/fwRSRGRLaJyCqrx7KKy+Xi9OnTvPPOO3ZHUUoFoD7bI5f54IMPmDhxIjExMaSkpDBo0CA++eQTbrrpJubPn88zzzzDzp07SUxMpHPnzuzfv59HHnmEtWvX0rRpU4tfcfWCsYf/KLAHsOcV1oMhQ4bQoUMHcnJyfE2RlFKBq2lP3Er10R7ZHwMHDmTTpk2sXr2ayZMnM3PmTKZMmcKOHTtYt24dc+bM4c0337TlMLGle/gi0h64E3jVynGs5nA4mD59Ou+9994V26cqpUJTfbRHLm/AgAEsW7aM0tJSTp48yaZNm+jTpw9ff/01KSkp3H///bhcLj799FNOnTqF2+1m7NixPPvss3z66adWvcxaWX1I5yXg50DYN6SZPn06QEh1vlNK+a8+2iOXN3r0aNLS0khPT+fWW2/lhRdeoE2bNuTm5pKenk5mZiYrVqzg0Ucf5ejRo2RnZ5ORkcG0adP43e9+Z9XLrJUYY6xZschdwAhjzE9FJBt4whhzVzXzPQA8AJCSktJr6dKlAY1XVFRk+fdjn3jiCY4ePcrixYtxOPz7rAxGrkBorrrRXHVTXa6kpCS6dOliUyKP0tJSYmJibM1QHX9z7du3j4KCggrTBg8evNUY09uvgYwxltyA3wFHgIPAt0Ax8Hpty/Tq1csEasOGDQEv668lS5YYwLz33nt+LxOMXIHQXHWjueqmuly7d+8OfpBKzp49a3eEavmbq7ptCGwxftZlyw7pGGOeNMa0N8Z0BCYA/2OMmWTVeMEwatQokpOT9Tv5SqmwpH94VQcJCQncd999vPXWW9pQTakAGIsOIUeD+th2QSn4xphcU83x+3DkdDq5cOECb7zxht1RlAorCQkJfPfdd1r0A2CM4bvvviMhIeGq1qN/aVtHmZmZZGZmMm/ePB566CG74ygVNtq3b8+RI0c4efKkbRnOnz9/1UXTCv7kSkhIoH379lc1jhb8ALhcLh5++GG2bdtGZmam3XGUCgtxcXF06tTJ1gy5ubkh+Z4NVi49hh+Ae++9l/j4eD15q5QKK1rwA5CcnMyYMWNYvHjxVf8ZtlJKBYsW/AA5nU7OnDmjDdWUUmFDC36Abr31Vq699lq9yLlSKmxowQ9QWUO1999/n4MHD9odRymlrkgL/lUoa6j22muv2RtEKaX84FfBF5F+ItLYe3+SiPxBRK61Nlro69ChA0OHDmX+/Pm43WHfEFQpFeH83cP/v0CxiKTjaXf8NbDQslRhxOl0cujQId5//327oyilVK38Lfgl3q5s/wL8yRjzJyDRuljhY9SoUTRv3lxP3iqlQp6/Bb9QRJ4EJgGrRcQBxFkXK3zEx8dz33338fbbb3P69Gm74yilVI38LfjjgQuAyxjzLdAe+L1lqcKMy+Xi4sWLLF682O4oSilVI7/38PEcytksItcBGcAS62KFl/T0dHr27KmtFpRSIc3fgr8JiBeRdsD7wHTgNatChSOXy8X27dttuzixUkpdib8FX4wxxcAY4BVjzGigu3Wxws+9995LQkKCnrxVSoUsvwu+iNwM3AesruOyUaFZs2aMGTOGN954g3PnztkdRymlqvC3aD8GPAm8bYz5XEQ6AxusixWeXC4X+fn5vP3223ZHUUqpKvwq+MaYjcaYkcAcEWlijNlvjPmZxdnCTnZ2Np06ddKTt0qpkORva4UeIrIN+BzYLSJbRaSbtdHCT/mGagcOHLA7jlJKVeDvIZ2/ADOMMdcaYzoA/wr8t3Wxwte0adMQEW2oppQKOf4W/MbGGN8xe2NMLtDYkkRhLjU1ldtvv5358+dTWlpqdxyllPLxt+DvF5GnRaSj9/YUsN/KYOHM5XJx+PBh1q9fb3cUpZTy8bfgO4FWwFveWyvvNFWNkSNH0qJFCz15q5QKKf5+S+eMMeZnxpie3tujxpgztS0jIgki8g8R2SEin4vIr+sncuiLj49n0qRJvPPOOxQUFNgdRymlgCsUfBF5V0RW1nS7wrovALcaY9Lx9N4ZJiJZ9RU81DmdTi5evKiHdZRSISP2Cs/PDnTF3v75Rd6Hcd6bCXR94SYtLY3evXuzZs0aXn75ZUTE7khKqShX6x6+9w+uKtyAwnL3ayUiMSKyHTgBvGeM+biecocFl8vF/v37taGaUiokiGdHvA4LiHxqjOlZx2WaAW8DjxhjdlV67gHgAYCUlJReS5curVOeMkVFRTRp0iSgZa1SVFTE2LFjGTZsGI8//rjdcSoIxe0FmquuNFfdRGKuwYMHbzXG9PZrZmNMnW7Atrou413uV8ATtc3Tq1cvE6gNGzYEvKyVhg4dapKSkkxxcbHdUSoI1e2luepGc9VNJOYCthg/63AgHS/9+raNiLTy7tkjIg2BocDeAMYLa8OHD6egoIC33nrL7ihKqSjnby+d0SKSBGCMeUdEmonIqCss1hbYICKfAZ/gOYa/6urihp/09HQ6d+6sffKVUrbzdw//340xvi+UG2PygX+vbQFjzGfGmExjTJoxprsx5j+uJmi4KmuotmHDBvbv1z9OVkrZx9+CX918V/pKp/KaNm0aDoeD+fPn2x1FKRXF/C34W0TkDyLyA+/tD8BWK4NFkvbt23PHHXfw2muvaUM1pZRt/C34jwAXgWXAUuA88JBVoSKR0+nkyJEjvPfee3ZHUUpFKb8OyxhjvgdmWZwloo0cOZKWLVsyb948hg0bZnccpVQU8vdbOu+VfcXS+zhZRNZZFyvyNGjQgEmTJvG3v/2NU6dO2R1HKRWF/D2k09L7zRzA0z0TaG1NpMjlcrm4dOkSr7/+ut1RlFJRyN+C7xaRDmUPRKQjUdQIrb50796dm266iZycnLK/PlZKqaDxt+D/G/CBiCwSkdeBjcCT1sWKXC6Xi507d7Jlyxa7oyilooy/F0BZC/QGvgCW4LmI+TkLc0WsCRMm0LBhQ/3LW6VU0Pl70vbHwPt4Cv0TwCLgGetiRa6kpCTuvvtulixZQnFxsd1xlFJRxN9DOo8CNwFfG2MGA5nASctSRTiXy8XZs2dZsWKF3VGUUlHE34J/3hhzHkBE4o0xe4HrrYsV2QYOHMgPfvADvci5Uiqo/C34R7zfw38HeE9E/gZ8Y12syCYiOJ1OcnNz+eqrr+yOo5SKEv6etB1tjMk3xjwDPA3MA67UHlnVYurUqdpQTSkVVHW+AIrxXM92pTHmohWBokW7du0YNmyYNlRTSgVNIFe8UvXE5XJx9OhR1q3TLhVKKetpwbfRXXfdRatWrfTkrVIqKLTg26hBgwZMnjyZlStXcvKkfstVKWUtLfg2czqd2lBNKRUUWvBt1q1bN/r27cu8efO0oZpSylJa8EOAy+Xi888/55NPPrE7ilIqgmnBDwHjx4+nUaNG2lBNKWUpLfghoGnTpowbN44lS5bw/fff2x1HKRWhLCv4IpIqIhtEZLeIfC4ij1o1ViRwOp0UFhZqQzWllGWs3MMvAf7VGNMVyAIeEpGuFo4X1gYMGECXLl30sI5SyjKWFXxjzDFjzKfe+4XAHqCdVeOFu7KGaps2beLLL7+0O45SKgIF5Ri+9xq4mcDHwRgvXGlDNaWUlcTq736LSBM818D9rTHmrWqefwB4ACAlJaXX0qVLAxqnqKiIJk2aXE1US9Q11y9/+Uu+/PJLli5dSkxMTMjkChbNVTeaq24iMdfgwYO3GmN6+zWzMcayGxAHrANm+DN/r169TKA2bNgQ8LJWqmuut956ywBm1apV1gTyipTtFSyaq240V91cTS5gi/GzJlv5LR3B0zd/jzHmD1aNE2nuuusuWrdurSdvlVL1zspj+P2AycCtIrLdexth4XgRIS4ujsmTJ/Puu+9y4sQJu+MopSKIld/S+cAYI8aYNGNMhve2xqrxIonT6aSkpIRFixbZHUUpFUH0L21DUNeuXcnKyiInJ0cbqiml6o0W/BDlcrnYvXs3H3+s32RVStUPLfghShuqKaXqmxb8EJWYmMg999zD0qVLtaGaUqpeaMEPYS6Xi6KiIv7617/aHUUpFQG04Iewfv36cd111+lFzpVS9UILfggra6i2efNm/vnPf9odRykV5rTgh7gpU6YQExOjDdWUUldNC36Ia9u2LSNGjGDBggWUlJTYHUcpFca04IcBl8vFsWPH+Pvf/253FKVUGNOCHwZGjBhBSkqKnrxVSl0VLfhhIC4ujilTprBq1SqOHz9udxylVJjSgh8mtKGaUupqacEPEzfccAO33HIL8+bN04ZqSqmAaMEPI06nk71795KXl2d3FKVUGNKCH0buueceGjdurA3VlFIB0YIfRhITExk/fjzLli2jqKjI7jhKqTCjBT/MOJ1ObaimlAqIFvwwc8stt3D99dfrYR2lVJ1pwQ8zZQ3VPvzwQ7744gu74yilwogW/DBU1lBN//JWKVUXWvDDUJs2bbjrrrtYsGABly5dsjuOUipMaMEPU06nk+PHj2tDNaWU37Tgh6kRI0bQpk0bPXmrlPKbZQVfRHJE5ISI7LJqjGgWGxvLlClTWL16Nd9++63dcZRSYcDKPfzXgGEWrj/qOZ1OSktLWbhwod1RlFJhINaqFRtjNolIR6vWf1UqNx+z8LFcvAjnz1uy/uuvuYZ+WVnkvPoqM3/yE0TkisvnFwh79gpb8xK4cKrw8nPGVFyu/HTK7l9hvkrTjaHqvL7Hxrfa8tMNWOFyAAASc0lEQVS/+OIi5/cevOJ8/o1R6fGV5qtl+oED33Ex71McDnCIKffT4JDK06r+jHGYK85T03pjYio+JwJl/9TJO3ZACF4J7apyXU1zwCssm7xjB1y86PtndhvB7S73s+xmpOpz3p+l7pqfq3ZaDesqdV++//WRk2RnB/6y/WVZwfeXiDwAPACQkpJCbm5undfRf8QIBp0/76sLEkLdJAdZvH4X4AT+t1kz+pWbfpKW7OFGdtO1wu0Y13jnyLI4WaCutTtADTLtDlCB4MaBGwcDvT9rvsVQesV56nt5oSNuCikl5ipGvnyrj/V41nErbhyYEDt9mSKpXD8h1/JxbC/4xpi5wFyA3r17m+xAPuaefJKDX35Jx44dL08r2wWy+fH+/fvp3LmzJes3BgZ8F0PDF5/kX9tmk9nuOXYfb8Hu48059X0j36yNG1yka5vT3J5ymq4pX3FjymnO5e8jtX17Kuwulh+j8vTanqtxGZAq033/V+169h88SOdOneq0jIgfGSs/V5fXAez94p/88IfXV7OnBm63XP5ZZS+u9uev9LO0tLbn4ZtjJ2jdOqXiGOV+lpaWPY7BbWJqnK/an7692YrTS0wNe7Ll5ikuPk/jxgk4HMbz2021v8lAbNlvMlV+Vp3mW0+Ny7hxiLvSPBXHPXHiW9pdk1J9nmrHrn6+mJqWd0g106/wm58D9h34guzssVjN9oJfL55+moO5uXQMxu9EdXQoN5fOV5nLGDh8GHbv9tz27Ll8Pz8fYBcfH1rK3oLOdOuWyKhboWvXy7f27Rsg0gZo41tnbm4uN4fg9jqfm0tWCOY6l5vPLdnpdseoIjd3P9nZve2OUUVu7icEtPNmsdzcfSG5vc7nngrKOJFR8CNEaSkcPFixoJcV+PLNMVu18hTyiRPhxhsBXPzsZ/OYPftNfvxjl03plVKhzrKCLyJLgGygpYgcAf7dGKNfGsdzLuurryoW9d27Ye/eiud3r7nGU9CnT7+8t37jjZ6CX54xWfz5zzcwf36OFnylVI2s/JbORKvWHS4uXIADBxpz4kTFvfUvvoDyHRE6dPAU81tvrVjYmzXzbxwRweVyMXPmTPbs2cONnt1+pZSqQA/p1IPiYk8Rr3x8fd8+KC29CfCc++vc2VPM77zzclG/4QZITLz6DJMnT+bJJ59k/vz5vPDCC1e/QqVUxNGCXweFhZ7DLpUPxRw4cPnrvzEx8MMfQrducPfdALu5++6uXH89NGxoXbaUlBRfQ7Xf/va3xMXFWTeYUiosacGvxpkz1Z84PXTo8jwNGsB110Hv3jBlyuVDMT/8oee5Mrm5J8jI6BqU3C6Xi3feeYfVq1czatSooIyplAofUV3wT56sehhm9244duzyPAkJnkMvAwZU/Kpj584QG2Jbb9iwYbRt25acnBwt+EqpKkKsZNU/Y+Dbb6sehtm9G06V++prkyaeQn7HHRVPnF57recwTTiIjY1l6tSp/P73v+fYsWO0bdvW7khKqRASMQXfGM8hl+oKe0HB5fmaNfMcXx892lPQL/9xUtU/Zg1H06dP57nnnmPhwoX84he/sDuOUiqEhH3BLymBfv1g587+nDt3eXrr1p6Cfu+9FQ/FpKRERmGvyXXXXceAAQPIycnh5z//+eWGakqpqBf2BT82Frp0gfbtv+X229v7DsW0bGl3Mvs4nU6mT5/Ohx9+SP/+/e2Oo5QKEaHVMi5AixfDI4/s4yc/8ZxcjeZiDzBu3DgSExP1alhKqQoiouCriho3bsyECRN48803OXv2rN1xlFIhQgt+hHI6nRQXF/Pmm2/aHUUpFSK04Eeovn370rVrVz2so5Ty0YIfoUQEp9NJXl4eu3fvtjuOUioEaMGPYJMnTyY2NpacnBy7oyilQoAW/AjWunVrfvSjH7Fw4UIuXrxodxyllM204Ec4l8vFyZMnWb16td1RlFI204If4e644w6uueYaPXmrlAr/v7RVtStrqPb888/zzTffcM0119gdSUWQwsJCDh065LsdPnyYQ4cOcfToUdauXUtqaiodOnTw3Zo1a6btPmykBT8KOJ1Ofve737FgwQKefPJJu+OoMHHp0iW++eabCoW8cnHPz8+vsExMTAzt27fn0qVLfPDBB1XOHTVp0qTKh0CHDh1809q3b098fHwwX2ZU0YIfBbp06cLAgQPJyclh1qxZuoelMMZw+vTpGgv5oUOH+Oabb3C73RWWa968OR06dKBTp04MGjSoQrHu0KEDbdu2JSYmhtzcXAYOHMiJEyeqXfehQ4fYtm0bJ06cqJKtTZs2VT4Iyt9atWql/w0HSAt+lHC5XEydOpXNmzczcOBAu+Moi50/f54jR45UW2jLbsXFxRWWadCgga+o3nbbbVWKbWpqKo0bN/Y7g8PhoE2bNrRp04Y+ffr4lbN83l27drFmzZoqOePj46vNFmjOaKIFP0qMHTuWhx9+mHnz5mnBD3Nut9u357xx40a2bdtWpVgeP368ynJle87dunVj+PDhVQpmq1atcDiC+z2OhIQEunTpQpcuXap9vrbfRA4dOsT69eur/U2kRYsW1f52cPLkSbp06eL7TSTaaMGPEo0bN2bixIksWrSIV155xe44qhZFRUW1Hmo5fPhwlWPjjRs39hW1zMzMiDk2LiK0aNGCFi1akJGRUe085c81VP5t5sCBA2zcuJGCcldBeuSRR3znGmo6bJSamkpSUlLEHTqytOCLyDDgT0AM8Kox5jkrx1O1c7lczJ07l6VLl3LdddfZHScqlZSUcOzYsVoPtZw5c6bCMg6Hg3bt2tGhQwf69OnD2LFjfYXp+PHjjB07Nqq//RIXF8e1117LtddeW+M8Z8+e5fDhw6xatYpmzZpV2O4fffQRf/3rX7l06VKFZRITE2s8bNShQwfatWtHgwYNrH559cqygi8iMcAcYChwBPhERFYaY7Sxi01uuukmunXrRk5ODs89p5+99c0YQ35+fq1750ePHqW0tLTCcsnJyb4i0q9fvypF5pprriE2tvq3am5uLsnJycF4eWGtadOmdOvWjZMnT5KdnV3lebfbzfHjx2v8d9uyZQsnT56ssIyIVDjBXN0HQ8uWLUPqg9jKPfw+wD5jzH4AEVkK/AtQ7wU/4TcJlLhLcGx2ICIIng18pfv+/HSIw7ecQy6vv+y+A4dvvvLzOHDgEAfFxcUk7k3EIY4K89R2K1t/+VuMI6bG+WMkBofD+9P7WEQuP3bE+O63GdCG9//rff74wR/JNbk4HJfHLBuj/P2ydVV3v+y1xjhifK83xhHjy182vXx23/0apu8r3EfL4543Sawj1vd8rCPWN3asI9YznqPm+2U/69PFixernGCsXNyLiooqLNOgQQNSU1NJTU0lOzu7SmFITU0lMTGxXnOqunM4HLRt25a2bdvSt2/fauc5d+4chw8frvYD/bPPPmPVqlWcK3+dVTznKGo7bJSamkqjRo2C8RIBEGOMNSsWuRsYZoz5sffxZKCvMebhmpbp3bu32bJlS53Huu6V6zj7/VniE+IxxuA2bgwGYwwGz2MMvvvlnyv/E6h1Wtn9snUZvNPKP+dVebuWf85W3wMvAnHeW2382TG50jxX+7zd6yibfg7Ptqv8z9gYSLp8kySp8Ng0MojDuj08g/HtkNQ2T7i50muq07rK7WEbY/za467L+GXrM8ZAMVAApsBU/Jnv+UkRVf8bagSOVg5KD5YSCBHZaozp7c+8tp+0FZEHgAcAUlJSyM3NrfM65vaYS1FREU2aNKnndFfP31xut5uy/5W4Szz33G5KKaXUXUoppbiNmxJTgtu4KTWlGAylptT32I3bN2/ZdGOMb9myaZ/++FMOfXGImJgY3weX70Ormg++sulV7pvLxaTKhyJ4Pmi9qvvwrG69buOu8gb1Fazy41X3IVvNun3zGKrMX+N6Kn9YG4PEC7HJscQ2iyWmWQwxzWKIaxaHxFUsDP4UCn+LSXXzVS5W7lI3jpiKv8lUu1yAY/p7OKLycqXuUmIcMTV+2FS33X3PVfo3rWlZf5avPN3tdtf4mq64rmr+G6qyTDzQrOryvp8lBnehm9L8Ut/NXeDGYRwB1b46M8ZYcgNuBtaVe/wk8GRty/Tq1csEasOGDQEvayXNVTeaq240V91EYi5gi/GzLlv5pdtPgB+KSCcRaQBMAFZaOJ5SSqlaWHZIxxhTIiIPA+vwfC0zxxjzuVXjKaWUqp2lx/CNMWuANVaOoZRSyj/aD18ppaKEFnyllIoSWvCVUipKaMFXSqkooQVfKaWihGWtFQIhIieBrwNcvCVwqh7j1BfNVTeaq240V91EYq5rjTGt/JkxpAr+1RCRLcbPfhLBpLnqRnPVjeaqm2jPpYd0lFIqSmjBV0qpKBFJBX+u3QFqoLnqRnPVjeaqm6jOFTHH8JVSStUukvbwlVJK1SLsCr6IDBORL0Rkn4jMqub5eBFZ5n3+YxHpGCK5ponISRHZ7r39OAiZckTkhIjsquF5EZGXvZk/E5GeVmfyM1e2iBSU21a/ClKuVBHZICK7ReRzEXm0mnmCvs38zBX0bSYiCSLyDxHZ4c3162rmCfr70c9cQX8/lhs7RkS2iciqap6zdnv52zg/FG542ix/BXQGGgA7gK6V5vkp8F/e+xOAZSGSaxrw/wV5ew0EegK7anh+BPB3PBfyywI+DpFc2cAqG/77agv09N5PBP5Zzb9j0LeZn7mCvs2826CJ934c8DGQVWkeO96P/uQK+vux3NgzgDeq+/eyenuF2x6+78LoxpiLQNmF0cv7F2CB9/5yYIj4e502a3MFnTFmE3C6lln+BVhoPPKAZiLSNgRy2cIYc8wY86n3fiGwB2hXabagbzM/cwWddxuUXbW97CrJlU8KBv396GcuW4hIe+BO4NUaZrF0e4VbwW8HHC73+AhV/8P3zWOMKcFz6eAWIZALYKz3MMByEUm1OJM//M1th5u9v5L/XUS6BXtw76/SmXj2DsuzdZvVkgts2GbewxPbgRPAe8aYGrdXEN+P/uQCe96PLwE/B9w1PG/p9gq3gh/O3gU6GmPSgPe4/CmuqvoUz5+LpwOvAO8Ec3ARaQKsAB4zxpwN5ti1uUIuW7aZMabUGJMBtAf6iEj3YIx7JX7kCvr7UUTuAk4YY7ZaPVZNwq3gHwXKfxK3906rdh4RiQWSgO/szmWM+c4Yc8H78FWgl8WZ/OHP9gw6Y8zZsl/JjeeqaXEi0jIYY4tIHJ6iutgY81Y1s9iyza6Uy85t5h0zH9gADKv0lB3vxyvmsun92A8YKSIH8Rz2vVVEXq80j6XbK9wKvj8XRl8JTPXevxv4H+M9A2JnrkrHeUfiOQ5rt5XAFO83T7KAAmPMMbtDiUibsuOWItIHz3+nlhcJ75jzgD3GmD/UMFvQt5k/uezYZiLSSkSaee83BIYCeyvNFvT3oz+57Hg/GmOeNMa0N8Z0xFMj/scYM6nSbJZuL0uvaVvfTA0XRheR/wC2GGNW4nljLBKRfXhODE4IkVw/E5GRQIk31zSrc4nIEjzf3mgpIkeAf8dzAgtjzH/hud7wCGAfUAxMtzqTn7nuBv6PiJQA54AJQfjQBs8e2GRgp/f4L8AvgQ7lstmxzfzJZcc2awssEJEYPB8wbxpjVtn9fvQzV9DfjzUJ5vbSv7RVSqkoEW6HdJRSSgVIC75SSkUJLfhKKRUltOArpVSU0IKvlFJRQgu+UvVAPN0qq3Q/VCqUaMFXSqkooQVfRRURmeTtlb5dRP7ibbJVJCIvisinIvK+iLTyzpshInneBltvi0iyd3oXEVnvbVT2qYj8wLv6Jt5GXHtFZHEQurQqVSda8FXUEJEbgfFAP29jrVLgPqAx8KkxpiewEc9f/gIsBH7hbbC1s9z0xcAcb6OyW4Cy1gqZwGNAVzzXRuhn+YtSqg7CqrWCUldpCJ4mWZ94d74b4mmf6waWeed5HXhLRJKAZsaYjd7pC4C/ikgi0M4Y8zaAMeY8gHd9/zDGHPE+3g50BD6w/mUp5R8t+CqaCLDAGPNkhYkiT1eaL9B+IxfK3S9F318qxOghHRVN3gfuFpHWACLSXESuxfM+uNs7z73AB8aYAuCMiAzwTp8MbPReceqIiIzyriNeRBoF9VUoFSDdA1FRwxizW0SeAv6fiDiAS8BDwPdANxHZiucKQ+O9i0wF/stb0PdzuTPmZOAv3i6Hl4BxQXwZSgVMu2WqqCciRcaYJnbnUMpqekhHKaWihO7hK6VUlNA9fKWUihJa8JVSKkpowVdKqSihBV8ppaKEFnyllIoSWvCVUipK/P+Pl9gkJIvBCgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12be1e38fd0>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcFNW58PHfM/sMwzaDsiOoRAUdQUDxgwYw3gTUqyaYuOFyXxPivZoYNTeSNwlxuXldrwu4YqIxJoq7QR3FJYyIG4uggIAMCDJssg3MDLP19Hn/6O6anp7u6Z6eru6urufrh49d3VXdp89U11PnOXVOiTEGpZRSCiAr1QVQSimVPjQoKKWUsmhQUEopZdGgoJRSyqJBQSmllEWDglJKKYsGBaWUUhYNCkoppSwaFJRSSllyUl2AzurTp48ZOnRoXNvW1dXRrVu3xBbIobQufLQeWmldtMrEuli+fPkeY8xh0dZzXFAYOnQoy5Yti2vbiooKJk2alNgCOZTWhY/WQyuti1aZWBcisiWW9TR9pJRSyqJBQSmllEWDglJKKYvj+hSUUpmrubmZqqoqGhoaUlqOnj17snbt2pSWIV4FBQUMGjSI3NzcuLbXoKCUShtVVVV0796doUOHIiIpK0dNTQ3du3dP2efHyxjD3r17qaqqYtiwYXG9h6aPlFJpo6GhgdLS0pQGBCcTEUpLS7vU0tKgoJRKKxoQuqar9eeaoLD4m8U88fUTNLU0pbooSimVtlwTFD7e+jFPf/M0zS3NqS6KUipNVVdX8/DDD8e17VlnnUV1dXXM6998883cc889cX2WnVwTFAIMJtVFUEqlqY6Cgsfj6XDb8vJyevXqZUexkso1QSGQZzNGg4JSKryZM2eyceNGJkyYwH//939TUVHB5MmTueSSSygrKwPg/PPPZ8yYMYwcOZK5c+da2w4dOpQ9e/awefNmjjvuOH72s58xcuRIvv/971NfX9/h565cuZLx48dTVlbGD3/4Q/bv3w/A7NmzGTFiBGVlZVx00UUAvP/++4waNYpRo0YxevRoampqEloHrrkkVdDOK6Wc5Fdv/YqVO1cm9D1H9RvF/VPuj/j6HXfcwerVq/nwww/p3r07FRUVLFmyhNWrV1uXeD7xxBOUlJRQX1/PuHHjmDZtGqWlpW3eZ8OGDTz77LM8/vjj/OQnP+Gll15i+vTpET/38ssvZ86cOUycOJFZs2Zxyy23cP/993PHHXfw9ddfk5+fb6Wm7rnnHh566CEmTJhAbW0tBQUFCaiZVq5pKQRo+kgp1Rknn3xym2v+Z8+ezYknnsj48ePZunUrGzZsaLfNsGHDGDVqFABjxoxh8+bNEd//wIEDVFdXM3HiRACuuOIKFi1aBEBZWRmXXnopf//738nJ8Z3DT5gwgRtuuIHZs2dTXV1tPZ8o7mkpaPpIKUfp6Iw+mYKn0K6oqODdd9/l448/pqioiEmTJoUdE5Cfn289zs7Ojpo+iuSNN95g0aJFzJ8/n9tuu401a9Ywc+ZMzj77bMrLyxk/fjzvvvsuxx57bFzvH45rWgqaPlJKRdO9e/cOc/QHDhygd+/eFBUVsW7dOj755JMuf2bPnj3p3bs3H3zwAQBPP/00EydOxOv1snXrViZPnsxdd91FdXU1tbW1bNy4kRNOOIGbbrqJsWPHsm7dui6XIZhrWgoBmj5SSkVSWlrKhAkTOOWUUzj77LM5++yz27w+ZcoUHn30UcrKyjjmmGMYP358Qj73qaee4uqrr+bQoUMceeSRPPnkk7S0tDB9+nQOHDiAMYbrr7+eXr168Yc//IGFCxeSlZXFyJEjmTp1akLKEOCaoKCjJJVSsXjmmWfazH0UfLOd/Px83nzzzbDbBfoN+vTpw+rVq63nf/3rX4dd/+abb7Yejxo1KmyrY/Hixe2emzNnTrSv0CWuSR8FaJ+CUkpF5pqgEOhT0PSRUkpF5p6goOkjpRxBW/Nd09X6c01QCNAdTqn0VVBQwN69e/V3GqfA/RS6MqDNPR3NekmqUmlv0KBBVFVVsXv37pSWo6GhIeEjhZMlcOe1eLkmKARon4JS6Ss3NzfuO4YlUkVFBaNHj051MVLCNekjHdGslFLRuScoaPpIKaWick1QCND0kVJKReaaoKDpI6WUis49QUHTR0opFZVrgkKApo+UUioy1wQFHdGslFLRuSYoBGifglJKReaaoKAT4imlVHTuCQqaPlJKqahsDQoiMkVE1otIpYjM7GC9aSJiRGSsneUBTR8ppVRHbAsKIpINPARMBUYAF4vIiDDrdQeuAz61qyyg6SOllIqFnS2Fk4FKY8wmY0wTMA84L8x6twF3Ag02lkXTR0opFQM7g8JAYGvQcpX/OYuInAQMNsa8YWM52tD0kVJKRZayqbNFJAu4F7gyhnVnADMA+vbtS0VFRac/b/2O9QB8/PHHbCzY2OntM01tbW1c9ZhptB5aaV20cnNd2BkUtgGDg5YH+Z8L6A4cD1T4Uzv9gPkicq4xZlnwGxlj5gJzAcaOHWsmTZrU6cJs/GwjfAXjTx3PkJ5DOr19pqmoqCCeesw0Wg+ttC5aubku7EwfLQWGi8gwEckDLgLmB140xhwwxvQxxgw1xgwFPgHaBYRE0QnxlFIqOtuCgjHGA1wLLADWAs8bY9aIyK0icq5dnxuJToinlFLR2dqnYIwpB8pDnpsVYd1JdpbF+hy9JFUppSLSEc1KKaUsrgkKAdqnoJRSkbkmKOiIZqWUis49QUHTR0opFZVrgkKApo+UUioy1wQFTR8ppVR07gkKmj5SSqmoXBMUAjR9pJRSkbkmKDhxRPPeQ3upaaxJdTGUUi7imqAQ4KQ+hT5392HYA8NSXQyllIu4Jig4dUK8vfV7U10EpZSLuCcoODB9pJRSyeaaoBDgpPSRUkolm2uCglPTR0oplUzuCQqaPlJKqahcExQCNH2klFKRuSYo6IhmpZSKzjVBIUD7FJRSKjLXBAWdEE8ppaJzT1DQ9JFSSkXlmqAQ4MT0kRPLrJRyJtcEBSdfkqopL6VUsrgmKAQ48QCrLQWlVLK4Jig4eUSzEwOZUsqZ3BMUnJw+cmAgU0o5k2uCQoATz7qdWGallDO5Jig4On3kwDIrpZzJPUHByekjbSkopZLENUEhwIkHWG0pKKWSxTVBwUkjmn/zzm+QW1rL68RAppRyJtcEhQAnnHXf/dHdbZadUGalVGZwTVBw8oR4TiyzUsqZ3BMUHJQ+CqUtBaVUsrgmKAQ48QCrLQWlVLLYGhREZIqIrBeRShGZGeb1q0VklYisFJHFIjLCtrI4OX3kwECmlHIm24KCiGQDDwFTgRHAxWEO+s8YY04wxowC7gLutbE8dr11wjl5TIVSytnsbCmcDFQaYzYZY5qAecB5wSsYYw4GLXYD+0/jnXjW7cTWjVLKmXJsfO+BwNag5SrglNCVROQa4AYgDzgj3BuJyAxgBkDfvn2pqKjodGFW7V0FwPLPllNfWd/p7VPpgw8+oHtu94S+Z21tbVz1mGm0HlppXbRyc13YGRRiYox5CHhIRC4Bfg9cEWaducBcgLFjx5pJkyZ1+nPqN9TDajjppJMYP2h81wptM1kkbVo0E06bQElhSUI/o6KignjqMdNoPbTSumjl5rqwM320DRgctDzI/1wk84Dz7SqMkybEC+1TcEKZlVKZwc6gsBQYLiLDRCQPuAiYH7yCiAwPWjwb2GBXYZzceat9CkqpZLEtfWSM8YjItcACIBt4whizRkRuBZYZY+YD14rImUAzsJ8wqaOEl8sBB1gRadPlri0FpVSy2NqnYIwpB8pDnpsV9Pg6Oz8/mJMuSQ3lhECmlMoMOqI5DWmfglIqVVwTFBw9otmBZVZKOZN7goKT00faUlBKJYlrgkKAEw6woQFMWwpKqWSJKSiIyAQR6eZ/PF1E7hWRI+wtWmI5On3kgECmlMoMsbYUHgEOiciJwG+ALcDfbCuVDZyUPmrX0ezAQKaUcqZYg4LH+E5XzwMeMMY8ACR2Mp4kccJZd7v0kQPKrJTKDLGOU6gRkd8C04HvikgWkGtfsRJPRzQrpVR0sbYULgQagauMMTvxzWN0d8ebpCcnHGB1nIJSKlVibingSxu1iMh3gGOBZ+0rVuI5aUK8UE4IZEqpzBBrS2ERkC8iA4H3gP8A/mpXoezgpPSR9ikopVIl1qAgxphDwI+AOcaYHwLH21cs+zjxrNuJZVZKOVPMQUFETgUuBd7o5LZpwdHpIweWWSnlTLEe2H8F/BZ4xT/99ZHAQvuKlXiOSh85qKxKqcwSU0ezMeZ94H0RKRaRYmPMJuCX9hbNHk5MxTixzEopZ4p1mosTRGQFsAb4UkSWi8hIe4uWWI4a0awdzUqpFIk1ffQYcIMx5ghjzBDgRuBx+4plHyccYHWaC6VUqsQaFLoZY6w+BGNMBdDNlhLZRCfEU0qp6GIdvLZJRP4APO1fng5ssqdI9nB0+siBgUwp5UyxthT+D3AY8LL/32H+5xzHiWfdTiyzUsqZYr36aD8OvdoowMmXeWpLQSmVLB0GBRF5DSIfkYwx5ya8RDZzwgFWJ8RTSqVKtJbCPUkpRRI4ekSzAwKZUiozdBgU/IPW2hCRk4wxn9lXJHs4KX2k4xSUUqkSz/xFf054KZLICWfdOk5BKZUq8QQF55xyB3F0+siBZVZKOVM8QeGWhJciCRydPtKWglIqSWKd++iHItITwBjzqoj0EpHz7S2aPZx4gNWWglIqWWJtKfzRGHMgsGCMqQb+aE+R7OGoEc3ap6CUSpFYg0K49WKdIiOtOPGs24llVko5U6xBYZmI3CsiR/n/3Qsst7NgieboCfEcWGallDPFGhR+ATQBzwHzgAbgGrsKZQdHpY90nIJSKkVinfuoDphpc1mSwokHWG0pKKWSJdarj94RkV5By71FZEEM200RkfUiUiki7YKKiNwgIl+KyBci8p6IHNG54sfO0ekjBwYypZQzxZo+6uO/4giwZk09vKMNRCQbeAiYCowALhaRESGrrQDGGmPKgBeBu2IteGc5KX3U3NLcZtmJgUwp5UyxBgWviAwJLIjIUDqYPdXvZKDSGLPJGNOEry/ivOAVjDELjTGH/IufAINiLE/cnHDW3djSmOoiKKVcKtbLSn8HLBaR9/FNc3E6MCPKNgOBrUHLVcApHax/FfBmjOXpNCeNaG70tA0KTghkSqnMEGtH81siMhZfIFgBvArUJ6oQIjIdGAtMjPD6DP9n07dvXyoqKjr9GZW1lQCsWr2K3rt6x1vUpAhNFy3/bDmNGxPbeqitrY2rHjON1kMrrYtWbq6LmIKCiPwUuA5femclMB74GDijg822AYODlgf5nwt97zPxtUQmGmPCHvmMMXOBuQBjx441kyZNiqXYbZTsKoHlMHLkSCaN6Pz2SRUyYfno0aOZMGRCQj+ioqKCeOox02g9tNK6aOXmuoi1T+E6YBywxRgzGRgN7I6yzVJguIgME5E84CJgfvAKIjIaeAw41xjzbadK3klOSh+VFJa0WdaOZqVUssQaFBqMMQ0AIpJvjFkHHNPRBsYYD3AtsABYCzxvjFkjIreKSOA2nncDxcALIrJSROZHeLuEccIBdlS/UW2WtU9BKZUssXY0V/nHKbwKvCMi+4Ht0TYyxpQD5SHPzQp6fGYnytolTrokNTQIOCGQKaUyQ6wdzT/0P7xZRBYCPYG3bCuVjZxw1h0aBJxQZqVUZuj0TXaMMe8bY+b7xx44hpNGNGtLQaWToj8VMefTOakuhkqSeO685kiOSh9pS0GlCWMM9Z56fvnWL1NdFJUkrgkKAU44wGpLQaUL3ffcxzVBwVHpI20pqDSh+577uCcoOCh95DXeNstOCGQqM+m+5z6uCQoBTjjzaZc+ckCZVWbSfc99XBMUnDSiuV36KIaztR01O8i9LZel25baVSzlQtpScB/XBIUAJ+zk8bQU3t30Lh6vhwc+fcCuYikX0paC+7gmKAT6FJywk8fTUnBSn4lyDiecRKnEck9QcFL6SPsUVJoIvehBZT7XBIUAJ5z5xNNSiGddpaLRExL3cU1QcFT6KI6WgpNaQso59CTDfdwTFBx00OxKS0GpRHLCSZRKLNcEhQAnHGC7ksfVH7FKJCf8XlRiuSYoOOnqnLjSRw76fso59CTDfVwTFAKcsJMbDIN7DG6z3JltlUoU3Z/cxzVBwVET4hnDmAFjeOHHL1jL0Tipz0Q5hxNOolRiuSYoZInvqzphJzcYBOGo3kcB8PQXT6e4RMqtnHASpRLLdUGhxbSkuCTRGWMQEauf4KW1L8UczJwQ9JRz6P7kPq4JCtlZ2YAzRmgGWgrBKaFo5bbGYeiZnUogJ/xeVGK5JigEWgrpvpMbY9hzaA8983u2uaLI4/WksFTKrfQkw31cFxRavOmdPtpZu5Nv675lVL9RbVoKTkh7qcyj6SP3cU1QyBZnpI8G3DsAgP7d+1uBDFqD2fA5w7n05UvbbadXHyk7aEvBfVwTFJySPgrIkiwKcgqs5UD6qHJfJc+seibidnpml1qNnkbu/+T+jEn36f7kPq4LCk5JwwjSJihEK7eOaE4Pdyy+g+sXXM+TK55MdVESQlsK7uOaoOCkq4+gfUsh3ftCQsktwjnPnJPqYiRddUM1ALVNtSkuSWJoS8F9XBMUnJY+EpGw6aNo0unM7o0Nb6S6CKqL7N6fXlv/GnKLsL1mu62fo2LnuqDglDPuLMkiPyffWo6aPsI594tQzmH3/vTo8kcBWLFjha2fo2LnmqDglKuPAgQhJyvHWnZKMFOZJZ1anio5XBMUnJY+Cr4cFXTwmkoNp/xeVOK4Lig45uqjkKuJYr36SM/sUivT6l/Tke7jmqAQOGg65cwntKUQLX2kg9eUHVIR5J5Z9Qxyi7D1wNakf7ayOSiIyBQRWS8ilSIyM8zr3xWRz0TEIyIX2FkWgCyyHBMUQg/ymj5SqZCKlsJTnz8FwJe7v0z6Zysbg4KIZAMPAVOBEcDFIjIiZLVvgCuByEN0EyhLshzTYRtoKYw8bCQQPX2UaWkLlR50v3IfO1sKJwOVxphNxpgmYB5wXvAKxpjNxpgvgKScvgvinJaCP91155l3AtHTR4EzOs0Bp1ampfF0f3IfO4PCQCA4KVjlfy5lsiXbMUEh0FIIjMT2eD0dlj1wRqdndqmVafWfad9HRZcTfZXUE5EZwAyAvn37UlFREd/7IGzZuiXu7ZNp5cqVsBlW71sNwGsfvUbdYXXW66HfYfW3vvX27N4T0/erra1NSj2ke10nuh6qqqoAqNxYSUVj4t43GcLVxTeHvrEe2/G33Lt3LwBfrPqCbtu7AbBv3z4APv/ic/Kr8iNua6dk/T7SkZ1BYRswOGh5kP+5TjPGzAXmAowdO9ZMmjQprgJlLc5iwMABxLv9wq8XMmHIBPKy8+LaPibv+/43ZvQYTj/idMzXBlbB7etuZ9a0WfCB7/XQ77Bj1Q5YC30O6xPT96uoqIi7HmLi/x62fkYCJLoeXm14FbbB0UcdzaRTE/e+yRCuLtbtWQdLfY/t+FuWbi+FfVB2QhmTvuN7/5KqEtgPZWVlTDo68Z8ZC9t/H2nMzvTRUmC4iAwTkTzgImC+jZ8XVZbEf/XRsu3LOONvZzDz3XYXUdki0KcQSB9Bx1cgWekjzQGrBErF/pRp/TJOY1tQMMZ4gGuBBcBa4HljzBoRuVVEzgUQkXEiUgX8GHhMRNbYVR7oWkfz7rrdAKzdszaRRYrI6lOQ1qDQUWezBoP0kGkHNKf0wanEsbVPwRhTDpSHPDcr6PFSfGmlpOjKJanJ7nALHFza3H2tg8tSAz9e7RhMrUyrf7u/j57MpB/XjGiGrqWPki0QDBo8DdZzHbYU0uhgpD/0zKF/S/dxV1Bw0ohmf5/CoeZD1nMLNi6IuH46/XjTKUCprrH7bxnujoG6/6SWq4KCiMQ9IV6yc8WBlkLwD+SyVy6LuL6VPkqD4OCUwGunTLk9qt37Uzrsr6otVwWFzrQU/vb53+h1Ry/rip9U9SlMPXpqTOun09mV/tAzpw5SuV9lWqe9U7grKHSiT+EXb/6CA40HqGuqa/N8snbU4BHN046bFnX9dDo7T6eyqK6xO7hlSosqk7gqKAjxp4+SLfiqo9zs3KjrW3MfJeDMbsWOFdz78b1xb69BIXOk8uqjdGr9uokjprlIlHjmPkrVjhl8BpWbFT0o/GXFXxL22SfNPQmAG069Ia7tNShkjkxJg6nYuaulIJ0fvJaqA1xwSyGWJvan2z4F0uNHrEEhc9Iiqbj6SPsSUstdQQHp9OC1VB3ggn8YqboHRLzfXYNCegTnREjW31JTRenDVUEhnvRRMg/IwQeSWEcyJ9r3n/5+6+fG+d01KGSOZAW3cPtMpgRWp3FVUOjM3EeBM/VkHpCDz5aCm9WdOTh39YzrnU3vWI/jvQWonvVljmT9LcMFAN2PUsNVQSFLsmI+yAd2yGQOCgsOWMEtheCpLqIJLufibxb7pj6OU7xBQVsKmSOVLQXdj1LDfUGhkymR0PXt7EAM/gEG9ymMGzAurvc7/cnTOe6h4+IujwaF+GlHc+f878f/2y4A6X6UGq4KCjmSQ7O3Oep66/ass3bI0B3TzjOnSC2FG069gRtPvdG2z41Eg4JKVkvhw60fsnDzwjbP6X6UGq4KCnlZeVFTMV/u/pLjHjqO2qZaoLVPIRlnTJH6FLrldetwVHPwD7eppSlh5dGgoJKZ1w/9bep+lBquCgq5kkujp7HDdbZUb2mzHEgfJWMHjdRSADimzzERtws+eC/dvpScW3OY/ensLpdHg4JK5hVAoeMTdD9KDVcFhbysPBpbOg4KoR3RoWmkVPQpAJQUlrRZXr59OX9c+EeANimxg40HaTEtXPfWdV0uT9xXH+mlhBkjmS2FwGcFfmMaFFLDVUEhNys3avootGM5ECRS3VIIdcELF3DrolvZdnAbzS0d95Os3R3bLURDD+baUlCpDPC6H6WG64JCtPRR6I4YCBLJGMQWqU8hnM3VmwFYsXNF1M7zEQ+PAOCrvV/x15V/jbhe6HdPVFBo8bZQ31wPwLTnpyUktZVu9tfvb9OfkymtpWQemAOt40DdaVBIDVcFhVg6mqOlj+zUmZZCwLd137KrdldM657+5On8xz//I2LLIjS4JCooTHt+GkX/rwiAl9e+nJDUVropuauEf3/2363lTBl4lYr0kbWcIYHVaVw1S2puVm70PoUUpo866lOI5Kr5V8X8/vvr9wNQua8y7OuhQcDj9bDm2zWUFJbQv3v/mD8ntK7+uf6fMW/rZG9vfJvj+vjGhWTKWa6mj9zHXS0FyYuaPgo9MKbL1UdddeWrV9KzoCfgSyeF+7GHtiDW713P8Y8cz8B7B3bqsyLVVbwtDyfKlANaKls8mVKHTuOuoOBPH3V09hO6IwaWkzEHUvBnh+tTGN1vdNzv/dTnT9Ejv4e1fOf6O9utE3rQvviliwHfgeGKV6/g27pvAViybUnMdbhx30brcWem63CSsPP2ZEjqQ1sK7uOqoJCblYvBdHhwCj34JzV9FHRWFq6l8N7l73HThJvifv+i3CLr8YJdCzj32XO58MULrQ7Sjs7k//b537jt/dt4c8ObnPLnU3hs+WMR1w3+Hsc/crz1ONDZnGnCdfRnygEtmS2FSCdkKrlcFRT6FfQD4PWvXo+4Tmh6KTR9ZOcNQNq0FMJ8Tu/C3pwx7IwO3+PkgSdHfC10tPNrX73G82ue591N73LPR/fw0tqXOnzvnKwcayqC9XvWA77BfoG+inDfIzgAH2o+1OH7O1W4jvuM6Wi2uaUQ/P6h/XkaFFLDVR3NY3qPAWDPoT0R1wntiE7m1UeR7qcQLFuyAThtyGks/mZxu9c/ueoTKvdV8p0Hv9Puta/2fhX2PS975TL21e+LWr687Dzu+ugu6zHA0AeG0r+4P9tv3M5jyx7jmD7HtBtoF1DTVBP1M5xIWwqJEWiVJ3vw2o6aHcxdPpdZE2dlzESGXeGqlkJelu9AVu+JnMZo11Iwbccp2PkjidanEKzs8LKwz4sIw0uHc1L/k2L+3FgCAkBuduu9ovNz8q3HO2p3AHD1G1cz+anJEX/MBxsPxlwmO13+yuXILYn78YdLu2VMULC5pRC8n4fWY6AOtx3cxqb9m2wrw+WvXs7N79/Mkm1LbPsMJ3FnUOggtx2a4kjm1UfR+hQAzhh2Bn/+9z9zz/fvYcRhIyK+lx0/oj998Cfr8W2LbuPsZ84Ou14sQaG6oTqxheuEp794OqHvFzZ9lCkdzTa3FILrKdwl0Re+eCGD7hvEUbOPsq0MNY2+Fmwyb6iVzlwVFHIlF0E67GgOPZttN/dRCvsUwHdmddVJV1GYW8iiKxex6MpF1mu9C3pbj2//3u10y+3GxCMmxvz5+dn50VcKUr6h3HocXKePLns07PrBB+Nxj8d3j4h0FJw+yrTRuMn8HqFB4Zdv/ZLn1zyftM9XPq4KCiJCYW5hh+mj0KCQqsFrsYxTKC0q5fQjTreW11+73np89dirqf2/tbx28Wsxf/6C6QsAuO6U6xjVb1TM2wFtbubz+GePh13nmVXPWI8jDaBLpkSNmwhuKQT2k8Vb2/f3OFEy00fRppLpqC8wETKldddVrgoKAAU5BR2mj0I7Q1M1TiGewWvd8rq1e657fnfmTZsXdv3vHvHddsueP3i4f8r9LJi+gGvHXQtgjdLtaPvAXExOkqhLZNu0FPzpln99/S9eWftKQt4/lVKZPgp12N2H2VqWWG7A5QauCwqFOZ1sKYRekmrn1NmdmBAvWL9i36W2BTkFYV+/8PgLeeknLzFzwkzruQWnL+C9y9+jb7e+nPOdc1h/7XpEhOws39VNh3c7nAemPsCjZz/Kip+vsLb7n8n/Yz3+6eifxlzGcCKlmezwyNJH2vWBdLQfdEZwS+HtjW9bj7tyf+x0kcyz55qmGl5d92rKztgzdXBlZ7nqklSAwtzCTvUpBFoOgaCQittxRrP0Z0tZvn15h9v86Lgf8aPjfkSDp4GRh48k72AeOVk57Pz1zojbZEkWPx/78zbP3XTaTfx+4e8BuOzEyxhguByiAAAOHUlEQVTWexgPLnmQ59Y8F3N5A/7zjf/k+MOPZ3jJcPoW9+309h1p8DRQ11RHaVEpAP9V/l/t1rGjpbBh3wbrcbR5tpzA7pZC8MnPHYvvYPeh3bZ+Xkdl0KDgY2tLQUSmiMh6EakUkZlhXs8Xkef8r38qIkPtLA/400cdnCHurd/bZvmKV6+g0dOYthPiAQzqMYjzjj0vpnXvm3IfPz0p/jP8nKy25xGnDTmNO8+8k3EDxlH5i8j9BKGXyN73g/sA38ytJz56YtzlieSsf5xFn7v78PDSh/lo60fW88E//ND9oLmlmbnL53a6ryHSrLOJvDVqqth5EmSMafP+XQ0Ixhju+/i+uK+8izYvmlvYFhREJBt4CJgKjAAuFpHQayivAvYbY44G7gPaT8iTYKWFpXy5+0teW9++A9ZrvGE7QLfVbLPSSM3eZtvy53ZOiJcIU4+eCvimyzj/2POt54/odQRLfraEo0qOsvohQt155p386YzWS1qL84qtx7vqdvHw0odjvmeF13h5ee3LEdffUr3FGnl9Tfk1THhigvVa8Ojr0JbCw0sf5uev/5zHl4fvKI8kUi769sW3I7dIWnSqxyu4pdDZeunIi1++SNatWWw5sCX6ykGuf+t6PtvxWdjXdtXt4oa3b2gzhXlndLal8NXer/hF+S8ybqJHO488JwOVxphNxpgmYB4Qejp7HvCU//GLwPfE5iGFZX3L+GrvV5w771yWbV/W5kzljwv/GPYP/PnOz/l026eAL2c87IFh/OOLf1gHleaW5g7PqGqbamPKL3dm8FqymT8a3rjkDQDq/m8dr1wYvhN1zllz+P3pv2/3/JlHnsn0sun0KerDumvW8eMRP27z+jXl15BzWw73fXwf15Zfy3Orn2Prga3UN9fT4m3hmwPf4DVe9hzaw+/e+x3Tnp/GX1b8hYONB62/2a7aXdQ11TH0gaERv0fwnE1vb3ybWk8t6/esZ/76+dYZZtXBqnZ/T6/xUt9c3+YS5eC/f0eGzxnOre/fSm1TLeAbC+OUK12Cyznj9RnsObSHis0VvLL2FT7Y8gFPrXwqpmAe2iqYu3wuAF/u/rJT5bn/0/sZM3cM5RvKafA0sG7POhZ/s5imliaWb19uveeSbUto8DTwr6//FXNndmCyTK/x0uRtwhjDpv2bWLlzZdj1f/D3H/Dg0gf5fOfnMZW90dPIpS9fapUzXYldO6eIXABMMcb81L98GXCKMebaoHVW+9ep8i9v9K8T8dqzsWPHmmXLlsVVpoqKCgqOLuAnL/yErQe3AtAzvyclhSUU5BSwdk9st60MVpRbxKHmQ/Qr7keP/B7UN9cjInTL7WadZW2v2c7BxoMM6TmE3XW76VnQkx75PdqkiER8Z5SBndY7y2trYKioqGDSpEm2vT/AJS9dwrOrn+Wxcx5jxpgZ7V73eD3cuOBGZi+J/05sgq9zvCi3KGEjprMlm7zsPPJz8snPzqc4r5h99fuobarFa7wU5hZijMHj9TCwx0DqmurYVRf9RkcFOQUU5xWz59Aeehf0pjC3EK/xkiVZ1j9BaDEtYS8aCJdSDLePWHcw8+9/xhgMhvrmeorziiPuV4cOHaKoqKjNcwcbD7K9ZnvU79a/uD9ZkmV9ZmFOIQZjffau2l2UFJZQlFuE13jZuH9j2Pe5/MTLuf17t1PdUM2bG97k1+/8mtys3KhXBmVLdsQrBIvziinIKaAwp5Bddbvokd8Dj9dDr4JeCMLX1V8D0KeoD8YYappqaGppIi87z0oBDuw+kNzsXL6t+5a+3foiIm3SVEf1PgoRafebDtbgaeCbA98AcHTJ0daUNZ1x86Sbuej4izq9nb88y40xY6Ou54SgICIzgBkAffv2HTNvXvhLLKOpra2luNiXtli0exGL9y6mKLuIOk8dHuM7GJfmlXJirxN5YesLXDn0Smo9tayoXsGG2g1MKJ1A/8L+7KjfwYd7P2RI0RCKc4rJlVx2NPimesjPyseLl0avLz8pCM3eZmo8NZTkldA7tzdNpolDHt/IaeuHiwEDe5r2UNazjBlHtj+IJlJwXaSLnQ072XpoK5W1ldS31LPqwCr6FfYjPyufgqwC8rPzqW6q5oDnAHWeOgYUDGB342565fUiW7LZUreF4pxiBhYOpMnbxMa6jQwuHExuVi45kkNtSy1DCofQI7cHuxp2Ub6znFN6noLkCHsb9zK8eDi983qzvWE7DS0N1kG6oaWBRm8j9S31DCgcQJ2njiZvE/Ut9fTK9X1234K+DOs2jCGFQ9jfvJ/eub3Jz85nYOFA/rn9n6w6sIqCrAJaaKE4u5gaTw3Zku07cBjw4sVrvHjxkkWWtT8GhO3wDfuU77/Awck6SIl/6viWhjavB/N4POTktL/2pLK2kuN7HE9pfiker4eDnoP0zO1Ji2nhYPNBvMZ3ApMt2QjiO9M2TWSR1W4eI4PxPY+w5uAacrJy+LfD/40f9PsBHq+H0vxSa+YB8AU0EcHj9bC2Zi1HFx9N+Y5y5m2dx/Di4b7gnZVPj5we7GrcxefVnzO4aDADCgeQJ3msrVnLyB4j8RovzaaZvY17Kc0vpSi7iFpPLYJgMKyvWc/w7sPJlmyyJZtdh3bRt7AvXrwc8hyiMLuQFlooyi6ivqUeQciRHGo8NRRkF7T/G4X8bQJ1vurgKk7ocQKIr046Oxj2rH5nMbYk6nE9rMmTJ8cUFKxmXaL/AacCC4KWfwv8NmSdBcCp/sc5wB78gSrSvzFjxph4LVy4MO5tM43WhY/WQyuti1aZWBfAMhPDsdvOPoWlwHARGSYiecBFwPyQdeYDV/gfXwD8y194pZRSKWDbOAVjjEdErsXXGsgGnjDGrBGRW/FFrPnAX4CnRaQS2IcvcCillEoRWwevGWPKgfKQ52YFPW4Afhy6nVJKqdRIv4vhlVJKpYwGBaWUUhYNCkoppSwaFJRSSlk0KCillLLYNqLZLiKyG+jcLFqt+uAbIKe0LgK0HlppXbTKxLo4whgT9U5FjgsKXSEiy0wsw7xdQOvCR+uhldZFKzfXhaaPlFJKWTQoKKWUsrgtKMxNdQHSiNaFj9ZDK62LVq6tC1f1KSillOqY21oKSimlOuCaoCAiU0RkvYhUisjMVJfHTiIyWEQWisiXIrJGRK7zP18iIu+IyAb//3v7nxcRme2vmy9E5KTUfoPEE5FsEVkhIq/7l4eJyKf+7/ycf3p3RCTfv1zpf31oKsudSCLSS0ReFJF1IrJWRE516z4hItf7fxurReRZESlw4z4RjiuCgohkAw8BU4ERwMUiMiK1pbKVB7jRGDMCGA9c4/++M4H3jDHDgff8y+Crl+H+fzOAR5JfZNtdBwTfb/VO4D5jzNHAfuAq//NXAfv9z9/nXy9TPAC8ZYw5FjgRX324bp8QkYHAL4Gxxpjj8U3tfxHu3Cfai+VOPE7/Rwx3gcvkf8A/gX8D1gP9/c/1B9b7Hz8GXBy0vrVeJvwDBuE74J0BvA4IvoFJOaH7B3HcDdAJ/4CewNeh38WN+wQwENgKlPj/xq8DP3DbPhHpnytaCrTuBAFV/ucynr+pOxr4FOhrjNnhf2kn0Nf/ONPr537gN4DXv1wKVBtj3Qg5+PtadeF//YB/facbBuwGnvSn0f4sIt1w4T5hjNkG3AN8A+zA9zdejvv2ibDcEhRcSUSKgZeAXxljDga/ZnynPRl/6ZmInAN8a4xZnuqypFgOcBLwiDFmNFBHa6oIcNU+0Rs4D1+gHAB0A6aktFBpxC1BYRswOGh5kP+5jCUiufgCwj+MMS/7n94lIv39r/cHvvU/n8n1MwE4V0Q2A/PwpZAeAHqJSODOg8Hf16oL/+s9gb3JLLBNqoAqY8yn/uUX8QUJN+4TZwJfG2N2G2OagZfx7Sdu2yfCcktQWAoM919dkIevU2l+istkGxERfPe/XmuMuTfopfnAFf7HV+Drawg8f7n/ipPxwIGglIKjGWN+a4wZZIwZiu/v/i9jzKXAQuAC/2qhdRGoowv86zv+7NkYsxPYKiLH+J/6HvAlLtwn8KWNxotIkf+3EqgLV+0TEaW6UyNZ/4CzgK+AjcDvUl0em7/rafjSAF8AK/3/zsKXB30P2AC8C5T41xd8V2dtBFbhuyoj5d/DhnqZBLzuf3wksASoBF4A8v3PF/iXK/2vH5nqcifw+48Clvn3i1eB3m7dJ4BbgHXAauBpIN+N+0S4fzqiWSmllMUt6SOllFIx0KCglFLKokFBKaWURYOCUkopiwYFpZRSFg0KSoUhIkNFZHUn1r9SRAbEsM6DXS+dUvbRoKBUYlyJb8oEpRxNg4JSkeWIyFP++wm86B8BO0tElvrn4Z/rH/F7ATAW+IeIrBSRQhEZJyIficjnIrJERLr733OAiLzlv3/BXSn8bkqFpUFBqciOAeYaY8qAg8B/AQ8aY8YZ3zz8hcA5xpgX8Y0UvtQYMwpoAZ4DrjPGnIhvrp16/3uOAi4ETgAuFJHBKJVGNCgoFdlWY8yH/sd/xzd9yGT/3bdW4Ztcb2SY7Y4BdhhjlgIYYw6a1imZ3zPGHDDGNOCbb+cIe7+CUp2TE30VpVwrdA4YAzyMbx6grSJyM755cTqjMehxC/obVGlGWwpKRTZERE71P74EWOx/vMd/r4oLgtatAQL9BuuB/iIyDkBEugdNyaxUWtMdVanI1gFXiMhj+GYRfQTfzKKrgM34pmQP+CvwqIjU47uV44XAHBEpxNefcGbyiq1U/HSWVKWUUhZNHymllLJoUFBKKWXRoKCUUsqiQUEppZRFg4JSSimLBgWllFIWDQpKKaUsGhSUUkpZ/j+br3fgL6mW8gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12c14a34320>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "inception(num_class=5,epochs=20)\t#binary classificaiton\n",
    "history_this.loss_plot('epoch')\n",
    "history_this.loss_plot('batch')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Validation accuracy of Inception: 100.0%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model 7: InceptionResnet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def inceptionresnet( num_class, epochs,savepath='./inceptionresnet.h5',history=history_this,input_shape=INPUT_SIZE):\n",
    "    model_fine_tune = InceptionResNetV2(include_top=False, weights='imagenet', input_shape=input_shape)\n",
    "    for layer in model_fine_tune.layers[:500]:\t#the number of frozen layers for transfer learning, have tuned from 400-550\n",
    "        layer.trainable = False\n",
    "    for layer in model_fine_tune.layers[500:]:\t#the number of trainable layers for transfer learning\n",
    "        layer.trainable = True\n",
    "    model = GlobalAveragePooling2D()(model_fine_tune.output)\n",
    "    model=Dense(units=256,activation='relu')(model)\n",
    "    model=Dropout(0.5)(model)\n",
    "    model = Dense(num_class, activation='softmax')(model)\n",
    "    model = Model(model_fine_tune.input, model, name='resnet')\n",
    "    opt = keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08)\t#tuned learning rate to be 0.001\n",
    "    model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy']) #set the loss function to be binary crossentropy\n",
    "    #train model\n",
    "    earlyStopping = kcallbacks.EarlyStopping(\n",
    "        monitor='val_acc', patience=2, verbose=1, mode='auto')\t#set early stop patience to save training time\n",
    "    saveBestModel = kcallbacks.ModelCheckpoint(\n",
    "        filepath=savepath,\n",
    "        monitor='val_acc',\n",
    "        verbose=1,\n",
    "        save_best_only=True,\n",
    "        mode='auto')\n",
    "    hist = model.fit_generator(\n",
    "        train_generator,\n",
    "        steps_per_epoch=len(train_generator),\n",
    "        epochs=epochs,\n",
    "        validation_data=validation_generator,\n",
    "        validation_steps=len(validation_generator),\n",
    "        #use_multiprocessing=True, \n",
    "        callbacks=[earlyStopping, saveBestModel, history],\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0076 - acc: 0.9979Epoch 00001: val_acc improved from -inf to 0.99538, saving model to ./inceptionresnet.h5\n",
      "183/183 [==============================] - 360s 2s/step - loss: 0.0076 - acc: 0.9980 - val_loss: 0.0172 - val_acc: 0.9954\n",
      "Epoch 2/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0049 - acc: 0.9994Epoch 00002: val_acc improved from 0.99538 to 0.99966, saving model to ./inceptionresnet.h5\n",
      "183/183 [==============================] - 134s 731ms/step - loss: 0.0049 - acc: 0.9994 - val_loss: 0.0011 - val_acc: 0.9997\n",
      "Epoch 3/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 2.7993e-04 - acc: 1.0000Epoch 00003: val_acc did not improve\n",
      "183/183 [==============================] - 135s 739ms/step - loss: 2.7842e-04 - acc: 1.0000 - val_loss: 0.2168 - val_acc: 0.9864\n",
      "Epoch 4/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 0.0022 - acc: 0.9998Epoch 00004: val_acc improved from 0.99966 to 0.99993, saving model to ./inceptionresnet.h5\n",
      "183/183 [==============================] - 137s 748ms/step - loss: 0.0022 - acc: 0.9998 - val_loss: 0.0011 - val_acc: 0.9999\n",
      "Epoch 5/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 6.2310e-04 - acc: 0.9999Epoch 00005: val_acc did not improve\n",
      "183/183 [==============================] - 139s 761ms/step - loss: 6.1971e-04 - acc: 0.9999 - val_loss: 0.0011 - val_acc: 0.9999\n",
      "Epoch 6/20\n",
      "182/183 [============================>.] - ETA: 0s - loss: 8.3696e-04 - acc: 0.9999Epoch 00006: val_acc did not improve\n",
      "183/183 [==============================] - 140s 763ms/step - loss: 8.5028e-04 - acc: 0.9999 - val_loss: 0.0011 - val_acc: 0.9999\n",
      "Epoch 00006: early stopping\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4FFW+//H3N52QAAm7ohdUQL0qkA2i4CDbKApyB3BB9LK5jF4fxRG9V8V1VMZl0JlhYHAcdFBcGUVRHBGUGUJgBJUlgIL+RgUF3AABEyCQdJ/fH90pOpCEJqTSCfm8nqeeqjp9qup7svS3q+r0KXPOISIiApAQ7wBERKT2UFIQERGPkoKIiHiUFERExKOkICIiHiUFERHxKCmIiIhHSUFERDxKCiIi4kmMdwCHq1WrVq5du3ZV2nbXrl00bty4egOq5dTm+kFtrh+OpM3Lly/f6pw75lD16lxSaNeuHcuWLavStrm5ufTp06d6A6rl1Ob6QW2uH46kzWb2VSz1dPlIREQ8SgoiIuJRUhAREU+du6cgIkev4uJiNm3aRFFR0SHrNm3alHXr1tVAVLVHLG1OSUmhbdu2JCUlVekYSgoiUmts2rSJtLQ02rVrh5lVWregoIC0tLQaiqx2OFSbnXNs27aNTZs20b59+yodw7fLR2Y2zcx+MLOPK3jdzGySmX1uZqvNrItfsYhI3VBUVETLli0PmRCkfGZGy5YtYzrTqoif9xSeBfpX8voA4NTIdB3wZx9jEZE6QgnhyBzpz8+3y0fOuTwza1dJlcHAcy78PNClZtbMzI53zn3rS0AFBSRv2QIbN4JzEAqF59FTeWVHWNeFHCXFjmAQSkogWOIoCVrZssrmQYtahmDpesjK1gla2TpBIxgyfvhhB188k0fjBsU0Ti6hUVJ43rhB8f4paR+NGpSQYK70l1en5+02bIB//vPgv4Hy/llqaVkoBHtKkijc14BdxUns2teAXfuS2LWv/LIfthWwsPnisvuryqN2D3ebaq5//q+b8k1CQfQGFdYNlgQpSPzp8I5fxyU3DoLPV8zieU+hDbAxan1TpOygpGBm1xE+m6B169bk5uYe9sHWT1jFknca8hTvESRACYkHzcsrO9I6rg518GrIbhqzK4Yp1nr7p0SCNdIGZ8ZJHPxWYj48i9wBe2h4UGsLSa10PZay3dSvb+qW6nbHOr4piN99goKCHcyd+xJDh95w2NvefPOF/OY3L5GW1iym+lOn3k/DhqmMHPl/MR/jeLeNgoKCQ9YrKiqq0vsk1JEbzc65qcBUgJycHFeVb/StffcU3shLI7lBgMSAI5DgSEyIzAOh/WVR84YJjsQA3npiwBEIRMpK1xOCJAZKCAT2kpgY/VpkngiBBMLzSHl4OWoecOF5knnlZeok2sHbJNnBrzVIKLufJPjgw+V07NyNXbuNXXsSKp/vNnbtacyu3ans2r2/7JvS13bjlRUXH94paoMGjsaNoXGj0jk0bhy9vH9qdMB6mSnVDponJ+//kG2U/danc7B3L+zaFZ4KC/cvl1tW6PaX7S4tCy+Xt61zsf8cAgG3vx2R9qemQtNG8B+NnfdzSE11NG5U4v2swuvs//k1Cm+3/2fpWLJ0Eb169Tr4oJVdSqjotTiWf/opnH7GQSm93Op+3GjesGEHb7/9BI8+enBSKCkpITGx4rfMxYvnHNaxjj8eUlOha9fYtykoSIqpzSkpKWRnZx9WPKXimRQ2AydErbeNlPnihofb0vH8+ve1+JRtX3Bit+Orfb/FxWXfHHfvPvgNs+xkB8zD07bt8PWmsnX37Dm8WBISyiaOoqKzCIX2v+GHQrHvy8wOSkSpqeH5scceXFbRenllycl2wPth9V07dynJWKOG1ba/uLHYr4mbVZ7zquLOO8fxxRdfkJ2dRb9+/Rg4cCAPPPAAxx9/PPn5+axdu5YhQ4awceNGioqKuPnmm7nuuuuA/UPwFBYWMmDAAM455xzef/992rRpw5tvvknDhmV/P6Xxm0F+fj7XX389u3fv5uSTT2batGk0b96cSZMm8eSTT5KYmEjHjh156qmnWLhwITfffHNkH0ZeXl61Jsd4JoXZwBgzmwF0A3b6dj9Bql1SEjRrFp6qWygUS5KpePr++wLat29UpTfuhg2r/41Gqmbs3LHkf5df4evBYJBAIHBY+8w6LouJ/SdW+Pqjjz7Kxx9/TH5++Li5ubl8+OGHfPzxx14Xz2nTptGiRQv27NnDmWeeySWXXELLli3L7Off//43L7/8Mk899RSXXXYZr732GiNGjKjwuKNGjWLy5Mn07t2b++67jwceeICJEyfy6KOPsn79epKTk9mxYwcAjz/+OFOmTKFHjx4UFhaSkpJyWD+DQ/EtKZjZy0AfoJWZbQJ+DSQBOOeeBOYAFwKfA7uBq/yKReqWhITwm3VqatW2z81dR58+ras3KKm3zjrrrDJ9/idNmsSsWbMA2LhxI//+978PSgrt27cnKysLgK5du7Jhw4YK979z50527NhB7969ARg9ejRDhw4FICMjg+HDhzNkyBCGDBmCc44ePXpw6623Mnz4cC6++GLatm1bnc31tffRFYd43QE3+nV8EanbKvtEDzX35bXooapzc3OZP38+S5YsoVGjRvTp06fc7wQkJyd7y4FAgD2He0004u233yYvL4/Zs2czfvx4li5dyrhx4xg4cCBz5syhe/fuzJ8/n9NPP71K+y9P3ekaIyLis7S0tEp79+zcuZPmzZvTqFEjPv30U5YuXXrEx2zatCnNmzdn0aJFADz//PP07t2bUCjExo0b6du3LxMmTGDHjh0UFhbyxRdfkJ6ezh133EFOTg6ffvrpEccQrU70PhIRqQktW7akR48edO7cmQEDBjBw4MAyr/fv358nn3ySjIwMTjvtNLp3714tx50+fbp3o7lDhw4888wzBINBRowYwc6dO3HOccstt9CsWTPuuusuFixYQEJCAp06dWLAgAHVEkMpJQURkSgvvfRSmfXoHovJycm888475W5Xet+gVatWfPzx/tF9/u//yv8ewv333+8tZ2VllXvWsXhx2S8kFhQUMHny5MrCP2K6fCQiIh4lBRER8SgpiIiIR0lBREQ8SgoiIuJRUhAREY+SgohIxI4dO3jiiSeqtO2FF17ojU9UlykpiIhEVJYUSkpKKt12zpw5NPNjhMgapqQgIhIxblx46OysrCxuu+02cnNz6du3L//93/9NRkYGAEOGDKFr16506tSJqVOnetu2a9eOrVu3smHDBs444wyuvfZaOnXqxPnnn1/u2EdvvfUW3bp1Izs7m/POO4/vv/8egMLCQq666irS09PJyMjgtddeA2Du3Ln07NmTzMxMzj33XN9+BvpGs4jUTmPHQn7FQ2c3DAbDT5c6HFlZMLF2DJ19zjnnsHTpUsyMp59+mgkTJvC73/2O8ePH07RpU9asWQPA9u3b2bJlC9deey1z5swhPT2dH3/88fDafRiUFEREKuHX0NmbNm1i2LBhfPvtt+zbt887xvz585kxY4ZXr3nz5rz11lv06tWLdu3aAdCiRYvqbGIZSgoiUjtV8okeYE8dHzr7pptu4tZbb2XQoEHk5uaWGQspnnRPQUQkoiaHzt65cydt2rQBwqOklurXrx9Tpkzx1rdv30737t3Jy8vzzjj8vHykpCAiEhE9dPZtt9120Ov9+/enpKSEjIwM7r333iMaOvv+++9n6NCh9OzZk1atWnnl99xzD9u3b6dz585kZmayYMECjjnmGKZOncqIESPIzMxk2LBhVT7uoejykYhIlJoaOnvw4MEMHjz4oPLU1NQyZw6lBgwYwDnnnOP7JTOdKYiIiEdJQUREPEoKIiLiUVIQERGPkoKIiHiUFERExKOkICISUZNDZ99///08/vjjVTqWn5QUREQiNHS2koKIiKcmh86Olp+fT/fu3cnIyOCiiy5i+/btQHjwvY4dO5KRkcHll18OwMKFC8nKyiIrK4vs7OxKh+WoCn2jWURqpbFzx5L/XcVDZweDQQKHOXR21nFZTOxfO4bOjjZq1CgmT55M7969ue+++3jggQeYOHEijz76KOvXryc5Odm7NPX4448zZcoUevToQWFhISkpKYf1MzgUnSmIiFSivKGzMzMz6d69uzd09oFiGTq71M6dO9mxYwe9e/cGYPTo0eTl5QGQkZHB8OHDeeGFF0hMDH+G79GjB7feeiuTJk1ix44dXnl10ZmCiNRKlX2iByio40Nnx+Ltt98mLy+P2bNnM378eJYuXcq4ceMYOHAgc+bMoXv37syfP5/TTz+9Svsvj69nCmbW38w+M7PPzWxcOa+faGYLzGylma02swv9jEdEpDI1OXR2qaZNm9K8eXMWLVoEwPPPP0/v3r0JhUJs3LiRvn37MmHCBHbs2EFhYSFffPEF6enp3HHHHeTk5PDpp58ecQzRfDtTMLMAMAXoB2wCPjKz2c65tVHV7gFecc792cw6AnOAdn7FJCJSmeihswcMGMDAgQPLvN6/f3+efPJJMjIyOO20045o6Oxo06dP5/rrr2f37t106NCBZ555hmAwyIgRI9i5cyfOOW655RaaNWvGXXfdxYIFC0hISKBTp04MGDCgWmIo5eflo7OAz51zXwKY2QxgMBCdFBzQJLLcFPjGx3hERA6ppobOjn7SWlZWVrlnHYsXLy6zXlBQwOTJkysL/4j5mRTaABuj1jcB3Q6ocz/wrpndBDQGzvMxHhEROYR432i+AnjWOfc7MzsbeN7MOjvnQtGVzOw64DqA1q1bk5ubW6WDFRYWVnnbukptrh+OljY3bdo05n73wWCw2vvo13axtrmoqKjKfw9+JoXNwAlR620jZdGuAfoDOOeWmFkK0Ar4IbqSc24qMBUgJyfHRZ/OHY7c3Fyqum1dpTbXD0dLm9etWxdzj6Ka6n1Um8Ta5pSUFLKzs6t0DD97H30EnGpm7c2sAXA5MPuAOl8D5wKY2RlACrDFx5hERKQSviUF51wJMAaYB6wj3MvoEzN70MwGRar9L3Ctma0CXgaudM45v2ISEZHK+XpPwTk3h3A30+iy+6KW1wI9/IxBRERip2EuRESOQGpqarxDqFZKCiIi4lFSEBGJGDduHFOmTPHWSx+EU1hYyLnnnkuXLl1IT0/nzTffPOS+Khpie+7cuXTp0oXMzEzOPfdcINyl+KqrriI9PZ2MjAxee+216m9cjOL9PQURkXKNHQv5FY+cTTDYkMMcOZusLJhYyTh7w4YNY+zYsdx4440AvPLKK8ybN4+UlBRmzZpFkyZN2Lp1K927d2fQoEGYWYX7Km+I7VAoxLXXXkteXh7t27fnxx9/BGD8+PE0bdqUNWvWAHjPU4gHJQURkYjs7Gx++OEHvvnmG7Zs2ULz5s054YQTKC4u5q677iIvL4+EhAQ2b97M999/z3HHHVfhviZNmsSsWbMAvCG2t2zZQq9evbyhuFu0aAHA/PnzmTFjhrdt8+bNfWxl5ZQURKRWquwTPUBBwR5fvrw2dOhQZs6cyXfffcewYcMAePHFF9myZQvLly8nKSmJdu3alTtkdqlYh9iujXRPQUQkyrBhw5gxYwYzZ85k6NChQHjI7GOPPZakpCQWLFjAV199Vek+Khpiu3v37uTl5bF+/XoA7/JRv379ytzLiOflIyUFEZEonTp1oqCggDZt2nD88ccDMHz4cJYtW0ZOTg4vvvjiIR9q079/f0pKSsjIyODee+/1htg+5phjmDp1KhdffDGZmZnemcg999zD9u3b6dy5M5mZmSxYsMDfRlZCl49ERA5QesO3VKtWrViyZEm5dQsLCw8qq2yI7QEDBhz0DITU1FSmT59exWirl84URETEo6QgIiIeJQUREfEoKYiIiEdJQUREPEoKIiLiUVIQETkCFQ2dXVeH1FZSEBERj5KCiEhEdQ6dXco5x2233Ubnzp1JT0/nb3/7GwDffvstvXr1Iisri86dO7No0SKCwSBXXnmlV/cPf/hDtbfxUPSNZhGplcaOHUt+JWNnB4NBAoc5dnZWVhYTKxlprzqHzi71+uuvk5+fz6pVq9i6dStnnnkmvXr14qWXXuKCCy7g7rvvJhgMsnv3bvLz89m8eTMff/wxADt27Dis9lUHJQURkYjqHDq71OLFi7niiisIBAK0bt2a3r1789FHH3HmmWdy9dVXU1xczJAhQ8jKyqJDhw58+eWX3HTTTQwcOJDzzz+/BlpdlpKCiNRKlX2iBygoKKi1Q2fHolevXuTl5fH2228zcuRIbrvtNkaNGsWqVauYN28eU6ZM4ZVXXmHatGnV0ayY6Z6CiEiU6hg6O1rPnj3529/+RjAYZMuWLeTl5XHWWWfx1Vdf0bp1a6699lquueYaVqxYwdatWwmFQlxyySWMHz+eFStW+NXMCulMQUQkSkVDZ//iF78gJyeHrKysQw6dHe2iiy5iyZIlZGZmYmZMmDCB4447junTp/PYY4+RlJREamoqzz33HJs3b+aqq64iFAoB8Mgjj/jSxsooKYiIHOBIh86OLjczHnvsMR577LEyr48ePZrRo0cftF08zg6i6fKRiIh4lBRERMSjpCAitYpzLt4h1GlH+vNTUhCRWiMlJYVt27YpMVSRc45t27aRkpJS5X3oRrOI1Bpt27Zl06ZNbNmy5ZB1i4qKjujNry6Kpc0pKSm0bdu2ysdQUhCRWiMpKYn27dvHVDc3N5fs7GyfI6pdaqLNunwkIiIeX5OCmfU3s8/M7HMzG1dBncvMbK2ZfWJmL/kZj4iIVM63y0dmFgCmAP2ATcBHZjbbObc2qs6pwJ1AD+fcdjM71q94RETk0Pw8UzgL+Nw596Vzbh8wAxh8QJ1rgSnOue0AzrkffIxHREQOwfzq+mVmlwL9nXO/jKyPBLo558ZE1XkD+H9ADyAA3O+cm1vOvq4DrgNo3bp11xkzZlQppsLCwjr7iLyqUpvrB7W5fjiSNvft23e5cy7nUPXi3fsoETgV6AO0BfLMLN05V+bJEs65qcBUgJycHNenT58qHSw3N5eqbltXqc31g9pcP9REm2O6fGRmPcyscWR5hJn93sxOOsRmm4ETotbbRsqibQJmO+eKnXPrCZ81nBpb6CIiUt1ivafwZ2C3mWUCtwNfAc8dYpuPgFPNrL2ZNQAuB2YfUOcNwmcJmFkr4D+BL2OMSUREqlmsSaHEhW8+DAb+6Jz7I1DpI4+ccyXAGGAesA54xTn3iZk9aGaDItXmAdvMbC2wALjNObetKg0REZEjF+s9hQIzuxMYAfQyswQg6VAbOefmAHMOKLsvatkBt0YmERGJs1jPFIYBe4FrnHPfEb4/8Fjlm4iISF0T85kC4ctGQTP7T+B04GX/whIRkXiI9UwhD0g2szbAP4CrgGf9CkpEROIj1qRgzrndwMXAZOfcRUBn/8ISEZF4iDkpmNnZwHDg7cPcVkRE6ohY39jHEh64blakW2kHwl1IRUTkKBLTjWbn3EJgoZmlmlmqc+5L4Ff+hiYiIjUt1mEu0s1sJfAJsNbMlptZJ39DExGRmhbr5aO/ALc6505yzp0I/C/wlH9hiYhIPMSaFBo757x7CM65XKCxLxGJiEjcxPrltS/N7F7g+cj6CDRwnYjIUSfWM4WrgWOA1yPTMZEyERE5isTa+2g76m0kInLUqzQpmNlbQIXP63TODaroNRERqXsOdabweI1EISIitUKlSSHypbUyzKyLc26FfyGJiEi8VGX8oqerPQoREakVqpIUrNqjEBGRWqEqSeGBao9CRERqhVjHPrrIzJoCOOfeMLNmZjbE39BERKSmxXqm8Gvn3M7SFefcDuDX/oQkIiLxEmtSKK9erENkiIhIHRFrUlhmZr83s5Mj0++B5X4GJiIiNS/WpHATsA/4GzADKAJu9CsoERGJj1jHPtoFjPM5FhERibNYex+9Z2bNotabm9k8/8ISEZF4iPXyUatIjyPAGzX1WH9CEhGReIk1KYTM7MTSFTNrRyWjp4qISN0Ua7fSu4HFZraQ8DAXPYHrfItKRETiItYbzXPNLIdwIlgJvAHs8TMwERGpebHeaP4l8A/gf4H/I/ys5vtj2K6/mX1mZp+bWYW9l8zsEjNzkcQjIiJxEus9hZuBM4GvnHN9gWxgS2UbmFkAmAIMADoCV5hZx3LqpUX2/8FhxC0iIj6INSkUOeeKAMws2Tn3KXDaIbY5C/jcOfelc24f4S+9DS6n3njgt4S/ECciInEUa1LYFPmewhvAe2b2JvDNIbZpA2yM3kekzGNmXYATnHNvxxiHiIj4KNYbzRdFFu83swVAU2DukRzYzBKA3wNXxlD3OiK9nVq3bk1ubm6VjllYWFjlbesqtbl+UJvrhxpps3POlwk4G5gXtX4ncGfUelNgK7AhMhURPvvIqWy/Xbt2dVW1YMGCKm9bV6nN9YPaXD8cSZuBZS6G9+6qPHktVh8Bp5pZezNrAFwOzI5KRjudc62cc+2cc+2ApcAg59wyH2MSEZFK+JYUnHMlwBhgHrAOeMU594mZPWhmg/w6roiIVJ2vD8pxzs0B5hxQdl8Fdfv4GYuIiByan5ePRESkjlFSEBERj5KCiIh4lBRERMSjpCAiIh4lBRER8SgpiIiIR0lBREQ8SgoiIuJRUhAREY+SgoiIeJQURETEo6QgIiIeJQUREfEoKYiIiEdJQUREPEoKIiLiUVIQERGPkoKIiHiUFERExKOkICIiHiUFERHxKCmIiIhHSUFERDxKCiIi4lFSEBERj5KCiIh4lBRERMSjpCAiIh4lBRER8SgpyFEjFArxl7/8hVdffZWSkpJ4hyNSJ/maFMysv5l9Zmafm9m4cl6/1czWmtlqM/uHmZ3kZzxy9Nq8eTP9+/fn+uuv54knnqBXr1588cUX8Q5LpM7xLSmYWQCYAgwAOgJXmFnHA6qtBHKccxnATGCCX/HI0evVV18lPT2df/3rXzz55JPcfffdrF27lqysLKZNm4ZzLt4hitQZfp4pnAV87pz70jm3D5gBDI6u4Jxb4JzbHVldCrT1MR45yuzcuZPRo0dz2WWXccopp7By5Ur+53/+h/POO4/Vq1eTk5PDNddcw8UXX8zWrVvjHa5InWB+fYoys0uB/s65X0bWRwLdnHNjKqj/J+A759xvynntOuA6gNatW3edMWNGlWIqLCwkNTW1StvWVUdrm1evXs3DDz/Mli1bGDFiBCNHjiQxMRHY3+ZQKMSrr77KX//6V9LS0rj99tvp1q1bnCP3x9H6e66M2nx4+vbtu9w5l3PIis45XybgUuDpqPWRwJ8qqDuC8JlC8qH227VrV1dVCxYsqPK2ddXR1ua9e/e6cePGOTNzJ598snv//fcPqnNgm/Pz812nTp0c4G644Qa3a9euGoq25hxtv+dYqM2HB1jmYnjv9vPy0WbghKj1tpGyMszsPOBuYJBzbq+P8Ugdt3btWrp3786jjz7KNddcQ35+PmefffYht8vMzGTZsmXccsstPPHEE3Tp0oXly5fXQMQidY+fSeEj4FQza29mDYDLgdnRFcwsG/gL4YTwg4+xSB0WCoWYPHkyXbt2ZePGjbzxxhs89dRTh3UanZKSwu9//3vmz59PYWEh3bt356GHHiIYDPoYuUjd41tScM6VAGOAecA64BXn3Cdm9qCZDYpUewxIBV41s3wzm13B7qSe+uabbxgwYAC/+tWv+PnPf86aNWsYPHjwoTeswLnnnsvq1au55JJLuOeee+jduzfr16+vxohF6jZfv6fgnJvjnPtP59zJzrmHImX3OedmR5bPc861ds5lRaZBle9R6pOZM2eSnp7OokWL+POf/8zf//53jjvuuCPeb4sWLXj55Zd54YUXWLNmDRkZGTzzzDPquiqCvtEstdBPP/3ElVdeydChQ+nQoQMrV67k+uuvx8yq7RhmxvDhw1m9ejVdu3bl6quv5tJLL1XXVan3lBSkVlm0aBGZmZk8//zz3Hvvvbz//vucdtppvh3vpJNO4h//+AcTJkzgrbfeIj09nblz5/p2PJHaTklBaoV9+/Zx55130rt3bxISEli0aBEPPvggSUlJvh87EAhw22238eGHH9KyZUsGDBjATTfdxO7duw+9schRRklB4m7dunVeV9Orr76a/Px8fvazn9V4HFlZWSxbtoyxY8fypz/9ia5du7JixYoaj0MknpQUJG6cc0yePJkuXbqwceNGZs2axdNPP01aWlrcYkpJSeEPf/gD7733Hj/99BPdunXjkUceUddVqTeUFCQuorua9u3blzVr1jBkyJB4h+U577zzWLNmDRdffDF33XUXffr0UddVqReUFKTGvfbaa6Snp5OXl8eUKVN4++23q6WraXVr0aIFM2bM4Pnnn2f16tVkZmYyffp0dV2Vo5qSgtSYn376iauuuopLL72U9u3bs3LlSm644YZq7Wpa3cyMESNGsHr1arKzs72ustu2bYt3aCK+UFKQGrF48WIyMzN57rnnuOeee1iyZImvXU2r20knncQ///lPfvvb3zJ79mzS09OZN29evMMSqXZKCuKrffv2cffdd9O7d2/MjEWLFjF+/Pga6Wpa3QKBALfffjsffPABzZs3p3///vzqV79iz5498Q5NpNooKYhv1q1bx9lnn83DDz/MlVdeyapVq+LS1bS6ZWdns2zZMm6++WZvoD51XZWjhZKCVDvnHFOmTKFLly589dVXvP76696Dbo4WDRs2ZOLEibz77rvs3LmT7t27q+uqHBWUFKRaffvtt1x44YWMGTOGPn36sGbNGi666KJ4h+Wbfv36sXr1agYPHux1Xd2wYUO8wxKpMiUFqTavv/466enp5ObmMmXKFObMmcPxxx8f77B817JlS1555RWmT5/OqlWryMjIUNdVqbOUFOSIFRQUcPXVV3PJJZfQrl27OtHVtLqZGaNGjfK+z3DllVdy2WWXqeuq1DlKCnJE/vWvf3lf6rr77rt5//33Of300+MdVty0a9eO3NxcHnnkEd58803S09N599134x2WSMyUFKRKiouLueeee+jVqxcAeXl5/OY3v6FBgwZxjiz+AoEA48aN44MPPqBZs2ZccMEa5fRAAAAMcUlEQVQF3Hzzzeq6KnWCkoIctk8//ZSzzz6bhx56iNGjR5Ofn0+PHj3iHVatk52dzfLly7npppuYNGkSOTk5rFy5Mt5hiVRKSUFiFt3VdMOGDbz22mtMmzaNJk2axDu0Wqthw4ZMmjSJuXPnsn37drp168Zvf/tbdV2VWktJQWIS3dW0V69e3giiEpsLLriANWvWMGjQIMaNG0ffvn3VdVVqJSUFOaRZs2Z5XU0nT57MO++8Uy+6mla3li1b8uqrr/Lss8+Sn5/vPXZUXVelNlFSkAqVdjW9+OKLOfHEE1mxYgVjxoypV11Nq5uZMXr0aFatWkV6ejqjRo1i2LBh/Pjjj/EOTQRQUpAKvP/++2RlZfHss89y5513snTpUs4444x4h3XUaN++PQsXLuThhx/2zsTmz58f77BElBSkrNKupj179iQUCpGXl8fDDz+srqY+CAQCXsJt0qQJ/fr1Y+zYseq6KnGlpCCezz77jJ/97Gc89NBDjBw5klWrVnHOOefEO6yjXteuXVm+fDljxozhj3/8Izk5OeTn58c7LKmnlBQE5xxPPPEE2dnZfPnll8ycOZNnn31WXU1rUKNGjbyb+D/++CNnnXUWEyZMUNdVqXFKCvXcd999x8CBA7nxxhvp2bMna9as4ZJLLol3WPVW//79WbNmDb/4xS+44447+PnPf85XX30V77CkHlFSqMfeeOMN0tPTWbBgAZMmTeKdd97hP/7jP+IdVr3XqlUrZs6cyTPPPMOKFSvIyMjghRdeUNdVqRFKCvVQQUEBv/zlL7noooto27atNxRDQoL+HGoLM/OeVpeens7IkSO5/PLL1XVVfFdv3gW+//571q9fz9dff8327dspKSmJd0hxsWTJErKyspg2bZo3aFvHjh3jHZZUoEOHDixcuJCHHnqI119/nYyMDHVdFV8l+rlzM+sP/BEIAE875x494PVk4DmgK7ANGOac2+BHLM899xy33357mbKGDRuSlpZGkyZNSEtLO6Ll2t5ls7i4mPHjx/PQQw9xwgknsHDhQnr27BnvsCQGgUCAu+66iwsuuIDhw4d7XVcfeeQRUlJS4h2eHGV8SwpmFgCmAP2ATcBHZjbbObc2qto1wHbn3ClmdjnwW2CYH/G0yWlD3xv60qJJC/bt3kfxnmL27d7Hvt372Lt7L3t37+X7Xd/z9Zav2bNrD0W7iijaXcTePXtj2n9SgyTS0tJonNqY1LRUmqQ1Ia1JGk3SmtC0SVOaNGlCs6bNYkoyycnJ1fqt4c8++4yRI0fy0UcfMWrUKCZNmkTTpk2rbf9SM7p27cqKFSu4/fbbmThxIu+99x4vvvgimZmZ8Q5NjiJ+nimcBXzunPsSwMxmAIOB6KQwGLg/sjwT+JOZmfPhjtoL37zAgmMXhFcO58NVENgH7I2al7NcvLeYH/f+yI/7fgyXfQt8dUD9fTEeMwFIjkwNwnNLNm/dUgxrYOGyFMLLKUZCcsJB873L97L3nb1YotF0ZFPey3iPztM6E7AACZZAICFAwCJTQoDEhEQSLIHEhMSDpoAFSAokkZiQSFJCEkkJSSQGwq8lBZJokNCApEBSeLLwvEGgQZkpKZBEciCZBoEGJCcmkxxIJjkxmaSEJG85JTGF5EB43iCxASmJKSQm+HpSWyWhUIgQIUpCJYRCkXlk3StzJTjn9pe5EMFQ0KsXDAUJufCywx1UVlq3tCzogvT4nx406tiIqb+eSs6ZOQy5YQgnnnki61euJ5AQIIEELMFItETMjARLwDASEyLrhH/vCZbg/Q0kWEKl5QkJCd7fTOnfR5l6kTqJCYnlLicQWdd9q1rPz/+0NsDGqPVNQLeK6jjnSsxsJ9AS2Frdwfzm57/hDDuDU049haALev9wQRckGApSHComFAr/03mvu6BXVvpPXvp66bYhF/L+gUuXD3zNKwsGKS4qpnjP/qlkT0l4KiohWBQkuCfoLYeKQt48VBQiVBCZ7w1PxJg6A6cESLo4iX1N9rF3z15ChHDO4XBl5gAu1p3GmRE+kzIzDNs/x3DOYYvMa4s3j/qsEd3OAz+D1JWfAVcBb8HMP86MdyRSQzpe0ZFP+nzi6zFq38evcpjZdcB1AK1btyY3N7dK++nbpC+pu1IrOVBkgvBdkFrMOUdRURG7d++udEpLS+PCCy+s8ie0UChECSXsC+1jX2gfxaFib17s9i+XuJJwWfSyKw4nXFcc/qTsSsLLrqRM4i1xJQeVVTSVfloOEkm6hLwybz0UIpAQ/gV6qSIqaZQuAyRYglevdDkh0v+i9BN2me0jn7TLvF5JWel+y9tHafmB83JfP7CMcJmdYXy97Gu2fb2NhEACuHBSi06E0cnRhSuUrRP1oeBQ9UrLvPVyjlFp3Uh5meOUU17u/MCk7qj2wRlL/y5qetsKtz+g6IxTzqjy+1+s/EwKm4ETotbbRsrKq7PJzBKBpoRvOJfhnJsKTAXIyclxffr0qVJAubm5VHXbukptrgeuqodtRm32i58X+D4CTjWz9mbWALgcmH1AndnA6MjypcA//bifICIisfHtTCFyj2AMMI/wxZhpzrlPzOxBYJlzbjbwV+B5M/sc+JFw4hARkTjx9Z6Cc24OMOeAsvuilouAoX7GICIisVP/MBER8SgpiIiIR0lBREQ8SgoiIuJRUhAREY/Vta8FmNkWwqMKVUUrfBhCo5ZTm+sHtbl+OJI2n+ScO+ZQlepcUjgSZrbMOZcT7zhqktpcP6jN9UNNtFmXj0RExKOkICIinvqWFKbGO4A4UJvrB7W5fvC9zfXqnoKIiFSuvp0piIhIJepNUjCz/mb2mZl9bmbj4h2P38xsmpn9YGYfxzuWmmJmJ5jZAjNba2afmNnN8Y7Jb2aWYmYfmtmqSJsfiHdMNcHMAma20sz+Hu9YaoKZbTCzNWaWb2bLfD1Wfbh8ZGYB4P8B/Qg/FvQj4Arn3NpKN6zDzKwXUAg855zrHO94aoKZHQ8c75xbYWZpwHJgyFH+ezagsXOu0MySgMXAzc65pXEOzVdmdiuQAzRxzv1XvOPxm5ltAHKcc75/L6O+nCmcBXzunPvSObcPmAEMjnNMvnLO5RF+RkW94Zz71jm3IrJcAKwj/Bzwo5YLK4ysJkWmo/qTnpm1BQYCT8c7lqNRfUkKbYCNUeubOMrfLOo7M2sHZAMfxDcS/0UupeQDPwDvOeeO9jZPBG4HQvEOpAY54F0zWx55Zr1v6ktSkHrEzFKB14Cxzrmf4h2P35xzQedcFuHnoJ9lZkft5UIz+y/gB+fc8njHUsPOcc51AQYAN0YuD/uiviSFzcAJUettI2VylIlcV38NeNE593q846lJzrkdwAKgf7xj8VEPYFDkGvsM4Odm9kJ8Q/Kfc25zZP4DMIvwJXFf1Jek8BFwqpm1N7MGhJ8FPTvOMUk1i9x0/Suwzjn3+3jHUxPM7BgzaxZZbki4M8Wn8Y3KP865O51zbZ1z7Qj/H//TOTcizmH5yswaRzpOYGaNgfMB33oV1ouk4JwrAcYA8wjffHzFOfdJfKPyl5m9DCwBTjOzTWZ2TbxjqgE9gJGEPz3mR6YL4x2Uz44HFpjZasIfft5zztWLbpr1SGtgsZmtAj4E3nbOzfXrYPWiS6qIiMSmXpwpiIhIbJQURETEo6QgIiIeJQUREfEoKYiIiEdJQaQGmVmf+jKyp9RNSgoiIuJRUhAph5mNiDynIN/M/hIZdK7QzH5nZivM7B9mdkykbpaZLTWz1WY2y8yaR8pPMbP5kWcdrDCzkyO7TzWzmWb2qZm9GPkmtkitoKQgcgAzOwMYBvSIDDQXBIYDjYEVkYHJFgK/jmzyHHCHcy4DWBNV/iIwxTmXCfwM+DZSng2MBToCHQh/E1ukVkiMdwAitdC5QFfgo8iH+IaEh6UOAX+L1HkBeN3MmgLNnHMLI+XTgVcjY9W0cc7NAnDOFQFE9vehc25TZD0faEf44TgicaekIHIwA6Y75+4sU2h27wH1qjpGzN6o5SD6P5RaRJePRA72D+BSMzsWwMxamNlJhP9fLo3U+W9gsXNuJ7DdzHpGykcCCyNPfttkZkMi+0g2s0Y12gqRKtAnFJEDOOfWmtk9hJ90lQAUAzcCu4BOZrYc2En4vgPAaODJyJv+l8BVkfKRwF/M7MHIPobWYDNEqkSjpIrEyMwKnXOp8Y5DxE+6fCQiIh6dKYiIiEdnCiIi4lFSEBERj5KCiIh4lBRERMSjpCAiIh4lBRER8fx/zZXaQ9guf58AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12bdffca7b8>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt0HPV5//H3I9myfIuvRDHYWAYcEzsxciyMc9zEJk352aEHQkMbIDTQ0rrJr6QktCHm9BcCSf9IqEsIlwRMSy6kgTRAqROcEKCsCYkBGyKCje0gG4PMxeCLhCVLyJKe3x87u16ttNpdSaO9zOd1jo93Z2d3nu/Map75fp/ZGXN3REREACoKHYCIiBQPJQUREUlSUhARkSQlBRERSVJSEBGRJCUFERFJUlIQEZEkJQUREUlSUhARkaRRhQ4gX9OnT/fa2tpBvbetrY3x48cPb0BFpJzbV85tg/JuXzm3DUqnfc8888x+dz8u23wllxRqa2vZsmXLoN4bi8VYsWLF8AZURMq5feXcNijv9pVz26B02mdmL+cyn4aPREQkSUlBRESSlBRERCSp5GoKIlK+jh49yt69e+no6Ch0KDmbNGkS27dvL3QYSdXV1cycOZPRo0cP6v1KCiJSNPbu3cvEiROpra3FzAodTk4OHz7MxIkTCx0GAO7OgQMH2Lt3L3PmzBnUZ2j4SESKRkdHB9OmTSuZhFBszIxp06YNqaelpCAiRUUJYWiGuv4ikxSeeOUJ7nzpTjq7OwsdiohI0YpMUtjUtIm7XrmLo91HCx2KiBSp5uZmvvOd7wzqvR//+Mdpbm7Oef5rr72WtWvXDmpZYYpMUkhwvNAhiEiRGigpdHV1DfjeDRs2MHny5DDCGlGRSQoapxSRbNasWcOuXbuoq6vjS1/6ErFYjDPPPJOLLrqIhQsXAvCJT3yCxYsXs2DBAtatW5d8b21tLfv372fPnj28733v42//9m9ZsGABZ511Fu3t7QMut6GhgaVLl7Jw4ULOO+88Dh06BMBNN93E/PnzWbhwIRdccAEAGzdupK6ujrq6OhYtWsThw4eHdR1E7pRUd/UURErBF375BRreaBjWz6x7Tx03rrwx4+vf+MY32Lp1Kw0N8eXGYjGefvpptm7dmjzF884772Tq1Km0t7dz+umnc9ZZZ/U5JfXFF1/k7rvv5o477uAv/uIvuO+++7j44oszLvczn/kMN998M8uXL+eaa67huuuu48Ybb+Qb3/gGL730EmPGjEkOTa1du5Zbb72VZcuW0draSnV19VBXSy/R6SmgnoKI5G/JkiW9zvm/6aabOO2001i6dClNTU3s2rWrz3vmzJlDXV0dAIsXL2bPnj0ZP7+lpYXm5maWL18OwCWXXMLjjz8OwMKFC/n0pz/Nj370I0aNih/DL1u2jCuvvJKbbrqJ5ubm5PThEr2egmoKIiVhoCP6kZR6WexYLMYjjzzCpk2bGDduHCtWrOCdd97p854xY8YkH1dWVmYdPsrkwQcf5PHHH2f9+vV8/etfZ9u2baxZs4azzz6bDRs2sHTpUh555BFOPfXUQX1+f6LTU1BNQUSymDhx4oBj9C0tLUyZMoVx48axY8cOnnzyySEvc9KkSUyZMoVf//rXANx1110sX76cnp4empqaOPPMM7n++utpbm6mtbWVXbt28YEPfIAvf/nL1NfXs2PHjiHHkCp6PQXVFEQkg2nTprFs2TLe//73s2rVKs4+++xer69cuZLbbruNhQsXMm/ePJYuXTosy/3BD37AZz/7WY4cOcJJJ53E9773Pbq7u7n44otpaWnB3fniF7/I5MmT+cpXvsJjjz1GRUUFCxYsYNWqVcMSQ0JkkoJqCiKSix//+Me9nqfeQGfMmDH84he/6PV6omeRqBtMnz6drVu3Jl//p3/6p36Xc+211yYf19XV9dvreOKJJ/pMu/nmmweMf6giM3yUoJqCiEhmoSYFM1tpZjvNrNHM1vTz+qVm9paZNQT//ibEWML6aBGRshHa8JGZVQK3An8C7AU2m9l6d38hbdafuPvlYcUhIqXF3XUQNwRDrZuG2VNYAjS6+2537wTuAc4NcXk5UaFZpHhVV1dz4MAB/Z0OUuJ+CkP5QVuYheYTgKaU53uBM/qZ75Nm9hHgD8AX3b0pfQYzWw2sBqipqSEWi+UdTOPeRiBeuJk4ujhuiDHcWltbB7VuSkE5tw3Ku335tM3MGD9+PE1NfXYDRavYejbd3d20tbXx8ssvD+r9hT776GfA3e7+jpn9HfAD4KPpM7n7OmAdQH19vaeeDZCr5558DnbBsj9axtSxU4cWdZGKxWIMZt2UgnJuG5R3+8q5bVB+7Qtz+OhVYFbK85nBtCR3P+DuiZ8D/juwOKxgiimTi4gUqzCTwmZgrpnNMbMq4AJgfeoMZjYj5ek5QOh3v9ZYpYhIZqENH7l7l5ldDjwEVAJ3uvs2M/sasMXd1wP/YGbnAF3AQeDSsOLRj9dERLILtabg7huADWnTrkl5fDVwdZgx9IlJP14TEckoMr9oVk1BRCS7yCSFBNUUREQyi0xSUE1BRCS7yCSFBNUUREQyi0xSUE1BRCS7yCSFBNUUREQyi0xSUE1BRCS7yCSFBNUUREQyi0xSUE1BRCS7yCSFBNUUREQyi0xSUE1BRCS7yCSFBNUUREQyi0xSUE1BRCS7yCSFBNUUREQyi0xSUE1BRCS7yCQFERHJLnJJQYVmEZHMIpMUVGgWEckuMkkhQYVmEZHMIpMUVGgWEckuMkkhQTUFEZHMIpMUVFMQEckuMkkhQTUFEZHMIpMUVFMQEckuMkkhQTUFEZHMIpMUVFMQEckuMkkhQTUFEZHMIpMUVFMQEcku1KRgZivNbKeZNZrZmgHm+6SZuZnVhxkPqKYgIjKQ0JKCmVUCtwKrgPnAhWY2v5/5JgJXAE+FFUuwnDA/XkSkLITZU1gCNLr7bnfvBO4Bzu1nvq8D3wQ6QowlSTUFEZHMRoX42ScATSnP9wJnpM5gZh8EZrn7g2b2pUwfZGargdUANTU1xGKxvIPZsW8HAE8+9SRNY5uyzF2aWltbB7VuSkE5tw3Ku33l3DYov/aFmRQGZGYVwA3Apdnmdfd1wDqA+vp6X7FiRd7La3quCXbAGWecwSlTT8n7/aUgFosxmHVTCsq5bVDe7SvntkH5tS/M4aNXgVkpz2cG0xImAu8HYma2B1gKrA+r2KyagohIdmEmhc3AXDObY2ZVwAXA+sSL7t7i7tPdvdbda4EngXPcfUuIMammICIygNCSgrt3AZcDDwHbgf9y921m9jUzOyes5Wai3ymIiGQXak3B3TcAG9KmXZNh3hVhxiIiItlF5hfNCfrxmohIZpFJCio0i4hkF5mkkKBCs4hIZpFJCio0i4hkF5mkkKCagohIZpFJCqopiIhkF5mkkKCagohIZpFJCqopiIhkF5mkkKCagohIZpFJCqopiIhkF5mkkKCagohIZpFJCqopiIhkF5mkkKCagohIZpFJCqopiIhkF5mkkKCagohIZpFJCqopiIhkF5mkkKCagohIZpFJCqopiIhkF5mkkKCagohIZpFJCqopiIhkF5mkkKCagohIZpFJCqopiIhkF5mkICIi2UUuKajQLCKSWWSSggrNIiLZ5ZQUzGyZmY0PHl9sZjeY2exwQwuHCs0iIpnl2lP4LnDEzE4DrgJeBn4YWlQhUKFZRCS7XJNCl8cH488Fvu3u3wYmZnuTma00s51m1mhma/p5/bNm9ryZNZjZE2Y2P7/w86eagohIZrkmhcNmdjVwMfCgmVUAowd6g5lVArcCq4D5wIX97PR/7O4fcPc64Hrghryiz4NqCiIi2eWaFD4FvANc5u5vADOBf83yniVAo7vvdvdO4B7iPY0kd3875el4CH/AXzUFEZHMRuU432Hiw0bdZvZe4FTg7izvOQFoSnm+FzgjfSYz+3vgSqAK+GiO8eRNNQURkexyTQqPAx82synAo8AW4r2HTw81AHe/FbjVzC4C/h9wSfo8ZrYaWA1QU1NDLBbLeznPH3gegC1bttD6rtYhRFy8WltbB7VuSkE5tw3Ku33l3DYov/blmhTM3Y+Y2WXAze5+vZk9l+U9rwKzUp7PDKZlcg/xs5z6cPd1wDqA+vp6X7FiRY5hH9P2hzbYCosXL+b0E07P+/2lIBaLMZh1UwrKuW1Q3u0r57ZB+bUv15qCmdmHiPcMHszxvZuBuWY2x8yqgAuA9WkfOjfl6dnAiznGM2iqKYiIZJZrT+ELwNXAf7v7NjM7CXhsoDe4e5eZXQ48BFQCdwbv/Rqwxd3XA5eb2ceAo8Ah+hk6Gi6qKYiIZJdTUnD3jcBGM5tgZhPcfTfwDzm8bwOwIW3aNSmPr8gz3iHT7xRERDLL9TIXHzCz3wHbgBfM7BkzWxBuaMNLv1MQEcku15rC7cCV7j7b3U8E/hG4I7ywwqOagohIZrkmhfHunqwhuHuM+I/NSoZqCiIi2eVaaN5tZl8B7gqeXwzsDiekcKmmICKSWa49hb8GjgPuD/4dF0wrGaopiIhkl+vZR4fI4WyjUqCagohIZgMmBTP7GQNcpM7dzxn2iEKimoKISHbZegprRySKEaSagohIZgMmheBHa72Y2Qfd/dnwQgqHagoiItnlWmhO9e/DHoWIiBSFwSSFkj7kVqFZRCSzwSSF64Y9ihGgQrOISHa5XvvoPDObBODuD5jZZDP7RLihhUOFZhGRzHLtKXzV3VsST9y9GfhqOCGFQ4VmEZHsck0K/c2X6yUyiopqCiIimeWaFLaY2Q1mdnLw7wbgmTADG26qKYiIZJdrUvg80An8hPi9lDuAvw8rqDCppiAiklmu1z5qA9aEHEuoVFMQEcku17OPHjazySnPp5jZQ+GFFR7VFOCWp29h1X+uKnQYIlKEci0WTw/OOALiV001s3eHFFMoVFM45vO/+HyhQxCRIpVrTaHHzE5MPDGzWga4emoxU01BRCSzXHsK/ww8YWYbiV/m4sPA6tCiCoFqCiIi2eVaaP6lmdUTTwS/Ax4A2sMMLCyqKYiIZJZTUjCzvwGuAGYCDcBSYBPw0fBCG16qKYiIZJdrTeEK4HTgZXc/E1gEvBVaVCFSTUFEJLNck0KHu3cAmNkYd98BzAsvrOGnmoKISHa5Fpr3Br9TeAB42MwOAa+FF1Z4VFMQEcks10LzecHDa83sMWAS8MvQogqBagoiItnlfZMdd9/o7uvdvTPbvGa20sx2mlmjmfW5TIaZXWlmL5jZ783sUTObnW88+VJNQUQks8HceS0nZlYJ3AqsAuYDF5rZ/LTZfgfUu/tC4F7g+tDiUU1BRCSr0JICsARodPfdQa/iHuDc1Bnc/TF3PxI8fZL4Ka+hUk1BRCSzMJPCCUBTyvO9wbRMLgN+EVYwqimIiGRXFHdPM7OLgXpgeYbXVxNcVqOmpoZYLJb3Mra2bAXgueeeo6qparChFrXW1ta81s1g1mOh5Nu2UlPO7SvntkH5tS/MpPAqMCvl+cxgWi9m9jHi11Za7u7v9PdB7r4OWAdQX1/vK1asyDuYqqYqaICFCxey4pT8318KYrEYOa2bjfH/li9fXjI9qJzbVqLKuX3l3DYov/aFOXy0GZhrZnPMrAq4AFifOoOZLQJuB85x9zdDjEWF5n6oviIi6UJLCu7eBVwOPARsB/7L3beZ2dfM7Jxgtn8FJgA/NbMGM1uf4eOGLy7tCJN6vKfQIYhIkQm1puDuG4ANadOuSXn8sTCXn6pUhklGkpKCiKQLc/ioKOnHa8doXYhIusgkBdUU+krvKfzV//wVn/v55woUjYgUg8gkhQTVFI5JXxffb/g+tz1zW4GiEZFiEJmkoJpCX6opiEi6yCSFBI2jH6OkICLpIpMUVFPoSwlSRNJFJikkqKZwjHoKIpIuMklBNYW+lBREJF1kkkKChkyOUa9JRNJFJimoptCXegoiki4ySSFBR8fHqNckIukikxRUU+hLPQURSReZpJCgo+NjlBSKQ1NLE2duPJMHdjxQ6FBkiPY07+GttrcKHcaQRCYpqKbQl4bSisOzrz8LwPcavlfgSGSo5nx7DsffcHyhwxiSyCSFBMe5bctt2HXGwfaDhQ6noNRTEBl+XT1dhQ5hSCKTFFJrCt/d8l0g3m2PMiUFEUkXmaQgfam+IiLpIpcUtCM8Rj0FEUkXmaSgQnNfSgoiki46SSGoKfR4j3oLAZ19VFx+tetXhQ5BJDpJocLiTdXR8TFaF8Wlo6uj0CGIRCcpVFolEN8R6tfNceoxiUi6yCQF9RT60roQkXRKChGmdSEi6ZQUIkyFZhFJF7mk0O3dBY6keChBiki6yCSFyopjhWaJ07oQkXSRSQoaPupLZx+JSLpQk4KZrTSznWbWaGZr+nn9I2b2rJl1mdn5YcaipNCX1oWIpAstKZhZJXArsAqYD1xoZvPTZnsFuBT4cVhxJCgp9KV1ISLpRoX42UuARnffDWBm9wDnAi8kZnD3PcFroe+dlBT60tlHIpIuzOGjE4DUGxbsDaYVRPLso57iPPtoX+s+Ft2+iFdaXhmxZSpBiki6MHsKw8bMVgOrAWpqaojFYnl/xuGjhwHY+eJOWltbAdi8ZTOHJhwatjiH4p6me2h4o4Gr7r2Kz5782UF9Rmtra17r5plnn6FzV2ef6YNZv2HLt22l5Pn9zycfl2Mby3nbQf/tK+X2hpkUXgVmpTyfGUzLm7uvA9YB1NfX+4oVK/L+jJaOFvgtnHzyyYw/PB7aYPHixSyasWgwIQ27zb/ZDLth1qxZDKZ9EP8i5vTejfH/6urq+PDsD/eZPtjlhynntpWglh0tsC3+uBzbWM7bDtLaV8R/Q7kKc/hoMzDXzOaYWRVwAbA+xOUNKLWmkLgg3r/8+l8KFU5R0PCRiKQLLSm4exdwOfAQsB34L3ffZmZfM7NzAMzsdDPbC/w5cLuZbQsrntSkkDg///7t94e1uJKgQrOIpAu1puDuG4ANadOuSXm8mfiwUuh09lFfWhciki5yv2ju9m7dTyGgpCAi6SKTFFKvfaTLO8RpPRQfbRMptMgkBQ0f9aV1UXy0TaTQIpMUjPiQkW7HeYwKzcVH20QKLTpJwQzD+gwfRa27/qtdv0o+1lFp8dE2kUKLTFKAeG8h/TIXUfsjvHzD5cnHUWt7KYjaQYoUn0glhUqr7DN8FOU7sWkHVHyUqKXQIpUUzPoOHxXrBfLCkpoQM+2ATr7pZGJ7YiMUkaTWEVRTKF3lcpAVqaRQQUWfHWHYPYWrHr4Ku86K5ggw9YubKabdh3bzpYe/NFIhRV4u20SKX7kk9EglhURPodfwUcg9hX/b9G8jspzBKJcvcalLTQTlcrQZReWy7SKVFArRUxjp5WSTy/CRjKzU5KxtUrrK5SArUknBzOj27oLUFIqlp6ChiuLT6xTpMtmxRFG5/D1FKimop9DbSHZ3f7/v9zS80TBiyxuMzu5Onn392WH5rD3Ne3j98Os5zZtrT8Hd+eFzP6T9aPuQ48vmR7//EQfbD4a+nKF6q+0tGg82FjoMQMNHJanCKka8pjDSy8km1+GjxC/Ah8tpt53GotuL44ZGmVz50JUsXreYXQd3Dfmz5nx7DsffcHxO8+ZaU3h498Nc8sAlrHlkzZDjG8iLB17kL//7L7novotCXc5wmPWtWcy9eW6hwwDKp5dXErfjHC6G8R+/+w86u4/dgnKkunzF0lPQUEVmT736FAAH2w9yMieP2HJz3SYtHS0AvNb6WqjxtHfFeyKvHh7UjRJH1Dvd7xQ6hCQNH5Wg0RWjeyUEGMHhoyLpKaQqly/xcClU97/YCs3D3UuMCg0flaDRFaP7TBux4aMi6CkcOHKAw52Hk8+LYQckRPpaXOWkXHrekRo+qqqo6jMtSj2F6f86vddz7YB6K9TVc1OTsxJ16SqXv6do9RQs2j2FdNoBFYdiu8xFIoZy2cmNlHL5e4pWUuhv+ChCPYV05fIlLnXF9tsRJYPBKYaEPhwilRT6HT6KcE+hXL7Ew22kd8y9egpFsENOfFd1M6r8FMO2Gw5KCmXeU3D3jMvWNXd6S6yDkU7gxVZT6OrpKnQIJakYtt1wiFRS6K+mcLT76Igsu1A9hfN/ej6jvt7/+QS9kkJaryHKR4kjncCL7bcjpZgUiuGgphi23XCIVFKorKjsMy3xQ52wJI8+C9RTuH/7/Rlf06mQ/RvpnWKx/U4h0f5S+k4Uw/BsKa2vgUQqKXR0d/SZNhLXkYHi+NKmG6inEGUjnhRyTM4jtY1KsadQDDGXy99QpJLCn53wZ32mHTl6ZESWna2nEPYXqr+dTbEVOIvFSO9gcq0pjPR1ukppCLEYkkIx9PKGQ6SSQnVFdZ9pYQ8fJWTrKQznTrm7p5vVP1vNC2+9kJzW3zVi1FPoLbETHOoOJt+dQ66/UxipHV8x7GDzVQynfJfLgVWkftE8pnJMn2kjNXyU7Q9tOI8ytu/fzh3P3sFvm36bnNbW2TbgMsvlCz0UiXUw1J1ivu/PdfhISSGzYoi5XA6sQu0pmNlKM9tpZo1m1ud6v2Y2xsx+Erz+lJnVhhnPpNGT+kwLu6eQOPpM3UH3J7GDXrtpbfJqmIOV+KzUnX7qNY8SBjrr5elXn2b3od1DiqNUDbX+k+9Ra67DR0Pd8TUebMwp+edSaP67n/0dn9/w+SHFM5yKIimUyYFVaEnBzCqBW4FVwHzgQjObnzbbZcAhdz8F+BbwzbDiAXhP9XuYf1zvEK745RUsuWMJnd2ddHT1LUQPly8/8mV27N+R8fXUncHOAzuHtKzD7/RNAP1Ne+3wa/zhwB/6LD/hp9t+OqQ4+lMK464j3lNIGz5qeKOB27bc1me+oSSrJ/c+ydyb53LHs3dknTeX+Nc9u45bNt8y6HiGWzEkhbC/22+2vRnq5yeE2VNYAjS6+2537wTuAc5Nm+dc4AfB43uBP7aQq1vnnXpen2mbX9vMotsXcfy/5XZTlMFqeKOBrp4umjuauerhqzjUfgh3p8d7ev3Bv3jgxYxf8k1Nm1j727U8tfcp7nvhPh5/+XE2NW3ihbdeYFvLNs77yXn9noba37Xx125ay7xb5tF+tL3f4aU1j66h9sZalt25jD3Ne3B33J3dh3bzw+d+SI/3DHh01N+du4ph7DebQg4f9XgP9evq+dyDn2Pn/t4HB0OJK1FfevzlxzPOk9ipJb6LuSShROyJ3/sU6mh5sOumq6dr2GIOc/jowJED1Kyt4eanbg5tGQkW1kY0s/OBle7+N8HzvwTOcPfLU+bZGsyzN3i+K5hnf6bPra+v9y1btgwqplgsxhvT3+DC+y7MOM8pU09JXk++wir6nIHR2d1JV08X40ePz2mZ2/dvH/D1qWOn0tXTxdvvvN1r+oSqCRw/8XhaOlqYXD0ZgKM9R0MZ0plQNYHWztas840fPZ6xo8ey/8ixzTNjwoxkfKkOth9kX9s+aifXUlVZleyRvHfae6m0vr8XyebIkSOMGzcu7/flI7Gt3jPhPUyunpz8HuR7nNLV05Vs7/umvy/r/Il1BTB70mxebnk5+dq8afNw4gcOqbedzOVzUzV3NPN6a/z2oNPHTWf6uN5XzO3o6uC1w68xfdx0Dhw5kDwx4ZSpp1BplZgZhiXXRSLJHDfuOCorKnmz7U1OnHQih9oPMal6Uq+/j3y2Xb7rOhHHyVNOpqqy7xULsnn18KtUVVZx3Ljj8n5vQqJ9R3uOJrdRvtsnm/audvY072HjpRv5yOyPDOozzOwZd6/PNl9JFJrNbDWwGqCmpoZYLDaoz2ltbWVK9xQunX0p+97Zx7yJ89j41kZajrZQYfFO0+xRs5NHDj300x0cFb8JSb+v9eNdU9/F1patOE51ZTUfnPxB3uh4g61vb+Wj7/4obV1tdHs33WO7eb7leQxjStUUTplwCmMqxjBz3Ey66Y7/QVYYPeN6eLvrbaZWTaWxtZG6SXUcOnqISqukubOZ9p52Fk1eRENzA0unLWVMxRi2v72d6spqjnQfYe6EuUwZPQUzo9Iqqaqo4rX219jVtot9Hfv45Amf5PyZ5/PE/id4rSN+h6/f7P8N8981n4OdB3GcOWPmsPPwTuZNnEelVTLG+hbwp1ZP5VD7IWpH18YnTIgfSc2omJH/hgO6xnQxysL9uk6fPp3NBzdz6thTgeDIzyHvA0CD9rHtjBs1jnfbu7PO/u5x72Zf2z5mjp3JSVUnUTOlhqcPPc17J7yXGRUzqKACDGZMn8GWQ1tYMmUJ6ffByXZjnJqxNexv28/y45bT1tXWZ5v5KGf8+PGcMPYExk4cy6/2/Yp5E+YxtXJq8rvu7sl1MXvcbCqtkulV0xlVMYrqnmpOHHUiU8ZOYXzl+F6JP+dtN4h1veBdCzjYeZCZo2bm98ZA5Zj438BEmzio90NK+yrhSPURJo+ezHE2+CTTr9Gw8PiFdO7qJPZSbHg/O11iSGC4/wEfAh5KeX41cHXaPA8BHwoejwL2E/ReMv1bvHixD9Zjjz026PeWgnJuXzm3zb2821fObXMvnfYBWzyHfXeYNYXNwFwzm2NmVcAFwPq0edYDlwSPzwf+NwheREQKILT+uLt3mdnlxHsDlcCd7r7NzL5GPGOtB/4DuMvMGoGDxBOHiIgUSKiDtO6+AdiQNu2alMcdwJ+HGYOIiOQuUpe5EBGRgSkpiIhIkpKCiIgkKSmIiEiSkoKIiCSFdpmLsJjZW8DLWWfs33TiP5ArV+XcvnJuG5R3+8q5bVA67Zvt7ll/al1ySWEozGyL53Dtj1JVzu0r57ZBebevnNsG5dc+DR+JiEiSkoKIiCRFLSmsK3QAISvn9pVz26C821fObYMya1+kagoiIjKwqPUURERkAJFJCma20sx2mlmjma0pdDz5MrNZZvaYmb1gZtvM7Ipg+lQze9jMXgz+nxJMNzO7KWjv783sg4VtQXZmVmlmvzOznwfP55jZU0EbfhJcgh0zGxM8bwxery1k3Lkws8lmdq+Z7TCz7Wb2oTLbdl8MvpdbzexuM6su1e1nZnea2ZvBnSET0/LeVmZ2STD/i2Z2SX/LKkaRSApmVgncCqwC5gMXmtn8wkawRzrSAAAEwklEQVSVty7gH919PrAU+PugDWuAR919LvBo8BzibZ0b/FsNfHfkQ87bFUDq/Uu/CXzL3U8BDgGXBdMvAw4F078VzFfsvg380t1PBU4j3s6y2HZmdgLwD0C9u7+f+KXyL6B0t9/3gZVp0/LaVmY2FfgqcAbx+9V/NZFIil4ud+Ip9X/kcBe4UvsH/A/wJ8BOYEYwbQawM3h8O3BhyvzJ+YrxHzCT+B/bR4GfE7/h5H5gVPo2ZBB37Ctw2yYBL6XHWEbb7gSgCZgabI+fA/+nlLcfUAtsHey2Ai4Ebk+Z3mu+Yv4XiZ4Cx760CXuDaSUp6G4vAp4Catz99eClN4Ca4HGptflG4CpI3vx6GtDs7l3B89T4k20LXm8J5i9Wc4C3gO8Fw2P/bmbjKZNt5+6vAmuBV4DXiW+PZyif7Qf5b6uS2oapopIUyoaZTQDuA77g7m+nvubxQ5KSO53MzP4UeNPdnyl0LCEZBXwQ+K67LwLaODb8AJTutgMIhkXOJZ78jgfG03f4pWyU8rbKRVSSwqvArJTnM4NpJcXMRhNPCP/p7vcHk/eZ2Yzg9RnAm8H0UmrzMuAcM9sD3EN8COnbwGQzS9wdMDX+ZNuC1ycBB0Yy4DztBfa6+1PB83uJJ4ly2HYAHwNecve33P0ocD/xbVou2w/y31altg2TopIUNgNzg7MhqogXwdYXOKa8mJkRv6f1dne/IeWl9UDizIZLiNcaEtM/E5wdsRRoSen+FhV3v9rdZ7p7LfFt87/u/mngMeD8YLb0tiXafH4wf9Eeubn7G0CTmc0LJv0x8AJlsO0CrwBLzWxc8D1NtK8stl8g3231EHCWmU0JelJnBdOKX6GLGiP1D/g48AdgF/DPhY5nEPH/EfEu6++BhuDfx4mPxT4KvAg8AkwN5jfiZ1ztAp4nfmZIwduRQztXAD8PHp8EPA00Aj8FxgTTq4PnjcHrJxU67hzaVQdsCbbfA8CUctp2wHXADmArcBcwplS3H3A38drIUeK9vMsGs62Avw7a2Aj8VaHbles//aJZRESSojJ8JCIiOVBSEBGRJCUFERFJUlIQEZEkJQUREUlSUhDph5nVpl4lM4f5LzWz43OY55ahRycSHiUFkeFxKfFLPIiUNCUFkcxGmdkPguvk3xv8YvcaM9sc3DdgXfBL1vOBeuA/zazBzMaa2elm9lsze87MnjazicFnHm9mvwyusX99Adsm0i8lBZHM5gHr3H0h8Dbwf4Fb3P10j983YCzwp+5+L/FfK3/a3euAbuAnwBXufhrxawO1B59ZB3wK+ADwKTObhUgRUVIQyazJ3X8TPP4R8UuNnBncLex54hfuW9DP++YBr7v7ZgB3f9uPXUL6UXdvcfcO4tcHmh1uE0TyMyr7LCKRlX4NGAe+Q/z6Nk1mdi3x6/jk452Ux93ob1CKjHoKIpmdaGYfCh5fBDwRPN4f3Nfi/JR5DwOJusFOYIaZnQ5gZhNTLiEtUtT0RRXJbAdwiZndTvzqmN8lfnXT54E9xC/JnvB94DYzayd+68lPATeb2Vji9YSPjVzYIoOnq6SKiEiSho9ERCRJSUFERJKUFEREJElJQUREkpQUREQkSUlBRESSlBRERCRJSUFERJL+P+ExSpcxp3+lAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12ef46b52b0>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "inceptionresnet(num_class=5,epochs=20)\t# 5-class classificaiton\n",
    "history_this.loss_plot('epoch')\n",
    "history_this.loss_plot('batch')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Validation accuracy of InceptionResnet: 99.993%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hyperparameter Optimization \n",
    "Use VGG16 as an example.  \n",
    "\n",
    "Tuned hyperparameters of CNN: \n",
    "1. The number of frozen layers\n",
    "2. The number of epochs\n",
    "3. Early stop patience\n",
    "4. Learning rate\n",
    "5. Dropout rate\n",
    "\n",
    "Hyperparameter optimization methods:\n",
    "1. Random search\n",
    "2. Bayesian optimization - Tree Parzen Estimator(BO-TPE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def vgg16( num_class,epochs=20,frozen=15,lr=0.001,patience=2, dropout_rate=0.5,verbose=0, savepath='./VGG16.h5',history=history_this,input_shape=INPUT_SIZE):\n",
    "    model_fine_tune = VGG16(include_top=False, weights='imagenet', input_shape=input_shape)\n",
    "    for layer in model_fine_tune.layers[:frozen]:\t#the number of frozen layers for transfer learning, have tuned from 5-18\n",
    "        layer.trainable = False\n",
    "    for layer in model_fine_tune.layers[frozen:]:\n",
    "        layer.trainable = True\n",
    "    model = GlobalAveragePooling2D()(model_fine_tune.output)\n",
    "    model=Dense(units=256,activation='relu')(model)\n",
    "    model=Dropout(dropout_rate)(model)\n",
    "    model = Dense(num_class, activation='softmax')(model)\n",
    "    model = Model(model_fine_tune.input, model, name='vgg')\n",
    "    opt = keras.optimizers.Adam(lr=lr, beta_1=0.9, beta_2=0.999, epsilon=1e-08)\t#tuned learning rate to be 0.001\n",
    "    model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])\t#set the loss function to be binary crossentropy\n",
    "    #train model\n",
    "    earlyStopping = kcallbacks.EarlyStopping(\n",
    "        monitor='val_acc', patience=patience, verbose=verbose, mode='auto')\t#set early stop patience to save training time\n",
    "    saveBestModel = kcallbacks.ModelCheckpoint(\n",
    "        filepath=savepath,\n",
    "        monitor='val_acc',\n",
    "        verbose=verbose,\n",
    "        save_best_only=True,\n",
    "        mode='auto')\n",
    "    hist = model.fit_generator(\n",
    "        train_generator,\n",
    "        steps_per_epoch=len(train_generator),\n",
    "        epochs=epochs,\n",
    "        validation_data=validation_generator,\n",
    "        validation_steps=len(validation_generator),\n",
    "        #use_multiprocessing=True, \n",
    "        #workers=2,\n",
    "        callbacks=[earlyStopping, saveBestModel, history],\n",
    "        verbose = verbose\n",
    "    )\n",
    "    return hist\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def prediction(vgg_model):\n",
    "#read images from validation folder\n",
    "    rootdir = './test_224/'\n",
    "    test_laels = []\n",
    "    test_images=[]\n",
    "    for subdir, dirs, files in os.walk(rootdir):\n",
    "        for file in files:\n",
    "            if not (file.endswith(\".jpeg\"))|(file.endswith(\".jpg\"))|(file.endswith(\".png\")):\n",
    "                continue\n",
    "            test_laels.append(subdir.split('/')[-1])\n",
    "            test_images.append(os.path.join(subdir, file))\n",
    "\n",
    "    predict=[]\n",
    "    length=len(test_images)\n",
    "    label=validation_generator.class_indices\n",
    "    label={v: k for k, v in label.items()}\n",
    "    for i in range(length):\n",
    "        inputimg=test_images[i]\n",
    "        test_batch=[]\n",
    "        thisimg=np.array(Image.open(inputimg))/255 #read all the images in validation set\n",
    "        #print(thisimg)\n",
    "        test_shape=(1,)+thisimg.shape\n",
    "        thisimg=thisimg.reshape(test_shape)\n",
    "        vgg_model_batch=vgg_model.predict(thisimg) #use master model to process the input image\n",
    "        #generate result by model 1\n",
    "        prob=vgg_model_batch[0,np.argmax(vgg_model_batch,axis=1)[0]]\n",
    "        res=label[np.argmax(vgg_model_batch,axis=1)[0]]\n",
    "        predict.append(res)\n",
    "    acc=accuracy_score(test_laels,predict)\n",
    "    return acc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#define the objective function to be optimized\n",
    "import time\n",
    "from hyperopt import hp, fmin, tpe, rand, STATUS_OK, Trials\n",
    "import matplotlib.pyplot as plt\n",
    "import statistics \n",
    "\n",
    "def objective(params):\n",
    "    \n",
    "    params = {\n",
    "        'frozen': int(params['frozen']),\n",
    "        'epochs': int(params['epochs']),\n",
    "        'patience': int(params['patience']),\n",
    "        'lr': abs(float(params['lr'])),\n",
    "        'dropout_rate': abs(float(params['dropout_rate'])),\n",
    "    }\n",
    "    frozen=params['frozen']\n",
    "    epochs=params['epochs']\n",
    "    patience=params['patience']\n",
    "    lr=params['lr']\n",
    "    dropout_rate=params['dropout_rate']\n",
    "\n",
    "    vgg16(num_class=5, frozen=frozen,epochs=epochs,patience=patience, lr=lr, dropout_rate=dropout_rate)\n",
    "\n",
    "    acc=prediction(vgg_model=load_model('./VGG16.h5'))\n",
    "\n",
    "    print('accuracy:%s'%acc)\n",
    "    return {'loss': -acc, 'status': STATUS_OK }\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy:0.9322497861420017\n",
      "accuracy:1.0\n",
      "accuracy:1.0\n",
      "accuracy:0.89803250641574\n",
      "accuracy:0.8643284858853721\n",
      "accuracy:1.0\n",
      "accuracy:1.0\n",
      "accuracy:1.0\n",
      "accuracy:1.0\n",
      "accuracy:1.0\n",
      "100%|█████████████████████████████████████████████████████████████| 10/10 [58:16<00:00, 349.70s/trial, best loss: -1.0]\n",
      "Hyperopt estimated optimum {'lr': 0.002, 'patience': 4.0, 'epochs': 15.0, 'dropout_rate': 0.4, 'frozen': 17.0}\n",
      "Time: 3496.9893045425415\n"
     ]
    }
   ],
   "source": [
    "#Hyperparameter optimization by Bayesian optimization - Tree Parzen Estimator\n",
    "space = {\n",
    "    'frozen': hp.quniform('frozen', 15, 18, 1),\n",
    "    'epochs': hp.quniform('epochs', 5, 21, 5),\n",
    "    'patience': hp.quniform('patience', 2, 4, 1),\n",
    "    'lr': hp.quniform('lr', 0.001, 0.006, 0.001),\n",
    "    'dropout_rate': hp.quniform('dropout_rate', 0.3, 0.6, 0.1),\n",
    "}\n",
    "\n",
    "t1=time.time()\n",
    "best = fmin(fn=objective,\n",
    "            space=space,\n",
    "            algo=tpe.suggest,\n",
    "            max_evals=10)\n",
    "print(\"Hyperopt estimated optimum {}\".format(best))\n",
    "t2=time.time()\n",
    "print(\"Time: \"+str(t2-t1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy:0.8643284858853721\n",
      "accuracy:1.0\n",
      "accuracy:0.9707442258340462\n",
      "accuracy:1.0\n",
      "accuracy:1.0\n",
      "accuracy:1.0\n",
      "accuracy:0.9615055603079555\n",
      "accuracy:0.8643284858853721\n",
      "accuracy:1.0\n",
      "accuracy:0.8643284858853721\n",
      "100%|███████████████████████████████████████████████████████████| 10/10 [1:04:57<00:00, 389.76s/trial, best loss: -1.0]\n",
      "Hyperopt estimated optimum {'lr': 0.002, 'patience': 3.0, 'epochs': 15.0, 'dropout_rate': 0.6000000000000001, 'frozen': 16.0}\n",
      "Time: 3897.5851550102234\n"
     ]
    }
   ],
   "source": [
    "#Hyperparameter optimization by Random search\n",
    "space = {\n",
    "    'frozen': hp.quniform('frozen', 15, 18, 1),\n",
    "    'epochs': hp.quniform('epochs', 5, 21, 5),\n",
    "    'patience': hp.quniform('patience', 2, 4, 1),\n",
    "    'lr': hp.quniform('lr', 0.001, 0.006, 0.001),\n",
    "    'dropout_rate': hp.quniform('dropout_rate', 0.3, 0.6, 0.1),\n",
    "}\n",
    "\n",
    "t1=time.time()\n",
    "best = fmin(fn=objective,\n",
    "            space=space,\n",
    "            algo=rand.suggest,\n",
    "            max_evals=10)\n",
    "print(\"Hyperopt estimated optimum {}\".format(best))\n",
    "t2=time.time()\n",
    "print(\"Time: \"+str(t2-t1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/15\n",
      "183/183 [==============================] - 72s 395ms/step - loss: 0.0321 - acc: 0.9897 - val_loss: 9.1345e-07 - val_acc: 1.0000\n",
      "\n",
      "Epoch 00001: val_acc improved from -inf to 1.00000, saving model to ./VGG16.h5\n",
      "Epoch 2/15\n",
      "183/183 [==============================] - 70s 380ms/step - loss: 3.9581e-05 - acc: 1.0000 - val_loss: 3.1735e-07 - val_acc: 1.0000\n",
      "\n",
      "Epoch 00002: val_acc did not improve from 1.00000\n",
      "Epoch 3/15\n",
      "183/183 [==============================] - 70s 380ms/step - loss: 1.3703e-05 - acc: 1.0000 - val_loss: 1.7535e-07 - val_acc: 1.0000\n",
      "\n",
      "Epoch 00003: val_acc did not improve from 1.00000\n",
      "Epoch 4/15\n",
      "183/183 [==============================] - 70s 380ms/step - loss: 4.8536e-06 - acc: 1.0000 - val_loss: 2.1615e-07 - val_acc: 1.0000\n",
      "\n",
      "Epoch 00004: val_acc did not improve from 1.00000\n",
      "Epoch 00004: early stopping\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x28999cdc438>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Retrain the model by using the best hyperparameter values to obtain the best model\n",
    "vgg16(num_class=5, frozen=16,epochs=15,patience=3, lr=0.002, dropout_rate=0.6,verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}