3 Evaluation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we load the model we trained in the last step. If you're using a GPU, this will start it up, which can takeup to **10 seconds**. If you're using a GPU with less memory like the 750M, make sure to close the previous notebook or you will run out of GPU memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import keras\n",
    "from keras.models import model_from_json\n",
    "model = model_from_json(open('model.json').read())\n",
    "model.load_weights('weights.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's test the model by defining a function that evaluates a single image and prints a corresponding indicator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "def print_indicator(data, model, class_names, bar_width=50):\n",
    "    probabilities = model.predict(np.array([data]))[0]\n",
    "    print(probabilities)\n",
    "    left_count = int(probabilities[1] * bar_width)\n",
    "    right_count = bar_width - left_count\n",
    "    left_side = '-' * left_count\n",
    "    right_side = '-' * right_count\n",
    "    print class_names[0], left_side + '###' + right_side, class_names[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll test it on one image from our data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAAAAABWESUoAAACzklEQVR4nAXBW5LTRhQA0PvqbsmW\n7BkMhBBqWAHrzZ6ygVQ+qKQqBGY8DNiSLPXr9s05+Pvo2TELBmHBZpqgZMu322f/hqa3C9UEAIQG\ngNSKqRIioqZoz5ksolgxZkYkaMUaViOyy/M13lKaPd6CALIjBS0qnjypWP7x7SfsjnpJimsQg9Ly\nmtOW/XC676GCJbtj2ta43IFW4XhjRuh9sbLsfceW1xS3ZYtte3VqUXKUQZh1rbXGbU9o8+PTswYs\n5/b4a1Ypoc/nackugHQ0Ok+ruQPtOnx/Y+2vsu8v3z4vNr5/l6Hj6I3k1HGgSJx/LLsXgfUWhw9u\nf3J24i121ML9nQfLYBi+7kV2tH/XkDy3EGqoqsI9MopWy4PUUbzfaWlIRAwIBkrAiGzMQO3+ZRBi\nDk2BkE3BVD0JkbXmCakezyqAZiCEZg0QtPVg1oiZkSCM9y9SAhIgMChRytQygCJUNiLjMP4ryXtG\ndGDeSlJ/i52PpDZbCGwmJtW0CTOWbc0LH3gbGVYTus7dDs2qlNRVM4Plqq4tEzRX663nAefYxLZF\ntpCwaYyXNoYQrltcx/1dr9jhFNkmkpoiK9Q5jR3iEMqyyb5jawBu9TV10nJmslJCV4SbhCOZ5es8\nXY8fXH9FFsJCYtWFLWoxcb0vW+H16fxp6MuMQRyhC17T9y+Fd2OUSP5tv7M/4CG4rXIQEXFhB3nq\nHl6HUwsjxVDzP5eHI5RqCtKYq5oMH/fgZOUxu7ZOT+c3vzDkrHMvsTMy9eQPl69lCC9NskWUQ69Y\n02U6UdqalZSNwvHAl+9T444djwfXIJXn3MnsxWE2FOYBl1y1oPXBM1gt26NDiVtISFrISIIDdsLE\niFBlK1/m30SGqwMMLSE0L5k7T2QNoLGW89+xI3l9uVaU0FCUwDGzQEJAxPTzz+gAJHz8b0rtlSRx\nFqCSKBIAYZ7+Wit5/B+h5s6wCXMRIQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.0868205   0.91317958]\n",
      "Neutral ---------------------------------------------###----- Smiling\n"
     ]
    }
   ],
   "source": [
    "from utils import show_array\n",
    "X = np.load('X.npy')\n",
    "class_names = ['Neutral', 'Smiling']\n",
    "img = X[-7]\n",
    "show_array(255 * img)\n",
    "print_indicator(img, model, class_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's try it in realtime. One way to do this is to use OpenCV for video capture, then crop and resize every frame and process it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from IPython.display import clear_output\n",
    "from utils import crop_and_resize\n",
    "import cv2\n",
    "\n",
    "video_capture = cv2.VideoCapture(0)\n",
    "try:\n",
    "    while True:\n",
    "        ret, frame = video_capture.read()\n",
    "        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)\n",
    "        data = crop_and_resize(gray, 32, zoom=0.6)\n",
    "#         show_array(data)\n",
    "        data = data.astype(np.float) / 255.\n",
    "        print_indicator(data, model, class_names)\n",
    "        clear_output(wait=True)\n",
    "except KeyboardInterrupt:\n",
    "    pass\n",
    "video_capture.release()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, we can wait for video input with zmq, processing one frame every `display_rate` seconds. This should be running at the same time as the `ZMQVideoInput` app."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from IPython.display import clear_output\n",
    "from time import time\n",
    "import zmq\n",
    "\n",
    "display_rate = 1./10\n",
    "addr = 'tcp:https://localhost:14000'\n",
    "\n",
    "img_rows, img_cols = model.input_shape[1:]\n",
    "\n",
    "context = zmq.Context()\n",
    "subs = context.socket(zmq.SUB)\n",
    "subs.setsockopt(zmq.SUBSCRIBE, '')\n",
    "subs.connect(addr)\n",
    "start = time()\n",
    "previous = 0\n",
    "try:\n",
    "    while True:\n",
    "        raw = subs.recv()\n",
    "        duration = (time() - start) % display_rate\n",
    "        if duration < previous:\n",
    "            data = np.fromstring(raw, dtype=np.uint8).reshape(img_rows, img_cols)\n",
    "#             show_array(data) # uncomment this to see the image\n",
    "            data = data.astype(np.float) / 255.\n",
    "            print_indicator(data, model, class_names)\n",
    "            clear_output(wait=True)\n",
    "        previous = duration\n",
    "except KeyboardInterrupt:\n",
    "    pass"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}