NaiveBayes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Insult Classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise, we would like to filter out insulting comments on a web forum. \n",
    "\n",
    "To train our models, we have a list of historic comments with a judgement wether they're insulting or not."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Insult</th>\n",
       "      <th>Date</th>\n",
       "      <th>Comment</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>20120618192155Z</td>\n",
       "      <td>You fuck your dad.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>20120528192215Z</td>\n",
       "      <td>i really don't understand your point.  It seem...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Insult             Date                                            Comment\n",
       "0       1  20120618192155Z                                 You fuck your dad.\n",
       "1       0  20120528192215Z  i really don't understand your point.  It seem..."
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "path_to_insults = '/Users/Vignesh/Dev/ML/students/data/'\n",
    "data = pd.read_csv(path_to_insults + 'train-utf8.csv')\n",
    "data.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3947 comments, of which 1049 insults (26%)\n"
     ]
    }
   ],
   "source": [
    "print (\"%d comments, of which %d insults (%d%%)\" % \\\n",
    "    (len(data), data.Insult.sum(), 100 * data.Insult.mean()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Looking for known bad words"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One way to do this, is to load Google's bad word list and flag comments that contain one or more words.\n",
    "\n",
    "- Load `google_badlist.txt` from `data/insults/`\n",
    "- Add a column to `data` with a flag (0 or 1) if the comment contains a bad word\n",
    "- Compute the accuracy of this method - does this look good?\n",
    "- What would a naive classifier's score be (i.e., always predicting 0 or 1)?\n",
    "- Also compute the precision, recall, F1 score and AUC score\n",
    "- What is your verdict?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/Users/Vignesh/Dev/ML/students/data/google_badlist.txt'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filename = path_to_insults + 'google_badlist.txt'\n",
    "filename"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Creating a **badwords set** reading from the file - '/Users/Vignesh/Dev/ML/students/data/google_badlist.txt'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "with open(filename) as f:\n",
    "    content = f.readlines()\n",
    "content = [x.strip() for x in content]\n",
    "badwords = set(content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function to check a comment contain a bad word or not"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def has_bad_words(comments):\n",
    "    words = comments.strip().split()\n",
    "    if any(w in badwords for w in words):\n",
    "        return 1\n",
    "    else:\n",
    "        return 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "result = []\n",
    "for x in data['Comment']:\n",
    "    result += has_bad_words(x),"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Adding a new column - **Badwords** in the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "data['Badwords'] = np.array(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3947, 4)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Insult</th>\n",
       "      <th>Date</th>\n",
       "      <th>Comment</th>\n",
       "      <th>Badwords</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>20120618192155Z</td>\n",
       "      <td>You fuck your dad.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>20120528192215Z</td>\n",
       "      <td>i really don't understand your point.  It seem...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>A majority of Canadians can and has been wrong...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>listen if you dont wanna get married to a man ...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Insult             Date                                            Comment  \\\n",
       "0       1  20120618192155Z                                 You fuck your dad.   \n",
       "1       0  20120528192215Z  i really don't understand your point.  It seem...   \n",
       "2       0              NaN  A majority of Canadians can and has been wrong...   \n",
       "3       0              NaN  listen if you dont wanna get married to a man ...   \n",
       "\n",
       "   Badwords  \n",
       "0         1  \n",
       "1         0  \n",
       "2         0  \n",
       "3         0  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy = 70.8133%\n"
     ]
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "print ('Accuracy = {0:5.4f}%'.format(np.mean(data['Insult'] == data['Badwords'])*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### The Naive Classifier - (to predect a given comment is an Insult/ not Insult based on the badword content)\n",
    "\n",
    "* This kind of classifier will always **predict in term of 0 or 1**. If there is a badword in the comment, the classifier will predict as Insult. If not, will predict as Not Insult."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2510,  388],\n",
       "       [ 764,  285]])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.confusion_matrix(data['Insult'], data['Badwords'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.77      0.87      0.81      2898\n",
      "          1       0.42      0.27      0.33      1049\n",
      "\n",
      "avg / total       0.68      0.71      0.69      3947\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(metrics.classification_report(data['Insult'], data['Badwords']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "precision = metrics.precision_score(data['Insult'], data['Badwords'],average=None)\n",
    "recall = metrics.recall_score(data['Insult'], data['Badwords'],average=None)\n",
    "f1 = metrics.f1_score(data['Insult'], data['Badwords'],average=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "fpr, tpr, thresholds = metrics.roc_curve(data['Insult'], data['Badwords'])\n",
    "auc = metrics.auc(fpr, tpr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy = 70.8133%\n",
      "Precision = 0.5951\n",
      "Recall = 0.5689\n",
      "F1 = 0.5722\n",
      "AUC = 0.5689\n"
     ]
    }
   ],
   "source": [
    "print ('Accuracy = {0:5.4f}%'.format(np.mean(data['Insult'] == data['Badwords'])*100))\n",
    "print ('Precision = {0:5.4f}'.format(np.mean(precision)))\n",
    "print ('Recall = {0:5.4f}'.format(np.mean(recall)))\n",
    "print ('F1 = {0:5.4f}'.format(np.mean(f1)))\n",
    "print ('AUC = {0:5.4f}'.format(auc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### Summary\n",
    "\n",
    "For a random classifier the AUC will be 0.5. In this case, it is closer to the random classifier value.\n",
    "Thus, this model will not be so good in predict the comments as insult only based on the presence of Bad words in it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learning bad words on the fly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another way of doing this, is to learn the insulting words on the fly using `CountVectorizer`. \n",
    "\n",
    "Please refer to the scikit learn tutorial at 'http:https://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_data.html' if you need some help.\n",
    "\n",
    "Here is what you need to do:\n",
    "\n",
    "- Import `CountVectorizer` from `sklearn.feature_extraction.text`\n",
    "- Train the `CountVectorizer` on the insults and create a feature set $X$ representing words in the comments\n",
    "- Train `MultinomialNB` and `BernoulliNB` from `scikitsklearn`  on the new feature set $X$\n",
    "- Using cross-validation, compute the accuracy, precision, recall, F1 and AUC of your model\n",
    "- What is your verdict?\n",
    "\n",
    "NOTE: The F1 score is another useful score to compute when one of the two classes is very rare. We didn't go over it in class but it's basically the harmonic mean between precision and recall and goes from 0 (min) to 1 (max).  You can see more here: 'https://en.wikipedia.org/wiki/F1_score' "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.naive_bayes import BernoulliNB, MultinomialNB\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.model_selection import cross_val_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MultiNomial\n",
    "----\n",
    "\n",
    "* Splitting the data into Traning (80%) and Test (20%) set using the train_test_split method\n",
    "* Used Pipeline, CountVectorizer\n",
    "* MultiNomial Naive Bayes is choosen as the Classifier and trained the model on traning set.\n",
    "* Metrics of the model is check on the Test set\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(data['Comment'], data['Insult'], test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((3157,), (3157,))"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape, y_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((790,), (790,))"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test.shape, y_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Multi_clf = Pipeline([('vect', CountVectorizer()),\n",
    "                     ('clf', MultinomialNB()), ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Multi_clf = Multi_clf.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy = 82.0253%\n"
     ]
    }
   ],
   "source": [
    "predicted = Multi_clf.predict(X_test)\n",
    "print ('Accuracy = {0:5.4f}%'.format(np.mean(predicted == y_test)*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Multinomial Cross Validation \n",
    "----\n",
    "\n",
    "Doing cross validation on the whole data set, assuming we have only small data set. \n",
    "Thus, the average score of the KFold crossvalidation is choosen as the metrics for the choosen classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import cross_val_score\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn import metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((3947, 15457), (3947,))"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "count_vect = CountVectorizer()\n",
    "X_train = count_vect.fit_transform(data['Comment'])\n",
    "y = np.array(data['Insult'])\n",
    "X_train.shape, y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "clf = MultinomialNB()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5 fold cross validation is choosen here and their metrics is calculated.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "precision = cross_val_score(clf, X_train, y, scoring='precision',cv=5,n_jobs=1)\n",
    "accuracy = cross_val_score(clf, X_train, y, scoring='accuracy',cv=5,n_jobs=1)\n",
    "recall = cross_val_score(clf, X_train, y, scoring='recall',cv=5,n_jobs=1)\n",
    "f1 = cross_val_score(clf, X_train, y, scoring='f1',cv=5,n_jobs=1)\n",
    "roc_auc = cross_val_score(clf, X_train, y, scoring='roc_auc',cv=5,n_jobs=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy = 78.6418%\n",
      "Precision = 0.5903\n",
      "Recall = 0.6425\n",
      "F1 = 0.6149\n",
      "Roc_AUC = 0.8084\n"
     ]
    }
   ],
   "source": [
    "print ('Accuracy = {0:5.4f}%'.format(np.mean(accuracy)*100))\n",
    "print ('Precision = {0:5.4f}'.format(np.mean(precision)))\n",
    "print ('Recall = {0:5.4f}'.format(np.mean(recall)))\n",
    "print ('F1 = {0:5.4f}'.format(np.mean(f1)))\n",
    "print ('Roc_AUC = {0:5.4f}'.format(np.mean(roc_auc)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bernoulli\n",
    "\n",
    "***\n",
    "\n",
    "* Splitting the data into Traning (80%) and Test (20%) set using the train_test_split method, assuming we have **plenty of data**.\n",
    "* Used Pipeline, CountVectorizer\n",
    "* Bernoulii Naive Bayes is choosen as the Classifier and trained the model on traning set.\n",
    "* Metrics of the model is check on the Test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(data['Comment'], data['Insult'], test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((3157,), (3157,))"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape, y_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((790,), (790,))"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test.shape, y_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Bernoulli_clf = Pipeline([('vect', CountVectorizer()),\n",
    "                     ('clf', BernoulliNB()), ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy = 75.1899%\n"
     ]
    }
   ],
   "source": [
    "Bernoulli_clf = Bernoulli_clf.fit(X_train,y_train)\n",
    "predicted = Bernoulli_clf.predict(X_test)\n",
    "print ('Accuracy = {0:5.4f}%'.format(np.mean(predicted == y_test)*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bernoulli -Cross Validation\n",
    "----\n",
    "\n",
    "Doing cross validation on the whole data set, assuming we have only **small data set**. \n",
    "Thus, the average score of the KFold crossvalidation is choosen as the metrics for the choosen classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import cross_val_score\n",
    "from sklearn.naive_bayes import BernoulliNB\n",
    "from sklearn import metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((3947, 15457), (3947,))"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "count_vect = CountVectorizer()\n",
    "X_train= count_vect.fit_transform(data['Comment'])\n",
    "y = np.array(data['Insult'])\n",
    "X_train.shape, y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "clf = BernoulliNB()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5 fold cross validation is choosen here and their metrics is calculated.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "precision = cross_val_score(clf, X_train, y, scoring='precision',cv=5,n_jobs=1)\n",
    "accuracy = cross_val_score(clf, X_train, y, scoring='accuracy',cv=5,n_jobs=1)\n",
    "recall = cross_val_score(clf, X_train, y, scoring='recall',cv=5,n_jobs=1)\n",
    "f1 = cross_val_score(clf, X_train, y, scoring='f1',cv=5,n_jobs=1)\n",
    "roc_auc = cross_val_score(clf, X_train, y, scoring='roc_auc',cv=5,n_jobs=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy = 75.6273%\n",
      "Precision = 0.6497\n",
      "Recall = 0.1792\n",
      "F1 = 0.2808\n",
      "Roc_AUC = 0.8308\n"
     ]
    }
   ],
   "source": [
    "print ('Accuracy = {0:5.4f}%'.format(np.mean(accuracy)*100))\n",
    "print ('Precision = {0:5.4f}'.format(np.mean(precision)))\n",
    "print ('Recall = {0:5.4f}'.format(np.mean(recall)))\n",
    "print ('F1 = {0:5.4f}'.format(np.mean(f1)))\n",
    "print ('Roc_AUC = {0:5.4f}'.format(np.mean(roc_auc)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary\n",
    "\n",
    "---\n",
    "\n",
    "The Multinomial Naive bayes have more accuracy than the Bernoulli Naive bayes Classifier. \n",
    "With the help of k fold cross validation - we can predict the model performance on the given dataset, in which also the Multinomial Naive bayes classifer perform better than Bernoulli Naive bayes classifier."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}