Text_classification_Using_Tensorflow_Neural_Network.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "![giskard_logo.png](https://raw.githubusercontent.com/Giskard-AI/giskard/main/readme/Logo_full_darkgreen.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installing `giskard` and `tensorflow`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install giskard tensorflow==2.9.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect the external worker in daemon mode"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!giskard worker start -d"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Basic text classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Eg62Pmz3o83v",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "This tutorial demonstrates [text classification of Tensorflow](https://www.tensorflow.org/tutorials/keras/text_classification) modified for uploading on [Giskard](https://www.giskard.ai/) starting from plain text files stored on disk. You'll train a binary classifier to perform sentiment analysis on an IMDB dataset.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8RZOuS9LWQvv",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import shutil\n",
    "import tensorflow as tf\n",
    "import pandas as pd\n",
    "from tensorflow.keras import layers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NBTI1bi8qdFV",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Sentiment analysis\n",
    "\n",
    "This notebook trains a sentiment analysis model to classify movie reviews as *positive* or *negative*, based on the text of the review. This is an example of *binary*—or two-class—classification, an important and widely applicable kind of machine learning problem.\n",
    "\n",
    "You'll use the [Large Movie Review Dataset](https://ai.stanford.edu/~amaas/data/sentiment/) that contains the text of 50,000 movie reviews from the [Internet Movie Database](https://www.imdb.com/). These are split into 25,000 reviews for training and 25,000 reviews for testing. The training and testing sets are *balanced*, meaning they contain an equal number of positive and negative reviews.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iAsKG535pHep",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Download and explore the IMDB dataset\n",
    "\n",
    "Let's download and extract the dataset, then explore the directory structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "k7ZYnuajVlFN",
    "outputId": "bfd4caa3-1a0a-4087-8275-8d59cefccb52",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "url = \"https://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\"\n",
    "\n",
    "dataset = tf.keras.utils.get_file(\"aclImdb_v1\", url,\n",
    "                                    untar=True, cache_dir='.',\n",
    "                                    cache_subdir='')\n",
    "\n",
    "dataset_dir = os.path.join(os.path.dirname(dataset), 'aclImdb')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "355CfOvsV1pl",
    "outputId": "a7f1c39b-86d3-4482-e161-b6fbb32512e7",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "os.listdir(dataset_dir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7ASND15oXpF1",
    "outputId": "4515eb1e-5108-43b8-c9ec-13a8625bd92c",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "train_dir = os.path.join(dataset_dir, 'train')\n",
    "os.listdir(train_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ysMNMI1CWDFD",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "The `aclImdb/train/pos` and `aclImdb/train/neg` directories contain many text files, each of which is a single movie review. Let's take a look at one of them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "R7g8hFvzWLIZ",
    "outputId": "4744287d-bc67-4035-bd24-2f9f98259d38",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "sample_file = os.path.join(train_dir, 'pos/1181_9.txt')\n",
    "with open(sample_file) as f:\n",
    "  print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nQauv38Lnok3",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "To prepare a dataset for binary classification, you will need two folders on disk, corresponding to `class_a` and `class_b`. These will be the positive and negative movie reviews, which can be found in  `aclImdb/train/pos` and `aclImdb/train/neg`. As the IMDB dataset contains additional folders, you will remove them before using this utility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "VhejsClzaWfl",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "remove_dir = os.path.join(train_dir, 'unsup')\n",
    "shutil.rmtree(remove_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "95kkUdRoaeMw",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Next, you will use the `text_dataset_from_directory` utility to create a labeled `tf.data.Dataset`. [tf.data](https://www.tensorflow.org/guide/data) is a powerful collection of tools for working with data. \n",
    "\n",
    "When running a machine learning experiment, it is a best practice to divide your dataset into three splits: [train](https://developers.google.com/machine-learning/glossary#training_set), [validation](https://developers.google.com/machine-learning/glossary#validation_set), and [test](https://developers.google.com/machine-learning/glossary#test-set). \n",
    "\n",
    "The IMDB dataset has already been divided into train and test, but it lacks a validation set. Let's create a validation set using an 80:20 split of the training data by using the `validation_split` argument below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nOrK-MTYaw3C",
    "outputId": "2c0b74a1-9cf4-4975-8794-22bfa8b978aa",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "batch_size = 32\n",
    "seed = 42\n",
    "\n",
    "raw_train_ds = tf.keras.utils.text_dataset_from_directory(\n",
    "    'aclImdb/train', \n",
    "    batch_size=batch_size, \n",
    "    validation_split=0.2, \n",
    "    subset='training', \n",
    "    seed=seed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "JsMwwhOoqjKF",
    "outputId": "141a313b-d15e-4f11-99f7-87647ecf7aab",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "raw_val_ds = tf.keras.utils.text_dataset_from_directory(\n",
    "    'aclImdb/train', \n",
    "    batch_size=batch_size, \n",
    "    validation_split=0.2, \n",
    "    subset='validation', \n",
    "    seed=seed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rdSr0Nt3q_ns",
    "outputId": "c461a4f1-5e8e-49b0-9536-c6bd035e59c7",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "raw_test_ds = tf.keras.utils.text_dataset_from_directory(\n",
    "    'aclImdb/test', \n",
    "    batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8ukPIYm6mHhr",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## We convert this data to pandas dataframe to upload on Giskard"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "51wNaPPApk1K",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "train_dataset = {'Review':[], 'Label':[]}\n",
    "for text_batch, label_batch in raw_train_ds.take(625):\n",
    "  for i in range(32):\n",
    "    train_dataset['Review'].append(text_batch.numpy()[i])\n",
    "    train_dataset['Label'].append(label_batch.numpy()[i])\n",
    "\n",
    "train_df = pd.DataFrame.from_dict(train_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MlICTG8spyO2",
    "outputId": "382b5951-f328-483a-b60d-81f805aa267d",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "print(\"Label 0 corresponds to\", raw_train_ds.class_names[0])\n",
    "print(\"Label 1 corresponds to\", raw_train_ds.class_names[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "eSpOabEzdsER",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "val_dataset = {'Review':[], 'Label':[]}\n",
    "for text_batch, label_batch in raw_val_ds.take(157):\n",
    "  for i in range(8):\n",
    "    val_dataset['Review'].append(text_batch.numpy()[i])\n",
    "    val_dataset['Label'].append(label_batch.numpy()[i])\n",
    "\n",
    "val_df = pd.DataFrame.from_dict(val_dataset)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "gjVrRN00eRnT",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "test_dataset = {'Review':[], 'Label':[]}\n",
    "for text_batch, label_batch in raw_test_ds.take(782):\n",
    "  for i in range(8):\n",
    "    test_dataset['Review'].append(text_batch.numpy()[i])\n",
    "    test_dataset['Label'].append(label_batch.numpy()[i])\n",
    "  test_df = pd.DataFrame.from_dict(test_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qJmTiO0IYAjm",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Prepare the dataset for training\n",
    "\n",
    "Next, you will standardize, tokenize, and vectorize the data using the helpful `tf.keras.layers.TextVectorization` layer. \n",
    "\n",
    "Standardization refers to preprocessing the text, typically to remove punctuation or HTML elements to simplify the dataset. Tokenization refers to splitting strings into tokens (for example, splitting a sentence into individual words, by splitting on whitespace). Vectorization refers to converting tokens into numbers so they can be fed into a neural network. All of these tasks can be accomplished with this layer.\n",
    "\n",
    "As you saw above, the reviews contain various HTML tags like `<br />`. These tags will not be removed by the default standardizer in the `TextVectorization` layer (which converts text to lowercase and strips punctuation by default, but doesn't strip HTML). You will write a custom standardization function to remove the HTML."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZVcHl-SLrH-u",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Note: To prevent [training-testing skew](https://developers.google.com/machine-learning/guides/rules-of-ml#training-serving_skew) (also known as training-serving skew), it is important to preprocess the data identically at train and test time. To facilitate this, the `TextVectorization` layer can be included directly inside your model, as shown later in this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "max_features = 10000\n",
    "sequence_length = 250\n",
    "vectorize_layer = tf.keras.layers.TextVectorization(\n",
    "    standardize='lower_and_strip_punctuation',\n",
    "    max_tokens=max_features,\n",
    "    output_mode='int',\n",
    "    output_sequence_length=sequence_length)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "d2d3Aw8dsUux",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Next, you will create a `TextVectorization` layer. You will use this layer to standardize, tokenize, and vectorize our data. You set the `output_mode` to `int` to create unique integer indices for each token.\n",
    "\n",
    "Note that you're using the default split function, and the custom standardization function you defined above. You'll also define some constants for the model, like an explicit maximum `sequence_length`, which will cause the layer to pad or truncate sequences to exactly `sequence_length` values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Next, you will call `adapt` to fit the state of the preprocessing layer to the dataset. This will cause the model to build an index of strings to integers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lAhdjK7AtroA",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Note: It's important to only use your training data when calling adapt (using the test set would leak information)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "GH4_2ZGJsa_X",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Make a text-only dataset (without labels), then call adapt\n",
    "train_text = raw_train_ds.map(lambda x, y: x)\n",
    "vectorize_layer.adapt(train_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SHQVEFzNt-K_",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Let's create a function to see the result of using this layer to preprocess some data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "SCIg_T50wOCU",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "def vectorize_text(text, label):\n",
    "  text = tf.expand_dims(text, -1)\n",
    "  return vectorize_layer(text), label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "XULcm6B3xQIO",
    "outputId": "49b05f0d-ea0a-4c53-ae56-1604241c8385",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# retrieve a batch (of 32 reviews and labels) from the dataset\n",
    "text_batch, label_batch = next(iter(raw_train_ds))\n",
    "first_review, first_label = text_batch[0], label_batch[0]\n",
    "print(\"Review\", first_review)\n",
    "print(\"Label\", raw_train_ds.class_names[first_label])\n",
    "print(\"Vectorized review\", vectorize_text(first_review, first_label))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XD2H6utRydGv",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "You are nearly ready to train your model. As a final preprocessing step, you will apply the TextVectorization layer you created earlier to the train, validation, and test dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "2zhmpeViI1iG",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "train_ds = raw_train_ds.map(vectorize_text)\n",
    "val_ds = raw_val_ds.map(vectorize_text)\n",
    "test_ds = raw_test_ds.map(vectorize_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YsVQyPMizjuO",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Configure the dataset for performance\n",
    "\n",
    "These are two important methods you should use when loading data to make sure that I/O does not become blocking.\n",
    "\n",
    "`.cache()` keeps data in memory after it's loaded off disk. This will ensure the dataset does not become a bottleneck while training your model. If your dataset is too large to fit into memory, you can also use this method to create a performant on-disk cache, which is more efficient to read than many small files.\n",
    "\n",
    "`.prefetch()` overlaps data preprocessing and model execution while training. \n",
    "\n",
    "You can learn more about both methods, as well as how to cache data to disk in the [data performance guide](https://www.tensorflow.org/guide/data_performance)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "wMcs_H7izm5m",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "AUTOTUNE = tf.data.AUTOTUNE\n",
    "\n",
    "train_ds = train_ds.cache().prefetch(buffer_size=AUTOTUNE)\n",
    "val_ds = val_ds.cache().prefetch(buffer_size=AUTOTUNE)\n",
    "test_ds = test_ds.cache().prefetch(buffer_size=AUTOTUNE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LLC02j2g-llC",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Create the model\n",
    "\n",
    "It*'*s time to create your neural network:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "dkQP6in8yUBR",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "embedding_dim = 16"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xpKOoWgu-llD",
    "outputId": "0fcb6620-2adc-4576-95b0-b97f885182d4",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "model = tf.keras.Sequential([\n",
    "  layers.Embedding(max_features + 1, embedding_dim),\n",
    "  layers.Dropout(0.2),\n",
    "  layers.GlobalAveragePooling1D(),\n",
    "  layers.Dropout(0.2),\n",
    "  layers.Dense(2),\n",
    "  tf.keras.layers.Softmax()\n",
    "  ])\n",
    "\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6PbKQ6mucuKL",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "The layers are stacked sequentially to build the classifier:\n",
    "\n",
    "1. The first layer is an `Embedding` layer. This layer takes the integer-encoded reviews and looks up an embedding vector for each word-index. These vectors are learned as the model trains. The vectors add a dimension to the output array. The resulting dimensions are: `(batch, sequence, embedding)`.  To learn more about embeddings, check out the [Word embeddings](https://www.tensorflow.org/text/guide/word_embeddings) tutorial.\n",
    "2. Next, a `GlobalAveragePooling1D` layer returns a fixed-length output vector for each example by averaging over the sequence dimension. This allows the model to handle input of variable length, in the simplest way possible.\n",
    "3. This fixed-length output vector is piped through a fully-connected (`Dense`) layer with 16 hidden units. \n",
    "4. The last layer is densely connected with a single output node."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "L4EqVWg4-llM",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Loss function and optimizer\n",
    "\n",
    "A model needs a loss function and an optimizer for training. Since this is a binary classification problem and the model outputs a probability (a single-unit layer with a sigmoid activation), you'll use `losses.BinaryCrossentropy` loss function.\n",
    "\n",
    "Now, configure the model to use an optimizer and a loss function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Mr0GP-cQ-llN",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),\n",
    "              optimizer='adam',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "35jv_fzP-llU",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Train the model\n",
    "\n",
    "You will train the model by passing the `dataset` object to the fit method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tXSGrjWZ-llW",
    "outputId": "10c8eb0c-418b-4bf1-8bae-5efb1db612fa",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "epochs = 10\n",
    "history = model.fit(\n",
    "    train_ds,\n",
    "    validation_data=val_ds,\n",
    "    epochs=epochs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9EEGuDVuzb5r",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Evaluate the model\n",
    "\n",
    "Let's see how the model performs. Two values will be returned. Loss (a number which represents our error, lower values are better), and accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "zOMKywn4zReN",
    "outputId": "97fa836f-be51-44f1-e6b5-1db0d8463d24",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "loss, accuracy = model.evaluate(test_ds)\n",
    "\n",
    "print(\"Loss: \", loss)\n",
    "print(\"Accuracy: \", accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "z1iEXVTR0Z2t",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "This fairly naive approach achieves an accuracy of about 86%."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-to23J3Vy5d3",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Export the model\n",
    "\n",
    "In the code above, you applied the `TextVectorization` layer to the dataset before feeding text to the model. If you want to make your model capable of processing raw strings (for example, to simplify deploying it), you can include the `TextVectorization` layer inside your model. To do so, you can create a new model using the weights you just trained."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "FWXsMvryuZuq",
    "outputId": "22289970-24ed-4402-d6fc-a60709973e02",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "export_model = tf.keras.Sequential([\n",
    "  vectorize_layer,\n",
    "  model\n",
    "])\n",
    "\n",
    "export_model.compile(\n",
    "    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False), optimizer=\"adam\", metrics=['accuracy']\n",
    ")\n",
    "\n",
    "# Test it with `raw_test_ds`, which yields raw strings\n",
    "loss, accuracy = export_model.evaluate(raw_test_ds)\n",
    "print(accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TwQgoN88LoEF",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Inference on new data\n",
    "\n",
    "To get predictions for new examples, you can simply call `model.predict()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "QW355HH5L49K",
    "outputId": "6ba4291d-7f5c-462e-bfcb-f4c569c05e04",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "examples = [\n",
    "  \"The movie was great!\",\n",
    "  \"The movie was okay.\",\n",
    "  \"The movie was terrible...\"\n",
    "]\n",
    "\n",
    "export_model.predict(examples)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KdmMU2e9oNtN",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Initiate Project On Giskard"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Giskard require a prediction function which takes dataframe as input and provides prediction probability as output for classification models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "def predict(test_dataset):\n",
    "    test_dataset= test_dataset.squeeze(axis=1)\n",
    "    test_dataset = list(test_dataset)\n",
    "    predictions = export_model.predict(test_dataset)\n",
    "\n",
    "    return predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from giskard import GiskardClient\n",
    "\n",
    "url = \"http:https://localhost:19000\" #if Giskard is installed locally (for installation, see: https://docs.giskard.ai/start/guides/installation)\n",
    "#url = \"http:https://app.giskard.ai\" # If you want to upload on giskard URL\n",
    "token = \"YOUR GENERATED TOKEN\" #you can generate your API token in the Admin tab of the Giskard application (for installation, see: https://docs.giskard.ai/start/guides/installation)\n",
    "\n",
    "client = GiskardClient(url, token)\n",
    "\n",
    "# your_project = client.create_project(\"project_key\", \"PROJECT_NAME\", \"DESCRIPTION\")\n",
    "# Choose the arguments you want. But \"project_key\" should be unique and in lower case\n",
    "tensorflow_text_classification = client.create_project(\"tensorflow_text_classification\", \"Tensorflow_text_Classification\", \"Classification Of Text using Tensorflow Neural Network\")\n",
    "\n",
    "# If you've already created a project with the key \"tensorflow_text_classification\" use\n",
    "#tensorflow_text_classification = client.get_project(\"tensorflow_text_classification\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Declare the type of each column in the dataset(example: category, numeric, text)\n",
    "column_types = {'Review':\"text\",\n",
    "               'Label':\"category\"}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Lets upload the model and dataset on Giskard !"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "tensorflow_text_classification.upload_model_and_df(\n",
    "    prediction_function=predict, # Python function which takes pandas dataframe as input and returns probabilities for classification model OR returns predictions for regression model\n",
    "    model_type='classification', # \"classification\" for classification model OR \"regression\" for regression model\n",
    "    df=test_df, # the dataset you want to use to inspect your model\n",
    "    column_types=column_types, # A dictionary with columns names of df as key and types(category, numeric, text) of columns as values\n",
    "    target='Label', # The column name in df corresponding to the actual target variable (ground truth).\n",
    "    feature_names=['Review'], # List of the feature names of prediction_function\n",
    "    classification_labels=[0,1],  # List of the classification labels of your prediction\n",
    "    model_name='Tensorflow', # Name of the model\n",
    "    dataset_name='test_data' # Name of the dataset\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "tensorflow_text_classification.upload_df(\n",
    "    df=train_df, # The dataset you want to upload\n",
    "    column_types=column_types, # All the column types without the target\n",
    "    name=\"train_data\"  # Name of the dataset\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "tensorflow_text_classification.upload_df(\n",
    "    df=val_df, # The dataset you want to upload\n",
    "    column_types=column_types, # All the column types without the target\n",
    "    name=\"val_data\"  # Name of the dataset\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}