Newspaper_classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KV3Yqc-x0dKN"
   },
   "source": [
    "![giskard_logo.png](https://raw.githubusercontent.com/Giskard-AI/giskard/main/readme/Logo_full_darkgreen.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "htCJUiaF0iW8"
   },
   "source": [
    "# About Giskard\n",
    "\n",
    "Open-Source CI/CD platform for ML teams. Deliver ML products, better & faster. \n",
    "\n",
    "*   Collaborate faster with feedback from business stakeholders.\n",
    "*   Deploy automated tests to eliminate regressions, errors & biases.\n",
    "\n",
    "🏡 [Website](https://giskard.ai/)\n",
    "\n",
    "📗 [Documentation](https://docs.giskard.ai/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installing `giskard` and other packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install giskard torch torchdata torchtext"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect the external worker in daemon mode"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!giskard worker start -d\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gJVrEpvVoA10"
   },
   "source": [
    "\n",
    "# Text classification with the torchtext library\n",
    "\n",
    "In this tutorial, we will show how to use the torchtext library to build the dataset for the text classification analysis. Users will have the flexibility to\n",
    "\n",
    "   - Access to the raw data as an iterator\n",
    "   - Build data processing pipeline to convert the raw text strings into ``torch.Tensor`` that can be used to train the model\n",
    "   - Shuffle and iterate the data with [torch.utils.data.DataLoader](https://pytorch.org/docs/stable/data.html?highlight=dataloader#torch.utils.data.DataLoader)_\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "O01CE1ExoA10"
   },
   "source": [
    "## Access to the raw dataset iterators\n",
    "\n",
    "The torchtext library provides a few raw dataset iterators, which yield the raw text strings. For example, the ``AG_NEWS`` dataset iterators yield the raw data as a tuple of label and text.\n",
    "\n",
    "To access torchtext datasets, please install torchdata following instructions at https://github.com/pytorch/data. \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "eup4gpgVoA10"
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "from torchtext.datasets import AG_NEWS\n",
    "train_iter = iter(AG_NEWS(split='train'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "N7gaJx8PoA11"
   },
   "source": [
    "::\n",
    "\n",
    "    next(train_iter)\n",
    "    >>> (3, \"Fears for T N pension after talks Unions representing workers at Turner\n",
    "    Newall say they are 'disappointed' after talks with stricken parent firm Federal\n",
    "    Mogul.\")\n",
    "\n",
    "    next(train_iter)\n",
    "    >>> (4, \"The Race is On: Second Private Team Sets Launch Date for Human\n",
    "    Spaceflight (SPACE.com) SPACE.com - TORONTO, Canada -- A second\\\\team of\n",
    "    rocketeers competing for the  #36;10 million Ansari X Prize, a contest\n",
    "    for\\\\privately funded suborbital space flight, has officially announced\n",
    "    the first\\\\launch date for its manned rocket.\")\n",
    "\n",
    "    next(train_iter)\n",
    "    >>> (4, 'Ky. Company Wins Grant to Study Peptides (AP) AP - A company founded\n",
    "    by a chemistry researcher at the University of Louisville won a grant to develop\n",
    "    a method of producing better peptides, which are short chains of amino acids, the\n",
    "    building blocks of proteins.')\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hkjTYKuloA11"
   },
   "source": [
    "## Prepare data processing pipelines\n",
    "\n",
    "We have revisited the very basic components of the torchtext library, including vocab, word vectors, tokenizer. Those are the basic data processing building blocks for raw text string.\n",
    "\n",
    "Here is an example for typical NLP data processing with tokenizer and vocabulary. The first step is to build a vocabulary with the raw training dataset. Here we use built in\n",
    "factory function `build_vocab_from_iterator` which accepts iterator that yield list or iterator of tokens. Users can also pass any special symbols to be added to the\n",
    "vocabulary.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "GipzVvsAoA11"
   },
   "outputs": [],
   "source": [
    "from torchtext.data.utils import get_tokenizer\n",
    "from torchtext.vocab import build_vocab_from_iterator\n",
    "\n",
    "tokenizer = get_tokenizer('basic_english')\n",
    "train_iter = AG_NEWS(split='train')\n",
    "\n",
    "def yield_tokens(data_iter):\n",
    "    for _, text in data_iter:\n",
    "        yield tokenizer(text)\n",
    "\n",
    "vocab = build_vocab_from_iterator(yield_tokens(train_iter), specials=[\"<unk>\"])\n",
    "vocab.set_default_index(vocab[\"<unk>\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "VCQoWfdhtCJt"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UBc0uWZLoA12"
   },
   "source": [
    "The vocabulary block converts a list of tokens into integers.\n",
    "\n",
    "::\n",
    "\n",
    "    vocab(['here', 'is', 'an', 'example'])\n",
    "    >>> [475, 21, 30, 5297]\n",
    "\n",
    "Prepare the text processing pipeline with the tokenizer and vocabulary. The text and label pipelines will be used to process the raw data strings from the dataset iterators.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "MEoyEKVMoA12"
   },
   "outputs": [],
   "source": [
    "text_pipeline = lambda x: vocab(tokenizer(x))\n",
    "label_pipeline = lambda x: int(x) - 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dwLINdF3oA13"
   },
   "source": [
    "The text pipeline converts a text string into a list of integers based on the lookup table defined in the vocabulary. The label pipeline converts the label into integers. For example,\n",
    "\n",
    "::\n",
    "\n",
    "    text_pipeline('here is the an example')\n",
    "    >>> [475, 21, 2, 30, 5297]\n",
    "    label_pipeline('10')\n",
    "    >>> 9\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sguKd10EoA13"
   },
   "source": [
    "## Generate data batch and iterator\n",
    "\n",
    "[torch.utils.data.DataLoader](https://pytorch.org/docs/stable/data.html?highlight=dataloader#torch.utils.data.DataLoader)_\n",
    "is recommended for PyTorch users (a tutorial is [here](https://pytorch.org/tutorials/beginner/data_loading_tutorial.html)_).\n",
    "It works with a map-style dataset that implements the ``getitem()`` and ``len()`` protocols, and represents a map from indices/keys to data samples. It also works with an iterable dataset with the shuffle argument of ``False``.\n",
    "\n",
    "Before sending to the model, ``collate_fn`` function works on a batch of samples generated from ``DataLoader``. The input to ``collate_fn`` is a batch of data with the batch size in ``DataLoader``, and ``collate_fn`` processes them according to the data processing pipelines declared previously. Pay attention here and make sure that ``collate_fn`` is declared as a top level def. This ensures that the function is available in each worker.\n",
    "\n",
    "In this example, the text entries in the original data batch input are packed into a list and concatenated as a single tensor for the input of ``nn.EmbeddingBag``. The offset is a tensor of delimiters to represent the beginning index of the individual sequence in the text tensor. Label is a tensor saving the labels of individual text entries.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "dxmC1GoSoA13"
   },
   "outputs": [],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "def collate_batch(batch):\n",
    "    label_list, text_list, offsets = [], [], [0]\n",
    "    for (_label, _text) in batch:\n",
    "         label_list.append(label_pipeline(_label))\n",
    "         processed_text = torch.tensor(text_pipeline(_text), dtype=torch.int64)\n",
    "         text_list.append(processed_text)\n",
    "         offsets.append(processed_text.size(0))\n",
    "    label_list = torch.tensor(label_list, dtype=torch.int64)\n",
    "    offsets = torch.tensor(offsets[:-1]).cumsum(dim=0)\n",
    "    text_list = torch.cat(text_list)\n",
    "    return label_list.to(device), text_list.to(device), offsets.to(device)\n",
    "\n",
    "train_iter = AG_NEWS(split='train')\n",
    "dataloader = DataLoader(train_iter, batch_size=8, shuffle=False, collate_fn=collate_batch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "My6Fh7jsoA13"
   },
   "source": [
    "## Define the model\n",
    "\n",
    "The model is composed of the [nn.EmbeddingBag](https://pytorch.org/docs/stable/nn.html?highlight=embeddingbag#torch.nn.EmbeddingBag)_ layer plus a linear layer for the classification purpose. ``nn.EmbeddingBag`` with the default mode of \"mean\" computes the mean value of a “bag” of embeddings. Although the text entries here have different lengths, nn.EmbeddingBag module requires no padding here since the text lengths are saved in offsets.\n",
    "\n",
    "Additionally, since ``nn.EmbeddingBag`` accumulates the average across\n",
    "the embeddings on the fly, ``nn.EmbeddingBag`` can enhance the\n",
    "performance and memory efficiency to process a sequence of tensors.\n",
    "\n",
    "<img src=\"file:https://../_static/img/text_sentiment_ngrams_model.png\">\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "J66cyUvKoA14"
   },
   "outputs": [],
   "source": [
    "from torch import nn\n",
    "\n",
    "class TextClassificationModel(nn.Module):\n",
    "\n",
    "    def __init__(self, vocab_size, embed_dim, num_class):\n",
    "        super(TextClassificationModel, self).__init__()\n",
    "        self.embedding = nn.EmbeddingBag(vocab_size, embed_dim, sparse=True)\n",
    "        self.fc = nn.Linear(embed_dim, num_class)\n",
    "        self.init_weights()\n",
    "\n",
    "    def init_weights(self):\n",
    "        initrange = 0.5\n",
    "        self.embedding.weight.data.uniform_(-initrange, initrange)\n",
    "        self.fc.weight.data.uniform_(-initrange, initrange)\n",
    "        self.fc.bias.data.zero_()\n",
    "\n",
    "    def forward(self, text, offsets):\n",
    "        embedded = self.embedding(text, offsets)\n",
    "        return self.fc(embedded)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zgbK39d6oA14"
   },
   "source": [
    "## Initiate an instance\n",
    "\n",
    "The ``AG_NEWS`` dataset has four labels and therefore the number of classes is four.\n",
    "\n",
    "::\n",
    "\n",
    "   1 : World\n",
    "   2 : Sports\n",
    "   3 : Business\n",
    "   4 : Sci/Tec\n",
    "\n",
    "We build a model with the embedding dimension of 64. The vocab size is equal to the length of the vocabulary instance. The number of classes is equal to the number of labels,\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "j7x89PnVoA14"
   },
   "outputs": [],
   "source": [
    "train_iter = AG_NEWS(split='train')\n",
    "num_class = len(set([label for (label, text) in train_iter]))\n",
    "vocab_size = len(vocab)\n",
    "emsize = 64\n",
    "model = TextClassificationModel(vocab_size, emsize, num_class).to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yYNb6sDVoA14"
   },
   "source": [
    "## Define functions to train the model and evaluate results.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "tnYAEF9OoA14"
   },
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "def train(dataloader):\n",
    "    model.train()\n",
    "    total_acc, total_count = 0, 0\n",
    "    log_interval = 500\n",
    "    start_time = time.time()\n",
    "\n",
    "    for idx, (label, text, offsets) in enumerate(dataloader):\n",
    "        optimizer.zero_grad()\n",
    "        predicted_label = model(text, offsets)\n",
    "        loss = criterion(predicted_label, label)\n",
    "        loss.backward()\n",
    "        torch.nn.utils.clip_grad_norm_(model.parameters(), 0.1)\n",
    "        optimizer.step()\n",
    "        total_acc += (predicted_label.argmax(1) == label).sum().item()\n",
    "        total_count += label.size(0)\n",
    "        if idx % log_interval == 0 and idx > 0:\n",
    "            elapsed = time.time() - start_time\n",
    "            print('| epoch {:3d} | {:5d}/{:5d} batches '\n",
    "                  '| accuracy {:8.3f}'.format(epoch, idx, len(dataloader),\n",
    "                                              total_acc/total_count))\n",
    "            total_acc, total_count = 0, 0\n",
    "            start_time = time.time()\n",
    "\n",
    "def evaluate(dataloader):\n",
    "    model.eval()\n",
    "    total_acc, total_count = 0, 0\n",
    "\n",
    "    with torch.no_grad():\n",
    "        for idx, (label, text, offsets) in enumerate(dataloader):\n",
    "            predicted_label = model(text, offsets)\n",
    "            loss = criterion(predicted_label, label)\n",
    "            total_acc += (predicted_label.argmax(1) == label).sum().item()\n",
    "            total_count += label.size(0)\n",
    "    return total_acc/total_count"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "I7f_75fnoA14"
   },
   "source": [
    "## Split the dataset and run the model\n",
    "\n",
    "Since the original ``AG_NEWS`` has no valid dataset, we split the training\n",
    "dataset into train/valid sets with a split ratio of 0.95 (train) and\n",
    "0.05 (valid). Here we use\n",
    "[torch.utils.data.dataset.random_split](https://pytorch.org/docs/stable/data.html?highlight=random_split#torch.utils.data.random_split)_\n",
    "function in PyTorch core library.\n",
    "\n",
    "[CrossEntropyLoss](https://pytorch.org/docs/stable/nn.html?highlight=crossentropyloss#torch.nn.CrossEntropyLoss)_\n",
    "criterion combines ``nn.LogSoftmax()`` and ``nn.NLLLoss()`` in a single class.\n",
    "It is useful when training a classification problem with C classes.\n",
    "[SGD](https://pytorch.org/docs/stable/_modules/torch/optim/sgd.html)_\n",
    "implements stochastic gradient descent method as the optimizer. The initial\n",
    "learning rate is set to 5.0.\n",
    "[StepLR](https://pytorch.org/docs/master/_modules/torch/optim/lr_scheduler.html#StepLR)_\n",
    "is used here to adjust the learning rate through epochs.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "yO44xeGFoA14"
   },
   "outputs": [],
   "source": [
    "from torch.utils.data.dataset import random_split\n",
    "from torchtext.data.functional import to_map_style_dataset\n",
    "# Hyperparameters\n",
    "EPOCHS = 10 # epoch\n",
    "LR = 5  # learning rate\n",
    "BATCH_SIZE = 64 # batch size for training\n",
    "\n",
    "criterion = torch.nn.CrossEntropyLoss()\n",
    "optimizer = torch.optim.SGD(model.parameters(), lr=LR)\n",
    "scheduler = torch.optim.lr_scheduler.StepLR(optimizer, 1.0, gamma=0.1)\n",
    "total_accu = None\n",
    "train_iter, test_iter = AG_NEWS()\n",
    "train_dataset = to_map_style_dataset(train_iter)\n",
    "test_dataset = to_map_style_dataset(test_iter)\n",
    "num_train = int(len(train_dataset) * 0.95)\n",
    "split_train_, split_valid_ = \\\n",
    "    random_split(train_dataset, [num_train, len(train_dataset) - num_train])\n",
    "\n",
    "train_dataloader = DataLoader(split_train_, batch_size=BATCH_SIZE,\n",
    "                              shuffle=True, collate_fn=collate_batch)\n",
    "valid_dataloader = DataLoader(split_valid_, batch_size=BATCH_SIZE,\n",
    "                              shuffle=True, collate_fn=collate_batch)\n",
    "test_dataloader = DataLoader(test_dataset, batch_size=BATCH_SIZE,\n",
    "                             shuffle=True, collate_fn=collate_batch)\n",
    "\n",
    "for epoch in range(1, EPOCHS + 1):\n",
    "    epoch_start_time = time.time()\n",
    "    train(train_dataloader)\n",
    "    accu_val = evaluate(valid_dataloader)\n",
    "    if total_accu is not None and total_accu > accu_val:\n",
    "      scheduler.step()\n",
    "    else:\n",
    "       total_accu = accu_val\n",
    "    print('-' * 59)\n",
    "    print('| end of epoch {:3d} | time: {:5.2f}s | '\n",
    "          'valid accuracy {:8.3f} '.format(epoch,\n",
    "                                           time.time() - epoch_start_time,\n",
    "                                           accu_val))\n",
    "    print('-' * 59)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PaKWyDSgoA15"
   },
   "source": [
    "## Evaluate the model with test dataset\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "INVRiofwoA15"
   },
   "source": [
    "Checking the results of the test dataset…\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Uy9ziVhloA15"
   },
   "outputs": [],
   "source": [
    "print('Checking the results of test dataset.')\n",
    "accu_test = evaluate(test_dataloader)\n",
    "print('test accuracy {:8.3f}'.format(accu_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LTtEOI_7oA15"
   },
   "source": [
    "## Test on a random news\n",
    "\n",
    "Use the best model so far and test a golf news.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "OqLsJeQGoA15"
   },
   "outputs": [],
   "source": [
    "ag_news_label = {1: \"World\",\n",
    "                 2: \"Sports\",\n",
    "                 3: \"Business\",\n",
    "                 4: \"Sci/Tec\"}\n",
    "\n",
    "def predict(text, text_pipeline):\n",
    "    with torch.no_grad():\n",
    "        text = torch.tensor(text_pipeline(text))\n",
    "        output = model(text, torch.tensor([0]))\n",
    "        return output.argmax(1).item() + 1\n",
    "\n",
    "ex_text_str = \"MEMPHIS, Tenn. – Four days ago, Jon Rahm was \\\n",
    "    enduring the season’s worst weather conditions on Sunday at The \\\n",
    "    Open on his way to a closing 75 at Royal Portrush, which \\\n",
    "    considering the wind and the rain was a respectable showing. \\\n",
    "    Thursday’s first round at the WGC-FedEx St. Jude Invitational \\\n",
    "    was another story. With temperatures in the mid-80s and hardly any \\\n",
    "    wind, the Spaniard was 13 strokes better in a flawless round. \\\n",
    "    Thanks to his best putting performance on the PGA Tour, Rahm \\\n",
    "    finished with an 8-under 62 for a three-stroke lead, which \\\n",
    "    was even more impressive considering he’d never played the \\\n",
    "    front nine at TPC Southwind.\"\n",
    "\n",
    "model = model.to(\"cpu\")\n",
    "\n",
    "print(\"This is a %s news\" %ag_news_label[predict(ex_text_str, text_pipeline)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "SVa86jMS1IGU"
   },
   "outputs": [],
   "source": [
    "predict(ex_text_str, text_pipeline)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JQOB4tgkysCm"
   },
   "source": [
    "# Upload in Giskard"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xNjXaNvWzsoT"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "raw_data = { \"text\": [value[1] for value in test_dataset], \"label\": [ag_news_label[value[0]] for value in test_dataset]}\n",
    "df = pd.DataFrame(raw_data, columns=[\"text\", \"label\"])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "wsOyr0dOq8jc"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "def softmax(x):\n",
    "    \"\"\"Compute softmax values for each sets of scores in x.\"\"\"\n",
    "    return np.exp(x) / np.sum(np.exp(x), axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "vjgpoAL6qJXk"
   },
   "outputs": [],
   "source": [
    "def predict_proba(text):\n",
    "    with torch.no_grad():\n",
    "        text = torch.tensor(text_pipeline(text))\n",
    "        output = model(text, torch.tensor([0]))\n",
    "        np_output = output.numpy()[0]\n",
    "        return softmax(np_output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "pNt8qoqNreKJ"
   },
   "outputs": [],
   "source": [
    "def prediction_function(df):\n",
    "  series = df[\"text\"].apply(predict_proba)\n",
    "  return np.array(series.tolist())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "YyL40p4AcM-A"
   },
   "outputs": [],
   "source": [
    "# A quick test to verify the predict is working before uploading on Giskard\n",
    "feature_names = ['text']\n",
    "test_df = df[feature_names][:5]\n",
    "prediction_function(test_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "NwGzSUBbzEiT"
   },
   "outputs": [],
   "source": [
    "from giskard import GiskardClient\n",
    "\n",
    "url = \"http:https://localhost:19000\" # if docker image is running on your local\n",
    "#url = \"http:https://app.giskard.ai\" # If you want to upload on giskard URL \n",
    "token = \"XXX\" #Find your token in the Admin tab of your app (login: admin; password: admin)\n",
    "client = GiskardClient(url, token)\n",
    "\n",
    "newspaper = client.create_project(\"newspaper\", \"Classification of newspaper articles\", \"Project to classify newspaper articles\")\n",
    "\n",
    "# If you've already created a project with the key \"newspaper\" use\n",
    "# enron = client.get_project(\"newspaper\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "OnQKNsE8zmA4"
   },
   "outputs": [],
   "source": [
    "newspaper.upload_model_and_df(\n",
    "    prediction_function=prediction_function, \n",
    "    model_type='classification',\n",
    "    df=df, #the dataset you want to use to inspect your model\n",
    "    column_types = {\n",
    "        \"label\": \"category\",\n",
    "        \"text\": \"text\"\n",
    "    }, #all the column types of df\n",
    "    target='label', #the column name in df corresponding to the actual target variable (ground truth).\n",
    "    model_name='PyTorch',\n",
    "    dataset_name='dataset',\n",
    "    classification_labels = list(ag_news_label.values())\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}