tutorials/03_Basic_QA_Pipeline_without_Elasticsearch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Build a QA System Without Elasticsearch\n",
    "\n",
    "Haystack provides alternatives to Elasticsearch for developing quick prototypes.\n",
    "\n",
    "You can use an `InMemoryDocumentStore` or a `SQLDocumentStore`(with SQLite) as the document store.\n",
    "\n",
    "If you are interested in more feature-rich Elasticsearch, then please refer to the Tutorial 1. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Prepare environment\n",
    "\n",
    "#### Colab: Enable the GPU runtime\n",
    "Make sure you enable the GPU runtime to experience decent speed in this tutorial.\n",
    "**Runtime -> Change Runtime type -> Hardware accelerator -> GPU**\n",
    "\n",
    "<img src=\"https://raw.githubusercontent.com/deepset-ai/haystack/main/docs/img/colab_gpu_runtime.jpg\">\n",
    "\n",
    "You can double check whether the GPU runtime is enabled with the following command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    },
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To start, install the latest release of Haystack with `pip`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "pip install --upgrade pip\n",
    "pip install git+https://github.com/deepset-ai/haystack.git#egg=farm-haystack[colab]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Logging\n",
    "\n",
    "We configure how logging messages should be displayed and which log level should be used before importing Haystack.\n",
    "Example log message:\n",
    "INFO - haystack.utils.preprocessing -  Converting data/tutorial1/218_Olenna_Tyrell.txt\n",
    "Default log level in basicConfig is WARNING so the explicit parameter is not necessary but can be changed easily:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import logging\n",
    "\n",
    "logging.basicConfig(format=\"%(levelname)s - %(name)s -  %(message)s\", level=logging.WARNING)\n",
    "logging.getLogger(\"haystack\").setLevel(logging.INFO)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Document Store\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In-Memory Document Store\n",
    "from haystack.document_stores import InMemoryDocumentStore\n",
    "\n",
    "document_store = InMemoryDocumentStore()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Alternatively, uncomment the following to use the SQLite Document Store:\n",
    "\n",
    "# from haystack.document_stores import SQLDocumentStore\n",
    "# document_store = SQLDocumentStore(url=\"sqlite:https:///qa.db\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Preprocessing of documents\n",
    "\n",
    "Haystack provides a customizable pipeline for:\n",
    " - converting files into texts\n",
    " - cleaning texts\n",
    " - splitting texts\n",
    " - writing them to a Document Store\n",
    "\n",
    "In this tutorial, we download Wikipedia articles on Game of Thrones, apply a basic cleaning function, and index them in Elasticsearch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from haystack.utils import clean_wiki_text, convert_files_to_docs, fetch_archive_from_http\n",
    "\n",
    "\n",
    "# Let's first get some documents that we want to query\n",
    "# Here: 517 Wikipedia articles for Game of Thrones\n",
    "doc_dir = \"data/tutorial3\"\n",
    "s3_url = \"https://s3.eu-central-1.amazonaws.com/deepset.ai-farm-qa/datasets/documents/wiki_gameofthrones_txt3.zip\"\n",
    "fetch_archive_from_http(url=s3_url, output_dir=doc_dir)\n",
    "\n",
    "# convert files to dicts containing documents that can be indexed to our datastore\n",
    "# You can optionally supply a cleaning function that is applied to each doc (e.g. to remove footers)\n",
    "# It must take a str as input, and return a str.\n",
    "docs = convert_files_to_docs(dir_path=doc_dir, clean_func=clean_wiki_text, split_paragraphs=True)\n",
    "\n",
    "# We now have a list of dictionaries that we can write to our document store.\n",
    "# If your texts come from a different source (e.g. a DB), you can of course skip convert_files_to_dicts() and create the dictionaries yourself.\n",
    "# The default format here is: {\"name\": \"<some-document-name>\", \"content\": \"<the-actual-text>\"}\n",
    "\n",
    "# Let's have a look at the first 3 entries:\n",
    "print(docs[:3])\n",
    "\n",
    "# Now, let's write the docs to our DB.\n",
    "document_store.write_documents(docs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize Retriever, Reader & Pipeline\n",
    "\n",
    "### Retriever\n",
    "\n",
    "Retrievers help narrowing down the scope for the Reader to smaller units of text where a given question could be answered. \n",
    "\n",
    "With InMemoryDocumentStore or SQLDocumentStore, you can use the TfidfRetriever. For more retrievers, please refer to the tutorial-1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": false,
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# An in-memory TfidfRetriever based on Pandas dataframes\n",
    "from haystack.nodes import TfidfRetriever\n",
    "\n",
    "retriever = TfidfRetriever(document_store=document_store)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reader\n",
    "\n",
    "A Reader scans the texts returned by retrievers in detail and extracts the k best answers. They are based\n",
    "on powerful, but slower deep learning models.\n",
    "\n",
    "Haystack currently supports Readers based on the frameworks FARM and Transformers.\n",
    "With both you can either load a local model or one from Hugging Face's model hub (https://huggingface.co/models).\n",
    "\n",
    "**Here:** a medium sized RoBERTa QA model using a Reader based on FARM (https://huggingface.co/deepset/roberta-base-squad2)\n",
    "\n",
    "**Alternatives (Reader):** TransformersReader (leveraging the `pipeline` of the Transformers package)\n",
    "\n",
    "**Alternatives (Models):** e.g. \"distilbert-base-uncased-distilled-squad\" (fast) or \"deepset/bert-large-uncased-whole-word-masking-squad2\" (good accuracy)\n",
    "\n",
    "**Hint:** You can adjust the model to return \"no answer possible\" with the no_ans_boost. Higher values mean the model prefers \"no answer possible\"\n",
    "\n",
    "#### FARMReader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": [
    "from haystack.nodes import FARMReader\n",
    "\n",
    "\n",
    "# Load a  local model or any of the QA models on\n",
    "# Hugging Face's model hub (https://huggingface.co/models)\n",
    "reader = FARMReader(model_name_or_path=\"deepset/roberta-base-squad2\", use_gpu=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### TransformersReader\n",
    "\n",
    "Alternatively, we can use a Transformers reader:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from haystack.nodes import FARMReader, TransformersReader\n",
    "# reader = TransformersReader(model_name_or_path=\"distilbert-base-uncased-distilled-squad\", tokenizer=\"distilbert-base-uncased\", use_gpu=-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pipeline\n",
    "\n",
    "With a Haystack `Pipeline` you can stick together your building blocks to a search pipeline.\n",
    "Under the hood, `Pipelines` are Directed Acyclic Graphs (DAGs) that you can easily customize for your own use cases.\n",
    "To speed things up, Haystack also comes with a few predefined Pipelines. One of them is the `ExtractiveQAPipeline` that combines a retriever and a reader to answer our questions.\n",
    "You can learn more about `Pipelines` in the [docs](https://haystack.deepset.ai/docs/latest/pipelines)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": [
    "from haystack.pipelines import ExtractiveQAPipeline\n",
    "\n",
    "pipe = ExtractiveQAPipeline(reader, retriever)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Voilà! Ask a question!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": [
    "# You can configure how many candidates the reader and retriever shall return\n",
    "# The higher top_k for retriever, the better (but also the slower) your answers.\n",
    "prediction = pipe.run(\n",
    "    query=\"Who is the father of Arya Stark?\", params={\"Retriever\": {\"top_k\": 10}, \"Reader\": {\"top_k\": 5}}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# You can try asking more questions:\n",
    "\n",
    "# prediction = pipe.run(query=\"Who created the Dothraki vocabulary?\", params={\"Reader\": {\"top_k\": 5}})\n",
    "# prediction = pipe.run(query=\"Who is the sister of Sansa?\", params={\"Reader\": {\"top_k\": 5}})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now you can either print the object directly...\n",
    "from pprint import pprint\n",
    "\n",
    "pprint(prediction)\n",
    "\n",
    "# Sample output:\n",
    "# {\n",
    "#     'answers': [ <Answer: answer='Eddard', type='extractive', score=0.9919578731060028, offsets_in_document=[{'start': 608, 'end': 615}], offsets_in_context=[{'start': 72, 'end': 79}], document_id='cc75f739897ecbf8c14657b13dda890e', meta={'name': '454_Music_of_Game_of_Thrones.txt'}}, context='...' >,\n",
    "#                  <Answer: answer='Ned', type='extractive', score=0.9767240881919861, offsets_in_document=[{'start': 3687, 'end': 3801}], offsets_in_context=[{'start': 18, 'end': 132}], document_id='9acf17ec9083c4022f69eb4a37187080', meta={'name': '454_Music_of_Game_of_Thrones.txt'}}, context='...' >,\n",
    "#                  ...\n",
    "#                ]\n",
    "#     'documents': [ <Document: content_type='text', score=0.8034909798951382, meta={'name': '332_Sansa_Stark.txt'}, embedding=None, id=d1f36ec7170e4c46cde65787fe125dfe', content='\\n===\\'\\'A Game of Thrones\\'\\'===\\nSansa Stark begins the novel by being betrothed to Crown ...'>,\n",
    "#                    <Document: content_type='text', score=0.8002150354529785, meta={'name': '191_Gendry.txt'}, embedding=None, id='dd4e070a22896afa81748d6510006d2', 'content='\\n===Season 2===\\nGendry travels North with Yoren and other Night's Watch recruits, including Arya ...'>,\n",
    "#                    ...\n",
    "#                  ],\n",
    "#     'no_ans_gap':  11.688868522644043,\n",
    "#     'node_id': 'Reader',\n",
    "#     'params': {'Reader': {'top_k': 5}, 'Retriever': {'top_k': 5}},\n",
    "#     'query': 'Who is the father of Arya Stark?',\n",
    "#     'root_node': 'Query'\n",
    "# }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": false,
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# ...or use a util to simplify the output\n",
    "from haystack.utils import print_answers\n",
    "\n",
    "\n",
    "# Change `minimum` to `medium` or `all` to control the level of detail\n",
    "print_answers(prediction, details=\"minimum\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## About us\n",
    "\n",
    "This [Haystack](https://github.com/deepset-ai/haystack/) notebook was made with love by [deepset](https://deepset.ai/) in Berlin, Germany\n",
    "\n",
    "We bring NLP to the industry via open source!  \n",
    "Our focus: Industry specific language models & large scale QA systems.  \n",
    "  \n",
    "Some of our other work: \n",
    "- [German BERT](https://deepset.ai/german-bert)\n",
    "- [GermanQuAD and GermanDPR](https://deepset.ai/germanquad)\n",
    "- [FARM](https://github.com/deepset-ai/FARM)\n",
    "\n",
    "Get in touch:\n",
    "[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Discord](https://haystack.deepset.ai/community/join) | [GitHub Discussions](https://github.com/deepset-ai/haystack/discussions) | [Website](https://deepset.ai)\n",
    "\n",
    "By the way: [we're hiring!](https://www.deepset.ai/jobs)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.9 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.9"
  },
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}