tutorials/12_LFQA.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "bEH-CRbeA6NU"
   },
   "source": [
    "# Generative QA with Seq2SeqGenerator"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> As of version 1.16, `Seq2SeqGenerator` has been deprecated in Haystack and completely removed from Haystack as of v1.18. We recommend following the tutorial on [Creating a Generative QA Pipeline with Retrieval-Augmentation](https://haystack.deepset.ai/tutorials/22_pipeline_with_promptnode) instead. For more details about this deprecation, check out [our announcement](https://github.com/deepset-ai/haystack/discussions/4816) on Github."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Follow this tutorial to learn how to build and use a pipeline for Long-Form Question Answering (LFQA). LFQA is a variety of the generative question answering task. LFQA systems query large document stores for relevant information and then use this information to generate accurate, multi-sentence answers. In a regular question answering system, the retrieved documents related to the query (context passages) act as source tokens for extracted answers. In an LFQA system, context passages provide the context the system uses to generate original, abstractive, long-form answers."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "3K27Y5FbA6NV"
   },
   "source": [
    "\n",
    "## Preparing the Colab Environment\n",
    "\n",
    "- [Enable GPU Runtime](https://docs.haystack.deepset.ai/docs/enabling-gpu-acceleration#enabling-the-gpu-in-colab)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installing Haystack\n",
    "\n",
    "To start, let's install the latest release of Haystack with `pip`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "NM36kbRFA6Nc"
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "pip install --upgrade pip\n",
    "pip install farm-haystack[colab,faiss]==1.17.2"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Enabling Telemetry \n",
    "Knowing you're using this tutorial helps us decide where to invest our efforts to build a better product but you can always opt out by commenting the following line. See [Telemetry](https://docs.haystack.deepset.ai/docs/telemetry) for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from haystack.telemetry import tutorial_running\n",
    "\n",
    "tutorial_running(12)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Logging\n",
    "\n",
    "We configure how logging messages should be displayed and which log level should be used before importing Haystack.\n",
    "Example log message:\n",
    "INFO - haystack.utils.preprocessing -  Converting data/tutorial1/218_Olenna_Tyrell.txt\n",
    "Default log level in basicConfig is WARNING so the explicit parameter is not necessary but can be changed easily:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import logging\n",
    "\n",
    "logging.basicConfig(format=\"%(levelname)s - %(name)s -  %(message)s\", level=logging.WARNING)\n",
    "logging.getLogger(\"haystack\").setLevel(logging.INFO)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "q3dSo7ZtA6Nl"
   },
   "source": [
    "## Initializing the DocumentStore\n",
    "\n",
    "FAISS is a library for efficient similarity search on a cluster of dense vectors.\n",
    "The `FAISSDocumentStore` uses a SQL(SQLite in-memory be default) database under-the-hood\n",
    "to store the document text and other meta data. The vector embeddings of the text are\n",
    "indexed on a FAISS Index that later is queried for searching answers.\n",
    "The default flavour of FAISSDocumentStore is \"Flat\" but can also be set to \"HNSW\" for\n",
    "faster search at the expense of some accuracy. Just set the faiss_index_factor_str argument in the constructor.\n",
    "For more info on which suits your use case: https://github.com/facebookresearch/faiss/wiki/Guidelines-to-choose-an-index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1cYgDJmrA6Nv",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from haystack.document_stores import FAISSDocumentStore\n",
    "\n",
    "document_store = FAISSDocumentStore(embedding_dim=128, faiss_index_factory_str=\"Flat\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "06LatTJBA6N0",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Cleaning and Writing Documents\n",
    "\n",
    "Similarly to the previous tutorials, we download, convert and write some Game of Thrones articles to our DocumentStore."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "iqKnu6wxA6N1",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from haystack.utils import convert_files_to_docs, fetch_archive_from_http, clean_wiki_text\n",
    "\n",
    "\n",
    "# Let's first get some files that we want to use\n",
    "doc_dir = \"data/tutorial12\"\n",
    "s3_url = \"https://s3.eu-central-1.amazonaws.com/deepset.ai-farm-qa/datasets/documents/wiki_gameofthrones_txt12.zip\"\n",
    "fetch_archive_from_http(url=s3_url, output_dir=doc_dir)\n",
    "\n",
    "# Convert files to dicts\n",
    "docs = convert_files_to_docs(dir_path=doc_dir, clean_func=clean_wiki_text, split_paragraphs=True)\n",
    "\n",
    "# Now, let's write the dicts containing documents to our DB.\n",
    "document_store.write_documents(docs)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "wgjedxx_A6N6"
   },
   "source": [
    "## Initializing the Retriever\n",
    "\n",
    "We use a `DensePassageRetriever` and we invoke `update_embeddings` to index the embeddings of documents in the `FAISSDocumentStore`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "kFwiPP60A6N7",
    "pycharm": {
     "is_executing": true
    }
   },
   "outputs": [],
   "source": [
    "from haystack.nodes import DensePassageRetriever\n",
    "\n",
    "retriever = DensePassageRetriever(\n",
    "    document_store=document_store,\n",
    "    query_embedding_model=\"vblagoje/dpr-question_encoder-single-lfqa-wiki\",\n",
    "    passage_embedding_model=\"vblagoje/dpr-ctx_encoder-single-lfqa-wiki\",\n",
    ")\n",
    "\n",
    "document_store.update_embeddings(retriever)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "sMlVEnJ2NkZZ"
   },
   "source": [
    "Before we blindly use the `DensePassageRetriever` let's empirically test it to make sure a simple search indeed finds the relevant documents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "qpu-t9rndgpe"
   },
   "outputs": [],
   "source": [
    "from haystack.utils import print_documents\n",
    "from haystack.pipelines import DocumentSearchPipeline\n",
    "\n",
    "p_retrieval = DocumentSearchPipeline(retriever)\n",
    "res = p_retrieval.run(query=\"Tell me something about Arya Stark?\", params={\"Retriever\": {\"top_k\": 10}})\n",
    "print_documents(res, max_text_len=512)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "rnVR28OXA6OA"
   },
   "source": [
    "## Initializing the Generator\n",
    "\n",
    "Similar to previous Tutorials we now initalize our Generator.\n",
    "\n",
    "Here we use a `Seq2SeqGenerator` with the [*vblagoje/bart_lfqa*](https://huggingface.co/vblagoje/bart_lfqa) model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "fyIuWVwhA6OB"
   },
   "outputs": [],
   "source": [
    "from haystack.nodes import Seq2SeqGenerator\n",
    "\n",
    "\n",
    "generator = Seq2SeqGenerator(model_name_or_path=\"vblagoje/bart_lfqa\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "unhLD18yA6OF"
   },
   "source": [
    "## Initializing the Pipeline\n",
    "\n",
    "With a Haystack `Pipeline` you can stick together your building blocks to a search pipeline.\n",
    "Under the hood, `Pipelines` are Directed Acyclic Graphs (DAGs) that you can easily customize for your own use cases.\n",
    "To speed things up, Haystack also comes with a few predefined Pipelines. One of them is the `GenerativeQAPipeline` that combines a Retriever and a Generator to answer our questions.\n",
    "You can learn more about `Pipelines` in the [docs](https://docs.haystack.deepset.ai/docs/pipelines)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TssPQyzWA6OG"
   },
   "outputs": [],
   "source": [
    "from haystack.pipelines import GenerativeQAPipeline\n",
    "\n",
    "pipe = GenerativeQAPipeline(generator, retriever)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "bXlBBxKXA6OL"
   },
   "source": [
    "## Asking a Question\n",
    "We use the pipeline `run()` method to ask a question."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Zi97Hif2A6OM"
   },
   "outputs": [],
   "source": [
    "pipe.run(\n",
    "    query=\"How did Arya Stark's character get portrayed in a television adaptation?\", params={\"Retriever\": {\"top_k\": 3}}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "IfTP9BfFGOo6"
   },
   "outputs": [],
   "source": [
    "pipe.run(query=\"Why is Arya Stark an unusual character?\", params={\"Retriever\": {\"top_k\": 3}})"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "LFQA_via_Haystack.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3.8.9 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}