tutorials/24_Building_Chat_App.ipynb

{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "TEebQqubxa4G"
      },
      "source": [
        "# Tutorial: Building a Conversational Chat App\n",
        "\n",
        "- **Level**: Intermediate\n",
        "- **Time to complete**: 10 minutes\n",
        "- **Nodes Used**: `PromptNode`, `ConversationalAgent` and `ConversationSummaryMemory`\n",
        "- **Goal**: After completing this tutorial, you will have learned how to use ConversationalAgent to build a conversational chat application\n",
        "- **Prerequisites**: A [Hugging Face API Key](https://huggingface.co/settings/tokens)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "DQLWWW1Yy_Ta"
      },
      "source": [
        "## Overview\n",
        "\n",
        "A [ConversationalAgent](https://docs.haystack.deepset.ai/docs/agent#conversational-agent) is a type of Agent that is specifically implemented to create chat applications easily. With its memory integration, the ConversationalAgent enables human-like conversation with large language models (LLMs).\n",
        "\n",
        "This tutorial introduces you to the ConversationalAgent, ConversationSummaryMemory and explains how you can create your conversational chat application.\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "obaSuZBHy8PF"
      },
      "source": [
        "## Preparing the Colab Environment\n",
        "\n",
        "- [Enable GPU Runtime in Colab](https://docs.haystack.deepset.ai/docs/enabling-gpu-acceleration#enabling-the-gpu-in-colab)\n",
        "- [Set logging level to INFO](https://docs.haystack.deepset.ai/docs/log-level)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "2_nC2XoPzDVh"
      },
      "source": [
        "## Installing Haystack\n",
        "\n",
        "To start, install the latest release of Haystack with `pip`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JiZktTKoaHp5"
      },
      "outputs": [],
      "source": [
        "%%bash\n",
        "\n",
        "pip install --upgrade pip\n",
        "pip install farm-haystack[colab]"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "RrPgO_1vzWb6"
      },
      "source": [
        "\n",
        "### Enabling Telemetry\n",
        "Knowing you're using this tutorial helps us decide where to invest our efforts to build a better product, but you can always opt out by commenting the following line. See [Telemetry](https://docs.haystack.deepset.ai/docs/telemetry) for more details."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6YZjKAvOzZRq"
      },
      "outputs": [],
      "source": [
        "from haystack.telemetry import tutorial_running\n",
        "\n",
        "tutorial_running(24)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "n0pnO7S6tbyW"
      },
      "source": [
        "## Initializing the ConversationalAgent\n",
        "\n",
        "To initialize a ConversationalAgent, you'll first need to create a PromptNode to define the LLM that your chat application will use. Then, you'll add a memory to enable the application to store previous conversation and use this memory to make the interaction more human-like.\n",
        "\n",
        "Now, create necessary components for a ConversationalAgent:"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "1Omji8PK_675"
      },
      "source": [
        "### 1) Provide a Hugging Face API Key\n",
        "\n",
        "Hugging Face offers [a hosted Inference API](https://huggingface.co/docs/api-inference/index) which you can use to access machine learning models using simple HTTP requests. This way, you don't need to download models from the hub. To use the service, you need to provide an [API key](https://huggingface.co/settings/tokens) from Hugging Face:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "qWuRxFWGcAL4"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "from getpass import getpass\n",
        "\n",
        "model_api_key = os.getenv(\"HF_API_KEY\", None) or getpass(\"Enter HF API key:\")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "V4LI3vzH7Xvo"
      },
      "source": [
        "### 2) Create a PromptNode\n",
        "\n",
        "You'll initialize a PromptNode with the `model_name`, `api_key`, and `max_length` to control the output length of the model. In this tutorial, you'll use [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta), an open source chat Language Model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RckAPBT3bSoh"
      },
      "outputs": [],
      "source": [
        "from haystack.nodes import PromptNode\n",
        "\n",
        "model_name = \"https://huggingface.co/HuggingFaceH4/zephyr-7b-beta\"\n",
        "prompt_node = PromptNode(model_name, api_key=model_api_key, max_length=256, stop_words=[\"Human\"])"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "E1e15LLV8ULF"
      },
      "source": [
        "### 3) Create a ConversationSummaryMemory\n",
        "\n",
        "To have a chat application closer to a human interaction, you need to provide [memory](https://docs.haystack.deepset.ai/docs/agent#conversational-agent-memory) to the ConversationalAgent. There are two types of memory options in Haystack:\n",
        "\n",
        "1.   **ConversationMemory**: stores the conversation history (default).\n",
        "2.   **ConversationSummaryMemory**: stores the conversation history and periodically generates summaries.\n",
        "\n",
        "These memory nodes inject the conversation history into the prompt for the large language model with every run. Instead of using the full conversation history, you'll use ConversationSummaryMemory that sums up the conversation without losing important information, thus saving the token limit.\n",
        "\n",
        "You can use the same PromptNode in ConversationSummaryMemory, so the same `OpenAssistant/oasst-sft-1-pythia-12b` model generates chat summaries. By default, ConversationSummaryMemory summarizes the chat with every `3` runs using the predefined [`conversational-summary`](https://prompthub.deepset.ai/?prompt=deepset%2Fconversational-summary) PromptTemplate on PromptHub."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iBisS_dI8kan"
      },
      "outputs": [],
      "source": [
        "from haystack.agents.memory import ConversationSummaryMemory\n",
        "\n",
        "summary_memory = ConversationSummaryMemory(prompt_node)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "RpeKtIsSSNSh"
      },
      "source": [
        "> Optionally, you can define a separate PromptNode with another LLM and PromptTemplate for generating conversation summary and use it in the ConversationSummaryMemory."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "MiGGpDW98XjM"
      },
      "source": [
        "### 4) Create a ConversationalAgent\n",
        "\n",
        "Now that you have all the necessary components, you can initialize the ConversationalAgent. If you don't provide any tools, the ConversationalAgent uses the [`conversational-agent-without-tools`](https://prompthub.deepset.ai/?prompt=deepset%2Fconversational-agent-without-tools) prompt by default."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_gt2DqNzllPQ"
      },
      "outputs": [],
      "source": [
        "from haystack.agents.conversational import ConversationalAgent\n",
        "\n",
        "conversational_agent = ConversationalAgent(prompt_node=prompt_node, memory=summary_memory)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "brdbkCGryKe0"
      },
      "source": [
        "> You can add tools to your chat application using `tools` params of the ConversationalAgent:\n",
        "> ```python\n",
        "> conversational_agent = ConversationalAgent(\n",
        ">    prompt_node=prompt_node,\n",
        ">    memory=summary_memory,\n",
        ">    tools=[search_tool]\n",
        ">)\n",
        ">```\n",
        ">To learn how to create tools, check out [Haystack documentation](https://docs.haystack.deepset.ai/docs/agent#tools)."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "Mgeb6nlsGRTy"
      },
      "source": [
        "Now, your conversational agent is ready to chat!"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "Uq_yBlEXGe18"
      },
      "source": [
        "## Trying Out a Prompt"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gAi6DN-LySIH"
      },
      "outputs": [],
      "source": [
        "conversational_agent.run(\"Tell me three most interesting things about Istanbul, Turkey\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bBwcAcbgMTQN"
      },
      "outputs": [],
      "source": [
        "conversational_agent.run(\"Can you elaborate on the second item?\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-wl8dcWpMmNv"
      },
      "outputs": [],
      "source": [
        "conversational_agent.run(\"Can you turn this info into a twitter thread?\")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "mhrNe3xKIVsx"
      },
      "source": [
        "* At any point during the chat, you can use `load()` function to check the chat summary:\n",
        "\n",
        "```python\n",
        "print(conversational_agent.memory.load())\n",
        "```\n",
        "\n",
        "* To delete the whole chat history, call `clear()` method:\n",
        "\n",
        "```python\n",
        "conversational_agent.memory.clear()\n",
        "```\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "KGu3QLQXJo-z"
      },
      "source": [
        "Congratulations! 🎉 You've learned how to use ConversationalAgent to create a chat application with a summarized memory."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "wQyYPjmAG2gJ"
      },
      "source": [
        "## 💬 Example Application\n",
        "\n",
        "To take the chat experience to another level, check out this example application. Run the code cell below and use the textarea to interact with the conversational agent. Use the buttons on the right to load or delete the chat summary."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IEcpPCLKKasg"
      },
      "outputs": [],
      "source": [
        "import ipywidgets as widgets\n",
        "from IPython.display import clear_output, display\n",
        "\n",
        "## Text Input\n",
        "user_input = widgets.Textarea(\n",
        "    value=\"\",\n",
        "    placeholder=\"Type your prompt here\",\n",
        "    disabled=False,\n",
        "    style={\"description_width\": \"initial\"},\n",
        "    layout=widgets.Layout(width=\"100%\", height=\"100%\"),\n",
        ")\n",
        "\n",
        "## Submit Button\n",
        "submit_button = widgets.Button(\n",
        "    description=\"Submit\", button_style=\"success\", layout=widgets.Layout(width=\"100%\", height=\"80%\")\n",
        ")\n",
        "\n",
        "\n",
        "def on_button_clicked(b):\n",
        "    user_prompt = user_input.value\n",
        "    user_input.value = \"\"\n",
        "    print(\"\\nUser:\\n\", user_prompt)\n",
        "    conversational_agent.run(user_prompt)\n",
        "\n",
        "\n",
        "submit_button.on_click(on_button_clicked)\n",
        "\n",
        "## Show Memory Button\n",
        "memory_button = widgets.Button(\n",
        "    description=\"Show Memory\", button_style=\"info\", layout=widgets.Layout(width=\"100%\", height=\"100%\")\n",
        ")\n",
        "\n",
        "\n",
        "def on_memory_button_clicked(b):\n",
        "    memory = conversational_agent.memory.load()\n",
        "    if len(memory):\n",
        "        print(\"\\nMemory:\\n\", memory)\n",
        "    else:\n",
        "        print(\"Memory is empty\")\n",
        "\n",
        "\n",
        "memory_button.on_click(on_memory_button_clicked)\n",
        "\n",
        "## Clear Memory Button\n",
        "clear_button = widgets.Button(\n",
        "    description=\"Clear Memory\", button_style=\"warning\", layout=widgets.Layout(width=\"100%\", height=\"100%\")\n",
        ")\n",
        "\n",
        "\n",
        "def on_clear_button_button_clicked(b):\n",
        "    conversational_agent.memory.clear()\n",
        "    print(\"\\nMemory is cleared\\n\")\n",
        "\n",
        "\n",
        "clear_button.on_click(on_clear_button_button_clicked)\n",
        "\n",
        "## Layout\n",
        "grid = widgets.GridspecLayout(3, 3, height=\"200px\", width=\"800px\", grid_gap=\"10px\")\n",
        "grid[0, 2] = clear_button\n",
        "grid[0:2, 0:2] = user_input\n",
        "grid[2, 0:] = submit_button\n",
        "grid[1, 2] = memory_button\n",
        "display(grid)"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "machine_shape": "hm",
      "provenance": [],
      "toc_visible": true
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3.9.6 64-bit",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.12"
    },
    "vscode": {
      "interpreter": {
        "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}