haystack2x-demos/haystack_rag_serperdev_demo.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyOlv7POfUpUJiZnBfLWANQj",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/vblagoje/notebooks/blob/main/haystack2x-demos/haystack_rag_serperdev_demo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Introduction\n",
        "\n",
        "This notebook provides a detailed guide on leveraging Haystack 2.x and [serper.dev](https://serper.dev/) for implementing an OpenAPI service-based Retriever-Augmented Generation (RAG) workflow. It outlines an approach for processing user queries by integrating web search results into the context of Large Language Models (LLMs), enhancing their response generation.\n",
        "\n",
        "The pipeline starts by fetching the OpenAPI specification for serper.dev and converting it into OpenAI function-calling definitions. serper.dev is then used as an OpenAPI service to retrieve relevant information from the web based on the user's input. The pipeline incorporates these search results, along with system and user prompts, into the LLM to enhance response generation.\n",
        "\n",
        "The notebook will guide you through installing the necessary libraries, selecting an LLM provider, and constructing a Haystack 2.0 pipeline that orchestrates function calling, service requests, and LLM response generation.\n",
        "\n",
        "Note: To run this pipeline, a [serper.dev](https://serper.dev/) account is required. Signing up is quick and provides you with 2,500 free queries—no credit card needed."
      ],
      "metadata": {
        "id": "J8TDTtQwiSn6"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 1. Setup\n",
        "\n",
        "Let's install necessary libraries and import key modules to build the foundation for the subsequent steps."
      ],
      "metadata": {
        "id": "6zUWNgF-kUeU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install -q openapi3 jsonref git+https://github.com/deepset-ai/haystack.git@v2.0.x"
      ],
      "metadata": {
        "id": "bH8Lo7PSumLG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import getpass\n",
        "import os\n",
        "import json\n",
        "import requests\n",
        "\n",
        "from typing import Dict, Any, List\n",
        "from haystack import Pipeline\n",
        "from haystack.components.generators.utils import print_streaming_chunk\n",
        "from haystack.components.converters import OpenAPIServiceToFunctions, OutputAdapter\n",
        "from haystack.components.generators.chat import OpenAIChatGenerator\n",
        "from haystack.components.connectors import OpenAPIServiceConnector\n",
        "from haystack.components.fetchers import LinkContentFetcher\n",
        "from haystack.dataclasses import ChatMessage, ByteStream\n",
        "from haystack.utils import Secret"
      ],
      "metadata": {
        "id": "RhxdVLCb_D8R"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 2.  Enter the API keys for LLM provider and serper.dev"
      ],
      "metadata": {
        "id": "V8GwXloVDtg9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Enable support for multiple LLM providers and simplify switching between them\n",
        "\n",
        "llm_providers = {\"fireworks\": {\"api_base_url\": \"https://api.fireworks.ai/inference/v1\",\n",
        "                               \"text_model\": \"accounts/fireworks/models/mixtral-8x7b-instruct\",\n",
        "                               \"fc_model\": \"accounts/fireworks/models/firefunction-v1\"},\n",
        "                 \"openai\": {\"api_base_url\": \"https://api.openai.com/v1\",\n",
        "                            \"text_model\": \"gpt-4-turbo-preview\",\n",
        "                            \"fc_model\": \"gpt-3.5-turbo\"}\n",
        "                 }\n",
        "\n",
        "# change this value to another provider defined above - if needed\n",
        "selected_provider = \"fireworks\""
      ],
      "metadata": {
        "id": "Wqc7HHU4JE0O"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "llm_api_key = getpass.getpass(f\"Enter LLM api key for {selected_provider}:\")\n",
        "serper_dev_key = getpass.getpass(\"Enter serperdev api key:\")\n"
      ],
      "metadata": {
        "id": "TrK-E_CF__MG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 3. Build our RAG serper.dev pipeline\n",
        "\n"
      ],
      "metadata": {
        "id": "0N4yjfJHD8HU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# An OutputAdapter filter we'll use to setup function calling\n",
        "def prepare_fc_params(openai_functions_schema: Dict[str, Any]) -> Dict[str, Any]:\n",
        "    return {\n",
        "        \"tools\": [{\n",
        "            \"type\": \"function\",\n",
        "            \"function\": openai_functions_schema\n",
        "        }],\n",
        "        \"tool_choice\": {\n",
        "            \"type\": \"function\",\n",
        "            \"function\": {\"name\": openai_functions_schema[\"name\"]}\n",
        "        }\n",
        "    }\n",
        "\n",
        "pipe = Pipeline()\n",
        "pipe.add_component(\"spec_to_functions\", OpenAPIServiceToFunctions())\n",
        "pipe.add_component(\"functions_llm\", OpenAIChatGenerator(api_key=Secret.from_token(llm_api_key),\n",
        "                                                        model=llm_providers[selected_provider][\"fc_model\"],\n",
        "                                                        api_base_url=llm_providers[selected_provider][\"api_base_url\"]))\n",
        "pipe.add_component(\"openapi_container\", OpenAPIServiceConnector())\n",
        "pipe.add_component(\"prepare_fc_adapter\", OutputAdapter(\"{{functions[0] | prepare_fc}}\", Dict[str, Any], {\"prepare_fc\": prepare_fc_params}))\n",
        "pipe.add_component(\"openapi_spec_adapter\", OutputAdapter(\"{{specs[0]}}\", Dict[str, Any]))\n",
        "pipe.add_component(\"final_prompt_adapter\", OutputAdapter(\"{{system_message + service_response}}\", List[ChatMessage]))\n",
        "pipe.add_component(\"llm\", OpenAIChatGenerator(api_key=Secret.from_token(llm_api_key),\n",
        "                                              api_base_url=llm_providers[selected_provider][\"api_base_url\"],\n",
        "                                              model=llm_providers[selected_provider][\"text_model\"],\n",
        "                                              generation_kwargs={\"max_tokens\": 1024},\n",
        "                                              streaming_callback=print_streaming_chunk))\n",
        "\n",
        "pipe.connect(\"spec_to_functions.functions\", \"prepare_fc_adapter.functions\")\n",
        "pipe.connect(\"spec_to_functions.openapi_specs\", \"openapi_spec_adapter.specs\")\n",
        "pipe.connect(\"prepare_fc_adapter\", \"functions_llm.generation_kwargs\")\n",
        "pipe.connect(\"functions_llm.replies\", \"openapi_container.messages\")\n",
        "pipe.connect(\"openapi_spec_adapter\", \"openapi_container.service_openapi_spec\")\n",
        "pipe.connect(\"openapi_container.service_response\", \"final_prompt_adapter.service_response\")\n",
        "pipe.connect(\"final_prompt_adapter\", \"llm.messages\")"
      ],
      "metadata": {
        "id": "n7ErNg55_zJN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "As you can see in the pipeline graph above, for a given serper.dev query, our pipeline follows these steps:\n",
        "\n",
        "1. **Fetch OpenAPI Spec**: Retrieve the OpenAPI specification for serper.dev and convert it into OpenAI function-calling definitions.\n",
        "\n",
        "2. **Determine Parameters**: Use a function-calling model to identify the necessary parameters for the serper.dev service.\n",
        "\n",
        "3. **Query serper.dev**: Dispatch the request to serper.dev with the determined parameters and gather the responses.\n",
        "\n",
        "4. **Compile Results**: Organize and format the search results from serper.dev.\n",
        "\n",
        "5. **Generate Response**: Combine the system prompt, user query, and compiled search results, then pass them to LLM to generate the final response."
      ],
      "metadata": {
        "id": "1rawAmKSELBA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "system_prompt = requests.get(\"https://bit.ly/serper_dev_system\").text\n",
        "serper_spec = requests.get(\"https://bit.ly/serper_dev_spec\").text"
      ],
      "metadata": {
        "id": "msBEddD4Azfh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "query = \"Why did Elon Musk sue OpenAI?\"\n",
        "\n",
        "result = pipe.run(data={\"functions_llm\": {\"messages\":[ChatMessage.from_system(\"Only do function calling\"), ChatMessage.from_user(query)]},\n",
        "                        \"openapi_container\": {\"service_credentials\": serper_dev_key},\n",
        "                        \"spec_to_functions\": {\"sources\": [ByteStream.from_string(serper_spec)]},\n",
        "                        \"final_prompt_adapter\": {\"system_message\": [ChatMessage.from_system(system_prompt)]}})"
      ],
      "metadata": {
        "id": "OVcVXOSRAfrm"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}