notebooks/amazon_bedrock_for_documentation_qa.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YxZBCJn21Ygd"
   },
   "source": [
    "# PDF-Based Question Answering with Amazon Bedrock and Haystack\n",
    "\n",
    "*Notebook by [Bilge Yucel](https://www.linkedin.com/in/bilge-yucel/)*\n",
    "\n",
    "[Amazon Bedrock](https://aws.amazon.com/bedrock/) is a fully managed service that provides high-performing foundation models from leading AI startups and Amazon through a single API. You can choose from various foundation models to find the one best suited for your use case.\n",
    "\n",
    "In this notebook, we'll go through the process of **creating a generative question answering application** tailored for PDF files using the newly added [Amazon Bedrock integration](https://haystack.deepset.ai/integrations/amazon-bedrock) with [Haystack](https://github.com/deepset-ai/haystack) and [OpenSearch](https://haystack.deepset.ai/integrations/opensearch-document-store) to store our documents efficiently. The demo will illustrate the step-by-step development of a QA application designed specifically for the Bedrock documentation, demonstrating the power of Bedrock in the process 🚀\n",
    "\n",
    "## Setup the Development Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "v5dzhxUV1QwR"
   },
   "source": [
    "### Install dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "EX5oCws-etEH"
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "pip install opensearch-haystack amazon-bedrock-haystack pypdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WMJaEllC1Wat"
   },
   "source": [
    "### Download Files\n",
    "\n",
    "For this application, we'll use the user guide of Amazon Bedrock. Amazon Bedrock provides the [PDF form of its guide](https://docs.aws.amazon.com/pdfs/bedrock/latest/userguide/bedrock-ug.pdf). Run the code to download the PDF to `/content/bedrock-documentation.pdf` directory 👇🏼  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "chi-VAhGeuQn"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "import boto3\n",
    "from botocore import UNSIGNED\n",
    "from botocore.config import Config\n",
    "\n",
    "s3 = boto3.client('s3', config=Config(signature_version=UNSIGNED))\n",
    "s3.download_file('core-engineering', 'public/blog-posts/bedrock-documentation.pdf', '/content/bedrock-documentation.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ys3-RvVqqWdD"
   },
   "source": [
    "### Initialize an OpenSearch Instance on Colab\n",
    "\n",
    "[OpenSearch](https://opensearch.org/) is a fully open source search and analytics engine and is compatible with the [Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/what-is.html) that’s helpful if you’d like to deploy, operate, and scale your OpenSearch cluster later on.\n",
    "\n",
    "Let’s install OpenSearch and start an instance on Colab. For other installation options, check out [OpenSearch documentation](https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "vyWWR3Xye8l_"
   },
   "outputs": [],
   "source": [
    "!wget https://artifacts.opensearch.org/releases/bundle/opensearch/2.11.1/opensearch-2.11.1-linux-x64.tar.gz\n",
    "!tar -xvf opensearch-2.11.1-linux-x64.tar.gz\n",
    "!chown -R daemon:daemon opensearch-2.11.1\n",
    "# disabling security. Be mindful when you want to disable security in production systems\n",
    "!sudo echo 'plugins.security.disabled: true' >> opensearch-2.11.1/config/opensearch.yml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Vaxe75MXkMi2"
   },
   "outputs": [],
   "source": [
    "%%bash --bg\n",
    "cd opensearch-2.11.1 && sudo -u daemon -- ./bin/opensearch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YuN1y5WQ1jI9"
   },
   "source": [
    "> OpenSearch needs 30 seconds for a fully started server"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "f9gbVwRU_Y5Q"
   },
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "time.sleep(30)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pSBYYgYq1Ij3"
   },
   "source": [
    "### API Keys\n",
    "\n",
    "To use Amazon Bedrock, you need `aws_access_key_id`, `aws_secret_access_key`, and indicate the `aws_region_name`. Once logged into your account, locate these keys under the IAM user's \"Security Credentials\" section. For detailed guidance, refer to the documentation on [Managing access keys for IAM users](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "tZTz7cHwhZ-9"
   },
   "outputs": [],
   "source": [
    "from getpass import getpass\n",
    "\n",
    "os.environ[\"AWS_ACCESS_KEY_ID\"] = getpass(\"aws_access_key_id: \")\n",
    "os.environ[\"AWS_SECRET_ACCESS_KEY\"] = getpass(\"aws_secret_access_key: \")\n",
    "os.environ[\"AWS_DEFAULT_REGION\"] = input(\"aws_region_name: \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oa6aH6fB08d_"
   },
   "source": [
    "## Building the Indexing Pipeline\n",
    "\n",
    "Our indexing pipeline will convert the PDF file into a Haystack Document using [PyPDFToDocument](https://docs.haystack.deepset.ai/v2.0/docs/pypdftodocument) and preprocess it by cleaning and splitting it into chunks before storing them in [OpenSearchDocumentStore](https://docs.haystack.deepset.ai/v2.0/docs/opensearch-document-store).\n",
    "\n",
    "Let’s run the pipeline below and index our file to our document store:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "SrBctAl5e_Kf"
   },
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "from haystack import Pipeline\n",
    "from haystack.components.converters import PyPDFToDocument\n",
    "from haystack.components.preprocessors import DocumentCleaner, DocumentSplitter\n",
    "from haystack.components.writers import DocumentWriter\n",
    "from haystack.document_stores.types import DuplicatePolicy\n",
    "from haystack_integrations.document_stores.opensearch import OpenSearchDocumentStore\n",
    "\n",
    "## Initialize the OpenSearchDocumentStore\n",
    "document_store = OpenSearchDocumentStore()\n",
    "\n",
    "## Create pipeline components\n",
    "converter = PyPDFToDocument()\n",
    "cleaner = DocumentCleaner()\n",
    "splitter = DocumentSplitter(split_by=\"sentence\", split_length=10, split_overlap=2)\n",
    "writer = DocumentWriter(document_store=document_store, policy=DuplicatePolicy.SKIP)\n",
    "\n",
    "## Add components to the pipeline\n",
    "indexing_pipeline = Pipeline()\n",
    "indexing_pipeline.add_component(\"converter\", converter)\n",
    "indexing_pipeline.add_component(\"cleaner\", cleaner)\n",
    "indexing_pipeline.add_component(\"splitter\", splitter)\n",
    "indexing_pipeline.add_component(\"writer\", writer)\n",
    "\n",
    "## Connect the components to each other\n",
    "indexing_pipeline.connect(\"converter\", \"cleaner\")\n",
    "indexing_pipeline.connect(\"cleaner\", \"splitter\")\n",
    "indexing_pipeline.connect(\"splitter\", \"writer\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oJLXM8nM02AB"
   },
   "source": [
    "Run the pipeline with the files you want to index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "X7HrON1PFHos"
   },
   "outputs": [],
   "source": [
    "indexing_pipeline.run({\"converter\": {\"sources\": [Path(\"/content/bedrock-documentation.pdf\")]}})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UNmHvZLjA4Rv"
   },
   "source": [
    "## Building the Query Pipeline\n",
    "\n",
    "Let’s create another pipeline to query our application. In this pipeline, we’ll use [OpenSearchBM25Retriever](https://docs.haystack.deepset.ai/v2.0/docs/opensearchbm25retriever) to retrieve relevant information from the OpenSearchDocumentStore and an Amazon Titan model `amazon.titan-text-express-v1` to generate answers with [AmazonBedrockGenerator](https://docs.haystack.deepset.ai/v2.0/docs/amazonbedrockgenerator). You can select and test different models using the dropdown on right.\n",
    "\n",
    "Next, we'll create a prompt for our task using the Retrieval-Augmented Generation (RAG) approach with [PromptBuilder](https://docs.haystack.deepset.ai/v2.0/docs/promptbuilder). This prompt will help generate answers by considering the provided context. Finally, we'll connect these three components to complete the pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8Q3JYuyShRnQ"
   },
   "outputs": [],
   "source": [
    "from haystack.components.builders import PromptBuilder\n",
    "from haystack import Pipeline\n",
    "from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockGenerator\n",
    "from haystack_integrations.components.retrievers.opensearch import OpenSearchBM25Retriever\n",
    "\n",
    "## Create pipeline components\n",
    "retriever = OpenSearchBM25Retriever(document_store=document_store, top_k=15)\n",
    "\n",
    "## Initialize the AmazonBedrockGenerator with an Amazon Bedrock model\n",
    "bedrock_model = 'amazon.titan-text-express-v1' # @param [\"amazon.titan-text-express-v1\", \"amazon.titan-text-lite-v1\", \"anthropic.claude-instant-v1\", \"anthropic.claude-v1\", \"anthropic.claude-v2\",\"anthropic.claude-v2:1\", \"meta.llama2-13b-chat-v1\", \"meta.llama2-70b-chat-v1\", \"ai21.j2-mid-v1\", \"ai21.j2-ultra-v1\"]\n",
    "generator = AmazonBedrockGenerator(model=bedrock_model, max_length=500)\n",
    "template = \"\"\"\n",
    "{% for document in documents %}\n",
    "    {{ document.content }}\n",
    "{% endfor %}\n",
    "\n",
    "Please answer the question based on the given information from Amazon Bedrock documentation.\n",
    "\n",
    "{{question}}\n",
    "\"\"\"\n",
    "prompt_builder = PromptBuilder(template=template)\n",
    "\n",
    "## Add components to the pipeline\n",
    "rag_pipeline = Pipeline()\n",
    "rag_pipeline.add_component(\"retriever\", retriever)\n",
    "rag_pipeline.add_component(\"prompt_builder\", prompt_builder)\n",
    "rag_pipeline.add_component(\"llm\", generator)\n",
    "\n",
    "## Connect the components to each other\n",
    "rag_pipeline.connect(\"retriever\", \"prompt_builder.documents\")\n",
    "rag_pipeline.connect(\"prompt_builder\", \"llm\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5NywqZKo6msf"
   },
   "source": [
    "Ask your question and learn about the Amazon Bedrock service using Amazon Bedrock models!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mDYCSRRtiAy5",
    "outputId": "671da188-ed15-48e7-958f-0e1343faf458"
   },
   "outputs": [],
   "source": [
    "question = \"What is Amazon Bedrock?\"\n",
    "response = rag_pipeline.run({\"retriever\": {\"query\": question}, \"prompt_builder\": {\"question\": question}})\n",
    "\n",
    "print(response[\"llm\"][\"replies\"][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "giSWajzyAcNp",
    "outputId": "0ec1dcc6-4a28-4815-f9b4-7c073584c20e"
   },
   "outputs": [],
   "source": [
    "question = \"How can I setup Amazon Bedrock?\"\n",
    "response = rag_pipeline.run({\"retriever\": {\"query\": question}, \"prompt_builder\": {\"question\": question}})\n",
    "\n",
    "print(response[\"llm\"][\"replies\"][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ROhJ8VL_JdHc",
    "outputId": "76f73ed3-4fb3-4def-b88e-bbfde218cb7e"
   },
   "outputs": [],
   "source": [
    "question = \"How can I finetune foundation models?\"\n",
    "response = rag_pipeline.run({\"retriever\": {\"query\": question}, \"prompt_builder\": {\"question\": question}})\n",
    "\n",
    "print(response[\"llm\"][\"replies\"][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9ohsesYDgsSq",
    "outputId": "2ab326d4-4279-424a-a826-10dfcbe3c00d"
   },
   "outputs": [],
   "source": [
    "question = \"How should I form my prompts for Amazon Titan models?\"\n",
    "response = rag_pipeline.run({\"retriever\": {\"query\": question}, \"prompt_builder\": {\"question\": question}})\n",
    "\n",
    "print(response[\"llm\"][\"replies\"][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "__NGWZdqh_dJ",
    "outputId": "26c84bf3-9401-438a-bac5-1032a5d9c20e"
   },
   "outputs": [],
   "source": [
    "question = \"How should I form my prompts for Claude models?\"\n",
    "response = rag_pipeline.run({\"retriever\": {\"query\": question}, \"prompt_builder\": {\"question\": question}})\n",
    "\n",
    "print(response[\"llm\"][\"replies\"][0])"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}