cleaning up notebooks

deepset-ai · Jun 13, 2024 · 9dc0e5c · 9dc0e5c
1 parent 519aedb
commit 9dc0e5c
Show file tree

Hide file tree

Showing 3 changed files with 7 additions and 1,434 deletions.
diff --git a/...uations/analyze_aragog_eval_results.ipynb → evaluations/analyze_aragog.ipynb b/...uations/analyze_aragog_eval_results.ipynb → evaluations/analyze_aragog.ipynb
@@ -1,31 +1,5 @@
 {
  "cells": [
- {
- "cell_type": "markdown",
- "id": "de766774-9542-4798-987f-60ede7e7e9ca",
- "metadata": {
- "id": "de766774-9542-4798-987f-60ede7e7e9ca"
- },
- "source": [
- "In this tutorial, we will show you how to use Haystack to evaluate the performance of a RAG pipeline.\n",
- "\n",
- "We will use the dataset from the [ARAGOG - Advanced Retrieval Augmented Generation Output Grading (ARAGOG)](https://arxiv.org/pdf/2404.01037) paper.\n",
- "The dataset is composed of a collection of 13 public AI/LLM-ArXiv research papers and 107 question-answer (QA) pairs. The (QA) pairs generated with the assistance of GPT-4, and then each pair was validated/corrected by humans.\n",
- "\n",
- "We will use the following Haystack components to evaluate the performance of a RAG pipeline:\n",
- "\n",
- "- [ContextRelevance](https://docs.haystack.deepset.ai/docs/contextrelevanceevaluator)\n",
- "- [Faithfulness](https://docs.haystack.deepset.ai/docs/faithfulnessevaluator)\n",
- "- [Semantic Answer Similarity](https://docs.haystack.deepset.ai/docs/sasevaluator)\n",
- "\n",
- "\n",
- "We will build a RAG pipeline and then evaluate it using the ARAGOG dataset by varying three parameters:\n",
- "\n",
- "- `top_k`: the maximum number of documents returned by the retriever\n",
- "- `embedding_model`: the model used to encode the documents and the question\n",
- "- `chunk_size`: the number of tokens in the input text that the model can process at once"
- ]
- },
  {
  "cell_type": "markdown",
  "id": "00660aae-32be-4ff8-8c03-9de0df9dc1d3",
@@ -38,7 +12,7 @@
  },
  {
  "cell_type": "code",
- "execution_count": 25,
+ "execution_count": 1,
  "id": "013450e2-a49b-4111-8f04-6999376484f1",
  "metadata": {
  "id": "013450e2-a49b-4111-8f04-6999376484f1"
@@ -61,7 +35,7 @@
  },
  {
  "cell_type": "code",
- "execution_count": 26,
+ "execution_count": 2,
  "id": "67d86ea5-5d39-4cb8-90d2-470471571ac5",
  "metadata": {
  "id": "67d86ea5-5d39-4cb8-90d2-470471571ac5"
@@ -99,7 +73,7 @@
  },
  {
  "cell_type": "code",
- "execution_count": 28,
+ "execution_count": 3,
  "id": "a03966eb-658d-4e16-bce0-e198886eca35",
  "metadata": {
  "id": "a03966eb-658d-4e16-bce0-e198886eca35"
@@ -112,7 +86,7 @@
  },
  {
  "cell_type": "code",
- "execution_count": 29,
+ "execution_count": 4,
  "id": "a018bfb3-755b-4a4f-9f2d-cf69201f9f6d",
  "metadata": {
  "colab": {
@@ -460,7 +434,7 @@
  "26 3 256 "
  ]
  },
- "execution_count": 29,
+ "execution_count": 4,
  "metadata": {},
  "output_type": "execute_result"
  }