notebooks/techcrunch_news_digest_titanml_takeoff.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "08fea4db",
   "metadata": {},
   "source": [
    "# Getting a quick daily digest from your favorite tech websites"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e9ef71b",
   "metadata": {},
   "source": [
    "Motivation: We want to stay informed on the latest news in tech. However, with so many websites and news happening every day, it is impossible to keep track of what is going on. But what if we could summarize the latest developments and have all this run locally with an off-the-shelf LLM in a few lines of code?\n",
    "\n",
    "Let us see how Haystack together with TitanML's Takeoff Inference Server can help us achieve this."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "228b00a9",
   "metadata": {},
   "source": [
    "## Run Titan Takeoff Inference Server Image"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abfc2b6b",
   "metadata": {},
   "source": [
    "Remember that you must download this notebook and run it in your local environment. The Titan Takeoff Inference Server allows you to run modern open-source LLMs in your infrastructure."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5016e3c",
   "metadata": {},
   "source": [
    "```bash\n",
    "docker run --gpus all -e TAKEOFF_MODEL_NAME=TheBloke/Llama-2-7B-Chat-AWQ \\\n",
    "                      -e TAKEOFF_DEVICE=cuda \\\n",
    "                      -e TAKEOFF_MAX_SEQUENCE_LENGTH=256 \\\n",
    "                      -it \\\n",
    "                      -p 3000:3000 tytn/takeoff-pro:0.11.0-gpu\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b7787f2",
   "metadata": {},
   "source": [
    "## Daily digest from top tech websites using Deepset Haystack and Titan Takeoff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44d543e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install feedparser\n",
    "!pip install takeoff_haystack"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "893b885e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Dict, List\n",
    "\n",
    "from haystack import Document, Pipeline\n",
    "from haystack.components.builders.prompt_builder import PromptBuilder\n",
    "from haystack.components.retrievers.in_memory import InMemoryBM25Retriever\n",
    "from haystack.document_stores.in_memory import InMemoryDocumentStore\n",
    "\n",
    "import feedparser\n",
    "\n",
    "#from takeoff_haystack import TakeoffGenerator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "61f7a5d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Dict of website RSS feeds\n",
    "urls = {\n",
    "    'theverge': 'https://www.theverge.com/rss/frontpage/',\n",
    "    'techcrunch': 'https://techcrunch.com/feed',\n",
    "    'mashable': 'https://mashable.com/feeds/rss/all',\n",
    "    'cnet': 'https://cnet.com/rss/news',\n",
    "    'engadget': 'https://engadget.com/rss.xml',\n",
    "    'zdnet': 'https://zdnet.com/news/rss.xml',\n",
    "    'venturebeat': 'https://feeds.feedburner.com/venturebeat/SZYF',\n",
    "    'readwrite': 'https://readwrite.com/feed/',\n",
    "    'wired': 'https://wired.com/feed/rss',\n",
    "    'gizmodo': 'https://gizmodo.com/rss',\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "71c3a908",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Configurable parameters\n",
    "NUM_WEBSITES = 3\n",
    "NUM_TITLES = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "d7231763",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_titles(urls: Dict[str, str], num_sites: int, num_titles: int) -> List[str]:\n",
    "    titles: List[str] = []\n",
    "    sites = list(urls.keys())[:num_sites]\n",
    "    for site in sites:\n",
    "        feed = feedparser.parse(urls[site])\n",
    "        entries = feed.entries[:num_titles]\n",
    "        for entry in entries:\n",
    "            titles.append(entry.title)\n",
    "    return titles\n",
    "\n",
    "titles = get_titles(urls, NUM_WEBSITES, NUM_TITLES)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "c0c38c22",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "document_store = InMemoryDocumentStore()\n",
    "document_store.write_documents(\n",
    "    [\n",
    "        Document(content=title) for title in titles\n",
    "    ]\n",
    ")\n",
    "\n",
    "template = \"\"\"\n",
    "HEADLINES:\n",
    "{% for document in documents %}\n",
    "    {{ document.content }}\n",
    "{% endfor %}\n",
    "\n",
    "REQUEST: {{ query }}\n",
    "\"\"\"\n",
    "\n",
    "pipe = Pipeline()\n",
    "\n",
    "pipe.add_component(\"retriever\", InMemoryBM25Retriever(document_store=document_store))\n",
    "pipe.add_component(\"prompt_builder\", PromptBuilder(template=template))\n",
    "pipe.add_component(\"llm\", TakeoffGenerator(base_url=\"http:https://localhost\", port=3000))\n",
    "pipe.connect(\"retriever\", \"prompt_builder.documents\")\n",
    "pipe.connect(\"prompt_builder\", \"llm\")\n",
    "\n",
    "query = f\"Summarize each of the {NUM_WEBSITES * NUM_TITLES} provided headlines in three words.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "184f41c6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Two words: poker roguelike - Former Twitter engineers are building Particle, an AI-powered news reader - Best laptops of MWC 2024, including a 2-in-1 that broke a world record'"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titles_string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "b6ae5032",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e73917616d6f48b6b2fe49b3b033f988",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Ranking by BM25...:   0%|          | 0/1 [00:00<?, ? docs/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['\\n\\n\\nANSWER:\\n\\n1. Poker Roguelike - Exciting gameplay\\n2. AI-powered news reader - Personalized feed\\n3. Best laptops MWC 2024 - Powerful devices']\n"
     ]
    }
   ],
   "source": [
    "response = pipe.run({\"prompt_builder\": {\"query\": query}, \"retriever\": {\"query\": query}})\n",
    "\n",
    "print(response[\"llm\"][\"replies\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1a2967e9",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}