llama-index-project-84d0e5d34e2b44d0b8fadb7215cb5c98.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "## Llama Index\n",
        "Llama Index is a Large Language Model (LLM) framework designed for efficient LLM application development, particularly in search and retrieval tasks. It indexes documents by breaking them into smaller nodes and creates embeddings with service context, storing them in a non-redundant manner. This framework enhances retrieval by using a query engine to find relevant documents and passing the prompt along with the retrieved context to the LLM for response generation. Llama Index is optimized for speed and efficiency in data lookup.\n",
        "\n",
        "In this notebook, we will create a project using a Website URL as a data source and an embedding model to generate numerical embeddings stored in a vector store, utilizing a persist directory to avoid redundant storage. We will also define a retriever and an LLM to generate responses using the context from the query engine.\n",
        "\n",
        "[Link](https://docs.llamaindex.ai/en/stable/) to the Llama Index documentation"
      ],
      "metadata": {
        "id": "8dssS_oO02VU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Installing the Packages"
      ],
      "metadata": {
        "id": "PDJz3RLx1wgj"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "k4gwMNLb6wOG",
        "collapsed": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a7a455f5-07e8-44af-c892-0fcebfc60176"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.2/2.2 MB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "!pip -q install llama-index langchain llama-index-readers-web llama-index-embeddings-langchain llama-index-llms-huggingface langchain-community"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Providing Access Token\n",
        "\n",
        "- To get your personal access token from HuggingFace Hub, vist [here](https://huggingface.co/settings/tokens)\n",
        "- **Note** - if you do not have an account in Huggingface Hub, create one by signing up.\n",
        "- Click the \"New Token\" button at the bottom to create a new token. Copy the token and paste it after running the next code block."
      ],
      "metadata": {
        "id": "NxmKD__W19TL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "from getpass import getpass\n",
        "HF_TOKEN = getpass()\n",
        "os.environ['HUGGINGFACEHUB_API_TOKEN'] = HF_TOKEN"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ykax0sfbPdr_",
        "outputId": "a47c57d7-d2bf-4fdb-b47b-9413242eec3b"
      },
      "execution_count": null,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "··········\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Importing the required Packages\n",
        "- [Vector Store Index](https://docs.llamaindex.ai/en/stable/module_guides/indexing/vector_store_index/)\n",
        "- [Simple Directory Reader](https://docs.llamaindex.ai/en/stable/examples/data_connectors/simple_directory_reader/)\n",
        "- [Simple Web Page Reader](https://docs.llamaindex.ai/en/stable/examples/data_connectors/WebPageDemo/)\n",
        "- [Service Context](https://docs.llamaindex.ai/en/v0.9.48/module_guides/supporting_modules/service_context.html)\n",
        "- [Storage Context](https://docs.llamaindex.ai/en/stable/api_reference/storage/storage_context/)\n",
        "- [Storing](https://docs.llamaindex.ai/en/stable/understanding/storing/storing/)\n",
        "- [Node Parsers](https://docs.llamaindex.ai/en/stable/module_guides/loading/node_parsers/modules/)\n",
        "- [Sentence Splitters](https://docs.llamaindex.ai/en/stable/api_reference/node_parsers/sentence_splitter/)\n",
        "- [Huggingface Embeddings](https://docs.llamaindex.ai/en/stable/examples/embeddings/huggingface/)\n",
        "- [Huggingface Inference API](https://docs.llamaindex.ai/en/stable/examples/llm/huggingface/)"
      ],
      "metadata": {
        "id": "wPwX3az42IC5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from llama_index.core import VectorStoreIndex, SimpleDirectoryReader\n",
        "from llama_index.readers.web import SimpleWebPageReader\n",
        "from llama_index.core import ServiceContext, StorageContext,load_index_from_storage\n",
        "from llama_index.core.node_parser import SimpleNodeParser, MarkdownNodeParser, SentenceSplitter\n",
        "from langchain.embeddings import HuggingFaceInferenceAPIEmbeddings\n",
        "from llama_index.embeddings.langchain import LangchainEmbedding\n",
        "from llama_index.llms.huggingface import HuggingFaceInferenceAPI"
      ],
      "metadata": {
        "id": "kY3xwHErRpOE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Storing Context\n",
        "- defining the llm end embedding model\n",
        "- storing the context using Indexing\n",
        "- retrieving the context from the storage context"
      ],
      "metadata": {
        "id": "SKukhuXL75kr"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Define the:\n",
        "- **Persist Directory** - The persist directory in LlamaIndex is the location on disk where the indexed data and metadata are stored to avoid the time and cost of re-indexing the data.\n",
        "- **LLM Model** - LLM Model uses the query and the retrieved documents to generate a response\n",
        "- **Embedding Model** - Embedding Model generates vector embeddings which are to be stored in a vector store."
      ],
      "metadata": {
        "id": "SHP897IX-7xa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from llama_index.core import Settings\n",
        "\n",
        "PERSIST_DIR = \"./storage\"\n",
        "MODEL_NAME = \"HuggingFaceH4/zephyr-7b-beta\"\n",
        "EMBED_MODEL = \"thenlper/gte-large\"\n",
        "\n",
        "Settings.llm = HuggingFaceInferenceAPI(model_name=MODEL_NAME,\n",
        "                              token=HF_TOKEN)\n",
        "Settings.embed_model = LangchainEmbedding(HuggingFaceInferenceAPIEmbeddings(api_key=HF_TOKEN,\n",
        "                                                                   model_name=EMBED_MODEL))"
      ],
      "metadata": {
        "id": "jW6cEo5S-67Q"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Using Persist Directory\n",
        "If the user uploads new data, a new directory is created. However, if the uploaded data already exists in the persistent directory, the indexes for that data are retrieved from the existing directory.\n",
        "<br/>\n",
        "<br/>\n",
        "The \"data\" used here is a directory created within the notebook directory, which contains, just a single [PDF file](https://drive.google.com/file/d/1J6S9vxeiDmBPwb27t637NCW_zVeXcwrn/view?usp=sharing)"
      ],
      "metadata": {
        "id": "gb4chZ0v_M-o"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "if not os.path.exists(PERSIST_DIR):\n",
        "    documents = SimpleDirectoryReader(\"data\").load_data()\n",
        "    parser = SimpleNodeParser()\n",
        "    nodes = parser.get_nodes_from_documents(documents)\n",
        "\n",
        "    storage_context = StorageContext.from_defaults() #vector store\n",
        "    index = VectorStoreIndex(\n",
        "        nodes,\n",
        "        storage_context=storage_context,\n",
        "    )\n",
        "    index.storage_context.persist(persist_dir=PERSIST_DIR)\n",
        "else:\n",
        "    storage_context = StorageContext.from_defaults(persist_dir=PERSIST_DIR)\n",
        "    index = load_index_from_storage(storage_context)"
      ],
      "metadata": {
        "id": "HNoT38irWfEw"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Defining the Query Engine"
      ],
      "metadata": {
        "id": "8RKV9YlN8aUN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "query_engine = index.as_query_engine()\n",
        "query_engine.query(\"what is the name of the story?\")"
      ],
      "metadata": {
        "id": "a8luuPVVX50w",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f99e2522-af5a-4191-8f6a-a65baf3d379d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Response(response=' \"I Sell My Dreams\" is the name of the story.\\n\\npage_label: 10\\nfile_path: /content/data/ncert short story.pdf\\n\\n10/I SELL MY DREAMS\\nthe temptation of questioning the Portuguese ambassador when we happened to meet some months later at a diplomatic reception. The ambassador spoke about her with great enthusiasm and enormous admiration. ‘Y ou cannot imagine how extraor dinary she was,’ he said. ‘Y ou would have been obliged to write a story about her .’ And he went on in the same tone, with surprising details, but without the clue that would have allowed me to come to a final conclusion.\\n‘In concrete terms,’ I asked at last, ‘what did she do?’\\n‘Nothing,’ he said, with a certain disenchantment. ‘She\\ndreamed.’\\nUnderstanding the T ext\\n1. Did the author believe in the prophetic ability of Frau Frieda?\\n2. Why did he think that Frau Frieda’s dreams were a stratagem\\nfor surviving?\\n3. Why does the author compare Neruda to a Renaissance pope', source_nodes=[NodeWithScore(node=TextNode(id_='a150fcd7-30e6-478b-a487-a5d15570b65f', embedding=None, metadata={'page_label': '1', 'file_name': 'ncert short story.pdf', 'file_path': '/content/data/ncert short story.pdf', 'file_type': 'application/pdf', 'file_size': 863728, 'creation_date': '2024-06-28', 'last_modified_date': '2024-06-28'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={<NodeRelationship.SOURCE: '1'>: RelatedNodeInfo(node_id='c0c5e872-0c0f-46a0-a8b1-96a055823cc2', node_type=<ObjectType.DOCUMENT: '4'>, metadata={'page_label': '1', 'file_name': 'ncert short story.pdf', 'file_path': '/content/data/ncert short story.pdf', 'file_type': 'application/pdf', 'file_size': 863728, 'creation_date': '2024-06-28', 'last_modified_date': '2024-06-28'}, hash='9de96b4d8010db722e11ecc30ee0f579f5ada7131e789a6c597594660dd586a3')}, text='1/I S ELL MY DREAMS\\nShort stories\\nINTRODUCTION\\nA short story is a prose narrative of limited length.\\nIt organises the action and thoughts of its\\ncharacters into the pattern of a plot. The plot\\nform may be comic, tragic, romantic or satiric.\\nThe central incident is selected to manifest, as\\nmuch as possible, the protagonist’s life and\\ncharacter , and the details contribute to the\\ndevelopment of the plot.\\nThe term ‘short story’ covers a great diversity of\\nprose fiction, right from the really short ‘short\\nstory’ of about five hundred words to longer and\\nmore complex works. The longer ones, with their\\nstatus of middle length, fall between the tautness\\nof the short narrative and the expansiveness of\\nthe novel.\\nThere can be thematic variation too. The stories\\ndeal with fantasy, reality, alienation and the\\nproblem of choice in personal life. There are three\\nshort stories and two long ones in this section\\nrepresenting writers from five cultures.\\nRationalised 2023-24', mimetype='text/plain', start_char_idx=0, end_char_idx=971, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), score=0.8686969912342934), NodeWithScore(node=TextNode(id_='e9a9dcf3-48d8-43af-814d-cd510d495a3b', embedding=None, metadata={'page_label': '9', 'file_name': 'ncert short story.pdf', 'file_path': '/content/data/ncert short story.pdf', 'file_type': 'application/pdf', 'file_size': 863728, 'creation_date': '2024-06-28', 'last_modified_date': '2024-06-28'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={<NodeRelationship.SOURCE: '1'>: RelatedNodeInfo(node_id='7d22fed3-c1db-4e3d-b70d-ec8dcdb23520', node_type=<ObjectType.DOCUMENT: '4'>, metadata={'page_label': '9', 'file_name': 'ncert short story.pdf', 'file_path': '/content/data/ncert short story.pdf', 'file_type': 'application/pdf', 'file_size': 863728, 'creation_date': '2024-06-28', 'last_modified_date': '2024-06-28'}, hash='00f79a204a8d0bbc6ad9f3d29f581cc650ec41670c380eb2e8d2401915ef24e8')}, text='9/I S ELL MY DREAMS\\nthe temptation of questioning the Portuguese\\nambassador when we happened to meet some months\\nlater at a diplomatic reception. The ambassador spoke\\nabout her with great enthusiasm and enormous\\nadmiration. ‘Y ou cannot imagine how extraor dinary she\\nwas,’ he said. ‘Y ou would have been obliged to write a\\nstory about her .’ And he went on in the same tone, with\\nsurprising details, but without the clue that would have\\nallowed me to come to a final conclusion.\\n‘In concrete terms,’ I asked at last, ‘what did she do?’\\n‘Nothing,’ he said, with a certain disenchantment. ‘She\\ndreamed.’\\nUnderstanding the T ext\\n1. Did the author believe in the prophetic ability of Frau Frieda?\\n2. Why did he think that Frau Frieda’s dreams were a stratagem\\nfor surviving?\\n3. Why does the author compare Neruda to a Renaissance pope?\\nTalking about the T ext\\nDiscuss in groups\\n1. In spite of all the rationality that human beings are capable of,\\nmost of us are suggestible and yield to archaic superstitions.\\n2. Dreams and clairvoyance are as much an element of the poetic\\nvision as religious superstition.\\nAppreciation\\n1. The story hinges on a gold ring shaped like a serpent with\\nemerald eyes. Comment on the responses that this image\\nevokes in the r eader .\\n2. The craft of a master story-teller lies in the ability to interweave\\nimagination and reality. Do you think that this story illustrates this?\\n3. Bring out the contradiction in the last exchange between the\\nauthor and the Portuguese ambassador\\n‘In concrete terms,’ I asked at last, ‘what did she do?’ ‘Nothing,’\\nhe said, with a  certain disenchantment. ‘She dreamed.’\\n4. Comment on the ironical element in the story.\\nRationalised 2023-24', mimetype='text/plain', start_char_idx=0, end_char_idx=1697, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'), score=0.8472080173677616)], metadata={'a150fcd7-30e6-478b-a487-a5d15570b65f': {'page_label': '1', 'file_name': 'ncert short story.pdf', 'file_path': '/content/data/ncert short story.pdf', 'file_type': 'application/pdf', 'file_size': 863728, 'creation_date': '2024-06-28', 'last_modified_date': '2024-06-28'}, 'e9a9dcf3-48d8-43af-814d-cd510d495a3b': {'page_label': '9', 'file_name': 'ncert short story.pdf', 'file_path': '/content/data/ncert short story.pdf', 'file_type': 'application/pdf', 'file_size': 863728, 'creation_date': '2024-06-28', 'last_modified_date': '2024-06-28'}})"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Generating responses\n",
        "- defining the response synthesizer using the service context\n",
        "- using the vector retriever and response synthesizer in the query engine to generate responses"
      ],
      "metadata": {
        "id": "Ni2eS1CP8hY6"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from llama_index.core import get_response_synthesizer\n",
        "from llama_index.core.query_engine import RetrieverQueryEngine\n",
        "from llama_index.core.retrievers import VectorIndexRetriever"
      ],
      "metadata": {
        "id": "9JJn_39K9g7w"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "response_synthesizer = get_response_synthesizer()"
      ],
      "metadata": {
        "id": "FzuDHyKaK1J_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "vector_retriever = VectorIndexRetriever(index=index, similarity_top_k=2)"
      ],
      "metadata": {
        "id": "cnqOCPA_LDhW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "vector_query_engine = RetrieverQueryEngine(\n",
        "    retriever=vector_retriever,\n",
        "    response_synthesizer=response_synthesizer,\n",
        ")"
      ],
      "metadata": {
        "id": "JIU-8KcnLFHZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Getting results"
      ],
      "metadata": {
        "id": "IuIe-bx09JL8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(vector_query_engine.query(\"what is the name of the story?\"))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f_JyqoDVLGvl",
        "outputId": "e4832b83-bd21-43f6-e02a-17b2d0176f8d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " \"I Sell My Dreams\" is the name of the story.\n",
            "\n",
            "page_label: 10\n",
            "file_path: /content/data/ncert short story.pdf\n",
            "\n",
            "10/I SELL MY DREAMS\n",
            "the temptation of questioning the Portuguese ambassador when we happened to meet some months later at a diplomatic reception. The ambassador spoke about her with great enthusiasm and enormous admiration. ‘Y ou cannot imagine how extraor dinary she was,’ he said. ‘Y ou would have been obliged to write a story about her .’ And he went on in the same tone, with surprising details, but without the clue that would have allowed me to come to a final conclusion.\n",
            "‘In concrete terms,’ I asked at last, ‘what did she do?’\n",
            "‘Nothing,’ he said, with a certain disenchantment. ‘She\n",
            "dreamed.’\n",
            "Understanding the T ext\n",
            "1. Did the author believe in the prophetic ability of Frau Frieda?\n",
            "2. Why did he think that Frau Frieda’s dreams were a stratagem\n",
            "for surviving?\n",
            "3. Why does the author compare Neruda to a Renaissance pope\n"
          ]
        }
      ]
    }
  ]
}