Sankshepika_Final_code_ML_Presentation.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/SwethaMagesh/sankshepika-mlpro/blob/main/Sankshepika_Final_code_ML_Presentation.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7T8W5pFXvkpS"
      },
      "outputs": [],
      "source": [
        "!pip install transformers"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4KcQXjE0nKmR"
      },
      "source": [
        "#Download dataset n extract"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_Vch3JmAnBG6",
        "outputId": "fb430429-f1bb-4d3a-9f06-33bfcaa8dc9d"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2023-11-28 10:41:20--  https://zenodo.org/records/7152317/files/dataset.zip?download=1\n",
            "Resolving zenodo.org (zenodo.org)... 188.184.98.238, 188.185.79.172, 188.184.103.159, ...\n",
            "Connecting to zenodo.org (zenodo.org)|188.184.98.238|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 97822590 (93M) [application/octet-stream]\n",
            "Saving to: ‘dataset.zip?download=1.2’\n",
            "\n",
            "dataset.zip?downloa 100%[===================>]  93.29M  1.22MB/s    in 80s     \n",
            "\n",
            "2023-11-28 10:42:41 (1.16 MB/s) - ‘dataset.zip?download=1.2’ saved [97822590/97822590]\n",
            "\n",
            "Archive:  /content/dataset.zip?download=1\n",
            "replace dataset/IN-Abs/test-data/judgement/1181.txt? [y]es, [n]o, [A]ll, [N]one, [r]ename: "
          ]
        }
      ],
      "source": [
        "!wget https://zenodo.org/records/7152317/files/dataset.zip?download=1\n",
        "!unzip '/content/dataset.zip?download=1'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3Sslki7cIwY-"
      },
      "source": [
        "## Preprocess the data into dataframe"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "id": "oLEeemPsIue3",
        "outputId": "d440c9c8-ad9e-4b51-bd8d-ad76193178d4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                               document  \\\n",
              "0     Criminal Appeal No. 208 of 1978. From the Judg...   \n",
              "1     tition (CRL) No. 1171 of 1982. Under article 3...   \n",
              "2     No. 528, of 1959. Appeal from the judgment and...   \n",
              "3     Appeals Nos. 37 & 38 of 1957. Appeals from the...   \n",
              "4     ivil Appeal No. 3927 of 1986. From the Judgmen...   \n",
              "...                                                 ...   \n",
              "6975  vil Appeals Nos. 596 597 (NL) of 1986. From th...   \n",
              "6976  N: Civil Appeal No. 748 of 1975. From the Judg...   \n",
              "6977  Appeal No. 1240 of 1993. From the Judgment and...   \n",
              "6978  Appeal No. 704 of 1975. (Appeal by Special Lea...   \n",
              "6979  tition No. 13636/83 & 11760 66/84. (Under Arti...   \n",
              "\n",
              "                                                summary  \n",
              "0     The appellant, who was a Patwari, had been dem...  \n",
              "1     The petitioner, an advocate of the Supreme Cou...  \n",
              "2     A consent decree was passed against the assess...  \n",
              "3     Section 2(5) of the , properly construed, does...  \n",
              "4     The respondent was a Deputy Registrar of the a...  \n",
              "...                                                 ...  \n",
              "6975  Under the , the State Government referred to t...  \n",
              "6976  The appellant is a commission agent, engaged i...  \n",
              "6977  The respondent who was employed as an Executiv...  \n",
              "6978  The respondent filed a Writ Petition in the Hi...  \n",
              "6979  Staff Artists of All India Radio/Doordarshan, ...  \n",
              "\n",
              "[6980 rows x 2 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-6dbe6340-3f32-46b4-95d9-8482b3ea8cff\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>document</th>\n",
              "      <th>summary</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Criminal Appeal No. 208 of 1978. From the Judg...</td>\n",
              "      <td>The appellant, who was a Patwari, had been dem...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>tition (CRL) No. 1171 of 1982. Under article 3...</td>\n",
              "      <td>The petitioner, an advocate of the Supreme Cou...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>No. 528, of 1959. Appeal from the judgment and...</td>\n",
              "      <td>A consent decree was passed against the assess...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Appeals Nos. 37 &amp; 38 of 1957. Appeals from the...</td>\n",
              "      <td>Section 2(5) of the , properly construed, does...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>ivil Appeal No. 3927 of 1986. From the Judgmen...</td>\n",
              "      <td>The respondent was a Deputy Registrar of the a...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6975</th>\n",
              "      <td>vil Appeals Nos. 596 597 (NL) of 1986. From th...</td>\n",
              "      <td>Under the , the State Government referred to t...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6976</th>\n",
              "      <td>N: Civil Appeal No. 748 of 1975. From the Judg...</td>\n",
              "      <td>The appellant is a commission agent, engaged i...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6977</th>\n",
              "      <td>Appeal No. 1240 of 1993. From the Judgment and...</td>\n",
              "      <td>The respondent who was employed as an Executiv...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6978</th>\n",
              "      <td>Appeal No. 704 of 1975. (Appeal by Special Lea...</td>\n",
              "      <td>The respondent filed a Writ Petition in the Hi...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6979</th>\n",
              "      <td>tition No. 13636/83 &amp; 11760 66/84. (Under Arti...</td>\n",
              "      <td>Staff Artists of All India Radio/Doordarshan, ...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>6980 rows × 2 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6dbe6340-3f32-46b4-95d9-8482b3ea8cff')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-6dbe6340-3f32-46b4-95d9-8482b3ea8cff button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-6dbe6340-3f32-46b4-95d9-8482b3ea8cff');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-86cad339-2d74-4ab5-b7ef-5ff6aba5511d\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-86cad339-2d74-4ab5-b7ef-5ff6aba5511d')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-86cad339-2d74-4ab5-b7ef-5ff6aba5511d button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 116
        }
      ],
      "source": [
        "import os\n",
        "import random\n",
        "import pandas as pd\n",
        "from transformers import AutoTokenizer, AutoModelForSeq2SeqLM\n",
        "def prepare_val_list():\n",
        "    val_files = []  # Add the validation files to be used\n",
        "    for i in range(1, 50):\n",
        "        x = random.randint(1, 7028)\n",
        "        if x not in val_files:\n",
        "            val_files.append(x)\n",
        "\n",
        "    val_files = list(map(lambda x: str(x) + '.txt', val_files))\n",
        "    return val_files\n",
        "\n",
        "def getData(dataPath,  val=0):\n",
        "\n",
        "    documentPath = f'{dataPath}/judgement'\n",
        "    summaryPath = f'{dataPath}/summary'\n",
        "    dataset = {'document': [], 'summary': []}\n",
        "    count = 0\n",
        "    for file in os.listdir(documentPath):\n",
        "        count += 1\n",
        "        if os.stat(f'{documentPath}/{file}').st_size == 0 or os.stat(f'{summaryPath}/{file}').st_size == 0:\n",
        "            continue\n",
        "        doc_in = open(f'{documentPath}/{file}', 'r', encoding='utf8')\n",
        "        doc_lines = [line.strip() for line in doc_in.readlines()]\n",
        "        summ_in = open(f'{summaryPath}/{file}', 'r', encoding='utf8')\n",
        "        summ_lines = [line.strip() for line in summ_in.readlines()]\n",
        "        if len(doc_lines) == 0 or len(summ_lines) == 0:\n",
        "            continue\n",
        "\n",
        "        if val == 0 and file not in val_files:\n",
        "            dataset['document'].append(' '.join(doc_lines))\n",
        "            dataset['summary'].append(' '.join(summ_lines))\n",
        "        if val == 1 and file in val_files:\n",
        "            dataset['document'].append(' '.join(doc_lines))\n",
        "            dataset['summary'].append(' '.join(summ_lines))\n",
        "\n",
        "    df = pd.DataFrame(dataset)\n",
        "    return df\n",
        "\n",
        "\n",
        "dataPath = \"/content/dataset/IN-Abs\"\n",
        "val_files = prepare_val_list()\n",
        "train_df = getData( f'{dataPath}/train-data')\n",
        "val_df = getData(f'{dataPath}/train-data',val=1)\n",
        "\n",
        "train_df\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YhsiH6MYKICG"
      },
      "outputs": [],
      "source": [
        "#  TEST DATA\n",
        "def get_test_data(dataPath):\n",
        "    documentPath = f'{dataPath}/judgement'\n",
        "    summaryPath = f'{dataPath}/summary'\n",
        "    dataset = {'document': [], 'summary': []}\n",
        "    count = 0\n",
        "    for file in os.listdir(documentPath):\n",
        "        count += 1\n",
        "        if os.stat(f'{documentPath}/{file}').st_size == 0 or os.stat(f'{summaryPath}/{file}').st_size == 0:\n",
        "            continue\n",
        "        doc_in = open(f'{documentPath}/{file}', 'r', encoding='utf8')\n",
        "        doc_lines = [line.strip() for line in doc_in.readlines()]\n",
        "        summ_in = open(f'{summaryPath}/{file}', 'r', encoding='utf8')\n",
        "        summ_lines = [line.strip() for line in summ_in.readlines()]\n",
        "        if len(doc_lines) == 0 or len(summ_lines) == 0:\n",
        "            continue\n",
        "\n",
        "        dataset['document'].append(' '.join(doc_lines))\n",
        "        dataset['summary'].append(' '.join(summ_lines))\n",
        "\n",
        "\n",
        "    df = pd.DataFrame(dataset)\n",
        "    return df\n",
        "\n",
        "dataPath = \"/content/dataset/IN-Abs\"\n",
        "test_df = get_test_data( f'{dataPath}/test-data')\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "test_df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "id": "1T6cvqyfPfZo",
        "outputId": "4a21c908-d9d7-486b-a835-204a10dafaf8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                             document  \\\n",
              "0   Civil Appeals Nos. 1264 67/71 etc. From the Ju...   \n",
              "1   Civil Appeal No. 1433 of 1977 Appeal by Specia...   \n",
              "2   vil Appeal No. 6202 (NT) of 1983. From the Jud...   \n",
              "3   ivil Appeals Nos. 259 and 260 of 1958. Appeals...   \n",
              "4   iminal Appeals Nos. 160 to 162 of 1960. Appeal...   \n",
              "..                                                ...   \n",
              "95  Appeal No. 1375 of 1966. Appeal from the judgm...   \n",
              "96  it Petition No. 459 & 460 of 1988. (Under Arti...   \n",
              "97  Appeal No. 68 of 1964. Appeal from the judgmen...   \n",
              "98  ON: Criminal Appeal No. 126 of 1955. Appeal by...   \n",
              "99  etition (Civil) Nos. 12676 77 of 1985 etc. (Un...   \n",
              "\n",
              "                                              summary  \n",
              "0   Section 28 of the Gold Control Act, 1968 bars ...  \n",
              "1   Section 33 of the Punjab Panchayat Samitis and...  \n",
              "2   Each of the appellants/petitioners is a regist...  \n",
              "3   Section 32(1) of the Travancore Income tax Act...  \n",
              "4   The appellant, a postman, and I,is father were...  \n",
              "..                                                ...  \n",
              "95  The appellant Panchayat levied octroi duty on ...  \n",
              "96  The petitioners, carrying,on import export of ...  \n",
              "97  The respondent was the head of a Hindu undivid...  \n",
              "98  Appellant was detained under section 3(1)(a)(i...  \n",
              "99  The Petitioners in these petitions have challe...  \n",
              "\n",
              "[100 rows x 2 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-957c71b7-2593-4994-9d5f-3e8b10df7e7d\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>document</th>\n",
              "      <th>summary</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Civil Appeals Nos. 1264 67/71 etc. From the Ju...</td>\n",
              "      <td>Section 28 of the Gold Control Act, 1968 bars ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Civil Appeal No. 1433 of 1977 Appeal by Specia...</td>\n",
              "      <td>Section 33 of the Punjab Panchayat Samitis and...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>vil Appeal No. 6202 (NT) of 1983. From the Jud...</td>\n",
              "      <td>Each of the appellants/petitioners is a regist...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>ivil Appeals Nos. 259 and 260 of 1958. Appeals...</td>\n",
              "      <td>Section 32(1) of the Travancore Income tax Act...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>iminal Appeals Nos. 160 to 162 of 1960. Appeal...</td>\n",
              "      <td>The appellant, a postman, and I,is father were...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>95</th>\n",
              "      <td>Appeal No. 1375 of 1966. Appeal from the judgm...</td>\n",
              "      <td>The appellant Panchayat levied octroi duty on ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>96</th>\n",
              "      <td>it Petition No. 459 &amp; 460 of 1988. (Under Arti...</td>\n",
              "      <td>The petitioners, carrying,on import export of ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>97</th>\n",
              "      <td>Appeal No. 68 of 1964. Appeal from the judgmen...</td>\n",
              "      <td>The respondent was the head of a Hindu undivid...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>98</th>\n",
              "      <td>ON: Criminal Appeal No. 126 of 1955. Appeal by...</td>\n",
              "      <td>Appellant was detained under section 3(1)(a)(i...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>99</th>\n",
              "      <td>etition (Civil) Nos. 12676 77 of 1985 etc. (Un...</td>\n",
              "      <td>The Petitioners in these petitions have challe...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>100 rows × 2 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-957c71b7-2593-4994-9d5f-3e8b10df7e7d')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-957c71b7-2593-4994-9d5f-3e8b10df7e7d button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-957c71b7-2593-4994-9d5f-3e8b10df7e7d');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-8ebe922b-1691-448c-95b4-e6f5473594b0\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-8ebe922b-1691-448c-95b4-e6f5473594b0')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-8ebe922b-1691-448c-95b4-e6f5473594b0 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 118
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dAeDK2YxmTON"
      },
      "source": [
        "\n",
        "\n",
        "# LEGAL LED + Law data mapping"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "z0VsgtGuklJC"
      },
      "outputs": [],
      "source": [
        "from transformers import AutoTokenizer, AutoModelForSeq2SeqLM\n",
        "\n",
        "def get_LED_PRE(text):\n",
        "\n",
        "  tokenizer = AutoTokenizer.from_pretrained(\"nsi319/legal-led-base-16384\")\n",
        "  model = AutoModelForSeq2SeqLM.from_pretrained(\"nsi319/legal-led-base-16384\")\n",
        "\n",
        "  padding = \"max_length\"\n",
        "\n",
        "\n",
        "  input_tokenized = tokenizer.encode(text, return_tensors='pt',padding=padding,pad_to_max_length=True, max_length=6144,truncation=True)\n",
        "  summary_ids = model.generate(input_tokenized,\n",
        "                                    num_beams=4,\n",
        "                                    no_repeat_ngram_size=3,\n",
        "                                    length_penalty=2,\n",
        "                                    min_length=350,\n",
        "                                    max_length=500)\n",
        "  summary = [tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids][0]\n",
        "  return summary\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "get_LED_PRE(test_df.iloc[23]['document'])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 147
        },
        "id": "gVWZKdzpjuPN",
        "outputId": "3be3b91b-14aa-4813-a73f-c205b75c8467"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"On April 21, 2019, the Honorable B. H. Dhebar of the Bombay High Court entered a final judgment against five bus conductors who were found guilty of violating the anti-fraud provisions of Section 33(1) of the Securities Act of 1933.  The court also issued a temporary restraining order against the bus conductor Louis Pereira, who was also found liable for the same conduct.  According to the court's order, Pereira was sentenced to a fine of $250,000 and ordered to appear before a conciliation officer or a Board or of any proceeding before a Labour Court or Tribunal or National Tribunal in respect of an industrial dispute.  Pereira agreed to settle the case without admitting or denying the allegations of the court.  In a parallel action, the U.S. Attorney's Office for the Southern District of New York today announced criminal charges against Pereira.  As alleged in the SEC's Complaint, filed on March 18, 1952, the Conciliation Officer wrote to the bus driver saying that no useful purpose would be served by holding any further meetings. On May 9, the Union had also indicated to the conciliation Officer that the negotiations had failed.  On June 2, the court granted the Commission's request for an emergency asset freeze and ordered Pereira to be barred from appearing before the Commission as an officer or director of a public company for a period of more than 14 days as from January 17, 1952. The Commission's investigation was conducted by Hardayal Hardy, H. J. Umrigar, and B.H.Dhebar, and supervised by Chedly C. Dumornay. The SEC's litigation will be led by Mr. Hardy and Mr. Dhabar. The Court will determine the amount of disgorgement, prejudgment interest, and civil penalties to be determined by the Commission upon motion of the Commission.  For further information, see Litigation Release No. 46 (Feb. 4, 2019).  The Commission appreciates the assistance of the United Kingdom Secretary of State's Office, the British Columbia Secretary of Commerce, the United States Secret Service, the Internal Revenue Service, and the Financial Industry Regulatory Authority (FINRA) in this matter.\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 161
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q4UKFUWkf_Do"
      },
      "source": [
        "### Few shot dictionary formation for US, UK to Indian Law\n",
        "### Replace the pretrained output with dictionary k,v"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rI98EP4Ctvwh",
        "outputId": "ed81dabf-8f43-4649-d678-450edd9a0bf8"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "U.S. District Court in India is: Indian District Court\n"
          ]
        }
      ],
      "source": [
        "legal_term_mapping = {\n",
        "    # U.S. Legal Terms\n",
        "    \"U.S. District Court\": \"Indian District Court\",\n",
        "    \"U.S. Attorney's Office\": \"Indian Public Prosecutor's Office or Court Prosecutor's Office\",\n",
        "    \"Southern District of New York\": \"Relevant Jurisdiction in India (e.g., Delhi High Court)\",\n",
        "    \"Southern District\": \"Relevant Jurisdiction in India\",\n",
        "    \"SEC\": \"SEBI (Securities and Exchange Board of India)\",\n",
        "    \"Exchange Act\": \"SEBI Act\",\n",
        "    \"Civil penalty\": \"Monetary Penalty\",\n",
        "    \"Judgment\": \"Court Decision\",\n",
        "    \"Asset freeze\": \"Asset Restraint\",\n",
        "    \"Emergency asset freeze\": \"Emergency Asset Restraint\",\n",
        "    \"Enforcement Division\": \"Regulatory Enforcement Division\",\n",
        "    \"Litigation Release\": \"Legal Notice\",\n",
        "    \"SEC investigation\": \"SEBI Inquiry\",\n",
        "    \"Retail Strategy Task Force\": \"Retail Market Surveillance Unit\",\n",
        "    \"Attorney\":\"Prosecutor\",\n",
        "    \"SEC\": \"SEBI\",\n",
        "    \"United States\":\"Indian\",\n",
        "    \"U.S.\":\"India\",\n",
        "    \"New York\":\"Indian city\",\n",
        "\n",
        "    # U.K. Legal Terms\n",
        "    \"U.K. Attorney's Office\": \"British Public Prosecutor's Office or Court Prosecutor's Office\",\n",
        "    \"Home Office\": \"Ministry of Home Affairs (in the context of India)\",\n",
        "    \"Order suspending\": \"Order halting\",\n",
        "    \"Hearing\": \"Court Hearing\",\n",
        "    \"Court approval\": \"Judicial Approval\",\n",
        "    \"Receiver\": \"Court-Appointed Administrator\",\n",
        "\n",
        "    # Generic Legal Terms\n",
        "    \"Final judgment\": \"Conclusive Decision\",\n",
        "    \"Temporary restraining order\": \"Interim Restraining Order\",\n",
        "    \"Parallel action\": \"Simultaneous Legal Action\",\n",
        "    \"Criminal charges\": \"Criminal Allegations\",\n",
        "    \"Complaint\": \"Legal Complaint\",\n",
        "    \"Allegations\": \"Accusations\",\n",
        "    \"Consent\": \"Agreement\",\n",
        "    \"Disgorgement\": \"Restitution\",\n",
        "    \"Prejudgment interest\": \"Anticipatory Interest\",\n",
        "    \"Civil penalties\": \"Monetary Penalties\",\n",
        "    \"Injunction\": \"Restraining Order\",\n",
        "    \"Hearing on the matter\": \"Court Proceedings\",\n",
        "    \"Court's order\": \"Judicial Directive\",\n",
        "    \"Court's judgment\": \"Court's Ruling\",\n",
        "    \"Court's decision\": \"Judicial Decision\",\n",
        "\n",
        "    # Other Country-Specific Terms\n",
        "    \"District Court for the Central District of Kerala\": \"Kerala Central District Court\",\n",
        "    \"U.S. District Court for the District of New Delhi\": \"District Court for New Delhi\",\n",
        "    \"Railway Board\": \"Indian Railway Board\",\n",
        "    \"New York Regional Office\": \"Regional Office (Relevant Region in India)\",\n",
        "    \"Department of Railways\": \"Ministry of Railways (India)\",\n",
        "    \"Office of Personnel and Inspections\": \"Office of Personnel and Inspections (India)\",\n",
        "\n",
        "    # Other References\n",
        "    \"Press Release\": \"Media Statement\",\n",
        "    \"Public Finance Abuse Unit\": \"Public Finance Oversight Unit\",\n",
        "    \"Road Safety Task Force\": \"Road Safety Authority\",\n",
        "    \"Press Release No.\": \"Media Statement No.\",\n",
        "    \"Litigation Release No.\": \"Legal Notice No.\"\n",
        "}\n",
        "\n",
        "# Example usage\n",
        "us_term = \"U.S. District Court\"\n",
        "indian_equivalent = legal_term_mapping.get(us_term, \"No Indian equivalent found\")\n",
        "print(f\"{us_term} in India is: {indian_equivalent}\")\n",
        "\n",
        "def replace_legal_terms(pretrained_output):\n",
        "  c =0\n",
        "  for k, v in legal_term_mapping.items():\n",
        "    if k.lower() in pretrained_output.lower():\n",
        "      # print(k + \" -> \" + v)\n",
        "      c = c+1\n",
        "      pretrained_output = pretrained_output.replace(k,v)\n",
        "  return pretrained_output,c\n",
        "\n",
        "\n",
        "def abstractive1(test_doc, is_indian=True):\n",
        "  summary = get_LED_PRE(test_doc)\n",
        "  if is_indian:\n",
        "    replaced_sum,count = replace_legal_terms(summary)\n",
        "  return replaced_sum,count\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gpuqNyOqw6w-"
      },
      "outputs": [],
      "source": [
        "first,c = abstractive1(test_df.iloc[23]['document'])"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "first"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 276
        },
        "id": "cU0cypXviwot",
        "outputId": "5f008564-b84e-45de-8fa7-435746188773"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"On April 21, 2019, the Honorable B. H. Dhebar of the Bombay High Court entered a final judgment against five bus conductors who were found guilty of violating the anti-fraud provisions of Section 33(1) of the Securities Act of 1933.  The court also issued a temporary restraining order against the bus conductor Louis Pereira, who was also found liable for the same conduct.  According to the court's order, Pereira was sentenced to a fine of $250,000 and ordered to appear before a conciliation officer or a Board or of any proceeding before a Labour Court or Tribunal or National Tribunal in respect of an industrial dispute.  Pereira agreed to settle the case without admitting or denying the allegations of the court.  In a parallel action, the Indian Public Prosecutor's Office or Court Prosecutor's Office for the Relevant Jurisdiction in India (e.g., Delhi High Court) today announced criminal charges against Pereira.  As alleged in the SEBI's Legal Complaint, filed on March 18, 1952, the Conciliation Officer wrote to the bus driver saying that no useful purpose would be served by holding any further meetings. On May 9, the Union had also indicated to the conciliation Officer that the negotiations had failed.  On June 2, the court granted the Commission's request for an emergency asset freeze and ordered Pereira to be barred from appearing before the Commission as an officer or director of a public company for a period of more than 14 days as from January 17, 1952. The Commission's investigation was conducted by Hardayal Hardy, H. J. Umrigar, and B.H.Dhebar, and supervised by Chedly C. Dumornay. The SEBI's litigation will be led by Mr. Hardy and Mr. Dhabar. The Court will determine the amount of disgorgement, prejudgment interest, and civil penalties to be determined by the Commission upon motion of the Commission.  For further information, see Legal Notice No. 46 (Feb. 4, 2019).  The Commission appreciates the assistance of the United Kingdom Secretary of State's Office, the British Columbia Secretary of Commerce, the Indian Secret Service, the Internal Revenue Service, and the Financial Industry Regulatory Authority (FINRA) in this matter.\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 146
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ojm1dCHI6-F7"
      },
      "source": [
        "# Abstractive method 2 (train own model)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "len(test_df.iloc[23]['document'].split())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OAm57cg-MIiZ",
        "outputId": "ccdf6732-676f-479b-d1de-f669bd287534"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1384"
            ]
          },
          "metadata": {},
          "execution_count": 163
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def chunk_document_by_words(text, size=5000):\n",
        "    chunks = []\n",
        "    words = text.split()\n",
        "    current_chunk = []\n",
        "\n",
        "    for word in words:\n",
        "        if len(' '.join(current_chunk + [word])) <= size:\n",
        "            current_chunk.append(word)\n",
        "        else:\n",
        "            chunks.append(' '.join(current_chunk))\n",
        "            current_chunk = [word]\n",
        "\n",
        "    if current_chunk:\n",
        "        chunks.append(' '.join(current_chunk))\n",
        "\n",
        "    return chunks\n",
        "\n",
        "text = test_df.iloc[23]['document']\n",
        "chunks = chunk_document_by_words(text, size=5000)\n",
        "\n",
        "# second = ''\n",
        "# for i, chunk in enumerate(chunks):\n",
        "#     print(f\"Chunk {i + 1}, Word Count: {len(chunk.split())}\")\n",
        "#     second = second + get_output(chunk)\n"
      ],
      "metadata": {
        "id": "UocEIMmiLA3l"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IGfEa34-7CpQ"
      },
      "outputs": [],
      "source": [
        "second = '''The appellant was a member of the state of uttar pradesh. He was a clerk in the state of uttar pradesh. He was prosecuted for a declaration that he was not a foreigner. He was not a foreigner. He was found guilty of the appellant. He was prosecuted for a declaration that he was not a foreigner. He was convicted and sentenced to 7 years. He was also filed a suit for a declaration that he was not a foreigner. The trial court dismissed the suit. The high court dismissed the suit. In appeal to this court held 1. The appellant contended that the appellant was not a foreigner because 1 a foreigner as a foreigner as a foreigner as a foreigner in the definition of article 14 of the constitution. The appellant contended that the appellant wasThe appellant was a member of the state government of jammu and kashmir. He was a member of the state of the state of jammu and kashmir. He was a member of the state of orissa. He was a member of the state of the state of orissa. He was appointed as a member of the state. He was a member of the state. He was appointed as a member of the state. He was prosecuted for a public service. He was convicted for a declaration that he was not a foreigner as a foreigner as a foreigner as a foreigner. He was dismissed by the high court. The appellant contended that the appellant was not a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner in the state of bihar. TheThe appellant was a member of the state of orissa. He was a clerk in the state of uttar pradesh. He was a clerk in the state of uttar pradesh. He was a clerk in the state of punjab. He was a pakistani passport. He was prosecuted for a declaration that he was not a foreigner. He was dismissed by the high court. The appellant contended that the appellant was not a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner. The appellant was therefore void. The appellant contended that the appellant was not a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner in the definition of article 19 1 of the constitution. The appellant contended that the appellantThe appellant was a member of the state of orissa. He was a member of the state of orissa. He was assessed to income tax on the ground that he was not a foreigner. He was a foreigner as a member of the state of orissa. He was assessed to be a citizen of india. He was assessed to income tax on the ground that he was not a foreigner as a foreigner. He was also registered. He was assessed to income tax on the ground that he was not a foreigner as a foreigner as a foreigner as a foreigner. The appellant was declared elected. The appellant was also a foreigner as a foreigner as a result of the provisions of the act was repealed. The appellant was declared elected. The appellant was declared elected. The appellant'''"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "test_df.iloc[23]['document']"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 147
        },
        "id": "v--weuP_Y5q7",
        "outputId": "a1e55124-3b9f-40ea-dd74-a5dfd0586257"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'iminal Appeal No. 46 of 1957. Appeal by special leave from the judgment and order dated the February 4, 1955, of the Bombay High Court in Criminal Appeal No. 1256 of 1954, arising out of the judgment and order dated June 19, 1954, of the Chief Presidency Magistrate, Bombay, in Case No. 176/S of 1953. Hardayal Hardy, for the appellants. H. J. Umrigar and B. H. Dhebar, for the respondent. April 21. The Judgment of the Court was delivered by KAPUR, J. This is an appeal by special leave against the judgment and order of the High Court of Bombay reversing the judgment of the Chief Presidency Magistrate, Bombay, and thus convicting accused Nos. 1 & 5 under section 31(1) read with section 33(1) of the (XIV of 1947) (hereinafter called the Act) and sentencing accused No. 1 to a fine of Rs. 250 and accused No. 5 to a fine of Rs. 50, 736 The appellants are the Andheri Marol Kurla Bus Service who was accused No. 1 (now appellant No. 1) and its Manager H. M. Khan who was accused No. 5 (now appellant No. 2). Some disputes arose between the appellant No. 1 and its workmen. On December 13, 1951, the Conciliation Officer wrote to the appellant No. I and enclosed the demands of the Union which were dated August 9, 1951. On December 31, 1951, the appellant No. I was asked to appear before the Conciliation Officer on January 9, 1952, and after getting one adjournment the appellant No. I appeared before the Conciliation Officer on January 17, 1952, and filed its Written Statement and raised various objections. The next date of hearing was January 31, 1952, and the proceedings went on till June 2, 1952, when the appellant No. 1 wrote to the Conciliation Officer saying that no useful purpose would be served by holding any further meetings. On May 9, 1952, the Union had also indicated to the Conciliation Officer that the negotiations had failed. On March 18, 1952, the appellant dismissed Louis Pereira, a bus conductor and proceedings were taken on a complaint by Assistant Commissioner of Labour under section 33 read with section 31 of the Act against 5 accused persons the two appellants and the partners of appellant No. 1. The Chief Presidency Magistrate acquitted all the accused including the appellants and held that as the conciliation proceedings had continued for a period of more than 14 days as from January 17, 1952, further proceedings for conciliation were illegal and therefore the accused persons could not be convicted under section 31(1)of the Act. The State took an appeal to the High Court and the judgment of acquittal was reversed and of the accused persons the two appellants were convicted and the others were acquitted. The two appellants have appealed by special leave. The question for decision is whether the conciliation proceedings could be said to be pending when Louis Pereira was dismissed. If the answer is in the affirmative then the appellants have been properly convicted and if not the conviction must be set aside. Section 31(1) makes the contravention of the provision 737 of section 33 of the Act an offence punishable with imprisonment for a period which may extend to six months or with fine or with both. Section 33(1) pro K. vides: section 33(1) \" During the pendency of any conciliation proceedings before a conciliation officer or a Board or of any proceeding before a Labour Court or Tribunal or National Tribunal in respect of an industrial dispute, no employer shall: (a) in regard to any matter connected with the dispute, alter to the prejudice of the workmen concerned in such dispute, the conditions of service applicable to them immediately before the commencement of such proceedings; or (b) for any misconduct connected with the dispute, discharge or punish, whether by dismissal or otherwise, any workmen concerned in such dispute, save with the express permission in writing of the authority before which the proceeding is pending \". Therefore the question reduces itself to the meaning of the words \" pendency of any conciliation proceedings before a conciliation officer \". The argument raised on behalf of the appellant is that the object of conciliation is to get a settlement made with expedition and therefore under section 12 the Conciliation Officer was bound to make his report within 14 days of the commencement of the conciliation proceedings or within such shorter period fixed by the appropriate Government. From this it was submitted that as 14 days had expired before March 18, 1952, the dismissal could not be said to be one within the words \" pendency of conciliation proceedings \". The Act provides for commencement and conclusion of conciliation proceedings under section 20 but the first sub section of section 20 deals with what are called utility services and sub section 2 of that section provides as to when the conciliation proceedings conclude. That sub section is as follows: section 20(1). . . . . . . . . (2) A conciliation proceeding shall be deemed to have concluded 93 738 (a) where a settlement is arrived at, when a memorandum of the settlement is signed by the par ties to the dispute; (b) where no settlement is arrived at, when the report of the conciliation officer is received by the appropriate Government or when the report of the Board is published under s, 17, as the case may be; or (c) when a reference is made to a Court, Labour Court, Tribunal or National Tribunal under section 10 during the pendency of conciliation proceedings \". The provisions of sub section 2 apply to all conciliation proceedings whether in regard to utility services or otherwise. All conciliation proceedings under this sub section shall be deemed to have concluded in the case where no settlement is reached, when the report of the Conciliation Officer is received by the appropriate Government. The conciliation proceedings therefore do not end when the report under section 12( 6) is made by the Conciliation Officer but when that report is received by the appropriate Government. It was contended that the conciliation proceedings should be held to terminate when the Conciliation Officer is required under section 12(6) of the Act to submit his report but the provisions of the Act above quoted do not support this contention as the termination of the conciliation proceedings is deemed to take place when the report is received by the appropriate Government. This is how section 20(2)(b) was interpreted in Workers of the Industry Colliery, Dhanbad vs Management of the Industry Colliery (1). It was next contended that on this interpretation the conciliation proceedings could be prolonged much beyond what was contemplated by the Act and the termination would depend upon how soon a report is received by the appropriate Government. It is true that section 12(6) of the Act contemplates the submission of the report by the Conciliation Officer within 14 days but that does not affect the pendency of the conciliation proceedings and if for some reason the Conciliation Officer delays the submission of his report his action (1) ; 739 may be reprehensible but that will not affect the interpretation to be put on section 20(2)(b) of the Act. Section 12 lays down the duties of the Conciliation_ officer. He is required to bring about settlement between the parties and must begin his investigation without delay and if no settlement is arrived at he is to submit his report to the appropriate Government. No doubt section 12 contemplates that the report should be made and the proceedings closed within a fortnight and if proceedings are not closed but are carried on, as they were in the present case, or if the Conciliation Officer does not make his report within 14 days he may be guilty of a breach of duty but in law the proceedings do not automatically come to an end after 14 days but only terminate as provided in section 20(2)(b) of the Act. Colliery Mazdoor Congress, Asansol vs New Beerbhoom Coal Co. Ltd (1). As the conciliation proceedings were pending at the time when Louis Pereira was dismissed the appellants were rightly convicted under section 31(1) read with section 33 of the Act. The appeal is therefore dismissed. Appeal dismissed.'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 124
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IpKqRy0AvoiU"
      },
      "source": [
        "# Extractive Summary - TF IDF"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XNXn28w4r32r",
        "outputId": "53c4635e-5976-4031-d745-9cf90b83add4"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n",
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Package stopwords is already up-to-date!\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 125
        }
      ],
      "source": [
        "\n",
        "import nltk\n",
        "from nltk.tokenize import sent_tokenize, word_tokenize\n",
        "from nltk import PorterStemmer\n",
        "from nltk.corpus import stopwords\n",
        "import math\n",
        "from statistics import mean\n",
        "import argparse\n",
        "nltk.download('punkt')\n",
        "nltk.download('stopwords')\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fI5afVzruGN4"
      },
      "outputs": [],
      "source": [
        "\n",
        "def frquency_matrix(sentences):\n",
        "    matrix = {}\n",
        "    stop_wrd = stopwords.words(\"english\")\n",
        "    stemmer = PorterStemmer()\n",
        "\n",
        "    for sentence in sentences:\n",
        "        sent_freq = {}\n",
        "        words = word_tokenize(sentence)\n",
        "        for word in words:\n",
        "            word = word.lower()\n",
        "            word = stemmer.stem(word)\n",
        "            if word in stop_wrd:\n",
        "                continue\n",
        "            elif word in sent_freq.keys():\n",
        "                sent_freq[word] += 1\n",
        "            else:\n",
        "                sent_freq[word] = 1\n",
        "\n",
        "        matrix[sentence[:10]] = sent_freq\n",
        "    return matrix\n",
        "\n",
        "\n",
        "def term_freq_matrix(matrix_freq):\n",
        "    term_freq = {}\n",
        "    for s, table in matrix_freq.items():\n",
        "        sent_table = {}\n",
        "\n",
        "        word_count_in_sent = len(table)\n",
        "\n",
        "        for word, freq in table.items():\n",
        "            sent_table[word] = freq / word_count_in_sent\n",
        "\n",
        "        term_freq[s] = sent_table\n",
        "\n",
        "    return term_freq\n",
        "\n",
        "def total_word_count(matrix_freq):\n",
        "    total_word_freq = {}\n",
        "    for sent, wtable in matrix_freq.items():\n",
        "        for word, count in wtable.items():\n",
        "            if word in total_word_freq.keys():\n",
        "                total_word_freq[word] += 1\n",
        "            else:\n",
        "                total_word_freq[word] = 1\n",
        "\n",
        "    return total_word_freq\n",
        "\n",
        "\n",
        "def idf_matrix(matrix_freq, word_count, num_sent):\n",
        "    idf = {}\n",
        "\n",
        "    for sentence, freq in matrix_freq.items():\n",
        "        idf_sent = {}\n",
        "\n",
        "        for word in freq.keys():\n",
        "            idf_sent[word] = math.log10(num_sent/ float(word_count[word]))\n",
        "\n",
        "        idf[sentence] = idf_sent\n",
        "\n",
        "    return idf\n",
        "\n",
        "def tf_idf_matrix(term_freq_mat, matrix_idf):\n",
        "    matrix_tf_idf = {}\n",
        "\n",
        "    for (sentence1, tfreq1), (sentence2, tfreq2) in zip(term_freq_mat.items(), matrix_idf.items()):\n",
        "        tf_idf_sent = {}\n",
        "\n",
        "        for word, freq in tfreq1.items():\n",
        "            freq2 = tfreq2[word]\n",
        "            tf_idf_sent[word] = float(freq * freq2)\n",
        "\n",
        "        matrix_tf_idf[sentence1] = tf_idf_sent\n",
        "\n",
        "    return matrix_tf_idf\n",
        "\n",
        "def sentence_scores(matrix_tf_idf):\n",
        "    sent_score = {}\n",
        "\n",
        "    for sent, tf_idf_matrix in matrix_tf_idf.items():\n",
        "        total_score = 0\n",
        "        word_count = len(tf_idf_matrix)\n",
        "\n",
        "        for word, score in tf_idf_matrix.items():\n",
        "            total_score = total_score + score\n",
        "\n",
        "        sent_score[sent] = round(total_score/word_count, 2)\n",
        "\n",
        "    return sent_score\n",
        "\n",
        "def find_average_score(scores):\n",
        "    avg_score = mean(scores[sent] for sent in scores)\n",
        "\n",
        "    return round(avg_score, 2)\n",
        "\n",
        "def generate_summary(sentences, scores, threshold):\n",
        "    count = 0\n",
        "    summary = ''\n",
        "\n",
        "    print(round(threshold, 2))\n",
        "\n",
        "    for sentence in sentences:\n",
        "        if sentence[:10] in scores and scores[sentence[:10]] >= round(threshold, 2):\n",
        "            summary = summary + \" \" + sentence\n",
        "            count += 1\n",
        "\n",
        "    return summary\n",
        "\n",
        "\n",
        "def get_extractive_summary(input1):\n",
        "    sentences = sent_tokenize(input1)\n",
        "    num_sent = len(sentences)\n",
        "\n",
        "    matrix_freq = frquency_matrix(sentences)\n",
        "\n",
        "    term_freq_mat = term_freq_matrix(matrix_freq)\n",
        "\n",
        "    word_count = total_word_count(matrix_freq)\n",
        "\n",
        "    matrix_idf = idf_matrix(matrix_freq, word_count, num_sent)\n",
        "\n",
        "    matrix_tf_idf = tf_idf_matrix(term_freq_mat, matrix_idf)\n",
        "\n",
        "    scores = sentence_scores(matrix_tf_idf)\n",
        "\n",
        "    avg_score = find_average_score(scores)\n",
        "\n",
        "    summary = generate_summary(sentences, scores, 0.8*avg_score)\n",
        "\n",
        "    return summary\n",
        "    print(len(input1))\n",
        "    print(len(summary))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gVBWCdgKuHK_",
        "outputId": "4b02dafa-1530-417e-da5a-b4d18fa4ce84"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.1\n"
          ]
        }
      ],
      "source": [
        "third = get_extractive_summary(test_df.iloc[23]['document'])"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "third"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 202
        },
        "id": "eBFXXxk5pF7g",
        "outputId": "2426afb0-238e-4457-febd-dcfd9ceb2edd"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "' iminal Appeal No. 46 of 1957. 1256 of 1954, arising out of the judgment and order dated June 19, 1954, of the Chief Presidency Magistrate, Bombay, in Case No. 176/S of 1953. Hardayal Hardy, for the appellants. H. J. Umrigar and B. H. Dhebar, for the respondent. April 21. The Judgment of the Court was delivered by KAPUR, J. 1 to a fine of Rs. 250 and accused No. 5 to a fine of Rs. 50, 736 The appellants are the Andheri Marol Kurla Bus Service who was accused No. 1 (now appellant No. 1) and its Manager H. M. Khan who was accused No. 5 (now appellant No. 2). Some disputes arose between the appellant No. 1 and its workmen. On December 13, 1951, the Conciliation Officer wrote to the appellant No. I and enclosed the demands of the Union which were dated August 9, 1951. On December 31, 1951, the appellant No. 1 wrote to the Conciliation Officer saying that no useful purpose would be served by holding any further meetings. On May 9, 1952, the Union had also indicated to the Conciliation Officer that the negotiations had failed. 1. The two appellants have appealed by special leave. If the answer is in the affirmative then the appellants have been properly convicted and if not the conviction must be set aside. Therefore the question reduces itself to the meaning of the words \" pendency of any conciliation proceedings before a conciliation officer \". That sub section is as follows: section 20(1). Section 12 lays down the duties of the Conciliation_ officer. The appeal is therefore dismissed. Appeal dismissed.'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 165
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ybT3RGn3aRki"
      },
      "outputs": [],
      "source": [
        "!pip install openai==0.28.1"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Wq75DzUgaoIO"
      },
      "outputs": [],
      "source": [
        "\n",
        "ENDPOINT='https://open-ai-pro-iitb.openai.azure.com/'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gB_6UB7BczU2"
      },
      "source": [
        "# SYSTEM PROMPT"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2fiRDj0rc0v_"
      },
      "outputs": [],
      "source": [
        "system_prompt = '''\n",
        "YOU ARE A SUMMARIZATION ASSISTANT. FOR VERY LONG LEGAL DOCUMENTS, WHICH ARE NOT UNDERSTANDABLE BY HUMANS, WE GENERATE 3 SUMMARIES.\n",
        "YOU HAVE TO COMBINE THE THREE OF THEM INTO A SIMPLE SUMMARY.\n",
        "FIRST SUMMARY IS GENERATED BY A MODEL TRAINED FOR US DATA. IT HALLUCINATES AND IT IS HIGHLY NOT A CREDIBLE SOURCE. THE TARGET DOCUMENT IS FROM INDIAN LAW, SO THERE MIGHT BE SOME US LAW TERMS WHICH SHOULD BE REPLACED WITH THE CORRESPONDING INDIAN WORDS.\n",
        "SECOND SUMMARY IS GENERATED BY A TRANSFORMER MODEL TRAINED FROM A CUSTOM DATASET. IT SUFFERS FROM THE FLAW THAT SENTENCES ARE NOT MEANINGFUL ALWAYS AND SUFFER FROM GRAMMATICAL MISTAKES.\n",
        "THIRD SUMMARY IS GENERATED BY EXTRACTING THE IMPORTANT SENTENCES FROM THE DOCUMENT. MOSTLY BASE ON THIS ONE.\n",
        "TAKE ALL FACTS AND DATA TO BE USED FROM SUMMARY 3 MAJORLY AND FROM 2 AND MERGE IT WITH SUMMARY 1.\n",
        "'''"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W2LNLf4Uc0O4"
      },
      "source": [
        "# USER PROMPT"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "j7KvfLEycy5g"
      },
      "outputs": [],
      "source": [
        "user_prompt = '''\n",
        "COMBINE THE BELOW SUMMARIES INTO A MEANINGFUL, SHORT, LAYMAN UNDERSTANDABLE TEXT. DO NOT REPEAT POINTS.\n",
        "IF THE TIMELINE OR NAMES OF PEOPLE INVOLVED ARE AVAILABLE, THEN HIGHLIGHT THEM ABOVE THE COMBINED SUMMARY.\n",
        "STICK TO THE FACTS STRICTLY IN THE SUMMARIES, DO NOT ADD ANYTHING NEW.\n",
        "TAKE ALL FACTS, DATA, NUMBERS AND ALL DETAILS TO BE USED FROM SUMMARY 3 MAJORLY AND FROM 2 AND MERGE IT LIKE SUMMARY 1.\n",
        "USE ALL INFORMATION FROM SUMMARY 3.\n",
        "DO NOT USE DOLLAR AND OTHER NON-INDIAN WORDS FROM SUMMARY 1. THE LENGTH OF SUMMARY SHOULD BE AT LEAST AS BIG AS THIRD SUMMARY\n",
        "\n",
        "\n",
        "----\n",
        "FIRST SUMMARY\n",
        "{}\n",
        "-----\n",
        "SECOND SUMMARY\n",
        "{}\n",
        "-----\n",
        "THIRD SUMMARY\n",
        "{}\n",
        "-----\n",
        "\n",
        "SUMMARIZE THE LEGAL DOCUMENT USING THE SUMMARIES GENERATED BY DIFFERENT METHODS.GIVE ONLY SUMMARY. THE LENGTH OF SUMMARY SHOULD BE AT LEAST AS BIG AS THIRD SUMMARY\n",
        "\n",
        "OUTPUT\n",
        "------\n",
        "-------\n",
        "'''"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Yt333QHbp1Bq",
        "outputId": "0648f0bb-d59d-432c-be8e-e270cc6342e7"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "On April 21, 2019, the Honorable B. H. Dhebar of the Bombay High Court entered a final judgment against five bus conductors who were found guilty of violating the anti-fraud provisions of Section 33(1) of the Securities Act of 1933.  The court also issued a temporary restraining order against the bus conductor Louis Pereira, who was also found liable for the same conduct.  According to the court's order, Pereira was sentenced to a fine of $250,000 and ordered to appear before a conciliation officer or a Board or of any proceeding before a Labour Court or Tribunal or National Tribunal in respect of an industrial dispute.  Pereira agreed to settle the case without admitting or denying the allegations of the court.  In a parallel action, the Indian Public Prosecutor's Office or Court Prosecutor's Office for the Relevant Jurisdiction in India (e.g., Delhi High Court) today announced criminal charges against Pereira.  As alleged in the SEBI's Legal Complaint, filed on March 18, 1952, the Conciliation Officer wrote to the bus driver saying that no useful purpose would be served by holding any further meetings. On May 9, the Union had also indicated to the conciliation Officer that the negotiations had failed.  On June 2, the court granted the Commission's request for an emergency asset freeze and ordered Pereira to be barred from appearing before the Commission as an officer or director of a public company for a period of more than 14 days as from January 17, 1952. The Commission's investigation was conducted by Hardayal Hardy, H. J. Umrigar, and B.H.Dhebar, and supervised by Chedly C. Dumornay. The SEBI's litigation will be led by Mr. Hardy and Mr. Dhabar. The Court will determine the amount of disgorgement, prejudgment interest, and civil penalties to be determined by the Commission upon motion of the Commission.  For further information, see Legal Notice No. 46 (Feb. 4, 2019).  The Commission appreciates the assistance of the United Kingdom Secretary of State's Office, the British Columbia Secretary of Commerce, the Indian Secret Service, the Internal Revenue Service, and the Financial Industry Regulatory Authority (FINRA) in this matter.\n",
            "The appellant was a member of the state of uttar pradesh. He was a clerk in the state of uttar pradesh. He was prosecuted for a declaration that he was not a foreigner. He was not a foreigner. He was found guilty of the appellant. He was prosecuted for a declaration that he was not a foreigner. He was convicted and sentenced to 7 years. He was also filed a suit for a declaration that he was not a foreigner. The trial court dismissed the suit. The high court dismissed the suit. In appeal to this court held 1. The appellant contended that the appellant was not a foreigner because 1 a foreigner as a foreigner as a foreigner as a foreigner in the definition of article 14 of the constitution. The appellant contended that the appellant wasThe appellant was a member of the state government of jammu and kashmir. He was a member of the state of the state of jammu and kashmir. He was a member of the state of orissa. He was a member of the state of the state of orissa. He was appointed as a member of the state. He was a member of the state. He was appointed as a member of the state. He was prosecuted for a public service. He was convicted for a declaration that he was not a foreigner as a foreigner as a foreigner as a foreigner. He was dismissed by the high court. The appellant contended that the appellant was not a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner in the state of bihar. TheThe appellant was a member of the state of orissa. He was a clerk in the state of uttar pradesh. He was a clerk in the state of uttar pradesh. He was a clerk in the state of punjab. He was a pakistani passport. He was prosecuted for a declaration that he was not a foreigner. He was dismissed by the high court. The appellant contended that the appellant was not a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner. The appellant was therefore void. The appellant contended that the appellant was not a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner in the definition of article 19 1 of the constitution. The appellant contended that the appellantThe appellant was a member of the state of orissa. He was a member of the state of orissa. He was assessed to income tax on the ground that he was not a foreigner. He was a foreigner as a member of the state of orissa. He was assessed to be a citizen of india. He was assessed to income tax on the ground that he was not a foreigner as a foreigner. He was also registered. He was assessed to income tax on the ground that he was not a foreigner as a foreigner as a foreigner as a foreigner. The appellant was declared elected. The appellant was also a foreigner as a foreigner as a result of the provisions of the act was repealed. The appellant was declared elected. The appellant was declared elected. The appellant\n",
            " iminal Appeal No. 46 of 1957. 1256 of 1954, arising out of the judgment and order dated June 19, 1954, of the Chief Presidency Magistrate, Bombay, in Case No. 176/S of 1953. Hardayal Hardy, for the appellants. H. J. Umrigar and B. H. Dhebar, for the respondent. April 21. The Judgment of the Court was delivered by KAPUR, J. 1 to a fine of Rs. 250 and accused No. 5 to a fine of Rs. 50, 736 The appellants are the Andheri Marol Kurla Bus Service who was accused No. 1 (now appellant No. 1) and its Manager H. M. Khan who was accused No. 5 (now appellant No. 2). Some disputes arose between the appellant No. 1 and its workmen. On December 13, 1951, the Conciliation Officer wrote to the appellant No. I and enclosed the demands of the Union which were dated August 9, 1951. On December 31, 1951, the appellant No. 1 wrote to the Conciliation Officer saying that no useful purpose would be served by holding any further meetings. On May 9, 1952, the Union had also indicated to the Conciliation Officer that the negotiations had failed. 1. The two appellants have appealed by special leave. If the answer is in the affirmative then the appellants have been properly convicted and if not the conviction must be set aside. Therefore the question reduces itself to the meaning of the words \" pendency of any conciliation proceedings before a conciliation officer \". That sub section is as follows: section 20(1). Section 12 lays down the duties of the Conciliation_ officer. The appeal is therefore dismissed. Appeal dismissed.\n"
          ]
        }
      ],
      "source": [
        "print(first)\n",
        "print(second)\n",
        "print(third)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WJeRuCcvhr0k",
        "outputId": "cdf2cd96-dc49-40ae-b027-ebfd9e3b72a3"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "COMBINE THE BELOW SUMMARIES INTO A MEANINGFUL, SHORT, LAYMAN UNDERSTANDABLE TEXT. DO NOT REPEAT POINTS.\n",
            "IF THE TIMELINE OR NAMES OF PEOPLE INVOLVED ARE AVAILABLE, THEN HIGHLIGHT THEM ABOVE THE COMBINED SUMMARY.\n",
            "STICK TO THE FACTS STRICTLY IN THE SUMMARIES, DO NOT ADD ANYTHING NEW.\n",
            "TAKE ALL FACTS, DATA, NUMBERS AND ALL DETAILS TO BE USED FROM SUMMARY 3 MAJORLY AND FROM 2 AND MERGE IT LIKE SUMMARY 1.\n",
            "USE ALL INFORMATION FROM SUMMARY 3. \n",
            "DO NOT USE DOLLAR AND OTHER NON-INDIAN WORDS FROM SUMMARY 1. THE LENGTH OF SUMMARY SHOULD BE AT LEAST AS BIG AS THIRD SUMMARY\n",
            "\n",
            "\n",
            "----\n",
            "FIRST SUMMARY\n",
            "On April 21, 2019, the Honorable B. H. Dhebar of the Bombay High Court entered a final judgment against five bus conductors who were found guilty of violating the anti-fraud provisions of Section 33(1) of the Securities Act of 1933.  The court also issued a temporary restraining order against the bus conductor Louis Pereira, who was also found liable for the same conduct.  According to the court's order, Pereira was sentenced to a fine of $250,000 and ordered to appear before a conciliation officer or a Board or of any proceeding before a Labour Court or Tribunal or National Tribunal in respect of an industrial dispute.  Pereira agreed to settle the case without admitting or denying the allegations of the court.  In a parallel action, the Indian Public Prosecutor's Office or Court Prosecutor's Office for the Relevant Jurisdiction in India (e.g., Delhi High Court) today announced criminal charges against Pereira.  As alleged in the SEBI's Legal Complaint, filed on March 18, 1952, the Conciliation Officer wrote to the bus driver saying that no useful purpose would be served by holding any further meetings. On May 9, the Union had also indicated to the conciliation Officer that the negotiations had failed.  On June 2, the court granted the Commission's request for an emergency asset freeze and ordered Pereira to be barred from appearing before the Commission as an officer or director of a public company for a period of more than 14 days as from January 17, 1952. The Commission's investigation was conducted by Hardayal Hardy, H. J. Umrigar, and B.H.Dhebar, and supervised by Chedly C. Dumornay. The SEBI's litigation will be led by Mr. Hardy and Mr. Dhabar. The Court will determine the amount of disgorgement, prejudgment interest, and civil penalties to be determined by the Commission upon motion of the Commission.  For further information, see Legal Notice No. 46 (Feb. 4, 2019).  The Commission appreciates the assistance of the United Kingdom Secretary of State's Office, the British Columbia Secretary of Commerce, the Indian Secret Service, the Internal Revenue Service, and the Financial Industry Regulatory Authority (FINRA) in this matter.\n",
            "-----\n",
            "SECOND SUMMARY\n",
            "The appellant was a member of the state of uttar pradesh. He was a clerk in the state of uttar pradesh. He was prosecuted for a declaration that he was not a foreigner. He was not a foreigner. He was found guilty of the appellant. He was prosecuted for a declaration that he was not a foreigner. He was convicted and sentenced to 7 years. He was also filed a suit for a declaration that he was not a foreigner. The trial court dismissed the suit. The high court dismissed the suit. In appeal to this court held 1. The appellant contended that the appellant was not a foreigner because 1 a foreigner as a foreigner as a foreigner as a foreigner in the definition of article 14 of the constitution. The appellant contended that the appellant wasThe appellant was a member of the state government of jammu and kashmir. He was a member of the state of the state of jammu and kashmir. He was a member of the state of orissa. He was a member of the state of the state of orissa. He was appointed as a member of the state. He was a member of the state. He was appointed as a member of the state. He was prosecuted for a public service. He was convicted for a declaration that he was not a foreigner as a foreigner as a foreigner as a foreigner. He was dismissed by the high court. The appellant contended that the appellant was not a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner in the state of bihar. TheThe appellant was a member of the state of orissa. He was a clerk in the state of uttar pradesh. He was a clerk in the state of uttar pradesh. He was a clerk in the state of punjab. He was a pakistani passport. He was prosecuted for a declaration that he was not a foreigner. He was dismissed by the high court. The appellant contended that the appellant was not a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner. The appellant was therefore void. The appellant contended that the appellant was not a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner in the definition of article 19 1 of the constitution. The appellant contended that the appellantThe appellant was a member of the state of orissa. He was a member of the state of orissa. He was assessed to income tax on the ground that he was not a foreigner. He was a foreigner as a member of the state of orissa. He was assessed to be a citizen of india. He was assessed to income tax on the ground that he was not a foreigner as a foreigner. He was also registered. He was assessed to income tax on the ground that he was not a foreigner as a foreigner as a foreigner as a foreigner. The appellant was declared elected. The appellant was also a foreigner as a foreigner as a result of the provisions of the act was repealed. The appellant was declared elected. The appellant was declared elected. The appellant\n",
            "-----\n",
            "THIRD SUMMARY\n",
            " iminal Appeal No. 46 of 1957. 1256 of 1954, arising out of the judgment and order dated June 19, 1954, of the Chief Presidency Magistrate, Bombay, in Case No. 176/S of 1953. Hardayal Hardy, for the appellants. H. J. Umrigar and B. H. Dhebar, for the respondent. April 21. The Judgment of the Court was delivered by KAPUR, J. 1 to a fine of Rs. 250 and accused No. 5 to a fine of Rs. 50, 736 The appellants are the Andheri Marol Kurla Bus Service who was accused No. 1 (now appellant No. 1) and its Manager H. M. Khan who was accused No. 5 (now appellant No. 2). Some disputes arose between the appellant No. 1 and its workmen. On December 13, 1951, the Conciliation Officer wrote to the appellant No. I and enclosed the demands of the Union which were dated August 9, 1951. On December 31, 1951, the appellant No. 1 wrote to the Conciliation Officer saying that no useful purpose would be served by holding any further meetings. On May 9, 1952, the Union had also indicated to the Conciliation Officer that the negotiations had failed. 1. The two appellants have appealed by special leave. If the answer is in the affirmative then the appellants have been properly convicted and if not the conviction must be set aside. Therefore the question reduces itself to the meaning of the words \" pendency of any conciliation proceedings before a conciliation officer \". That sub section is as follows: section 20(1). Section 12 lays down the duties of the Conciliation_ officer. The appeal is therefore dismissed. Appeal dismissed.\n",
            "-----\n",
            "\n",
            "SUMMARIZE THE LEGAL DOCUMENT USING THE SUMMARIES GENERATED BY DIFFERENT METHODS.GIVE ONLY SUMMARY. THE LENGTH OF SUMMARY SHOULD BE AT LEAST AS BIG AS THIRD SUMMARY\n",
            "\n",
            "OUTPUT\n",
            "------\n",
            "-------\n",
            "\n"
          ]
        }
      ],
      "source": [
        "\n",
        "\n",
        "# generate messages\n",
        "sys_msg = {\n",
        "    \"role\":\"system\",\n",
        "    \"content\": system_prompt,\n",
        "}\n",
        "user_prompt = user_prompt.format(first, second, third)\n",
        "user_msg = {\n",
        "    \"role\":\"user\",\n",
        "    \"content\":user_prompt,\n",
        "}\n",
        "print(user_prompt)\n",
        "messages = [sys_msg, user_msg]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "A1urUmSzbUmU"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import openai\n",
        "openai.api_type = \"azure\"\n",
        "openai.api_base = ENDPOINT\n",
        "openai.api_key = API_KEY\n",
        "openai.api_version = \"2023-05-15\"\n",
        "\n",
        "response = openai.ChatCompletion.create(\n",
        "    engine=\"test\", # engine = \"deployment_name\".\n",
        "    messages=messages\n",
        ")\n",
        "\n",
        "final = (response['choices'][0]['message']['content'])\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "final\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 91
        },
        "id": "C_k_IEgNUKTD",
        "outputId": "6fa8da95-bd7f-414c-b1d5-db8145da99b8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"The Bombay High Court issued a final judgement against bus conductors who violated the anti-fraud provisions of the Securities Act of 1933. The Court sentenced the conductor Louis Pereira to a fine of Rs. 250,000 and ordered him to appear before a conciliation officer or a Board or of any proceeding before a Labour Court or Tribunal or National Tribunal in respect of an industrial dispute. The SEBI filed a legal complaint against Pereira and the Indian Public Prosecutor's office announced criminal charges against him. The court also issued a temporary restraining order against Pereira.\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 200
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "second"
      ],
      "metadata": {
        "id": "eb1d-IN6qMyM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "UFZUCKe1qMYJ"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7RmYCEHlSq73"
      },
      "source": [
        "# ROUGE SCORE"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TyZlzN8fTqUT"
      },
      "outputs": [],
      "source": [
        "!pip install py-rouge\n",
        "!pip install rouge_score"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1ZjQjXN0Sqc2"
      },
      "outputs": [],
      "source": [
        "from rouge_score import rouge_scorer\n",
        "\n",
        "def rouge_score(references, predictions):\n",
        "  print(references)\n",
        "  print(predictions)\n",
        "\n",
        "  # Initialize the Rouge Scorer\n",
        "  scorer = rouge_scorer.RougeScorer(['rouge1', 'rouge2', 'rougeL'], use_stemmer=True)\n",
        "\n",
        "  # Calculate ROUGE scores\n",
        "  scores = scorer.score(predictions, references)\n",
        "\n",
        "  # Print ROUGE scores\n",
        "  for metric, score in scores.items():\n",
        "      print(f\"{metric}: {score.fmeasure:.4f}\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yGs9oHgLn0t4"
      },
      "source": [
        "Legal LED + mappings"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "m2HBCfLHZkbh",
        "outputId": "3d7fe5ba-689e-4288-f000-3d8776c3789b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "On April 21, 2019, the Honorable B. H. Dhebar of the Bombay High Court entered a final judgment against five bus conductors who were found guilty of violating the anti-fraud provisions of Section 33(1) of the Securities Act of 1933.  The court also issued a temporary restraining order against the bus conductor Louis Pereira, who was also found liable for the same conduct.  According to the court's order, Pereira was sentenced to a fine of $250,000 and ordered to appear before a conciliation officer or a Board or of any proceeding before a Labour Court or Tribunal or National Tribunal in respect of an industrial dispute.  Pereira agreed to settle the case without admitting or denying the allegations of the court.  In a parallel action, the Indian Public Prosecutor's Office or Court Prosecutor's Office for the Relevant Jurisdiction in India (e.g., Delhi High Court) today announced criminal charges against Pereira.  As alleged in the SEBI's Legal Complaint, filed on March 18, 1952, the Conciliation Officer wrote to the bus driver saying that no useful purpose would be served by holding any further meetings. On May 9, the Union had also indicated to the conciliation Officer that the negotiations had failed.  On June 2, the court granted the Commission's request for an emergency asset freeze and ordered Pereira to be barred from appearing before the Commission as an officer or director of a public company for a period of more than 14 days as from January 17, 1952. The Commission's investigation was conducted by Hardayal Hardy, H. J. Umrigar, and B.H.Dhebar, and supervised by Chedly C. Dumornay. The SEBI's litigation will be led by Mr. Hardy and Mr. Dhabar. The Court will determine the amount of disgorgement, prejudgment interest, and civil penalties to be determined by the Commission upon motion of the Commission.  For further information, see Legal Notice No. 46 (Feb. 4, 2019).  The Commission appreciates the assistance of the United Kingdom Secretary of State's Office, the British Columbia Secretary of Commerce, the Indian Secret Service, the Internal Revenue Service, and the Financial Industry Regulatory Authority (FINRA) in this matter.\n",
            "Conciliation proceedings were started in January. 1952 with respect to some disputes between appellant 1 and its workmen. On May 9, 1952, the Union and on June 2, 1952, the appellant 1 indicated to the Conciliation Officer that the negotiations had failed. In the meantime on March ~I8, 1952, the appellant 1 dismissed (1) I.L.R. 1947 All. 735 one of its workmen. The two appellants and three others were prosecuted under section 31 Of the , for a breach of section 33 for dismissing a workman during the pendency of the conciliation proceedings. The appellants contended that since section 12(6) required the report of the conciliation proceedings to be submitted within 14 days of the commencement thereof, the proceedings had terminated on the expiry of the 14 days and the dismissal was, therefore, not during the pendency of the conciliation proceedings. Held that, in cases where no settlement was arrived at the conciliation proceedings terminated when the report of the Conciliation Officer was received by the appropriate Government and not on the expiry of 14 days from the commencement of the proceedings. The commencement and termination of conciliation proceedings were determined by section 20 and not by section 12(6). The dismissal of the workman was during the pendency of the conciliation proceedings and the appellants were guilty under section 31(1) of the Act. Workers of the Industry Colliery, Dhanbad vs Management Of the Industry Colliery, ; ; Colliery Mazdoor Congress, Asansol vs New Beerbhoom Coal Co. Ltd., , applied.\n",
            "rouge1: 0.3811\n",
            "rouge2: 0.1111\n",
            "rougeL: 0.1889\n"
          ]
        }
      ],
      "source": [
        "dataset_out = test_df.iloc[23]['summary']\n",
        "rouge_score(first, dataset_out)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z1yb6IPAnxFt"
      },
      "source": [
        "Transformer one"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g20FelqfnuXL",
        "outputId": "b7dc96fd-b327-4849-b10c-695268909308"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The appellant was a member of the state of uttar pradesh. He was a clerk in the state of uttar pradesh. He was prosecuted for a declaration that he was not a foreigner. He was not a foreigner. He was found guilty of the appellant. He was prosecuted for a declaration that he was not a foreigner. He was convicted and sentenced to 7 years. He was also filed a suit for a declaration that he was not a foreigner. The trial court dismissed the suit. The high court dismissed the suit. In appeal to this court held 1. The appellant contended that the appellant was not a foreigner because 1 a foreigner as a foreigner as a foreigner as a foreigner in the definition of article 14 of the constitution. The appellant contended that the appellant wasThe appellant was a member of the state government of jammu and kashmir. He was a member of the state of the state of jammu and kashmir. He was a member of the state of orissa. He was a member of the state of the state of orissa. He was appointed as a member of the state. He was a member of the state. He was appointed as a member of the state. He was prosecuted for a public service. He was convicted for a declaration that he was not a foreigner as a foreigner as a foreigner as a foreigner. He was dismissed by the high court. The appellant contended that the appellant was not a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner in the state of bihar. TheThe appellant was a member of the state of orissa. He was a clerk in the state of uttar pradesh. He was a clerk in the state of uttar pradesh. He was a clerk in the state of punjab. He was a pakistani passport. He was prosecuted for a declaration that he was not a foreigner. He was dismissed by the high court. The appellant contended that the appellant was not a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner. The appellant was therefore void. The appellant contended that the appellant was not a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner as a foreigner in the definition of article 19 1 of the constitution. The appellant contended that the appellantThe appellant was a member of the state of orissa. He was a member of the state of orissa. He was assessed to income tax on the ground that he was not a foreigner. He was a foreigner as a member of the state of orissa. He was assessed to be a citizen of india. He was assessed to income tax on the ground that he was not a foreigner as a foreigner. He was also registered. He was assessed to income tax on the ground that he was not a foreigner as a foreigner as a foreigner as a foreigner. The appellant was declared elected. The appellant was also a foreigner as a foreigner as a result of the provisions of the act was repealed. The appellant was declared elected. The appellant was declared elected. The appellant\n",
            "Conciliation proceedings were started in January. 1952 with respect to some disputes between appellant 1 and its workmen. On May 9, 1952, the Union and on June 2, 1952, the appellant 1 indicated to the Conciliation Officer that the negotiations had failed. In the meantime on March ~I8, 1952, the appellant 1 dismissed (1) I.L.R. 1947 All. 735 one of its workmen. The two appellants and three others were prosecuted under section 31 Of the , for a breach of section 33 for dismissing a workman during the pendency of the conciliation proceedings. The appellants contended that since section 12(6) required the report of the conciliation proceedings to be submitted within 14 days of the commencement thereof, the proceedings had terminated on the expiry of the 14 days and the dismissal was, therefore, not during the pendency of the conciliation proceedings. Held that, in cases where no settlement was arrived at the conciliation proceedings terminated when the report of the Conciliation Officer was received by the appropriate Government and not on the expiry of 14 days from the commencement of the proceedings. The commencement and termination of conciliation proceedings were determined by section 20 and not by section 12(6). The dismissal of the workman was during the pendency of the conciliation proceedings and the appellants were guilty under section 31(1) of the Act. Workers of the Industry Colliery, Dhanbad vs Management Of the Industry Colliery, ; ; Colliery Mazdoor Congress, Asansol vs New Beerbhoom Coal Co. Ltd., , applied.\n",
            "rouge1: 0.2569\n",
            "rouge2: 0.0725\n",
            "rougeL: 0.1721\n"
          ]
        }
      ],
      "source": [
        "rouge_score(second, dataset_out)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UgRaVdFcnyh4"
      },
      "source": [
        "Extractive"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TsTL6YFhnuzi",
        "outputId": "c55bce74-2126-4d1f-a8f7-8935d9888e5d"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " iminal Appeal No. 46 of 1957. 1256 of 1954, arising out of the judgment and order dated June 19, 1954, of the Chief Presidency Magistrate, Bombay, in Case No. 176/S of 1953. Hardayal Hardy, for the appellants. H. J. Umrigar and B. H. Dhebar, for the respondent. April 21. The Judgment of the Court was delivered by KAPUR, J. 1 to a fine of Rs. 250 and accused No. 5 to a fine of Rs. 50, 736 The appellants are the Andheri Marol Kurla Bus Service who was accused No. 1 (now appellant No. 1) and its Manager H. M. Khan who was accused No. 5 (now appellant No. 2). Some disputes arose between the appellant No. 1 and its workmen. On December 13, 1951, the Conciliation Officer wrote to the appellant No. I and enclosed the demands of the Union which were dated August 9, 1951. On December 31, 1951, the appellant No. 1 wrote to the Conciliation Officer saying that no useful purpose would be served by holding any further meetings. On May 9, 1952, the Union had also indicated to the Conciliation Officer that the negotiations had failed. 1. The two appellants have appealed by special leave. If the answer is in the affirmative then the appellants have been properly convicted and if not the conviction must be set aside. Therefore the question reduces itself to the meaning of the words \" pendency of any conciliation proceedings before a conciliation officer \". That sub section is as follows: section 20(1). Section 12 lays down the duties of the Conciliation_ officer. The appeal is therefore dismissed. Appeal dismissed.\n",
            "Conciliation proceedings were started in January. 1952 with respect to some disputes between appellant 1 and its workmen. On May 9, 1952, the Union and on June 2, 1952, the appellant 1 indicated to the Conciliation Officer that the negotiations had failed. In the meantime on March ~I8, 1952, the appellant 1 dismissed (1) I.L.R. 1947 All. 735 one of its workmen. The two appellants and three others were prosecuted under section 31 Of the , for a breach of section 33 for dismissing a workman during the pendency of the conciliation proceedings. The appellants contended that since section 12(6) required the report of the conciliation proceedings to be submitted within 14 days of the commencement thereof, the proceedings had terminated on the expiry of the 14 days and the dismissal was, therefore, not during the pendency of the conciliation proceedings. Held that, in cases where no settlement was arrived at the conciliation proceedings terminated when the report of the Conciliation Officer was received by the appropriate Government and not on the expiry of 14 days from the commencement of the proceedings. The commencement and termination of conciliation proceedings were determined by section 20 and not by section 12(6). The dismissal of the workman was during the pendency of the conciliation proceedings and the appellants were guilty under section 31(1) of the Act. Workers of the Industry Colliery, Dhanbad vs Management Of the Industry Colliery, ; ; Colliery Mazdoor Congress, Asansol vs New Beerbhoom Coal Co. Ltd., , applied.\n",
            "rouge1: 0.4769\n",
            "rouge2: 0.1583\n",
            "rougeL: 0.2154\n"
          ]
        }
      ],
      "source": [
        "rouge_score(third, dataset_out)\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "rouge_score(final, dataset_out)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "--U_MyJLTwv7",
        "outputId": "95338cd8-ceeb-4532-d0d1-ae5024b42d98"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The Bombay High Court issued a final judgement against bus conductors who violated the anti-fraud provisions of the Securities Act of 1933. The Court sentenced the conductor Louis Pereira to a fine of Rs. 250,000 and ordered him to appear before a conciliation officer or a Board or of any proceeding before a Labour Court or Tribunal or National Tribunal in respect of an industrial dispute. The SEBI filed a legal complaint against Pereira and the Indian Public Prosecutor's office announced criminal charges against him. The court also issued a temporary restraining order against Pereira.\n",
            "Conciliation proceedings were started in January. 1952 with respect to some disputes between appellant 1 and its workmen. On May 9, 1952, the Union and on June 2, 1952, the appellant 1 indicated to the Conciliation Officer that the negotiations had failed. In the meantime on March ~I8, 1952, the appellant 1 dismissed (1) I.L.R. 1947 All. 735 one of its workmen. The two appellants and three others were prosecuted under section 31 Of the , for a breach of section 33 for dismissing a workman during the pendency of the conciliation proceedings. The appellants contended that since section 12(6) required the report of the conciliation proceedings to be submitted within 14 days of the commencement thereof, the proceedings had terminated on the expiry of the 14 days and the dismissal was, therefore, not during the pendency of the conciliation proceedings. Held that, in cases where no settlement was arrived at the conciliation proceedings terminated when the report of the Conciliation Officer was received by the appropriate Government and not on the expiry of 14 days from the commencement of the proceedings. The commencement and termination of conciliation proceedings were determined by section 20 and not by section 12(6). The dismissal of the workman was during the pendency of the conciliation proceedings and the appellants were guilty under section 31(1) of the Act. Workers of the Industry Colliery, Dhanbad vs Management Of the Industry Colliery, ; ; Colliery Mazdoor Congress, Asansol vs New Beerbhoom Coal Co. Ltd., , applied.\n",
            "rouge1: 0.1609\n",
            "rouge2: 0.0173\n",
            "rougeL: 0.1092\n"
          ]
        }
      ]
    }
  ],
  "metadata": {
    "colab": {
      "toc_visible": true,
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}